Package apple.metalperformanceshaders

Class MPSRNNImageInferenceLayer

  • All Implemented Interfaces:
    NSCoding, NSCopying, NSSecureCoding, NSObject
    public class MPSRNNImageInferenceLayer
extends MPSCNNKernel
    MPSRNNImageInferenceLayer [@dependency] This depends on Metal.framework The MPSRNNImageInferenceLayer specifies a recurrent neural network layer for inference on MPSImages. Currently two types of recurrent layers are supported: ones that operate with convolutions on images: @ref MPSRNNImageInferenceLayer and one that operates on matrices: @ref MPSRNNMatrixInferenceLayer. The former can be often used to implement the latter by using 1x1-images, but due to image size restrictions and performance, it is advisable to use @ref MPSRNNMatrixInferenceLayer for linear recurrent layers. A MPSRNNImageInferenceLayer is initialized using a @ref MPSRNNLayerDescriptor, which further specifies the recurrent network layer, or an array of @ref MPSRNNLayerDescriptors, which specifies a stack of recurrent layers, that can operate in parallel a subset of the inputs in a sequence of inputs and recurrent outputs. Note that currently stacks with bidirectionally traversing encode functions do not support starting from a previous set of recurrent states, but this can be achieved quite easily by defining two separate unidirectional stacks of layers, and running the same input sequence on them separately (one forwards and one backwards) and ultimately combining the two result sequences as desired with auxiliary functions.

    • Constructor Detail

      • MPSRNNImageInferenceLayer

        protected MPSRNNImageInferenceLayer​(org.moe.natj.general.Pointer peer)

    • Method Detail

      • accessInstanceVariablesDirectly

        public static boolean accessInstanceVariablesDirectly()

      • allocWithZone

        public static java.lang.Object allocWithZone​(org.moe.natj.general.ptr.VoidPtr zone)

      • automaticallyNotifiesObserversForKey

        public static boolean automaticallyNotifiesObserversForKey​(java.lang.String key)

      • bidirectionalCombineMode

        public long bidirectionalCombineMode()
        [@property] bidirectionalCombineMode Defines how to combine the output-results, when encoding bidirectional layers using [@ref] encodeBidirectionalSequenceToCommandBuffer. Defaults to @ref MPSRNNBidirectionalCombineModeNone.

      • cancelPreviousPerformRequestsWithTarget

        public static void cancelPreviousPerformRequestsWithTarget​(java.lang.Object aTarget)

      • cancelPreviousPerformRequestsWithTargetSelectorObject

        public static void cancelPreviousPerformRequestsWithTargetSelectorObject​(java.lang.Object aTarget,
                                                                         org.moe.natj.objc.SEL aSelector,
                                                                         java.lang.Object anArgument)

      • classFallbacksForKeyedArchiver

        public static NSArray<java.lang.String> classFallbacksForKeyedArchiver()

      • classForKeyedUnarchiver

        public static org.moe.natj.objc.Class classForKeyedUnarchiver()

      • copyWithZoneDevice

        public java.lang.Object copyWithZoneDevice​(org.moe.natj.general.ptr.VoidPtr zone,
                                           MTLDevice device)
        Make a copy of this kernel for a new device - @see MPSKernel
        Overrides:
        copyWithZoneDevice in class MPSKernel
        Parameters:
        zone - The NSZone in which to allocate the object
        device - The device for the new MPSKernel. If nil, then use self.device.
        Returns:
        a pointer to a copy of this MPSKernel. This will fail, returning nil if the device is not supported. Devices must be MTLFeatureSet_iOS_GPUFamily2_v1 or later.

      • debugDescription_static

        public static java.lang.String debugDescription_static()

      • description_static

        public static java.lang.String description_static()

      • encodeBidirectionalSequenceToCommandBufferSourceSequenceDestinationForwardImagesDestinationBackwardImages

        public void encodeBidirectionalSequenceToCommandBufferSourceSequenceDestinationForwardImagesDestinationBackwardImages​(MTLCommandBuffer commandBuffer,
                                                                                                                      NSArray<? extends MPSImage> sourceSequence,
                                                                                                                      NSArray<? extends MPSImage> destinationForwardImages,
                                                                                                                      NSArray<? extends MPSImage> destinationBackwardImages)
        Encode an MPSRNNImageInferenceLayer kernel stack for an input image sequences into a command buffer bidirectionally. The operation proceeds as follows: The first source image x0 is passed through all forward traversing layers in the stack, ie. those that were initialized with MPSRNNSequenceDirectionForward, recurrent input is assumed zero. This produces forward output yf0 and recurrent states hf00, hf01, hf02, ... hf0n, one for each forward layer. Then x1 is passed to forward layers together with recurrent state hf00, hf01, ..., hf0n, which produces yf1, and hf10,... This procedure is iterated until the last image in the input sequence x_(N-1), which produces forward output yf(N-1). The backwards layers iterate the same sequence backwards, starting from input x_(N-1) (recurrent state zero), that produces yb(N-1) and recurrent output hb(N-1)0, hf(N-1)1, ... hb(N-1)m, one for each backwards traversing layer. Then the backwards layers handle input x_(N-2) using recurrent state hb(N-1)0, ..., et cetera, until the first image of the sequence is computed, producing output yb0. The result of the operation is either pair of sequences ({yf0, yf1, ... , yf(N-1)}, {yb0, yb1, ... , yb(N-1)}) or a combined sequence, {(yf0 + yb0), ... , (yf(N-1) + yb(N-1)) }, where '+' stands either for sum, or concatenation along feature channels, as specified by @ref bidirectionalCombineMode.
        Parameters:
        commandBuffer - A valid MTLCommandBuffer to receive the encoded filter
        sourceSequence - An array of valid MPSImage objects containing the source image sequence (x0, x1, ... x_n-1).
        destinationForwardImages - An array of valid MPSImages to be overwritten by result from forward input images. If bidirectionalCombineMode is either MPSRNNBidirectionalCombineModeAdd or MPSRNNBidirectionalCombineModeConcatenate, then will contain the combined results. destinationForwardImage may not alias with any of the source images.
        destinationBackwardImages - If bidirectionalCombineMode is MPSRNNBidirectionalCombineModeNone, then must be a valid MPSImage that will be overwritten by result from backward input image. Otherwise this parameter is ignored and can be nil. destinationBackwardImages may not alias to any of the source images.

      • encodeSequenceToCommandBufferSourceImagesDestinationImagesRecurrentInputStateRecurrentOutputStates

        public void encodeSequenceToCommandBufferSourceImagesDestinationImagesRecurrentInputStateRecurrentOutputStates​(MTLCommandBuffer commandBuffer,
                                                                                                               NSArray<? extends MPSImage> sourceImages,
                                                                                                               NSArray<? extends MPSImage> destinationImages,
                                                                                                               MPSRNNRecurrentImageState recurrentInputState,
                                                                                                               NSMutableArray<MPSRNNRecurrentImageState> recurrentOutputStates)
        Encode an MPSRNNImageInferenceLayer kernel (stack) for a sequence of inputs into a command buffer. Note that when encoding using this function the @see layerSequenceDirection is ignored and the layer stack operates as if all layers were forward feeding layers. In order to run bidirectional sequences use @ref encodeBidirectionalSequenceToCommandBuffer:sourceSequence: or alternatively run two layer stacks and combine results at the end using utility functions. [@code] MPSRNNRecurrentImageState* recurrent0 = nil; [filter encodeToCommandBuffer: cmdBuf sourceImage: source0 destinationImage: destination0 recurrentInputState: nil recurrentOutputState: &recurrent0]; [@endcode] Then use it for the next input in sequence: [@code] [filter encodeToCommandBuffer: cmdBuf sourceImage: source1 destinationImage: destination1 recurrentInputState: recurrent0 recurrentOutputState: &recurrent0]; [@endcode] And discard recurrent output of the third input: [@code] [filter encodeToCommandBuffer: cmdBuf sourceImage: source2 destinationImage: destination2 recurrentInputState: recurrent0 recurrentOutputState: nil]; [@endcode]
        Parameters:
        commandBuffer - A valid MTLCommandBuffer to receive the encoded filter
        sourceImages - An array of valid MPSImage objects containing the sequence of source images.
        destinationImages - An array valid MPSImages to be overwritten by result image sequence. destinationImages may not alias sourceImages.
        recurrentInputState - An optional state containing the output images and memory cells (for LSTMs) of the layer obtained from the previous input images in a sequence of inputs. Has to be the output of a previous call to this function or nil (assumed zero). Note: can be one of the states returned in @ref recurrentOutputStates.
        recurrentOutputStates - An optional array that will contain the recurrent output states. If nil then the recurrent output state is discarded. If @ref storeAllIntermediateStates is YES, then all intermediate states of the sequence are returned in the array, the first one corresponding to the first input in the sequence, otherwise only the last recurrent output state is returned. If recurrentOutputIsTemporary is YES and then all returned recurrent states will be temporary. @see MPSState:isTemporary. Example: In order to get a new state one can do the following:

      • hash_static

        public static long hash_static()

      • initWithCoderDevice

        public MPSRNNImageInferenceLayer initWithCoderDevice​(NSCoder aDecoder,
                                                     java.lang.Object device)
        NSSecureCoding compatability See @ref MPSKernel#initWithCoder.
        Overrides:
        initWithCoderDevice in class MPSCNNKernel
        Parameters:
        aDecoder - The NSCoder subclass with your serialized MPSRNNImageInferenceLayer
        device - The MTLDevice on which to make the MPSRNNImageInferenceLayer
        Returns:
        A new MPSRNNImageInferenceLayer object, or nil if failure.

      • initWithDevice

        public MPSRNNImageInferenceLayer initWithDevice​(java.lang.Object device)
        Description copied from class: MPSCNNKernel
        Standard init with default properties per filter type
        Overrides:
        initWithDevice in class MPSCNNKernel
        Parameters:
        device - The device that the filter will be used on. May not be NULL.
        Returns:
        A pointer to the newly initialized object. This will fail, returning nil if the device is not supported. Devices must be MTLFeatureSet_iOS_GPUFamily2_v1 or later.

      • initWithDeviceRnnDescriptor

        public MPSRNNImageInferenceLayer initWithDeviceRnnDescriptor​(MTLDevice device,
                                                             MPSRNNDescriptor rnnDescriptor)
        Initializes a convolutional RNN kernel
        Parameters:
        device - The MTLDevice on which this MPSRNNImageLayer filter will be used
        rnnDescriptor - The descriptor that defines the RNN layer
        Returns:
        A valid MPSRNNImageInferenceLayer object or nil, if failure.

      • initWithDeviceRnnDescriptors

        public MPSRNNImageInferenceLayer initWithDeviceRnnDescriptors​(MTLDevice device,
                                                              NSArray<? extends MPSRNNDescriptor> rnnDescriptors)
        Initializes a kernel that implements a stack of convolutional RNN layers
        Parameters:
        device - The MTLDevice on which this MPSRNNImageLayer filter will be used
        rnnDescriptors - An array of RNN descriptors that defines a stack of RNN layers, starting at index zero. The number of layers in stack is the number of entries in the array. All entries in the array must be valid MPSRNNDescriptors.
        Returns:
        A valid MPSRNNImageInferenceLayer object or nil, if failure.

      • inputFeatureChannels

        public long inputFeatureChannels()
        [@property] inputFeatureChannels The number of feature channels per pixel in the input image.

      • instanceMethodSignatureForSelector

        public static NSMethodSignature instanceMethodSignatureForSelector​(org.moe.natj.objc.SEL aSelector)

      • instancesRespondToSelector

        public static boolean instancesRespondToSelector​(org.moe.natj.objc.SEL aSelector)

      • isSubclassOfClass

        public static boolean isSubclassOfClass​(org.moe.natj.objc.Class aClass)

      • keyPathsForValuesAffectingValueForKey

        public static NSSet<java.lang.String> keyPathsForValuesAffectingValueForKey​(java.lang.String key)

      • new_objc

        public static java.lang.Object new_objc()

      • numberOfLayers

        public long numberOfLayers()
        [@property] numberOfLayers Number of layers in the filter-stack. This will be one when using initWithDevice:rnnDescriptor to initialize this filter and the number of entries in the array 'rnnDescriptors' when initializing this filter with initWithDevice:rnnDescriptors.

      • outputFeatureChannels

        public long outputFeatureChannels()
        [@property] outputFeatureChannels The number of feature channels per pixel in the output image.

      • recurrentOutputIsTemporary

        public boolean recurrentOutputIsTemporary()
        [@property] recurrentOutputIsTemporary How output states from @ref encodeSequenceToCommandBuffer are constructed. Defaults to NO. For reference @see MPSState.

      • resolveClassMethod

        public static boolean resolveClassMethod​(org.moe.natj.objc.SEL sel)

      • resolveInstanceMethod

        public static boolean resolveInstanceMethod​(org.moe.natj.objc.SEL sel)

      • setBidirectionalCombineMode

        public void setBidirectionalCombineMode​(long value)
        [@property] bidirectionalCombineMode Defines how to combine the output-results, when encoding bidirectional layers using [@ref] encodeBidirectionalSequenceToCommandBuffer. Defaults to @ref MPSRNNBidirectionalCombineModeNone.

      • setRecurrentOutputIsTemporary

        public void setRecurrentOutputIsTemporary​(boolean value)
        [@property] recurrentOutputIsTemporary How output states from @ref encodeSequenceToCommandBuffer are constructed. Defaults to NO. For reference @see MPSState.

      • setStoreAllIntermediateStates

        public void setStoreAllIntermediateStates​(boolean value)
        [@property] storeAllIntermediateStates If YES then calls to @ref encodeSequenceToCommandBuffer return every recurrent state in the array: recurrentOutputStates. Defaults to NO.

      • setVersion_static

        public static void setVersion_static​(long aVersion)

      • storeAllIntermediateStates

        public boolean storeAllIntermediateStates()
        [@property] storeAllIntermediateStates If YES then calls to @ref encodeSequenceToCommandBuffer return every recurrent state in the array: recurrentOutputStates. Defaults to NO.

      • superclass_static

        public static org.moe.natj.objc.Class superclass_static()

      • supportsSecureCoding

        public static boolean supportsSecureCoding()

      • _supportsSecureCoding

        public boolean _supportsSecureCoding()
        Description copied from interface: NSSecureCoding
        This property must return YES on all classes that allow secure coding. Subclasses of classes that adopt NSSecureCoding and override initWithCoder: must also override this method and return YES. The Secure Coding Guide should be consulted when writing methods that decode data.
        Specified by:
        _supportsSecureCoding in interface NSSecureCoding
        Overrides:
        _supportsSecureCoding in class MPSCNNKernel

      • version_static

        public static long version_static()