Package apple.metalperformanceshaders
Class MPSNNOptimizerAdam
- java.lang.Object
-
- org.moe.natj.general.NativeObject
-
- org.moe.natj.objc.ObjCObject
-
- apple.NSObject
-
- apple.metalperformanceshaders.MPSKernel
-
- apple.metalperformanceshaders.MPSNNOptimizer
-
- apple.metalperformanceshaders.MPSNNOptimizerAdam
-
- All Implemented Interfaces:
NSCoding,NSCopying,NSSecureCoding,NSObject
public class MPSNNOptimizerAdam extends MPSNNOptimizer
MPSNNOptimizerAdam The MPSNNOptimizerAdam performs an Adam Update Initialization time m[0] = 0 (Initialize initial 1st moment vector aka momentum, user is responsible for this) v[0] = 0 (Initialize initial 2nd moment vector aka velocity, user is responsible for this) t = 0 (Initialize timestep) https://arxiv.org/abs/1412.6980 At update time: t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t) m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) variable = variable - lr[t] * m[t] / (sqrt(v[t]) + epsilon) where, g is gradient of error wrt variable v[t] is velocity m[t] is momentum
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class apple.NSObject
NSObject.Function_instanceMethodForSelector_ret, NSObject.Function_methodForSelector_ret
-
-
Constructor Summary
Constructors Modifier Constructor Description protectedMPSNNOptimizerAdam(org.moe.natj.general.Pointer peer)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description boolean_supportsSecureCoding()This property must return YES on all classes that allow secure coding.static booleanaccessInstanceVariablesDirectly()static MPSNNOptimizerAdamalloc()static java.lang.ObjectallocWithZone(org.moe.natj.general.ptr.VoidPtr zone)static booleanautomaticallyNotifiesObserversForKey(java.lang.String key)doublebeta1()[@property] beta1 The beta1 at which we update values Default value is 0.9doublebeta2()[@property] beta2 The beta2 at which we update values Default value is 0.999static voidcancelPreviousPerformRequestsWithTarget(java.lang.Object aTarget)static voidcancelPreviousPerformRequestsWithTargetSelectorObject(java.lang.Object aTarget, org.moe.natj.objc.SEL aSelector, java.lang.Object anArgument)static NSArray<java.lang.String>classFallbacksForKeyedArchiver()static org.moe.natj.objc.ClassclassForKeyedUnarchiver()static java.lang.StringdebugDescription_static()static java.lang.Stringdescription_static()voidencodeToCommandBufferBatchNormalizationGradientStateBatchNormalizationSourceStateInputMomentumVectorsInputVelocityVectorsMaximumVelocityVectorsResultState(MTLCommandBuffer commandBuffer, MPSCNNBatchNormalizationState batchNormalizationGradientState, MPSCNNBatchNormalizationState batchNormalizationSourceState, NSArray<? extends MPSVector> inputMomentumVectors, NSArray<? extends MPSVector> inputVelocityVectors, NSArray<? extends MPSVector> maximumVelocityVectors, MPSCNNNormalizationGammaAndBetaState resultState)Encode an AMSGrad variant of MPSNNOptimizerAdam object to a command buffer to perform out of place update The following operations would be applied At update time: t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t) m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) maxVel[t] = max(maxVel[t-1],v[t]) variable = variable - lr[t] * m[t] / (sqrt(maxVel[t]) + epsilon)voidencodeToCommandBufferBatchNormalizationGradientStateBatchNormalizationSourceStateInputMomentumVectorsInputVelocityVectorsResultState(MTLCommandBuffer commandBuffer, MPSCNNBatchNormalizationState batchNormalizationGradientState, MPSCNNBatchNormalizationState batchNormalizationSourceState, NSArray<? extends MPSVector> inputMomentumVectors, NSArray<? extends MPSVector> inputVelocityVectors, MPSCNNNormalizationGammaAndBetaState resultState)Encode an MPSNNOptimizerAdam object to a command buffer to perform out of place update The following operations would be applied t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t) m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) variable = variable - lr[t] * m[t] / (sqrt(v[t]) + epsilon)voidencodeToCommandBufferBatchNormalizationStateInputMomentumVectorsInputVelocityVectorsMaximumVelocityVectorsResultState(MTLCommandBuffer commandBuffer, MPSCNNBatchNormalizationState batchNormalizationState, NSArray<? extends MPSVector> inputMomentumVectors, NSArray<? extends MPSVector> inputVelocityVectors, NSArray<? extends MPSVector> maximumVelocityVectors, MPSCNNNormalizationGammaAndBetaState resultState)Encode an AMSGrad variant of MPSNNOptimizerAdam object to a command buffer to perform out of place update The following operations would be applied At update time: t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t) m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) maxVel[t] = max(maxVel[t-1],v[t]) variable = variable - lr[t] * m[t] / (sqrt(maxVel[t]) + epsilon)voidencodeToCommandBufferBatchNormalizationStateInputMomentumVectorsInputVelocityVectorsResultState(MTLCommandBuffer commandBuffer, MPSCNNBatchNormalizationState batchNormalizationState, NSArray<? extends MPSVector> inputMomentumVectors, NSArray<? extends MPSVector> inputVelocityVectors, MPSCNNNormalizationGammaAndBetaState resultState)Encode an MPSNNOptimizerAdam object to a command buffer to perform out of place update The following operations would be applied t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t) m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) variable = variable - lr[t] * m[t] / (sqrt(v[t]) + epsilon)voidencodeToCommandBufferConvolutionGradientStateConvolutionSourceStateInputMomentumVectorsInputVelocityVectorsMaximumVelocityVectorsResultState(MTLCommandBuffer commandBuffer, MPSCNNConvolutionGradientState convolutionGradientState, MPSCNNConvolutionWeightsAndBiasesState convolutionSourceState, NSArray<? extends MPSVector> inputMomentumVectors, NSArray<? extends MPSVector> inputVelocityVectors, NSArray<? extends MPSVector> maximumVelocityVectors, MPSCNNConvolutionWeightsAndBiasesState resultState)Encode an AMSGrad variant of MPSNNOptimizerAdam object to a command buffer to perform out of place update The following operations would be applied At update time: t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t) m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) maxVel[t] = max(maxVel[t-1],v[t]) variable = variable - lr[t] * m[t] / (sqrt(maxVel[t]) + epsilon)voidencodeToCommandBufferConvolutionGradientStateConvolutionSourceStateInputMomentumVectorsInputVelocityVectorsResultState(MTLCommandBuffer commandBuffer, MPSCNNConvolutionGradientState convolutionGradientState, MPSCNNConvolutionWeightsAndBiasesState convolutionSourceState, NSArray<? extends MPSVector> inputMomentumVectors, NSArray<? extends MPSVector> inputVelocityVectors, MPSCNNConvolutionWeightsAndBiasesState resultState)Encode an MPSNNOptimizerAdam object to a command buffer to perform out of place update The following operations would be applied t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t) m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) variable = variable - lr[t] * m[t] / (sqrt(v[t]) + epsilon)voidencodeToCommandBufferInputGradientMatrixInputValuesMatrixInputMomentumMatrixInputVelocityMatrixMaximumVelocityMatrixResultValuesMatrix(MTLCommandBuffer commandBuffer, MPSMatrix inputGradientMatrix, MPSMatrix inputValuesMatrix, MPSMatrix inputMomentumMatrix, MPSMatrix inputVelocityMatrix, MPSMatrix maximumVelocityMatrix, MPSMatrix resultValuesMatrix)voidencodeToCommandBufferInputGradientMatrixInputValuesMatrixInputMomentumMatrixInputVelocityMatrixResultValuesMatrix(MTLCommandBuffer commandBuffer, MPSMatrix inputGradientMatrix, MPSMatrix inputValuesMatrix, MPSMatrix inputMomentumMatrix, MPSMatrix inputVelocityMatrix, MPSMatrix resultValuesMatrix)voidencodeToCommandBufferInputGradientVectorInputValuesVectorInputMomentumVectorInputVelocityVectorMaximumVelocityVectorResultValuesVector(MTLCommandBuffer commandBuffer, MPSVector inputGradientVector, MPSVector inputValuesVector, MPSVector inputMomentumVector, MPSVector inputVelocityVector, MPSVector maximumVelocityVector, MPSVector resultValuesVector)Encode an AMSGrad variant of MPSNNOptimizerAdam object to a command buffer to perform out of place update The following operations would be applied At update time: t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t) m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) maxVel[t] = max(maxVel[t-1],v[t]) variable = variable - lr[t] * m[t] / (sqrt(maxVel[t]) + epsilon)voidencodeToCommandBufferInputGradientVectorInputValuesVectorInputMomentumVectorInputVelocityVectorResultValuesVector(MTLCommandBuffer commandBuffer, MPSVector inputGradientVector, MPSVector inputValuesVector, MPSVector inputMomentumVector, MPSVector inputVelocityVector, MPSVector resultValuesVector)Encode an MPSNNOptimizerAdam object to a command buffer to perform out of place update The following operations would be applied t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t) m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) variable = variable - lr[t] * m[t] / (sqrt(v[t]) + epsilon)floatepsilon()[@property] epsilon The epsilon at which we update values This value is usually used to ensure to avoid divide by 0, default value is 1e-8static longhash_static()MPSNNOptimizerAdaminit()MPSNNOptimizerAdaminitWithCoder(NSCoder aDecoder)NS_DESIGNATED_INITIALIZERMPSNNOptimizerAdaminitWithCoderDevice(NSCoder aDecoder, java.lang.Object device)NSSecureCoding compatability While the standard NSSecureCoding/NSCoding method -initWithCoder: should work, since the file can't know which device your data is allocated on, we have to guess and may guess incorrectly.MPSNNOptimizerAdaminitWithDevice(java.lang.Object device)Standard init with default properties per filter typeMPSNNOptimizerAdaminitWithDeviceBeta1Beta2EpsilonTimeStepOptimizerDescriptor(MTLDevice device, double beta1, double beta2, float epsilon, long timeStep, MPSNNOptimizerDescriptor optimizerDescriptor)Full initialization for the adam updateMPSNNOptimizerAdaminitWithDeviceLearningRate(MTLDevice device, float learningRate)Convenience initialization for the adam updatestatic NSObject.Function_instanceMethodForSelector_retinstanceMethodForSelector(org.moe.natj.objc.SEL aSelector)static NSMethodSignatureinstanceMethodSignatureForSelector(org.moe.natj.objc.SEL aSelector)static booleaninstancesRespondToSelector(org.moe.natj.objc.SEL aSelector)static booleanisSubclassOfClass(org.moe.natj.objc.Class aClass)static NSSet<java.lang.String>keyPathsForValuesAffectingValueForKey(java.lang.String key)static java.lang.Objectnew_objc()static booleanresolveClassMethod(org.moe.natj.objc.SEL sel)static booleanresolveInstanceMethod(org.moe.natj.objc.SEL sel)voidsetTimeStep(long value)[@property] timeStep Current timeStep for the update, number of times update has occurredstatic voidsetVersion_static(long aVersion)static org.moe.natj.objc.Classsuperclass_static()static booleansupportsSecureCoding()longtimeStep()[@property] timeStep Current timeStep for the update, number of times update has occurredstatic longversion_static()-
Methods inherited from class apple.metalperformanceshaders.MPSNNOptimizer
applyGradientClipping, gradientClipMax, gradientClipMin, gradientRescale, learningRate, regularizationScale, regularizationType, setApplyGradientClipping, setLearningRate
-
Methods inherited from class apple.metalperformanceshaders.MPSKernel
copyWithZone, copyWithZoneDevice, device, encodeWithCoder, label, options, setLabel, setOptions
-
Methods inherited from class apple.NSObject
accessibilityActivate, accessibilityActivationPoint, accessibilityAssistiveTechnologyFocusedIdentifiers, accessibilityAttributedHint, accessibilityAttributedLabel, accessibilityAttributedUserInputLabels, accessibilityAttributedValue, accessibilityContainerType, accessibilityCustomActions, accessibilityCustomRotors, accessibilityDecrement, accessibilityDragSourceDescriptors, accessibilityDropPointDescriptors, accessibilityElementAtIndex, accessibilityElementCount, accessibilityElementDidBecomeFocused, accessibilityElementDidLoseFocus, accessibilityElementIsFocused, accessibilityElements, accessibilityElementsHidden, accessibilityFrame, accessibilityHint, accessibilityIncrement, accessibilityLabel, accessibilityLanguage, accessibilityNavigationStyle, accessibilityPath, accessibilityPerformEscape, accessibilityPerformMagicTap, accessibilityRespondsToUserInteraction, accessibilityScroll, accessibilityTextualContext, accessibilityTraits, accessibilityUserInputLabels, accessibilityValue, accessibilityViewIsModal, addObserverForKeyPathOptionsContext, attemptRecoveryFromErrorOptionIndex, attemptRecoveryFromErrorOptionIndexDelegateDidRecoverSelectorContextInfo, autoContentAccessingProxy, awakeAfterUsingCoder, awakeFromNib, class_objc, classForCoder, classForKeyedArchiver, copy, dealloc, debugDescription, description, dictionaryWithValuesForKeys, didChangeValueForKey, didChangeValueForKeyWithSetMutationUsingObjects, didChangeValuesAtIndexesForKey, doesNotRecognizeSelector, fileManagerShouldProceedAfterError, fileManagerWillProcessPath, finalize_objc, forwardingTargetForSelector, forwardInvocation, hash, indexOfAccessibilityElement, isAccessibilityElement, isEqual, isKindOfClass, isMemberOfClass, isProxy, methodForSelector, methodSignatureForSelector, mutableArrayValueForKey, mutableArrayValueForKeyPath, mutableCopy, mutableOrderedSetValueForKey, mutableOrderedSetValueForKeyPath, mutableSetValueForKey, mutableSetValueForKeyPath, observationInfo, observeValueForKeyPathOfObjectChangeContext, performSelector, performSelectorInBackgroundWithObject, performSelectorOnMainThreadWithObjectWaitUntilDone, performSelectorOnMainThreadWithObjectWaitUntilDoneModes, performSelectorOnThreadWithObjectWaitUntilDone, performSelectorOnThreadWithObjectWaitUntilDoneModes, performSelectorWithObject, performSelectorWithObjectAfterDelay, performSelectorWithObjectAfterDelayInModes, performSelectorWithObjectWithObject, prepareForInterfaceBuilder, provideImageDataBytesPerRowOrigin_Size_UserInfo, removeObserverForKeyPath, removeObserverForKeyPathContext, replacementObjectForCoder, replacementObjectForKeyedArchiver, respondsToSelector, self, setAccessibilityActivationPoint, setAccessibilityAttributedHint, setAccessibilityAttributedLabel, setAccessibilityAttributedUserInputLabels, setAccessibilityAttributedValue, setAccessibilityContainerType, setAccessibilityCustomActions, setAccessibilityCustomRotors, setAccessibilityDragSourceDescriptors, setAccessibilityDropPointDescriptors, setAccessibilityElements, setAccessibilityElementsHidden, setAccessibilityFrame, setAccessibilityHint, setAccessibilityLabel, setAccessibilityLanguage, setAccessibilityNavigationStyle, setAccessibilityPath, setAccessibilityRespondsToUserInteraction, setAccessibilityTextualContext, setAccessibilityTraits, setAccessibilityUserInputLabels, setAccessibilityValue, setAccessibilityViewIsModal, setIsAccessibilityElement, setNilValueForKey, setObservationInfo, setShouldGroupAccessibilityChildren, setValueForKey, setValueForKeyPath, setValueForUndefinedKey, setValuesForKeysWithDictionary, shouldGroupAccessibilityChildren, superclass, validateValueForKeyError, validateValueForKeyPathError, valueForKey, valueForKeyPath, valueForUndefinedKey, willChangeValueForKey, willChangeValueForKeyWithSetMutationUsingObjects, willChangeValuesAtIndexesForKey
-
-
-
-
Method Detail
-
accessInstanceVariablesDirectly
public static boolean accessInstanceVariablesDirectly()
-
alloc
public static MPSNNOptimizerAdam alloc()
-
allocWithZone
public static java.lang.Object allocWithZone(org.moe.natj.general.ptr.VoidPtr zone)
-
automaticallyNotifiesObserversForKey
public static boolean automaticallyNotifiesObserversForKey(java.lang.String key)
-
beta1
public double beta1()
[@property] beta1 The beta1 at which we update values Default value is 0.9
-
beta2
public double beta2()
[@property] beta2 The beta2 at which we update values Default value is 0.999
-
cancelPreviousPerformRequestsWithTarget
public static void cancelPreviousPerformRequestsWithTarget(java.lang.Object aTarget)
-
cancelPreviousPerformRequestsWithTargetSelectorObject
public static void cancelPreviousPerformRequestsWithTargetSelectorObject(java.lang.Object aTarget, org.moe.natj.objc.SEL aSelector, java.lang.Object anArgument)
-
classFallbacksForKeyedArchiver
public static NSArray<java.lang.String> classFallbacksForKeyedArchiver()
-
classForKeyedUnarchiver
public static org.moe.natj.objc.Class classForKeyedUnarchiver()
-
debugDescription_static
public static java.lang.String debugDescription_static()
-
description_static
public static java.lang.String description_static()
-
encodeToCommandBufferBatchNormalizationGradientStateBatchNormalizationSourceStateInputMomentumVectorsInputVelocityVectorsMaximumVelocityVectorsResultState
public void encodeToCommandBufferBatchNormalizationGradientStateBatchNormalizationSourceStateInputMomentumVectorsInputVelocityVectorsMaximumVelocityVectorsResultState(MTLCommandBuffer commandBuffer, MPSCNNBatchNormalizationState batchNormalizationGradientState, MPSCNNBatchNormalizationState batchNormalizationSourceState, NSArray<? extends MPSVector> inputMomentumVectors, NSArray<? extends MPSVector> inputVelocityVectors, NSArray<? extends MPSVector> maximumVelocityVectors, MPSCNNNormalizationGammaAndBetaState resultState)
Encode an AMSGrad variant of MPSNNOptimizerAdam object to a command buffer to perform out of place update The following operations would be applied At update time: t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t) m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) maxVel[t] = max(maxVel[t-1],v[t]) variable = variable - lr[t] * m[t] / (sqrt(maxVel[t]) + epsilon)- Parameters:
commandBuffer- A valid MTLCommandBuffer to receive the encoded kernel.batchNormalizationGradientState- A valid MPSCNNBatchNormalizationState object which specifies the input state with gradients for this update.batchNormalizationSourceState- A valid MPSCNNBatchNormalizationState object which specifies the input state with original gamma/beta for this update.inputMomentumVectors- An array MPSVector object which specifies the gradient momentum vectors which will be updated and overwritten. The index 0 corresponds to gamma, index 1 corresponds to beta, array can be of size 1 in which case beta won't be updatedinputVelocityVectors- An array MPSVector object which specifies the gradient velocity vectors which will be updated and overwritten. The index 0 corresponds to gamma, index 1 corresponds to beta, array can be of size 1 in which case beta won't be updatedmaximumVelocityVectors- An array MPSVector object which specifies the maximum velocity vectors which will be updated and overwritten. The index 0 corresponds to weights, index 1 corresponds to biases, array can be of size 1 in which case biases won't be updated. May be nil, if nil then normal Adam optimizer behaviour is followed.resultState- A valid MPSCNNNormalizationGammaAndBetaState object which specifies the resultValues state which will be updated and overwritten.
-
encodeToCommandBufferBatchNormalizationGradientStateBatchNormalizationSourceStateInputMomentumVectorsInputVelocityVectorsResultState
public void encodeToCommandBufferBatchNormalizationGradientStateBatchNormalizationSourceStateInputMomentumVectorsInputVelocityVectorsResultState(MTLCommandBuffer commandBuffer, MPSCNNBatchNormalizationState batchNormalizationGradientState, MPSCNNBatchNormalizationState batchNormalizationSourceState, NSArray<? extends MPSVector> inputMomentumVectors, NSArray<? extends MPSVector> inputVelocityVectors, MPSCNNNormalizationGammaAndBetaState resultState)
Encode an MPSNNOptimizerAdam object to a command buffer to perform out of place update The following operations would be applied t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t) m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) variable = variable - lr[t] * m[t] / (sqrt(v[t]) + epsilon)- Parameters:
commandBuffer- A valid MTLCommandBuffer to receive the encoded kernel.batchNormalizationGradientState- A valid MPSCNNBatchNormalizationState object which specifies the input state with gradients for this update.batchNormalizationSourceState- A valid MPSCNNBatchNormalizationState object which specifies the input state with original gamma/beta for this update.inputMomentumVectors- An array MPSVector object which specifies the gradient momentum vectors which will be updated and overwritten. The index 0 corresponds to gamma, index 1 corresponds to beta, array can be of size 1 in which case beta won't be updatedinputVelocityVectors- An array MPSVector object which specifies the gradient velocity vectors which will be updated and overwritten. The index 0 corresponds to gamma, index 1 corresponds to beta, array can be of size 1 in which case beta won't be updatedresultState- A valid MPSCNNNormalizationGammaAndBetaState object which specifies the resultValues state which will be updated and overwritten.
-
encodeToCommandBufferBatchNormalizationStateInputMomentumVectorsInputVelocityVectorsMaximumVelocityVectorsResultState
public void encodeToCommandBufferBatchNormalizationStateInputMomentumVectorsInputVelocityVectorsMaximumVelocityVectorsResultState(MTLCommandBuffer commandBuffer, MPSCNNBatchNormalizationState batchNormalizationState, NSArray<? extends MPSVector> inputMomentumVectors, NSArray<? extends MPSVector> inputVelocityVectors, NSArray<? extends MPSVector> maximumVelocityVectors, MPSCNNNormalizationGammaAndBetaState resultState)
Encode an AMSGrad variant of MPSNNOptimizerAdam object to a command buffer to perform out of place update The following operations would be applied At update time: t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t) m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) maxVel[t] = max(maxVel[t-1],v[t]) variable = variable - lr[t] * m[t] / (sqrt(maxVel[t]) + epsilon)- Parameters:
commandBuffer- A valid MTLCommandBuffer to receive the encoded kernel.batchNormalizationState- A valid MPSCNNBatchNormalizationState object which specifies the input state with gradients and original gamma/beta for this update.inputMomentumVectors- An array MPSVector object which specifies the gradient momentum vectors which will be updated and overwritten. The index 0 corresponds to gamma, index 1 corresponds to beta, array can be of size 1 in which case beta won't be updatedinputVelocityVectors- An array MPSVector object which specifies the gradient velocity vectors which will be updated and overwritten. The index 0 corresponds to gamma, index 1 corresponds to beta, array can be of size 1 in which case beta won't be updatedmaximumVelocityVectors- An array MPSVector object which specifies the maximum velocity vectors which will be updated and overwritten. The index 0 corresponds to weights, index 1 corresponds to biases, array can be of size 1 in which case biases won't be updated. May be nil, if nil then normal Adam optimizer behaviour is followed.resultState- A valid MPSCNNNormalizationGammaAndBetaState object which specifies the resultValues state which will be updated and overwritten.
-
encodeToCommandBufferBatchNormalizationStateInputMomentumVectorsInputVelocityVectorsResultState
public void encodeToCommandBufferBatchNormalizationStateInputMomentumVectorsInputVelocityVectorsResultState(MTLCommandBuffer commandBuffer, MPSCNNBatchNormalizationState batchNormalizationState, NSArray<? extends MPSVector> inputMomentumVectors, NSArray<? extends MPSVector> inputVelocityVectors, MPSCNNNormalizationGammaAndBetaState resultState)
Encode an MPSNNOptimizerAdam object to a command buffer to perform out of place update The following operations would be applied t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t) m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) variable = variable - lr[t] * m[t] / (sqrt(v[t]) + epsilon)- Parameters:
commandBuffer- A valid MTLCommandBuffer to receive the encoded kernel.batchNormalizationState- A valid MPSCNNBatchNormalizationState object which specifies the input state with gradients and original gamma/beta for this update.inputMomentumVectors- An array MPSVector object which specifies the gradient momentum vectors which will be updated and overwritten. The index 0 corresponds to gamma, index 1 corresponds to beta, array can be of size 1 in which case beta won't be updatedinputVelocityVectors- An array MPSVector object which specifies the gradient velocity vectors which will be updated and overwritten. The index 0 corresponds to gamma, index 1 corresponds to beta, array can be of size 1 in which case beta won't be updatedresultState- A valid MPSCNNNormalizationGammaAndBetaState object which specifies the resultValues state which will be updated and overwritten.
-
encodeToCommandBufferConvolutionGradientStateConvolutionSourceStateInputMomentumVectorsInputVelocityVectorsMaximumVelocityVectorsResultState
public void encodeToCommandBufferConvolutionGradientStateConvolutionSourceStateInputMomentumVectorsInputVelocityVectorsMaximumVelocityVectorsResultState(MTLCommandBuffer commandBuffer, MPSCNNConvolutionGradientState convolutionGradientState, MPSCNNConvolutionWeightsAndBiasesState convolutionSourceState, NSArray<? extends MPSVector> inputMomentumVectors, NSArray<? extends MPSVector> inputVelocityVectors, NSArray<? extends MPSVector> maximumVelocityVectors, MPSCNNConvolutionWeightsAndBiasesState resultState)
Encode an AMSGrad variant of MPSNNOptimizerAdam object to a command buffer to perform out of place update The following operations would be applied At update time: t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t) m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) maxVel[t] = max(maxVel[t-1],v[t]) variable = variable - lr[t] * m[t] / (sqrt(maxVel[t]) + epsilon)- Parameters:
commandBuffer- A valid MTLCommandBuffer to receive the encoded kernel.convolutionGradientState- A valid MPSCNNConvolutionGradientState object which specifies the input state with gradients for this update.convolutionSourceState- A valid MPSCNNConvolutionWeightsAndBiasesState object which specifies the input state with values to be updated.inputMomentumVectors- An array MPSVector object which specifies the gradient momentum vectors which will be updated and overwritten. The index 0 corresponds to weights, index 1 corresponds to biases, array can be of size 1 in which case biases won't be updatedinputVelocityVectors- An array MPSVector object which specifies the gradient velocity vectors which will be updated and overwritten. The index 0 corresponds to weights, index 1 corresponds to biases, array can be of size 1 in which case biases won't be updatedmaximumVelocityVectors- An array MPSVector object which specifies the maximum velocity vectors which will be updated and overwritten. The index 0 corresponds to weights, index 1 corresponds to biases, array can be of size 1 in which case biases won't be updated. May be nil, if nil then normal Adam optimizer behaviour is followed.resultState- A valid MPSCNNConvolutionWeightsAndBiasesState object which specifies the resultValues state which will be updated and overwritten.
-
encodeToCommandBufferConvolutionGradientStateConvolutionSourceStateInputMomentumVectorsInputVelocityVectorsResultState
public void encodeToCommandBufferConvolutionGradientStateConvolutionSourceStateInputMomentumVectorsInputVelocityVectorsResultState(MTLCommandBuffer commandBuffer, MPSCNNConvolutionGradientState convolutionGradientState, MPSCNNConvolutionWeightsAndBiasesState convolutionSourceState, NSArray<? extends MPSVector> inputMomentumVectors, NSArray<? extends MPSVector> inputVelocityVectors, MPSCNNConvolutionWeightsAndBiasesState resultState)
Encode an MPSNNOptimizerAdam object to a command buffer to perform out of place update The following operations would be applied t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t) m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) variable = variable - lr[t] * m[t] / (sqrt(v[t]) + epsilon)- Parameters:
commandBuffer- A valid MTLCommandBuffer to receive the encoded kernel.convolutionGradientState- A valid MPSCNNConvolutionGradientState object which specifies the input state with gradients for this update.convolutionSourceState- A valid MPSCNNConvolutionWeightsAndBiasesState object which specifies the input state with values to be updated.inputMomentumVectors- An array MPSVector object which specifies the gradient momentum vectors which will be updated and overwritten. The index 0 corresponds to weights, index 1 corresponds to biases, array can be of size 1 in which case biases won't be updatedinputVelocityVectors- An array MPSVector object which specifies the gradient velocity vectors which will be updated and overwritten. The index 0 corresponds to weights, index 1 corresponds to biases, array can be of size 1 in which case biases won't be updatedresultState- A valid MPSCNNConvolutionWeightsAndBiasesState object which specifies the resultValues state which will be updated and overwritten.
-
encodeToCommandBufferInputGradientMatrixInputValuesMatrixInputMomentumMatrixInputVelocityMatrixMaximumVelocityMatrixResultValuesMatrix
public void encodeToCommandBufferInputGradientMatrixInputValuesMatrixInputMomentumMatrixInputVelocityMatrixMaximumVelocityMatrixResultValuesMatrix(MTLCommandBuffer commandBuffer, MPSMatrix inputGradientMatrix, MPSMatrix inputValuesMatrix, MPSMatrix inputMomentumMatrix, MPSMatrix inputVelocityMatrix, MPSMatrix maximumVelocityMatrix, MPSMatrix resultValuesMatrix)
-
encodeToCommandBufferInputGradientMatrixInputValuesMatrixInputMomentumMatrixInputVelocityMatrixResultValuesMatrix
public void encodeToCommandBufferInputGradientMatrixInputValuesMatrixInputMomentumMatrixInputVelocityMatrixResultValuesMatrix(MTLCommandBuffer commandBuffer, MPSMatrix inputGradientMatrix, MPSMatrix inputValuesMatrix, MPSMatrix inputMomentumMatrix, MPSMatrix inputVelocityMatrix, MPSMatrix resultValuesMatrix)
-
encodeToCommandBufferInputGradientVectorInputValuesVectorInputMomentumVectorInputVelocityVectorMaximumVelocityVectorResultValuesVector
public void encodeToCommandBufferInputGradientVectorInputValuesVectorInputMomentumVectorInputVelocityVectorMaximumVelocityVectorResultValuesVector(MTLCommandBuffer commandBuffer, MPSVector inputGradientVector, MPSVector inputValuesVector, MPSVector inputMomentumVector, MPSVector inputVelocityVector, MPSVector maximumVelocityVector, MPSVector resultValuesVector)
Encode an AMSGrad variant of MPSNNOptimizerAdam object to a command buffer to perform out of place update The following operations would be applied At update time: t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t) m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) maxVel[t] = max(maxVel[t-1],v[t]) variable = variable - lr[t] * m[t] / (sqrt(maxVel[t]) + epsilon)- Parameters:
commandBuffer- A valid MTLCommandBuffer to receive the encoded kernel.inputGradientVector- A valid MPSVector object which specifies the input vector of gradients for this update.inputValuesVector- A valid MPSVector object which specifies the input vector of values to be updated.inputMomentumVector- A valid MPSVector object which specifies the gradient momentum vector which will be updated and overwritten.inputVelocityVector- A valid MPSVector object which specifies the gradient velocity vector which will be updated and overwritten.maximumVelocityVector- A valid MPSVector object which specifies the maximum velocity vector which will be updated and overwritten. May be nil, if nil then normal Adam optimizer behaviour is followed.resultValuesVector- A valid MPSCNNConvolutionWeightsAndBiasesState object which specifies the resultValues state which will be updated and overwritten.
-
encodeToCommandBufferInputGradientVectorInputValuesVectorInputMomentumVectorInputVelocityVectorResultValuesVector
public void encodeToCommandBufferInputGradientVectorInputValuesVectorInputMomentumVectorInputVelocityVectorResultValuesVector(MTLCommandBuffer commandBuffer, MPSVector inputGradientVector, MPSVector inputValuesVector, MPSVector inputMomentumVector, MPSVector inputVelocityVector, MPSVector resultValuesVector)
Encode an MPSNNOptimizerAdam object to a command buffer to perform out of place update The following operations would be applied t = t + 1 lr[t] = learningRate * sqrt(1 - beta2^t) / (1 - beta1^t) m[t] = beta1 * m[t-1] + (1 - beta1) * g v[t] = beta2 * v[t-1] + (1 - beta2) * (g ^ 2) variable = variable - lr[t] * m[t] / (sqrt(v[t]) + epsilon)- Parameters:
commandBuffer- A valid MTLCommandBuffer to receive the encoded kernel.inputGradientVector- A valid MPSVector object which specifies the input vector of gradients for this update.inputValuesVector- A valid MPSVector object which specifies the input vector of values to be updated.inputMomentumVector- A valid MPSVector object which specifies the gradient momentum vector which will be updated and overwritten.inputVelocityVector- A valid MPSVector object which specifies the gradient velocity vector which will be updated and overwritten.resultValuesVector- A valid MPSVector object which specifies the resultValues vector which will be updated and overwritten.
-
epsilon
public float epsilon()
[@property] epsilon The epsilon at which we update values This value is usually used to ensure to avoid divide by 0, default value is 1e-8
-
hash_static
public static long hash_static()
-
init
public MPSNNOptimizerAdam init()
- Overrides:
initin classMPSNNOptimizer
-
initWithCoder
public MPSNNOptimizerAdam initWithCoder(NSCoder aDecoder)
Description copied from interface:NSCodingNS_DESIGNATED_INITIALIZER- Specified by:
initWithCoderin interfaceNSCoding- Overrides:
initWithCoderin classMPSNNOptimizer
-
initWithCoderDevice
public MPSNNOptimizerAdam initWithCoderDevice(NSCoder aDecoder, java.lang.Object device)
Description copied from class:MPSKernelNSSecureCoding compatability While the standard NSSecureCoding/NSCoding method -initWithCoder: should work, since the file can't know which device your data is allocated on, we have to guess and may guess incorrectly. To avoid that problem, use initWithCoder:device instead.- Overrides:
initWithCoderDevicein classMPSNNOptimizer- Parameters:
aDecoder- The NSCoder subclass with your serialized MPSKerneldevice- The MTLDevice on which to make the MPSKernel- Returns:
- A new MPSKernel object, or nil if failure.
-
initWithDevice
public MPSNNOptimizerAdam initWithDevice(java.lang.Object device)
Description copied from class:MPSKernelStandard init with default properties per filter type- Overrides:
initWithDevicein classMPSNNOptimizer- Parameters:
device- The device that the filter will be used on. May not be NULL.- Returns:
- a pointer to the newly initialized object. This will fail, returning nil if the device is not supported. Devices must be MTLFeatureSet_iOS_GPUFamily2_v1 or later.
-
initWithDeviceBeta1Beta2EpsilonTimeStepOptimizerDescriptor
public MPSNNOptimizerAdam initWithDeviceBeta1Beta2EpsilonTimeStepOptimizerDescriptor(MTLDevice device, double beta1, double beta2, float epsilon, long timeStep, MPSNNOptimizerDescriptor optimizerDescriptor)
Full initialization for the adam update- Parameters:
device- The device on which the kernel will execute.beta1- The beta1 to update valuesbeta2- The beta2 to update valuesepsilon- The epsilon at which we update valuestimeStep- The timeStep at which values will start updatingoptimizerDescriptor- The optimizerDescriptor which will have a bunch of properties to be applied- Returns:
- A valid MPSNNOptimizerAdam object or nil, if failure.
-
initWithDeviceLearningRate
public MPSNNOptimizerAdam initWithDeviceLearningRate(MTLDevice device, float learningRate)
Convenience initialization for the adam update- Parameters:
device- The device on which the kernel will execute.learningRate- The learningRate at which we will update values- Returns:
- A valid MPSNNOptimizerAdam object or nil, if failure.
-
instanceMethodForSelector
public static NSObject.Function_instanceMethodForSelector_ret instanceMethodForSelector(org.moe.natj.objc.SEL aSelector)
-
instanceMethodSignatureForSelector
public static NSMethodSignature instanceMethodSignatureForSelector(org.moe.natj.objc.SEL aSelector)
-
instancesRespondToSelector
public static boolean instancesRespondToSelector(org.moe.natj.objc.SEL aSelector)
-
isSubclassOfClass
public static boolean isSubclassOfClass(org.moe.natj.objc.Class aClass)
-
keyPathsForValuesAffectingValueForKey
public static NSSet<java.lang.String> keyPathsForValuesAffectingValueForKey(java.lang.String key)
-
new_objc
public static java.lang.Object new_objc()
-
resolveClassMethod
public static boolean resolveClassMethod(org.moe.natj.objc.SEL sel)
-
resolveInstanceMethod
public static boolean resolveInstanceMethod(org.moe.natj.objc.SEL sel)
-
setTimeStep
public void setTimeStep(long value)
[@property] timeStep Current timeStep for the update, number of times update has occurred
-
setVersion_static
public static void setVersion_static(long aVersion)
-
superclass_static
public static org.moe.natj.objc.Class superclass_static()
-
supportsSecureCoding
public static boolean supportsSecureCoding()
-
_supportsSecureCoding
public boolean _supportsSecureCoding()
Description copied from interface:NSSecureCodingThis property must return YES on all classes that allow secure coding. Subclasses of classes that adopt NSSecureCoding and override initWithCoder: must also override this method and return YES. The Secure Coding Guide should be consulted when writing methods that decode data.- Specified by:
_supportsSecureCodingin interfaceNSSecureCoding- Overrides:
_supportsSecureCodingin classMPSNNOptimizer
-
timeStep
public long timeStep()
[@property] timeStep Current timeStep for the update, number of times update has occurred
-
version_static
public static long version_static()
-
-