org.nd4j.linalg.learning

Class AdaGradUpdater

java.lang.Object

org.nd4j.linalg.learning.AdaGradUpdater

All Implemented Interfaces:

GradientUpdater<AdaGrad>

public class AdaGradUpdater
extends Object
implements GradientUpdater<AdaGrad>

Vectorized Learning Rate used per Connection Weight

Adapted from: http://xcorr.net/2014/01/23/adagrad-eliminating-learning-rates-in-stochastic-gradient-descent/ See also http://cs231n.github.io/neural-networks-3/#ada

Author:

Adam Gibson

Field Summary

Fields

Modifier and Type	Field and Description
INDArray	historicalGradient
protected double	learningRate
protected int	numIterations
int[]	shape

Constructor Summary

Constructors

Constructor and Description
AdaGradUpdater(AdaGrad config)

Method Summary

Modifier and Type	Method and Description
void	applyUpdater(INDArray gradient, int iteration, int epoch) Gets feature specific learning rates Adagrad keeps a history of gradients being passed in.
void	setStateViewArray(INDArray viewArray, int[] gradientShape, char gradientOrder, boolean initialize) For the internal updater state (if any): set this to use the provided array.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.nd4j.linalg.learning.GradientUpdater

getConfig

Field Detail

historicalGradient

public INDArray historicalGradient

shape

public int[] shape

learningRate

protected double learningRate

numIterations

protected int numIterations

Constructor Detail

AdaGradUpdater

public AdaGradUpdater(AdaGrad config)

Method Detail

setStateViewArray

public void setStateViewArray(INDArray viewArray,
                              int[] gradientShape,
                              char gradientOrder,
                              boolean initialize)

Description copied from interface: GradientUpdater

For the internal updater state (if any): set this to use the provided array. Used during initialization, and when restoring the updater state (after serialization, for example)

Specified by:

setStateViewArray in interface GradientUpdater<AdaGrad>

Parameters:

viewArray - Array (that is a view of a larger array) to use for the state.

initialize - If true: the updater must initialize the view array. If false: no change to view array contents

applyUpdater

public void applyUpdater(INDArray gradient,
                         int iteration,
                         int epoch)

Gets feature specific learning rates Adagrad keeps a history of gradients being passed in. Note that each gradient passed in becomes adapted over time, hence the opName adagrad

Specified by:

applyUpdater in interface GradientUpdater<AdaGrad>

Parameters:

gradient - the gradient to get learning rates for

iteration -