org.nd4j.linalg.learning.config

Class Nesterovs

java.lang.Object

org.nd4j.linalg.learning.config.Nesterovs

All Implemented Interfaces:

Serializable, Cloneable, IUpdater

public class Nesterovs
extends Object
implements IUpdater

Nesterov's momentum. Keep track of the previous layer's gradient and use it as a way of updating the gradient.

Author:

Adam Gibson

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Nesterovs.Builder

Field Summary

Fields

Modifier and Type	Field and Description
static double	DEFAULT_NESTEROV_LEARNING_RATE
static double	DEFAULT_NESTEROV_MOMENTUM

Constructor Summary

Constructors

Constructor and Description
Nesterovs()
Nesterovs(double momentum)
Nesterovs(double learningRate, double momentum)
Nesterovs(double learningRate, ISchedule momentumSchedule)
Nesterovs(ISchedule learningRateSchedule)
Nesterovs(ISchedule learningRateSchedule, double momentum)
Nesterovs(ISchedule learningRateSchedule, ISchedule momentumSchedule)

Method Summary

Modifier and Type	Method and Description
Nesterovs	clone() Clone the updater
double	currentMomentum(int iteration, int epoch)
double	getLearningRate(int iteration, int epoch) Get the learning rate - if any - for the updater, at the specified iteration and epoch.
boolean	hasLearningRate()
GradientUpdater	instantiate(INDArray viewArray, boolean initializeViewArray) Create a new gradient updater
void	setLrAndSchedule(double lr, ISchedule lrSchedule) Set the learning rate and schedule.
long	stateSize(long numParams) Determine the updater state size for the given number of parameters.

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.nd4j.linalg.learning.config.IUpdater

equals

Field Detail

DEFAULT_NESTEROV_MOMENTUM

public static final double DEFAULT_NESTEROV_MOMENTUM

See Also:

Constant Field Values

DEFAULT_NESTEROV_LEARNING_RATE

public static final double DEFAULT_NESTEROV_LEARNING_RATE

See Also:

Constant Field Values

Constructor Detail

Nesterovs

public Nesterovs()

Nesterovs

public Nesterovs(double momentum)

Nesterovs

public Nesterovs(double learningRate,
                 double momentum)

Nesterovs

public Nesterovs(ISchedule learningRateSchedule)

Nesterovs

public Nesterovs(ISchedule learningRateSchedule,
                 double momentum)

Nesterovs

public Nesterovs(ISchedule learningRateSchedule,
                 ISchedule momentumSchedule)

Nesterovs

public Nesterovs(double learningRate,
                 ISchedule momentumSchedule)

Method Detail

stateSize

public long stateSize(long numParams)

Description copied from interface: IUpdater

Determine the updater state size for the given number of parameters. Usually a integer multiple (0,1 or 2) times the number of parameters in a layer.

Specified by:

stateSize in interface IUpdater

Parameters:

numParams - Number of parameters

Returns:

Updater state size for the given number of parameters

instantiate

public GradientUpdater instantiate(INDArray viewArray,
                                   boolean initializeViewArray)

Description copied from interface: IUpdater

Create a new gradient updater

Specified by:

instantiate in interface IUpdater

Parameters:

viewArray - The updater state size view away

initializeViewArray - If true: initialise the updater state

Returns:

clone

public Nesterovs clone()

Description copied from interface: IUpdater

Clone the updater

Specified by:

clone in interface IUpdater

Overrides:

clone in class Object

getLearningRate

public double getLearningRate(int iteration,
                              int epoch)

Description copied from interface: IUpdater

Get the learning rate - if any - for the updater, at the specified iteration and epoch. Note that if no learning rate is applicable (AdaDelta, NoOp updaters etc) then Double.NaN should be return

Specified by:

getLearningRate in interface IUpdater

Parameters:

iteration - Iteration at which to get the learning rate

epoch - Epoch at which to get the learning rate

Returns:

Learning rate, or Double.NaN if no learning rate is applicable for this updater

hasLearningRate

public boolean hasLearningRate()

Specified by:

hasLearningRate in interface IUpdater

Returns:

True if the updater has a learning rate hyperparameter, false otherwise

setLrAndSchedule

public void setLrAndSchedule(double lr,
                             ISchedule lrSchedule)

Description copied from interface: IUpdater

Set the learning rate and schedule. Note: may throw an exception if IUpdater.hasLearningRate() returns false.

Specified by:

setLrAndSchedule in interface IUpdater

Parameters:

lr - Learning rate to set (typically not used if LR schedule is non-null)

lrSchedule - Learning rate schedule to set (may be null)

currentMomentum

public double currentMomentum(int iteration,
                              int epoch)