org.nd4j.linalg.api.ops.impl.transforms

Class LeakyReLU

java.lang.Object

org.nd4j.autodiff.functions.DifferentialFunction

org.nd4j.linalg.api.ops.BaseOp

org.nd4j.linalg.api.ops.BaseTransformOp

org.nd4j.linalg.api.ops.impl.transforms.LeakyReLU

All Implemented Interfaces:

Op, TransformOp

public class LeakyReLU
extends BaseTransformOp

Leaky Rectified linear unit. Default alpha=0.01, cutoff=0
Out(x) = alpha*x if x<0
Out(x) = x if x >= 0
Leaky ReLU may avoid zero gradient "dying ReLU" problem by having non-zero gradient below 0.
See for example http://arxiv.org/abs/1505.00853 for a comparison of ReLU variants.

Author:

Alex Black

Nested Class Summary

Nested classes/interfaces inherited from interface org.nd4j.linalg.api.ops.Op

Op.Type

Field Summary

Fields

Modifier and Type	Field and Description
static double	DEFAULT_ALPHA

Fields inherited from class org.nd4j.linalg.api.ops.BaseOp

extraArgs, extraArgz, n, numProcessed, passThrough, x, xVertexId, y, yVertexId, z, zVertexId

Fields inherited from class org.nd4j.autodiff.functions.DifferentialFunction

dimensions, inPlace, sameDiff, scalarValue

Constructor Summary

Constructors

Constructor and Description
LeakyReLU()
LeakyReLU(INDArray x)
LeakyReLU(INDArray x, double alpha)
LeakyReLU(INDArray x, INDArray z)
LeakyReLU(INDArray x, INDArray z, double alpha)
LeakyReLU(INDArray x, INDArray y, INDArray z, long n)
LeakyReLU(INDArray x, INDArray y, INDArray z, long n, double alpha)
LeakyReLU(INDArray x, INDArray z, long n)
LeakyReLU(INDArray x, INDArray z, long n, double alpha)
LeakyReLU(SameDiff sameDiff, SDVariable i_v, boolean inPlace, double alpha)
LeakyReLU(SameDiff sameDiff, SDVariable i_v, int[] shape, boolean inPlace, Object[] extraArgs, double alpha)
LeakyReLU(SameDiff sameDiff, SDVariable i_v, Object[] extraArgs, double alpha)

Method Summary

Modifier and Type	Method and Description
List<SDVariable>	doDiff(List<SDVariable> i_v) The actual implementation for automatic differentiation.
void	init(INDArray x, INDArray y, INDArray z, long n) Initialize the operation based on the parameters
String	onnxName() The opName of this function in onnx
String	opName() The name of the op
int	opNum() The number of the op (mainly for old legacy XYZ ops like Op)
Map<String,Object>	propertiesForFunction() Returns the properties for a given function
String	tensorflowName() The opName of this function tensorflow

Methods inherited from class org.nd4j.linalg.api.ops.BaseTransformOp

calculateOutputShape, opType, z

Methods inherited from class org.nd4j.linalg.api.ops.BaseOp

equals, exec, exec, extraArgs, extraArgsBuff, extraArgsDataBuff, getOpType, hashCode, initFromOnnx, initFromTensorFlow, isExecSpecial, isPassThrough, n, numProcessed, outputVariables, setN, setX, setY, setZ, toCustomOp, toString, x, y

Methods inherited from class org.nd4j.autodiff.functions.DifferentialFunction

arg, args, asProperties, attributeAdaptersForFunction, configFieldName, diff, dup, f, getValue, hasPlaceHolderInputs, isConfigProperties, larg, mappingsForFunction, onnxNames, outputVariables, rarg, resolvePropertiesFromSameDiffBeforeExecution, setInstanceId, setValueFor, tensorflowNames

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.nd4j.linalg.api.ops.Op

exec, exec, extraArgs, extraArgsBuff, extraArgsDataBuff, isExecSpecial, isPassThrough, n, numProcessed, setExtraArgs, setN, setX, setY, setZ, toCustomOp, x, y

Field Detail

DEFAULT_ALPHA

public static final double DEFAULT_ALPHA

See Also:

Constant Field Values

Constructor Detail

LeakyReLU

public LeakyReLU(SameDiff sameDiff,
                 SDVariable i_v,
                 boolean inPlace,
                 double alpha)

LeakyReLU

public LeakyReLU(SameDiff sameDiff,
                 SDVariable i_v,
                 int[] shape,
                 boolean inPlace,
                 Object[] extraArgs,
                 double alpha)

LeakyReLU

public LeakyReLU(SameDiff sameDiff,
                 SDVariable i_v,
                 Object[] extraArgs,
                 double alpha)

LeakyReLU

public LeakyReLU()

LeakyReLU

public LeakyReLU(INDArray x,
                 double alpha)

LeakyReLU

public LeakyReLU(INDArray x,
                 INDArray z,
                 double alpha)

LeakyReLU

public LeakyReLU(INDArray x,
                 INDArray z,
                 long n,
                 double alpha)

LeakyReLU

public LeakyReLU(INDArray x,
                 INDArray y,
                 INDArray z,
                 long n,
                 double alpha)

LeakyReLU

public LeakyReLU(INDArray x,
                 INDArray z)

LeakyReLU

public LeakyReLU(INDArray x,
                 INDArray z,
                 long n)

LeakyReLU

public LeakyReLU(INDArray x,
                 INDArray y,
                 INDArray z,
                 long n)

LeakyReLU

public LeakyReLU(INDArray x)

Method Detail

propertiesForFunction

public Map<String,Object> propertiesForFunction()

Description copied from class: DifferentialFunction

Returns the properties for a given function

Overrides:

propertiesForFunction in class DifferentialFunction

Returns:

opNum

public int opNum()

Description copied from class: DifferentialFunction

The number of the op (mainly for old legacy XYZ ops like Op)

Specified by:

opNum in interface Op

Overrides:

opNum in class DifferentialFunction

Returns:

opName

public String opName()

Description copied from class: DifferentialFunction

The name of the op

Specified by:

opName in interface Op

Overrides:

opName in class DifferentialFunction

Returns:

the opName of this operation

init

public void init(INDArray x,
                 INDArray y,
                 INDArray z,
                 long n)

Description copied from interface: Op

Initialize the operation based on the parameters

Specified by:

init in interface Op

Overrides:

init in class BaseOp

Parameters:

x - the input

y - the pairwise transform ndarray

z - the resulting ndarray

n - the number of elements

onnxName

public String onnxName()

Description copied from class: DifferentialFunction

The opName of this function in onnx

Specified by:

onnxName in class DifferentialFunction

Returns:

tensorflowName

public String tensorflowName()

Description copied from class: DifferentialFunction

The opName of this function tensorflow

Specified by:

tensorflowName in class DifferentialFunction

Returns:

doDiff

public List<SDVariable> doDiff(List<SDVariable> i_v)

Description copied from class: DifferentialFunction

The actual implementation for automatic differentiation.

Specified by:

doDiff in class DifferentialFunction

Returns: