activation_relu
Activation functions
Description
relu(...)
: Applies the rectified linear unit activation function. elu(...)
: Exponential Linear Unit. selu(...)
: Scaled Exponential Linear Unit (SELU). hard_sigmoid(...)
: Hard sigmoid activation function. linear(...)
: Linear activation function (pass-through). sigmoid(...)
: Sigmoid activation function, sigmoid(x) = 1 / (1 + exp(-x))
. softmax(...)
: Softmax converts a vector of values to a probability distribution. softplus(...)
: Softplus activation function, softplus(x) = log(exp(x) + 1)
. softsign(...)
: Softsign activation function, softsign(x) = x / (abs(x) + 1)
. tanh(...)
: Hyperbolic tangent activation function. exponential(...)
: Exponential activation function. gelu(...)
: Applies the Gaussian error linear unit (GELU) activation function. swish(...)
: Swish activation function, swish(x) = x * sigmoid(x)
.
Usage
activation_relu(x, alpha = 0, max_value = NULL, threshold = 0)
activation_elu(x, alpha = 1)
activation_selu(x)
activation_hard_sigmoid(x)
activation_linear(x)
activation_sigmoid(x)
activation_softmax(x, axis = -1)
activation_softplus(x)
activation_softsign(x)
activation_tanh(x)
activation_exponential(x)
activation_gelu(x, approximate = FALSE)
activation_swish(x)
Arguments
Arguments | Description |
---|---|
x | Tensor |
alpha | Alpha value |
max_value | Max value |
threshold | Threshold value for thresholded activation. |
axis | Integer, axis along which the softmax normalization is applied |
approximate | A bool, whether to enable approximation. |
Details
Activations functions can either be used through layer_activation()
, or through the activation argument supported by all forward layers.
activation_selu()
to be used together with the initialization “lecun_normal”.activation_selu()
to be used together with the dropout variant “AlphaDropout”.
Section
References
activation_swish()
: Searching for Activation Functionsactivation_gelu()
: Gaussian Error Linear Units (GELUs)activation_selu()
: Self-Normalizing Neural Networksactivation_elu()
: Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
Value
Tensor with the same shape and dtype as x
.
See Also
https://www.tensorflow.org/api_docs/python/tf/keras/activations