Description
Return scale*expm1(x) for x<0 and zero otherwise, differentiating SELU with respect to alpha at fixed x and scale.
SELU multiplies an ELU-shaped function by an outer scale: the positive branch is scale*x and the negative branch is scale*alpha*(exp(x)-1). Input, alpha and scale derivatives differ, and one-sided input slopes at zero match only when alpha=1.
Return scale*expm1(x) for x<0 and zero otherwise, differentiating SELU with respect to alpha at fixed x and scale.
When to use this tool
Use SELU Alpha Partial Derivative when checking the stated scaled exponential-linear limit or derivative convention.
How to use the tool and how it works
- Enter x, scale. Use the units and conventions below.
- Run the tool. Correct any format, range, or cross-field input error before interpreting the output.
- Read derivative. Use the output convention below.
Require scale>0. Return scale*expm1(x) for x<0 and zero otherwise, differentiating SELU with respect to alpha at fixed x and scale.1
SELU combines an outer scale with an exponential negative branch. This local parameter or input term does not establish the distributional assumptions needed for network self-normalization.2
| Input field | Meaning and units | Accepted input |
|---|---|---|
| x | Finite scalar input coordinate; use the mathematical scale specified above. | Required. Finite number. |
| scale | Supplied scale parameter with its operation-specific meaning stated above. | Required. Finite number; strictly above 0. |
| Output field | Interpretation | Returned shape |
|---|---|---|
| derivative | Scalar derivative with respect to the variable and at the fixed coordinates specified above; not a tensor/model gradient. | Required. Finite number. |
{
"input": {
"x": -0.6931471805599453,
"scale": 2
},
"output": {
"derivative": -1
}
}Limitations and assumptions
- The absent alpha input is unnecessary for this partial derivative; no parameter update, fitted value or network gradient is computed.
- Inputs must satisfy both the field types and the cross-field conditions above. Invalid input causes an error; a successful false flag or documented placeholder is a different result.
- Numeric calculations use binary floating point unless exact decimal-integer arithmetic is explicitly stated. No hidden measurement tolerance or decimal rounding is added; nonfinite or out-of-contract results are rejected. Numeric signed-zero results are exported as zero.
Alternative or Complementary methods
Use the complete explicit function and the intended parameter, branch and numerical conventions for symbolic, tensor or model-level differentiation; assess training behavior independently.
References
-
Calculation and input contract: activation/selu-alpha-partial-derivative — tool.io
-
Self-Normalizing Neural Networks — Günter Klambauer and coauthors
Similar or alternative tools
- SELU Scale Partial Derivative
Return x for x>=0 and alpha*expm1(x) for x<0, the partial derivative of scaled ELU with respect to its outer scale.
- Activation: Selu Scalar Derivative
Return scale for x>0 and scale*alpha*exp(x) for x<0; at zero select scale*alpha on the left or scale on the right.