Description
Return alpha*exp(x) for x<0 and zero for x>=0 as the negative-branch curvature term.
ELU uses x on the nonnegative branch and alpha*(exp(x)-1) on the negative branch. Its negative asymptote is -alpha for positive alpha. Input derivatives, alpha partial derivatives and one-sided join conventions answer different mathematical questions.
Return alpha*exp(x) for x<0 and zero for x>=0 as the negative-branch curvature term.
When to use this tool
Use ELU Negative Curvature when checking the stated ELU inverse, curvature or partial derivative convention.
How to use the tool and how it works
- Enter x, alpha. Use the units and conventions below.
- Run the tool. Correct any format, range, or cross-field input error before interpreting the output.
- Read curvature. Use the output convention below.
Require alpha>=0. Return alpha*exp(x) for x<0 and zero for x>=0 as the negative-branch curvature term.1
ELU has a linear positive branch and an alpha-scaled exponential negative branch. The local derivative, inverse and join conventions above do not imply a particular network’s behavior.2
| Input field | Meaning and units | Accepted input |
|---|---|---|
| x | Finite scalar input coordinate; use the mathematical scale specified above. | Required. Finite number. |
| alpha | Nonnegative negative-branch parameter; zero is accepted. | Required. Finite number; minimum 0. |
| Output field | Interpretation | Returned shape |
|---|---|---|
| curvature | Negative-branch second-derivative term for x<0; zero is a branch placeholder at the join. | Required. Finite number. |
{
"input": {
"x": -1,
"alpha": 2
},
"output": {
"curvature": 0.7357588823428847
}
}Limitations and assumptions
- At zero the output is a branch convention, not an assertion of a two-sided second derivative.
- Inputs must satisfy both the field types and the cross-field conditions above. Invalid input causes an error; a successful false flag or documented placeholder is a different result.
- Numeric calculations use binary floating point unless exact decimal-integer arithmetic is explicitly stated. No hidden measurement tolerance or decimal rounding is added; nonfinite or out-of-contract results are rejected. Numeric signed-zero results are exported as zero.
Alternative or Complementary methods
Use the complete explicit function and the intended parameter, branch and numerical conventions for symbolic, tensor or model-level differentiation; assess training behavior independently.
References
-
Calculation and input contract: activation/elu-negative-curvature — tool.io
-
Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) — Djork-Arné Clevert, Thomas Unterthiner and Sepp Hochreiter
Similar or alternative tools
- ELU Negative Output Inverse
Return log1p(output/alpha), the finite negative-branch input of alpha*(exp(x)-1).
- ELU Alpha Partial Derivative
Return expm1(x) for x<0 and zero otherwise, the partial derivative of ELU with respect to alpha at fixed x.