Activation Tools
Browse all available activation tools.
Available tools
- Activation: CELU Alpha Partial Derivative Tool Evaluate the alpha partial of CELU alpha*(exp(x/alpha)-1) on its negative branch and zero on its linear branch, using supplied positive alpha.
- Activation: Elu Scalar Derivative Tool Evaluate one ELU derivative with explicit positive alpha and left/right convention at zero; does not assert differentiability when slopes differ.
- Activation: Glu Gate Partial Derivative Tool Return both local partial derivatives of value*sigmoid(gate), not an automatic-differentiation engine.
- Activation: Hardshrink Threshold Decision Tool Apply hardshrink with strict outside-threshold retention and zero on the inclusive threshold interval.
- Activation: HardTanh Inverse Preimage Tool Describe the preimage of hardtanh/clamp: lower/upper rays include their endpoint, interior is singleton and outside is empty; infinite rays are represented symbolically.
- Activation: Leaky ReLU Scalar Inverse Tool Invert a scalar leaky ReLU with explicitly nonzero negative-branch slope, retaining the two/no-preimage cases for a negative slope rather than claiming a global inverse.
- Activation: Maxout Two Value Tie Mask Tool Return maxout value and both active-branch flags, retaining ties rather than choosing a false unique gradient.
- Activation: Mish Scalar Derivative Tool Evaluate the scalar Mish derivative for x*tanh(softplus(x)) using stable softplus/sigmoid branches; no tensor differentiation.
- Activation: ReLU6 Scalar Value Tool Evaluate the bounded ReLU6 activation clamp in [0,6]; no quantized integer scale or neural-network engine.
- Activation: Selu Scalar Derivative Tool Evaluate one SELU derivative from supplied positive alpha/scale and explicit left/right zero convention; no assumed network normalization.
- Activation: Softmax Pairwise Logit Odds Tool Return the log of a softmax pair probability ratio directly from its two logits and explicit positive temperature, avoiding normalization/overflow.
- Activation: Softplus Linear Tail Switch Tool Report the strict numerical linear-tail decision for an explicitly beta-scaled softplus implementation; this is a branch diagnostic, not a second activation approximation.
- Activation: Softplus Scalar Second Derivative Tool Evaluate the second input derivative of beta-scaled softplus log(1+exp(beta*x))/beta for explicit positive beta.
- Activation: Softshrink Threshold Decision Tool Apply scalar softshrink with explicit nonnegative threshold and zero at either threshold endpoint.
- Activation: Softsign Scalar Derivative Tool Evaluate the scalar softsign derivative including its continuous derivative at zero.
- Activation: Softsign Scalar Inverse Tool Invert scalar softsign y=x/(1+abs(x)) only inside its open range (-1,1).
- Activation: Squareplus Scalar Derivative Tool Evaluate one squareplus derivative with explicit positive smoothing and a stable hypotenuse norm.
- Activation: Squareplus Scalar Inverse Tool Invert squareplus y=(x+sqrt(x*x+b))/2 for positive supplied smoothing b and y>0.
- Activation: Swish Scalar Derivative Beta Tool Evaluate the input derivative of beta-Swish x*sigmoid(beta*x) for explicit finite beta; beta is not learned or fitted.
- Activation: Tanh Scalar Second Derivative Tool Evaluate the analytic second scalar derivative of tanh using its bounded forward value.
- Bent Identity Negative Asymptote Tool Evaluate the negative-tail asymptotic line x/2-1/2 of bent identity.
- CELU Join Curvature Jump Tool Calculate celu join curvature jump from explicit caller-supplied inputs.
- ELU Alpha Partial Derivative Tool Return the ELU alpha partial under the x>=0 linear and x<0 exponential convention.
- ELU Negative Curvature Tool Evaluate ELU second derivative on the negative branch and zero on the positive branch; at zero use the positive-side value.
- ELU Negative Output Inverse Tool Invert the strictly negative ELU output branch with supplied positive alpha.
- Gaussian Activation Width Sensitivity Tool Differentiate exp(-(x/width)²) with respect to positive width.
- GELU Exact Scalar Second Derivative Tool Evaluate the analytic second derivative of exact GELU x*Phi(x), rather than a tanh approximation.
- Hardshrink Zero Preimage Interval Tool Return the closed interval mapped to zero by the strict-outside-threshold hard-shrink convention.
- HardSigmoid Linear Branch Preimage Tool Invert only the open linear branch of clamp(slope*x+intercept,0,1).
- HardSwish Breakpoint Distance Tool Measure distance to the fixed -3 and +3 hard-swish breakpoints, retaining an exact tie flag.
- HardSwish Negative Minimum Tool Find the minimum of x*clamp(x+scale,0,2*scale)/(2*scale) on its quadratic branch.
- HardTanh Saturation Distance Tool Measure input displacement from the closed linear interval of a hard-tanh activation.
- Hyperbolic Tangent Apply the hyperbolic tangent activation.
- Leaky ReLU Apply the leaky ReLU activation, scaling negatives by a small slope.
- Leaky ReLU Negative Inverse Slope Sensitivity Tool Calculate leaky relu negative inverse slope sensitivity from explicit caller-supplied inputs.
- Leaky ReLU Slope Partial Derivative Tool Return the negative-branch slope-parameter derivative of leaky ReLU, including zero at the origin.
- LogSoftmax One Logit Coordinate Derivative Tool Return one log-softmax logit derivative at fixed positive temperature from supplied probabilities.
- Mish Scalar Second Derivative Tool Evaluate the analytic second input derivative of x*tanh(softplus(x)) with reused stable forward cores.
- ReLU Apply the rectified linear activation, flooring negatives at zero.
- ReLU Squared Inverse Branch Tool Invert squared ReLU, flagging the complete nonpositive input ray at zero output.
- ReLU Zero Subgradient Choice Tool Return a ReLU slope away from zero and a caller-selected convex subgradient at zero.
- ReLU6 Output Preimage Interval Tool Represent the complete ReLU6 preimage with explicit unbounded flags and finite placeholder endpoints.
- SELU Alpha Partial Derivative Tool Differentiate SELU with respect to alpha at a fixed positive scale.
- SELU Negative Asymptote Tool Return the negative-input SELU asymptote for caller-supplied positive alpha and scale.
- SELU Scale Partial Derivative Tool Return the SELU scale partial while holding the input and positive alpha fixed.
- Sigmoid Apply the logistic sigmoid activation.
- Sigmoid Complement Product Tool Evaluate sigmoid(x)*(1−sigmoid(x)) without overflow on extreme inputs.
- Sigmoid Log Derivative Tool Evaluate log(sigmoid′(x)) stably without first forming an underflowing derivative.
- Sigmoid Logit Jacobian Sign Tool Calculate sigmoid logit jacobian sign from explicit caller-supplied inputs.
- Sigmoid Output Log Odds Contrast Subtract first from second log odds for two explicit interior sigmoid outputs without clipping endpoints.
- Sigmoid Scalar Third Derivative Tool Evaluate the third input derivative of the logistic sigmoid with stable forward evaluation.
- Sigmoid Temperature Sensitivity Tool Differentiate sigmoid(x/temperature) with respect to positive temperature using stable sigmoid branches.
- Softmax Convert a list of scores into a probability distribution with the numerically stable softmax.
- Softmax One Logit Coordinate Derivative Tool Return one softmax Jacobian entry from a validated probability vector and explicit positive temperature.
- Softplus Apply the numerically stable softplus activation.
- Softplus Beta Partial Derivative Tool Differentiate log(1+exp(beta*x))/beta with respect to positive beta at a fixed input.
- Softplus Curvature Inflection Location Tool For softplus log(1+exp(beta*x))/beta, locate its curvature maximum; the function has no curvature sign-change inflection.
- Softplus Scalar Third Derivative Tool Evaluate the third input derivative of beta-scaled softplus; zero is the inflection point.
- Softsign Curvature Magnitude Tool Return absolute second derivative of x/(1+|x|), preserving nondifferentiability of its derivative at zero.
- Softsign Output Distance To Saturation Tool Calculate softsign output distance to saturation from explicit caller-supplied inputs.
- Softsign Scalar Second Derivative Tool Evaluate the softsign second derivative with an explicit one-sided convention at its nondifferentiable second-derivative boundary.
- Softsign Scale Parameter Sensitivity Tool Differentiate x/(scale+abs(x)) with respect to its positive scale.
- Squareplus Bias Partial Derivative Tool Return the partial derivative of (x+sqrt(x²+bias))/2 with respect to positive bias.
- Squareplus Output Bias Feasibility Tool Recover a candidate nonnegative squareplus bias and flag whether the supplied output is feasible.
- Swish Beta Partial Derivative Tool Differentiate x*sigmoid(beta*x) with respect to beta, allowing zero and negative beta.
- Swish Beta Zero Limit Tool Return the beta=0 Swish value and its first beta derivative, from sigmoid(0)=1/2.
- Tanh Half Angle Ratio Tool Evaluate sinh(x)/(cosh(x)+1) using the stable equivalent tanh(x/2).
- Tanh Inflection Distance Tool Locate the unique inflection of tanh(slope*x+bias) under a nonzero supplied slope.
- Tanh Temperature Sensitivity Tool Differentiate tanh(x/temperature) with respect to positive temperature.
- Thresholded ReLU Jump Size Tool Return the one-sided discontinuity of x if x>threshold else zero.
- Thresholded ReLU Zero Input Policy Tool Report strict/inclusive threshold membership and derivative availability, using zero as a placeholder when undefined.