Description
For thresholded ReLU, compare zero to threshold using >= or > according to passAtThreshold; zeroOutput stays zero.
Thresholded ReLU returns the input only on a chosen side of a threshold and zero otherwise. Boundary inclusion, continuity and differentiability are separate questions: a jump can persist even if a value is assigned at the threshold.
For thresholded ReLU, compare zero to threshold using >= or > according to passAtThreshold; zeroOutput stays zero.
When to use this tool
Use Thresholded ReLU Zero Input Policy when checking the stated threshold and zero-input policy convention.
How to use the tool and how it works
- Enter threshold, passAtThreshold. Use the units and conventions below.
- Run the tool. Correct any format, range, or cross-field input error before interpreting the output.
- Read zeroPasses, zeroOutput, derivativeAtZero, derivativeDefined. Inspect zeroPasses, derivativeDefined before interpreting the associated values.
For thresholded ReLU, compare zero to threshold using >= or > according to passAtThreshold; zeroOutput stays zero. derivativeAtZero is one for negative thresholds, zero for positive thresholds, and a zero placeholder with derivativeDefined=false at threshold zero.1
| Input field | Meaning and units | Accepted input |
|---|---|---|
| threshold | Supplied symmetric or one-sided activation threshold as defined above. | Required. Finite number. |
| passAtThreshold | Pass At Threshold | Required. Boolean. |
| Output field | Interpretation | Returned shape |
|---|---|---|
| zeroPasses | Zero Passes | Required. Boolean. |
| zeroOutput | Zero Output | Required. Finite number. |
| derivativeAtZero | Zero placeholder at threshold=0; inspect derivativeDefined before interpreting it. | Required. Finite number. |
| derivativeDefined | Derivative Defined | Required. Boolean. |
{
"input": {
"threshold": 0,
"passAtThreshold": false
},
"output": {
"zeroPasses": false,
"zeroOutput": 0,
"derivativeAtZero": 0,
"derivativeDefined": false
}
}Limitations and assumptions
- Passing a threshold does not imply a defined derivative; the equality policy affects branch membership without changing zero's output.
- Inputs must satisfy both the field types and the cross-field conditions above. Invalid input causes an error; a successful false flag or documented placeholder is a different result.
- Numeric calculations use binary floating point unless exact decimal-integer arithmetic is explicitly stated. No hidden measurement tolerance or decimal rounding is added; nonfinite or out-of-contract results are rejected. Numeric signed-zero results are exported as zero.
Alternative or Complementary methods
Use the complete explicit function and the intended parameter, branch and numerical conventions for symbolic, tensor or model-level differentiation; assess training behavior independently.
References
Similar or alternative tools
- Thresholded ReLU Jump Size
Return leftLimit=0, rightLimit=threshold and their jump size threshold for the thresholded ReLU boundary.
- Activation: Hardshrink Threshold Decision Tool
Apply hardshrink with strict outside-threshold retention and zero on the inclusive threshold interval.