Softmax Logit Shift Invariance Residual

Description

Measure numerical softmax change after a supplied common logit shift.

Model and prompt records distinguish token indices, offset units, reserved capacity, code scales and numeric representations. A common softmax shift is mathematically invariant but can alter rounded inputs. Literal brace doubling and citation tokens are format conventions rather than universal template or evidence validation.

Measure numerical softmax change after a supplied common logit shift.

When to use this tool

Use Softmax Logit Shift Invariance Residual when you already have logits, shift and need to measure numerical softmax change after a supplied common logit shift.

How to use the tool and how it works

Enter logits, shift. Use the units and conventions below.
Run the tool. Correct any format, range, or cross-field input error before interpreting the output.
Read maximumProbabilityDifference. Use the output convention below.

Require a nonempty finite logit list and all logits+shift finite. Compute both softmax vectors by subtracting each vector's maximum before exponentiation, then return their largest absolute probability difference.^[1]

Equivalent softmax formulas can behave differently in floating-point arithmetic. Subtracting a vector maximum helps avoid exponential overflow, but it does not restore distinctions already lost when a common shift was rounded into the inputs.^[2]

Input field	Meaning and units	Accepted input
logits	Logits	Required. Array of finite number; maximum items 256.
shift	Shift	Required. Finite number.

Output field	Interpretation	Returned shape
maximumProbabilityDifference	Largest absolute softmax probability change after floating-point logit shifting; a large shift can erase distinctions.	Required. Finite number.

json

{
  "input": {
    "logits": [
      0,
      0
    ],
    "shift": 5
  },
  "output": {
    "maximumProbabilityDifference": 0
  }
}

Limitations and assumptions

Real-number softmax is invariant to a common shift. Floating-point addition can erase differences for a large shift; this output measures that numerical change, not a proof of model correctness or semantic invariance.
Inputs must satisfy both the field types and the cross-field conditions above. Invalid input causes an error; a successful false flag or documented placeholder is a different result.
Numeric calculations use binary floating point unless exact decimal-integer arithmetic is explicitly stated. No hidden measurement tolerance or decimal rounding is added; nonfinite or out-of-contract results are rejected. Numeric signed-zero results are exported as zero.

Alternative or Complementary methods

Use the complete tokenizer, template, model contract and evaluation method. Verify actual capacity, vocabulary and numerical behavior when supplied budgets, local masks or representation checks cannot establish correct model operation.

References

Calculation and input contract: artificial-intelligence/softmax-logit-shift-invariance-residual — tool.io
Accurate Computation of the Log-Sum-Exp and Softmax Functions (2019) — Pierre Blanchard, Desmond J. Higham and Nicholas J. Higham

Similar or alternative tools

Model Log Probability Exp Underflow Flag
Exponentiate a supplied log probability and flag exact numeric underflow.
Model Nonfinite Logit Flag
Identify nonfinite numeric values represented by supplied logit strings.