Description
Return a supplied beam length penalty and normalized score.
Autoregressive sequence log scores accumulate token log probabilities and commonly become more negative as a sequence grows. A length-normalization convention changes how candidate scores are compared. This helper computes a supplied length-penalty factor without searching candidates, normalizing a distribution or choosing the penalty parameter.
Return a supplied beam length penalty and normalized score.
When to use this tool
Use Beam Score Length Penalty Factor when checking the stated probability, score, guidance or scale contribution.
How to use the tool and how it works
- Enter logScore, length, alpha. Use the units and conventions below.
- Run the tool. Correct any format, range, or cross-field input error before interpreting the output.
- Read penalty, normalizedScore. Use the output convention below.
Return a supplied beam length penalty and normalized score. Require whole length >= 1 and alpha >= 0; penalty = ((5+length)/6)^alpha, normalizedScore = logScore/penalty.1
GNMT uses a length-normalization factor of the form implemented here. This local factor does not include coverage penalties or perform beam search.2
| Input field | Meaning and units | Accepted input |
|---|---|---|
| logScore | Log Score | Required. Finite number. |
| length | Length | Required. Safe integer; minimum 1. |
| alpha | Alpha | Required. Finite number; minimum 0. |
| Output field | Interpretation | Returned shape |
|---|---|---|
| penalty | ((5+length)/6)^alpha under this length-normalization convention. | Required. Finite number. |
| normalizedScore | Normalized Score | Required. Finite number. |
{
"input": {
"logScore": -12,
"length": 7,
"alpha": 1
},
"output": {
"penalty": 2,
"normalizedScore": -6
}
}Limitations and assumptions
- This is one scoring convention and supplied length; no beam search, tokenizer or optimal sequence is computed.
- Inputs must satisfy both the field types and the cross-field conditions above. Invalid input causes an error; a successful false flag or documented placeholder is a different result.
- Numeric calculations use binary floating point unless exact decimal-integer arithmetic is explicitly stated. No hidden measurement tolerance or decimal rounding is added; nonfinite or out-of-contract results are rejected. Numeric signed-zero results are exported as zero.
Alternative or Complementary methods
Use the full distribution and model-specific decoding or training objective for normalized probabilities, preference optimization or actual generation.
References
Similar or alternative tools
- Log Probability Nats Bits Crosswalk
Convert a nonpositive log probability from nats to bits.
- Inference Batch Sequence Waste Term
Return ragged-batch token totals and padding waste.