Description
Resolve one supplied length-prefix boundary in a single caller-defined indexing unit.
Parsing depends on the chosen grammar, character encoding, offset unit, and treatment of escapes and delimiters. UTF-16 code-unit positions, Unicode code points, and UTF-8 byte offsets are not interchangeable. A boundary or token check supplies one part of a parser rather than validating an entire file.
Resolve one supplied length-prefix boundary in a single caller-defined indexing unit.
When to use this tool
Use Length Prefixed Token Boundary when you need the stated token boundary, offset, prefix, delimiter, or count component inside a larger parsing workflow.
How to use the tool and how it works
- Enter prefixLength, tokenLength, sourceLength. Use the units and conventions below.
- Run the tool. Correct any format, range, or cross-field input error before interpreting the output.
- Read tokenStart, tokenEnd, fits. Inspect fits before interpreting the associated values.
For nonnegative integer prefixLength, tokenLength, and sourceLength return tokenStart=prefixLength, tokenEnd=prefixLength+tokenLength using exact integer addition first, and fits when the end<=sourceLength.1
| Input field | Meaning and units | Accepted input |
|---|---|---|
| prefixLength | Prefix size in the caller-chosen coordinate unit. | Required. Safe integer; minimum 0. |
| tokenLength | Payload size in that same unit. | Required. Safe integer; minimum 0. |
| sourceLength | Available source size in that same unit. | Required. Safe integer; minimum 0. |
| Output field | Interpretation | Returned shape |
|---|---|---|
| tokenStart | Token Start | Required. Safe integer. |
| tokenEnd | Token End | Required. Safe integer. |
| fits | Fits | Required. Boolean. |
{
"input": {
"prefixLength": 2,
"tokenLength": 5,
"sourceLength": 10
},
"output": {
"tokenStart": 2,
"tokenEnd": 7,
"fits": true
}
}Limitations and assumptions
- Lengths must count the same coordinate unit, such as bytes or UTF-16 code units; the tool does not choose or convert it. An end beyond the safe numeric output range is rejected even when fits would be false.
- Inputs must satisfy both the field types and the cross-field conditions above. Invalid input causes an error; a successful false flag or documented placeholder is a different result.
- Numeric calculations use binary floating point unless exact decimal-integer arithmetic is explicitly stated. No hidden measurement tolerance or decimal rounding is added; nonfinite or out-of-contract results are rejected. Signed zero is exported as zero.
Alternative or Complementary methods
Choose a parser for the intended complete grammar when context, nesting, encodings, or semantic validation matters. Keep byte, code-unit, and code-point coordinates explicit when combining results.
References
Similar or alternative tools
- Token Lookahead Span
For nonnegative integer tokenCount and lookaheadCount and currentIndex within zero through tokenCount return endExclusive=min(tokenCount,currentIndex+lookaheadCount) and available=min(tokenCount-currentIndex,lookaheadCount).
- Escaped Delimiter Parity
Inspect an explicitly selected UTF-16 delimiter position under a repeated single-character escape parity convention.