Description
Return token-span width with explicit end inclusion.
Language-processing records depend on tokenization, indexing, null sentinels and text representation. A UTF-8 byte offset, UTF-16 index, dependency link or literal feature label can be handled precisely without inferring linguistic correctness or semantic equivalence.
Return token-span width with explicit end inclusion.
When to use this tool
Use Parse Node Span Width when checking a supplied linguistic annotation, span, alignment or text-offset convention.
How to use the tool and how it works
- Enter startToken, endToken, inclusiveEnd. Use the units and conventions below.
- Run the tool. Correct any format, range, or cross-field input error before interpreting the output.
- Read width. Use the output convention below.
Return token-span width with explicit end inclusion. Require whole 0 <= startToken <= endToken; width = end-start+(inclusiveEnd ? 1 : 0).1
| Input field | Meaning and units | Accepted input |
|---|---|---|
| startToken | Start Token | Required. Safe integer; minimum 0. |
| endToken | End Token | Required. Safe integer. |
| inclusiveEnd | Inclusive End | Required. Boolean. |
| Output field | Interpretation | Returned shape |
|---|---|---|
| width | Width | Required. Safe integer. |
{
"input": {
"startToken": 2,
"endToken": 5,
"inclusiveEnd": false
},
"output": {
"width": 3
}
}Limitations and assumptions
- A coincident inclusive span has width one; token count and sentence bounds are not checked.
- Inputs must satisfy both the field types and the cross-field conditions above. Invalid input causes an error; a successful false flag or documented placeholder is a different result.
- Numeric calculations use binary floating point unless exact decimal-integer arithmetic is explicitly stated. No hidden measurement tolerance or decimal rounding is added; nonfinite or out-of-contract results are rejected. Numeric signed-zero results are exported as zero.
Alternative or Complementary methods
Use the appropriate tokenizer, Unicode and annotation-format conventions and validate the complete text or tree for full linguistic processing.
References
Similar or alternative tools
- Token Offset Domain Gap
Slice a supplied string under a code-point or UTF-16 coordinate convention.
- Quoted Passage Span Share
Return a half-open quotation's share of a supplied source length.