CoNLL-U Range Token Boundary

Description

Parse a supplied CoNLL-U-style multiword range ID.

Language-processing records distinguish tokens, code points, UTF-16 positions and UTF-8 byte offsets. Tag grammars and local transition checks do not validate a full annotation sequence. Log-loss units determine perplexity conversion, while identifier suffixes may be textual labels rather than ordinary decimal numbers.

Parse a supplied CoNLL-U-style multiword range ID.

When to use this tool

Use CoNLL-U Range Token Boundary when you already have rangeToken and need to parse a supplied CoNLL-U-style multiword range ID.

How to use the tool and how it works

Enter rangeToken. Use the units and conventions below.
Run the tool. Correct any format, range, or cross-field input error before interpreting the output.
Read startId, endId, coveredWordCount. Use the output convention below.

Require two nonzero-leading positive decimal integers joined by one hyphen; both must be safe and end>start. Return endpoints and inclusive end-start+1 count.^[1]

CoNLL-U multiword tokens use a range of word IDs. Parsing that range is separate from verifying its position and coverage within a complete sentence record.^[2]

Input field	Meaning and units	Accepted input
rangeToken	Positive non-leading-zero word IDs separated by one hyphen; the end must exceed the start.	Required. Text; maximum UTF-16 length 8192.

Output field	Interpretation	Returned shape
startId	Start Id	Required. Safe integer.
endId	End Id	Required. Safe integer.
coveredWordCount	Covered Word Count	Required. Safe integer.

json

{
  "input": {
    "rangeToken": "3-5"
  },
  "output": {
    "startId": 3,
    "endId": 5,
    "coveredWordCount": 3
  }
}

Limitations and assumptions

A singleton range is rejected. No actual words, sentence bounds, overlaps, ordering or complete CoNLL-U record is validated.
Inputs must satisfy both the field types and the cross-field conditions above. Invalid input causes an error; a successful false flag or documented placeholder is a different result.

Alternative or Complementary methods

Validate the complete corpus format, tokenizer, annotation sequence and scoring convention. Use a full parser or model evaluation when needed, preserving the exact offset unit, vocabulary and log base rather than inferring them from a local transformation.

References

Calculation and input contract: computational-linguistics/conllu-range-token-boundary — tool.io
CoNLL-U Format: Word, range and empty-node identifiers — Universal Dependencies

Similar or alternative tools

CoNLL-U Decimal Empty Node Key
Split a supplied CoNLL-U-style decimal empty-node ID.
Dependency One Based Head Crosswalk
Crosswalk a supplied one-based dependency head with a root sentinel.