Word Gap Length Table
Tabulate whitespace gaps between non-whitespace runs, excluding leading/trailing whitespace; positions and lengths use UTF-16 units.
Description
Tabulate whitespace gaps between non-whitespace runs, excluding leading/trailing whitespace; positions and lengths use UTF-16 units.
Whitespace gaps between nonwhitespace tokens can expose alignment and spacing inconsistencies. Keeping their text alongside lengths distinguishes spaces, tabs and line breaks.
When to use this tool
Tabulate whitespace gaps between non-whitespace runs, excluding leading/trailing whitespace; positions and lengths use UTF-16 units.
- Prepare the supplied text fields for this operation.
- Compare the complete example with the representation used by your own input before carrying the result into another step.
How to use the tool and how it works
- Enter text. Use the exact text syntax, units and index conventions explained below.
- Run the tool. If input validation fails, correct the supplied fields before interpreting any result.
- Read result.gaps. Interpret false, null and empty lists using the distinctions below; they can be valid outputs.
Find every maximal JavaScript nonwhitespace token. For each adjacent pair, return the intervening half-open UTF-16 span, length and original gap text.1
| Input field | Meaning and representation | Accepted input |
|---|---|---|
| text | text. | Text; at most 8,192 UTF-16 code units, subject to the grammar above |
| Output field | Interpretation |
|---|---|
| result | Object containing the named outputs below. |
| result.gaps | The gaps list in the ordering and representation described above. |
{
"input": {
"text": "a b\tc"
},
"output": {
"result": {
"gaps": [
{
"start": 1,
"end": 3,
"length": 2,
"text": " "
},
{
"start": 4,
"end": 5,
"length": 1,
"text": "\t"
}
]
}
}
}Limitations and assumptions
- Leading and trailing whitespace are omitted because they do not lie between two tokens. Length is code units rather than visual width, and tokenization is not linguistic word segmentation.
- Text offsets and lengths use UTF-16 code units unless explicitly described as bytes or code points. CRLF, CR and LF are recognized as line endings; a terminal newline creates a final empty line in the shared line model.
- Every declared top-level field is required and unexpected fields are rejected. Numeric inputs must be finite; ordinary text fields are bounded to 8,192 UTF-16 code units, with tighter operation-specific limits where stated.
- Ordinary string lists have at most 128 entries of at most 128 code units each; numeric lists and matrices have separate bounded size checks. Nonfinite results and excessively large serialized outputs are rejected.
- Indexes are zero-based and spans have an exclusive end unless explicitly stated otherwise. A valid false, null or empty-list result is distinct from malformed input.
Alternative or Complementary methods
Use a parser or language-aware editor when quoting, syntax or nested constructs matter. For plain text transforms, keep UTF-16 offsets, Unicode code points, grapheme clusters and UTF-8 byte offsets distinct; apply changes to the same source version used to calculate spans.
References
Similar or alternative tools
- average word length
Calculate the mean character length of whitespace-delimited words in text.
- word length histogram
Build a histogram mapping each word length to the number of whitespace-delimited words of that length.