UTF-8 Byte Offset to Code Point Index

Convert UTF-8 byte offset to zero-based Unicode scalar index; reject unpaired surrogates and offsets inside multibyte encodings.

Description

Convert UTF-8 byte offset to zero-based Unicode scalar index; reject unpaired surrogates and offsets inside multibyte encodings.

A UTF-8 byte offset can identify a code point only at a complete encoding boundary. An offset in the middle of a multibyte character must be rejected.^[1]

When to use this tool

Convert UTF-8 byte offset to zero-based Unicode scalar index; reject unpaired surrogates and offsets inside multibyte encodings.

Prepare the supplied text, byteOffset fields for this operation.
Compare the complete example with the representation used by your own input before carrying the result into another step.

How to use the tool and how it works

Enter text, byteOffset. Use the exact text syntax, units and index conventions explained below.
Run the tool. If input validation fails, correct the supplied fields before interpreting any result.
Read result.codePointIndex. Interpret false, null and empty lists using the distinctions below; they can be valid outputs.

Validate scalar text and an integer byteOffset within its total UTF-8 size. Accumulate each code point's encoded bytes until the requested boundary is reached, returning its code-point index; reject overshoot.^[2]

Input field	Meaning and representation	Accepted input
text	text.	Text; at most 8,192 UTF-16 code units, subject to the grammar above
byteOffset	byte offset.	Integer from 0 to utf8(text inclusive

Output field	Interpretation
result	Object containing the named outputs below.
result.codePointIndex	Zero-based Unicode code-point boundary index, including the final end boundary when selected.

json

{
  "input": {
    "text": "A\ud83d\ude00B",
    "byteOffset": 5
  },
  "output": {
    "result": {
      "codePointIndex": 2
    }
  }
}

Limitations and assumptions

Both start and end boundaries are accepted. The input is text rather than a raw byte buffer, so invalid UTF-8 byte sequences are not decoded or diagnosed.
Text offsets and lengths use UTF-16 code units unless explicitly described as bytes or code points. CRLF, CR and LF are recognized as line endings; a terminal newline creates a final empty line in the shared line model.
Every declared top-level field is required and unexpected fields are rejected. Numeric inputs must be finite; ordinary text fields are bounded to 8,192 UTF-16 code units, with tighter operation-specific limits where stated.
Ordinary string lists have at most 128 entries of at most 128 code units each; numeric lists and matrices have separate bounded size checks. Nonfinite results and excessively large serialized outputs are rejected.
Indexes are zero-based and spans have an exclusive end unless explicitly stated otherwise. A valid false, null or empty-list result is distinct from malformed input.

Alternative or Complementary methods

Use a parser or language-aware editor when quoting, syntax or nested constructs matter. For plain text transforms, keep UTF-16 offsets, Unicode code points, grapheme clusters and UTF-8 byte offsets distinct; apply changes to the same source version used to calculate spans.

References

RFC 3629: UTF-8 encoding — RFC Editor
Tool behavior and example: UTF-8 Byte Offset to Code Point Index — tool.io

Similar or alternative tools

Code Point Index to UTF-8 Byte Offset
Convert zero-based Unicode scalar index to UTF-8 byte offset; reject unpaired surrogates and count scalars rather than grapheme clusters.
byte count
Count the UTF-8 bytes required to encode Unicode text.