UTF-16 BOM Encoding Check

Match initial UTF-16 and UTF-32 BOM byte prefixes as encoding hints, preferring the longest signature; FF FE 00 00 remains ambiguous with UTF-16 LE followed by NUL.

Description

A byte order mark at the beginning of a byte stream can hint at its Unicode encoding and byte order.

A byte order mark at the beginning of a byte stream can hint at its Unicode encoding and byte order. Some signatures overlap: FF FE 00 00 can indicate UTF-32 little-endian or UTF-16 little-endian followed by a NUL.^{[1, 2]}

Match initial UTF-16 and UTF-32 BOM byte prefixes as encoding hints, preferring the longest signature; FF FE 00 00 remains ambiguous with UTF-16 LE followed by NUL.

When to use UTF-16 BOM Encoding Check

Use this tool to match initial UTF-16 and UTF-32 BOM byte prefixes as encoding hints, preferring the longest signature; FF FE 00 00 remains ambiguous with UTF-16 LE followed by NUL.

Choose the original CSV text, decoded cell strings, byte values or counts requested by this tool. Use the stated delimiter, encoding, indexing and number/date convention.

How to use UTF-16 BOM Encoding Check and understand its results

Choose the original CSV text, decoded cell strings, byte values or counts requested by this tool. Use the stated delimiter, encoding, indexing and number/date convention.
Enter bytes using the field table and calculation rules, then run the tool.
Read encodingHint, ambiguous. The explanation below defines the units, signs, ordering and any flags or placeholder values.
If the tool reports an input issue, correct the values and run it again. Apply the stated assumptions when interpreting a successfully returned result.

Input field	Meaning	Value type; see rules below	Requirement
bytes	Bytes	list of integer; minimum: 0; maximum: 255; minimum entries: 0; maximum entries: 100000	Required

Provide integer byte values from 0 through 255. The tool checks FF FE 00 00 and 00 00 FE FF before the shorter FF FE and FE FF signatures, returning utf32-le, utf32-be, utf16-le or utf16-be respectively. Only the utf32-le match sets ambiguous true because of the stated overlap. Without a supported prefix it returns unknown; short and empty arrays are allowed.

hint = longest matching initial UTF-16/UTF-32 signature

Units, indexing, boundary conditions and exceptional cases are defined in the calculation rules above.

Output field	Meaning	Value type
encodingHint	Encoding hint	string; one of utf16-le, utf16-be, utf32-le, utf32-be, unknown
ambiguous	Prefix can have multiple interpretations	boolean

json

{
  "input": {
    "bytes": [
      255,
      254,
      65,
      0
    ]
  },
  "output": {
    "encodingHint": "utf16-le",
    "ambiguous": false
  }
}

Limitations and assumptions

This is an initial-byte hint, not full-stream decoding or encoding verification. UTF-8 signatures are not recognized by this tool, and unknown does not rule out Unicode text without a BOM. An ambiguity flag does not inspect the remaining content to choose an encoding.

Numeric inputs must be finite. Follow the calculation-specific rules for overflow, underflow and exceptional cases. Missing required fields, extra fields, wrong value types and violations of the calculation rules are input errors. A successfully returned false flag is an evaluated outcome; its meaning is explained above.

Alternative or Complementary methods

Use BOM Remover First Bytes: Remove an initial UTF-8 EF BB BF signature from byte values and preserve every remaining byte, including interior signatures and other encoding BOMs. For a complementary task, use BOM Trim From First Cell: Remove exactly one initial U+FEFF from the first decoded cell only, preserving interior marks and all subsequent cells.

Further background: Unicode encoding signatures and byte order marks.

References

UTF-16 BOM Encoding Check — tool.io
Unicode encoding signatures and byte order marks — Unicode Consortium

Similar or alternative tools

BOM Remover First Bytes
Remove an initial UTF-8 EF BB BF signature from byte values and preserve every remaining byte, including interior signatures and other encoding BOMs.
BOM Trim From First Cell
Remove exactly one initial U+FEFF from the first decoded cell only, preserving interior marks and all subsequent cells.