Random Unicode Byte Round-Trip Corpus Generator

Description

Sample supplied Unicode strings into UTF-8 round trips and supplied byte controls while distinguishing bytes, UTF-16 units, and code points.

A parser corpus pairs literal input spellings or structured documents with observable parser results or explicitly constructed expectations. Successful parsing, semantic validity, and exact round-trip preservation are separate properties, so test records retain the relevant evidence.

Sample supplied Unicode strings into UTF-8 round trips and supplied byte controls while distinguishing bytes, UTF-16 units, and code points.

When to use this tool

Use this generator to create reproducible unicode byte round trip corpus cases from the supplied model or bank. Inspect the returned evidence before using cases in a test or exercise.

How to use the tool and how it works

Supply texts, byteControls, count in the field shapes below. Set seed when you need an explicit replay.
Run the tool and correct any input error. For generated controls, distinguish a successful fixture containing an invalid case from failure to generate the fixture.
Read cases, controls. Use the conventions below and keep answer-bearing fields separate when presenting an exercise.

Draw supplied Unicode strings, delegate UTF-8 text/byte round trips and code-point formatting, and return Base64 byte representations with byte, UTF-16-unit and code-point counts. Decode every supplied byte control and retain its actual structured result.^[1]

Delegated operations used for this result: text to bytes; bytes to text. Their calculation or parsing conventions apply to the corresponding result fields.^{[2, 3]}

Input field	Meaning and units	Accepted input
texts	Texts	Required. Array of Text; maximum UTF-16 length 4096.; minimum items 1; maximum items 128.
byteControls	Byte Controls	Required. Array of Text; maximum UTF-16 length 4096.; minimum items 0; maximum items 32.
count	Requested count; the mechanics explain whether one draw expands into several returned records.	Required. Safe integer; minimum 1; maximum 128.
seed	Optional numeric seed; takes precedence over context randomness. Without either, use the fixed tool.io default seed.	Optional. Safe integer.

Output field	Interpretation	Returned shape
cases	Cases	Required. Array of Object; no unknown keys.; minimum items 1; maximum items 128.
cases[].text	Text	Required. Text; maximum UTF-16 length 4096.
cases[].bytes	Bytes	Required. Text.
cases[].codePoints	Code Points	Required. Text.
cases[].codePointCount	Code Point Count	Required. Safe integer.
cases[].codeUnitCount	Code Unit Count	Required. Safe integer.
cases[].byteCount	Byte Count	Required. Safe integer.
cases[].decoded	Decoded	Required. Text; maximum UTF-16 length 4096.
cases[].identical	Identical	Required. Boolean.
controls	Controls	Required. Array of Object; no unknown keys.; minimum items 0; maximum items 32.
controls[].bytes	Bytes	Required. Text; maximum UTF-16 length 4096.
controls[].result	Result	Required. Object; no unknown keys. or Object; no unknown keys.
randomness	Replay metadata: source is seed or context; seed is the normalized 32-bit seed or null for a context source.	Required. Object; no unknown keys.
randomness.seed	Optional numeric replay seed; normalization and precedence are explained below.	Required. Safe integer. or Null.
randomness.source	Source	Required. Text; one of "context", "seed".

An explicit safe-integer seed takes precedence over injected context randomness and is normalized to an unsigned 32-bit integer. With no seed and no context source, replay uses the fixed tool.io default seed. A context source must return finite values from zero inclusive to one exclusive; its advancing state must be retained separately. randomness records the source and normalized seed, or null for context. These generators are not cryptographic random sources.

json

{
  "input": {
    "texts": [
      "Aé",
      "🌙"
    ],
    "byteControls": [
      "QQ=="
    ],
    "count": 2,
    "seed": 42
  },
  "output": {
    "cases": [
      {
        "text": "🌙",
        "bytes": "8J+MmQ==",
        "codePoints": "U+1F319",
        "codePointCount": 1,
        "codeUnitCount": 2,
        "byteCount": 4,
        "decoded": "🌙",
        "identical": true
      },
      {
        "text": "Aé",
        "bytes": "QcOp",
        "codePoints": "U+0041 U+00E9",
        "codePointCount": 2,
        "codeUnitCount": 2,
        "byteCount": 3,
        "decoded": "Aé",
        "identical": true
      }
    ],
    "controls": [
      {
        "bytes": "QQ==",
        "result": {
          "ok": true,
          "output": {
            "text": "A"
          }
        }
      }
    ],
    "randomness": {
      "seed": 42,
      "source": "seed"
    }
  }
}

Limitations and assumptions

These length measures differ, particularly for non-ASCII and supplementary characters. Lone surrogates or invalid byte controls follow the actual encoder/decoder behavior; a sampled case is not a universal round-trip proof.
A replay requires the same complete input and implementation. Randomized construction does not guarantee uniform coverage of every feasible output; finite samples can repeat unless a distinct-selection rule is explicitly stated.
The field tables state structural bounds; cross-field constraints and bounded search or output budgets also apply. Unknown keys are rejected where the object contract disallows them. Input errors are distinct from null fields and deliberate invalid controls in a successful output.

Alternative or Complementary methods

For a supplied example, inspect the returned construction or delegated check directly. To evaluate a consumer, also test cases chosen for its own boundaries; a generated finite corpus does not establish exhaustive coverage.

References

Tool behavior and example: Random Unicode Byte Round-Trip Corpus Generator — tool.io
Calculation or validation: convert/text-to-bytes — tool.io
Calculation or validation: convert/bytes-to-text — tool.io

Similar or alternative tools

Random JSON Flattening Round-Trip Corpus Generator
Reuse the random nested-document generator for its safe dot-path/contiguous-array subset, then actually flatten and rebuild every generated document with both existing cores; retain intermediate leaves and original/rebuilt forms, excluding sparse/mixed containers, reserved segments, literal-dot keys and empty-container loss cases rather than claiming arbitrary JSON round trips.
Random Graph Representation Round-Trip Corpus Generator
Generate graphs with optional duplicate listings and delegate edge normalization and matrix round trips, comparing semantic edges independently of listing order.