Description
Construct token runs with bounded sampled lengths and guaranteed adjacent token changes, retaining exclusive segment boundaries and exact global frequencies independently of text encoding.
An operation trace pairs an ordered request stream with the state after each step. Empty operations, ties, evictions, and boundary indexes are part of the modeled contract rather than incidental display details.
Construct token runs with bounded sampled lengths and guaranteed adjacent token changes, retaining exclusive segment boundaries and exact global frequencies independently of text encoding.
When to use this tool
Use this generator to create reproducible run segmentation workload cases from the supplied model or bank. Inspect the returned evidence before using cases in a test or exercise.
How to use the tool and how it works
- Supply tokens, runCount, minimumRunLength, maximumRunLength in the field shapes below. Set seed when you need an explicit replay.
- Run the tool and correct any input error. For generated controls, distinguish a successful fixture containing an invalid case from failure to generate the fixture.
- Read stream, segments, boundaries, counts. Use the conventions below and keep answer-bearing fields separate when presenting an exercise.
Choose each run's token from tokens different from the preceding run, draw its inclusive length, and concatenate the stream. Return zero-based starts, exclusive ends, all boundary offsets including zero and final length, and token counts.1
| Input field | Meaning and units | Accepted input |
|---|---|---|
| tokens | Tokens | Required. Array of Text; maximum UTF-16 length 64.; minimum items 2; maximum items 8. |
| runCount | Run Count | Required. Safe integer; minimum 1; maximum 16. |
| minimumRunLength | Minimum Run Length | Required. Safe integer; minimum 1; maximum 8. |
| maximumRunLength | Maximum Run Length | Required. Safe integer; minimum 1; maximum 8. |
| seed | Optional numeric seed; takes precedence over context randomness. Without either, use the fixed tool.io default seed. | Optional. Safe integer. |
| Output field | Interpretation | Returned shape |
|---|---|---|
| stream | Stream | Required. Array of Text; maximum UTF-16 length 64.; minimum items 0; maximum items 128. |
| segments | Segments | Required. Array of Object; no unknown keys.; minimum items 0; maximum items 16. |
| segments[].token | Token | Required. Text; maximum UTF-16 length 64. |
| segments[].start | Start | Required. Safe integer; minimum 0; maximum 127. |
| segments[].endExclusive | End Exclusive | Required. Safe integer; minimum 1; maximum 128. |
| segments[].length | Length | Required. Safe integer; minimum 1; maximum 8. |
| boundaries | Boundaries | Required. Array of Safe integer; minimum 0; maximum 128.; minimum items 2; maximum items 17. |
| counts | Counts | Required. Array of Object; no unknown keys.; minimum items 0; maximum items 8. |
| counts[].key | Key | Required. Text; maximum UTF-16 length 64. |
| counts[].count | Requested count; the mechanics explain whether one draw expands into several returned records. | Required. Safe integer; minimum 1; maximum 128. |
| randomness | Replay metadata: source is seed or context; seed is the normalized 32-bit seed or null for a context source. | Required. Object; no unknown keys. |
| randomness.seed | Optional numeric replay seed; normalization and precedence are explained below. | Required. Safe integer. or Null. |
| randomness.source | Source | Required. Text; one of "context", "seed". |
An explicit safe-integer seed takes precedence over injected context randomness and is normalized to an unsigned 32-bit integer. With no seed and no context source, replay uses the fixed tool.io default seed. A context source must return finite values from zero inclusive to one exclusive; its advancing state must be retained separately. randomness records the source and normalized seed, or null for context. These generators are not cryptographic random sources.
{
"input": {
"tokens": [
"A",
"B"
],
"runCount": 3,
"minimumRunLength": 1,
"maximumRunLength": 3,
"seed": 42
},
"output": {
"stream": [
"B",
"B",
"A",
"A",
"A",
"B",
"B"
],
"segments": [
{
"token": "B",
"start": 0,
"endExclusive": 2,
"length": 2
},
{
"token": "A",
"start": 2,
"endExclusive": 5,
"length": 3
},
{
"token": "B",
"start": 5,
"endExclusive": 7,
"length": 2
}
],
"boundaries": [
0,
2,
5,
7
],
"counts": [
{
"key": "B",
"count": 4
},
{
"key": "A",
"count": 3
}
],
"randomness": {
"seed": 42,
"source": "seed"
}
}
}Limitations and assumptions
- Adjacent runs cannot merge because their tokens differ. Worst-case stream length is bounded to 128 entries; these are constructed boundaries rather than segmentation inferred from arbitrary observations.
- A replay requires the same complete input and implementation. Randomized construction does not guarantee uniform coverage of every feasible output; finite samples can repeat unless a distinct-selection rule is explicitly stated.
- The field tables state structural bounds; cross-field constraints and bounded search or output budgets also apply. Unknown keys are rejected where the object contract disallows them. Input errors are distinct from null fields and deliberate invalid controls in a successful output.
Alternative or Complementary methods
For a supplied example, inspect the returned construction or delegated check directly. To evaluate a consumer, also test cases chosen for its own boundaries; a generated finite corpus does not establish exhaustive coverage.
References
Similar or alternative tools
- Random Hash Table Collision Workload Generator
Solve residues for the caller's toy linear modular hash, construct distinct planted colliding integer keys and bounded noise, then retain bucket counts and every actual collision pair; no cryptographic collision claim.
- Random Top K Frequency Workload Generator
Construct repeated token counts with optional sampled extras, shuffle the stream and retain exact top-k answers under explicit bank, UTF-16 lexical or first-occurrence tie ordering.