Random Cardinality Sketch Workload Generator

Description

Construct a seeded shuffled stream with an exact caller-selected number of distinct strings and duplicate occurrences, then delegate the existing fixed-hash HyperLogLog estimate; report signed and relative errors separately from exact cardinality, with null duplicate/relative fractions when their denominator is zero.

An operation trace pairs an ordered request stream with the state after each step. Empty operations, ties, evictions, and boundary indexes are part of the modeled contract rather than incidental display details.

When to use this tool

Use this generator to create reproducible cardinality sketch workload cases from the supplied model or bank. Inspect the returned evidence before using cases in a test or exercise.

How to use the tool and how it works

Supply universe, streamLength, distinctCount, precision in the field shapes below. Set seed when you need an explicit replay.
Run the tool and correct any input error. For generated controls, distinguish a successful fixture containing an invalid case from failure to generate the fixture.
Read stream, exact, sketch. Use the conventions below. Keep answer-bearing fields separate when presenting an exercise.

Deduplicate the supplied universe, choose exactly distinctCount values, include each once, and fill and shuffle the remaining stream with repeats. Run HyperLogLog and return exact distinct/duplicate counts, estimate, and signed/relative error.^[1]

Input field	Meaning and units	Accepted input
universe	Universe	Required. Array of Text; maximum UTF-16 length 64.; minimum items 1; maximum items 512.
streamLength	Stream Length	Required. Safe integer; minimum 0; maximum 2048.
distinctCount	Distinct Count	Required. Safe integer; minimum 0; maximum 512.
precision	Precision	Required. Safe integer; minimum 4; maximum 16.
seed	Optional numeric seed; takes precedence over context randomness. Without either, use the fixed tool.io default seed.	Optional. Safe integer.

Output field	Interpretation	Returned shape
stream	Stream	Required. Array of Text; maximum UTF-16 length 64.; maximum items 2048.
exact	Exact	Required. Object; no unknown keys.
exact.uniqueCount	Unique Count	Required. Safe integer; minimum 0; maximum 512.
exact.duplicateCount	Duplicate Count	Required. Safe integer; minimum 0; maximum 2048.
exact.duplicateFraction	Duplicate Fraction	Required. Finite number; minimum 0; maximum 1. or Null.
sketch	Sketch	Required. Object; no unknown keys.
sketch.estimate	Estimate	Required. Finite number; minimum 0.
sketch.registers	Registers	Required. Safe integer; minimum 16; maximum 65536.
sketch.signedError	Signed Error	Required. Finite number.
sketch.relativeError	Relative Error	Required. Finite number. or Null.
randomness	Replay source and normalized 32-bit seed, or null seed for an advancing context source.	Required. Object; no unknown keys.
randomness.seed	Optional safe-integer replay seed; normalization and precedence are explained below.	Required. Safe integer. or Null.
randomness.source	Source	Required. Text; one of "context", "seed".

An explicit safe-integer seed takes precedence over injected context randomness and is normalized to an unsigned 32-bit integer. With no seed and no context source, replay uses the fixed tool.io default seed. A context source must return finite values from zero inclusive to one exclusive; its advancing state must be retained separately. randomness records the source and normalized seed, or null for context. These generators are not cryptographic random sources.

json

{
  "input": {
    "universe": [
      "a",
      "b",
      "c",
      "d"
    ],
    "streamLength": 8,
    "distinctCount": 3,
    "precision": 4,
    "seed": 42
  },
  "output": {
    "stream": [
      "d",
      "d",
      "a",
      "b",
      "a",
      "d",
      "b",
      "a"
    ],
    "exact": {
      "uniqueCount": 3,
      "duplicateCount": 5,
      "duplicateFraction": 0.625
    },
    "sketch": {
      "estimate": 2.136502281992361,
      "registers": 16,
      "signedError": -0.8634977180076389,
      "relativeError": -0.28783257266921297
    },
    "randomness": {
      "seed": 42,
      "source": "seed"
    }
  }
}

Limitations and assumptions

An empty stream has null duplicate fraction and relative error. Precision controls the linked sketch rather than an exact-count guarantee; the output is a synthetic workload with known truth, not observed traffic.
A replay requires the same complete input and implementation. The stated sampling convention applies within its eligible space. A finite draw does not exhaust all possible outputs; repeats remain possible unless a distinct-selection rule is explicit.
The field tables state structural bounds; cross-field constraints and bounded search or output budgets also apply. Unknown keys are rejected where the object contract disallows them. Input errors are distinct from null fields and deliberate invalid controls in a successful output.

Alternative or Complementary methods

For a supplied example, inspect the returned construction or delegated check directly. To evaluate a consumer, also test cases chosen for its own boundaries; a generated finite corpus does not establish exhaustive coverage.

References

Tool behavior and example: Random Cardinality Sketch Workload Generator — tool.io

Similar or alternative tools

Random Bloom Filter Workload Generator
Deduplicate a supplied string universe, sample a distinct insertion set and controlled present/absent query pools, and delegate deterministic Bloom responses; report exact labels and observed false positives separately from the core's 8-significant-digit ideal-hash probability approximation, not a guaranteed error rate.
Random Counting Bloom Filter Workload Generator
Generate with-replacement string insertions, remove only eligible remaining occurrences, and query a delegated counting Bloom filter against exact retained multiplicities; query estimates may overcount through collisions, and the fixed batch is not an arbitrary interleaved deletion trace.