Skip to content

Main content

Random Multiple-Testing Test Case Generator

Generate synthetic beta-distributed probability values with the existing 12-significant-digit sampler and apply delegated Benjamini–Hochberg and Bonferroni corrections, preserving original order and threshold decisions; these are not calibrated study p-values and no actual false-discovery-rate guarantee is inferred.

Loading tool…

Description

Generate synthetic beta-distributed probability values with the existing 12-significant-digit sampler and apply delegated Benjamini–Hochberg and Bonferroni corrections, preserving original order and threshold decisions; these are not calibrated study p-values and no actual false-discovery-rate guarantee is inferred.

Synthetic statistical data separates a specified generating model from the values observed in a finite draw. A model parameter, a sample summary, and the result of a fitted analysis answer different questions; retaining them separately makes numerical and statistical checks easier to interpret.

Generate synthetic beta-distributed probability values with the existing 12-significant-digit sampler and apply delegated Benjamini–Hochberg and Bonferroni corrections, preserving original order and threshold decisions; these are not calibrated study p-values and no actual false-discovery-rate guarantee is inferred.

When to use this tool

Use this generator to create reproducible multiple testing test case cases from the supplied model or bank. Inspect the returned evidence before using cases in a test or exercise.

How to use the tool and how it works

  1. Supply betaAlpha, betaBeta, significanceLevel, count in the field shapes below. Set seed when you need an explicit replay.
  2. Run the tool and correct any input error. For generated controls, distinguish a successful fixture containing an invalid case from failure to generate the fixture.
  3. Read model, samplerSeed, probabilities, benjaminiHochberg, bonferroni, decisions. Use the conventions below and keep answer-bearing fields separate when presenting an exercise.

Derive a sampler seed and draw synthetic probabilities from the specified beta distribution. Delegate Benjamini-Hochberg and Bonferroni adjustment and compare original and adjusted probabilities with significanceLevel using an inclusive threshold.1

Delegated operations used for this result: beta distribution sampler; benjamini hochberg. Their calculation or parsing conventions apply to the corresponding result fields.2, 3

Input fieldMeaning and unitsAccepted input
betaAlphaBeta AlphaRequired. Finite number; strictly above 0.
betaBetaBeta BetaRequired. Finite number; strictly above 0.
significanceLevelSignificance LevelRequired. Finite number; minimum 0; maximum 1.
countRequested count; the mechanics explain whether one draw expands into several returned records.Required. Safe integer; minimum 1; maximum 512.
seedOptional numeric seed; takes precedence over context randomness. Without either, use the fixed tool.io default seed.Optional. Safe integer.
Output fieldInterpretationReturned shape
modelModelRequired. Object; no unknown keys.
model.betaAlphaBeta AlphaRequired. Finite number; strictly above 0.
model.betaBetaBeta BetaRequired. Finite number; strictly above 0.
model.significanceLevelSignificance LevelRequired. Finite number; minimum 0; maximum 1.
samplerSeedSampler SeedRequired. Text; minimum UTF-16 length 1.
probabilitiesProbabilitiesRequired. Array of Finite number; minimum 0; maximum 1.; minimum items 1; maximum items 512.
benjaminiHochbergBenjamini HochbergRequired. Array of Finite number; minimum 0; maximum 1.; minimum items 1; maximum items 512.
bonferroniBonferroniRequired. Array of Finite number; minimum 0; maximum 1.; minimum items 1; maximum items 512.
decisionsDecisionsRequired. Object; no unknown keys.
decisions.unadjustedUnadjustedRequired. Array of Boolean.; minimum items 1; maximum items 512.
decisions.benjaminiHochbergBenjamini HochbergRequired. Array of Boolean.; minimum items 1; maximum items 512.
decisions.bonferroniBonferroniRequired. Array of Boolean.; minimum items 1; maximum items 512.
randomnessReplay metadata: source is seed or context; seed is the normalized 32-bit seed or null for a context source.Required. Object; no unknown keys.
randomness.seedOptional numeric replay seed; normalization and precedence are explained below.Required. Safe integer. or Null.
randomness.sourceSourceRequired. Text; one of "context", "seed".

An explicit safe-integer seed takes precedence over injected context randomness and is normalized to an unsigned 32-bit integer. With no seed and no context source, replay uses the fixed tool.io default seed. A context source must return finite values from zero inclusive to one exclusive; its advancing state must be retained separately. randomness records the source and normalized seed, or null for context. These generators are not cryptographic random sources.

json
{
  "input": {
    "count": 4,
    "betaAlpha": 2,
    "betaBeta": 5,
    "significanceLevel": 0.2,
    "seed": 42
  },
  "output": {
    "model": {
      "betaAlpha": 2,
      "betaBeta": 5,
      "significanceLevel": 0.2
    },
    "samplerSeed": "multiple-testing:2581720956",
    "probabilities": [
      0.365540836018,
      0.172622514081,
      0.406420307559,
      0.402638240148
    ],
    "benjaminiHochberg": [
      0.406420307559,
      0.406420307559,
      0.406420307559,
      0.406420307559
    ],
    "bonferroni": [
      1,
      0.690490056324,
      1,
      1
    ],
    "decisions": {
      "unadjusted": [
        false,
        true,
        false,
        false
      ],
      "benjaminiHochberg": [
        false,
        false,
        false,
        false
      ],
      "bonferroni": [
        false,
        false,
        false,
        false
      ]
    },
    "randomness": {
      "seed": 42,
      "source": "seed"
    }
  }
}

Limitations and assumptions

  • These probabilities are synthetic inputs, not p-values calculated from experiments or validated null models. Decision flags demonstrate correction behavior rather than establish scientific discoveries or error control for real data.
  • A replay requires the same complete input and implementation. Randomized construction does not guarantee uniform coverage of every feasible output; finite samples can repeat unless a distinct-selection rule is explicitly stated.
  • The field tables state structural bounds; cross-field constraints and bounded search or output budgets also apply. Unknown keys are rejected where the object contract disallows them. Input errors are distinct from null fields and deliberate invalid controls in a successful output.

Alternative or Complementary methods

For a supplied example, inspect the returned construction or delegated check directly. To evaluate a consumer, also test cases chosen for its own boundaries; a generated finite corpus does not establish exhaustive coverage.

References

  1. Tool behavior and example: Random Multiple-Testing Test Case Generator — tool.io

  2. Calculation or validation: probability/beta-distribution-sampler — tool.io

  3. Calculation or validation: statistics/benjamini-hochberg — tool.io

Similar or alternative tools

  • Random Heavy-Tail Comparison Experiment Generator

    Compare matched-size Cauchy and Pareto Type I samples with visible derived sampler seeds, observed extremes, medians, and linearly interpolated empirical quantiles; preserve Pareto sampler 10-significant-digit rounding and epsilon tail clipping, and do not report a finite Cauchy population mean or variance.

  • Random Transaction Itemset Instance Generator

    Generate exact-count set-valued transactions by choosing a uniform size and a without-replacement subset from distinct supplied item labels; transactions may repeat and imported frequencies count transaction presence.