Description
Generate paired batches of independent synthetic binomial conversion counts under supplied visitor totals/rates and delegate the existing pooled two-proportion z-test plus Wald interval; retain 12-significant-digit summaries, including degenerate endpoint conventions, not exact small-sample inference or real-study advice.
Synthetic statistical data separates a specified generating model from the values observed in a finite draw. A model parameter, a sample summary, and the result of a fitted analysis answer different questions; retaining them separately makes numerical and statistical checks easier to interpret.
Generate paired batches of independent synthetic binomial conversion counts under supplied visitor totals/rates and delegate the existing pooled two-proportion z-test plus Wald interval; retain 12-significant-digit summaries, including degenerate endpoint conventions, not exact small-sample inference or real-study advice.
When to use this tool
Use this generator to create reproducible ab test experiment cases from the supplied model or bank. Inspect the returned evidence before using cases in a test or exercise.
How to use the tool and how it works
- Supply visitorsA, visitorsB, probabilityA, probabilityB, confidenceLevel, experimentCount in the field shapes below. Set seed when you need an explicit replay.
- Run the tool and correct any input error. For generated controls, distinguish a successful fixture containing an invalid case from failure to generate the fixture.
- Read model, samplerSeeds, experiments. Use the conventions below. Keep answer-bearing fields separate when presenting an exercise.
Derive separate binomial sampler seeds for groups A and B, draw conversion counts, and delegate a pooled two-proportion z test with a two-sided p-value and an unpooled normal difference interval. confidenceLevel is a fraction and the reported difference is rateB minus rateA.1
Delegated operations used for this result: binomial sampler; ab test. Their calculation or parsing conventions apply to the corresponding result fields.2, 3
| Input field | Meaning and units | Accepted input |
|---|---|---|
| visitorsA | Visitors A | Required. Safe integer; minimum 1; maximum 1000000. |
| visitorsB | Visitors B | Required. Safe integer; minimum 1; maximum 1000000. |
| probabilityA | Probability A | Required. Finite number; minimum 0; maximum 1. |
| probabilityB | Probability B | Required. Finite number; minimum 0; maximum 1. |
| confidenceLevel | Confidence Level | Required. Finite number; strictly above 0; strictly below 1. |
| experimentCount | Experiment Count | Required. Safe integer; minimum 1; maximum 128. |
| seed | Optional numeric seed; takes precedence over context randomness. Without either, use the fixed tool.io default seed. | Optional. Safe integer. |
| Output field | Interpretation | Returned shape |
|---|---|---|
| model | Model | Required. Object; no unknown keys. |
| model.visitorsA | Visitors A | Required. Safe integer; minimum 1; maximum 1000000. |
| model.visitorsB | Visitors B | Required. Safe integer; minimum 1; maximum 1000000. |
| model.probabilityA | Probability A | Required. Finite number; minimum 0; maximum 1. |
| model.probabilityB | Probability B | Required. Finite number; minimum 0; maximum 1. |
| model.confidenceLevel | Confidence Level | Required. Finite number; strictly above 0; strictly below 1. |
| samplerSeeds | Sampler Seeds | Required. Object; no unknown keys. |
| samplerSeeds.a | A | Required. Text; minimum UTF-16 length 1. |
| samplerSeeds.b | B | Required. Text; minimum UTF-16 length 1. |
| experiments | Experiments | Required. Array of Object; no unknown keys.; minimum items 1; maximum items 128. |
| experiments[].conversionsA | Conversions A | Required. Safe integer; minimum 0. |
| experiments[].conversionsB | Conversions B | Required. Safe integer; minimum 0. |
| experiments[].analysis | Analysis | Required. Object; no unknown keys. |
| experiments[].analysis.rateA | Conversion rate of variant A. | Required. Finite number. |
| experiments[].analysis.rateB | Conversion rate of variant B. | Required. Finite number. |
| experiments[].analysis.difference | Difference in rates, B minus A. | Required. Finite number. |
| experiments[].analysis.standardError | Pooled standard error of the difference. | Required. Finite number. |
| experiments[].analysis.zScore | z-score of the observed difference. | Required. Finite number. |
| experiments[].analysis.pValue | Two-sided p-value of the z-test. | Required. Finite number. |
| experiments[].analysis.significant | Whether the difference is significant at the level. | Required. Boolean. |
| experiments[].analysis.ciLow | Lower bound of the difference confidence interval. | Required. Finite number. |
| experiments[].analysis.ciHigh | Upper bound of the difference confidence interval. | Required. Finite number. |
| randomness | Replay source and normalized 32-bit seed, or null seed for an advancing context source. | Required. Object; no unknown keys. |
| randomness.seed | Optional safe-integer replay seed; normalization and precedence are explained below. | Required. Safe integer. or Null. |
| randomness.source | Source | Required. Text; one of "context", "seed". |
An explicit safe-integer seed takes precedence over injected context randomness and is normalized to an unsigned 32-bit integer. With no seed and no context source, replay uses the fixed tool.io default seed. A context source must return finite values from zero inclusive to one exclusive; its advancing state must be retained separately. randomness records the source and normalized seed, or null for context. These generators are not cryptographic random sources.
{
"input": {
"experimentCount": 4,
"visitorsA": 1,
"visitorsB": 1,
"probabilityA": 0.5,
"probabilityB": 0.5,
"confidenceLevel": 0.95,
"seed": 42
},
"output": {
"model": {
"visitorsA": 1,
"visitorsB": 1,
"probabilityA": 0.5,
"probabilityB": 0.5,
"confidenceLevel": 0.95
},
"samplerSeeds": {
"a": "ab-A:2581720956",
"b": "ab-B:1925393290"
},
"experiments": [
{
"conversionsA": 1,
"conversionsB": 0,
"analysis": {
"ciHigh": -1,
"ciLow": -1,
"difference": -1,
"pValue": 0.15729920705,
"rateA": 1,
"rateB": 0,
"significant": false,
"standardError": 0.707106781187,
"zScore": -1.41421356237
}
},
{
"conversionsA": 1,
"conversionsB": 0,
"analysis": {
"ciHigh": -1,
"ciLow": -1,
"difference": -1,
"pValue": 0.15729920705,
"rateA": 1,
"rateB": 0,
"significant": false,
"standardError": 0.707106781187,
"zScore": -1.41421356237
}
},
{
"conversionsA": 0,
"conversionsB": 0,
"analysis": {
"ciHigh": 0,
"ciLow": 0,
"difference": 0,
"pValue": 1,
"rateA": 0,
"rateB": 0,
"significant": false,
"standardError": 0,
"zScore": 0
}
},
{
"conversionsA": 0,
"conversionsB": 1,
"analysis": {
"ciHigh": 1,
"ciLow": 1,
"difference": 1,
"pValue": 0.15729920705,
"rateA": 0,
"rateB": 1,
"significant": false,
"standardError": 0.707106781187,
"zScore": 1.41421356237
}
}
],
"randomness": {
"seed": 42,
"source": "seed"
}
}
}Limitations and assumptions
- This is fixed-model simulation rather than a live causal experiment. Normal approximations can be poor for small or extreme samples. The core sets zScore to zero when pooled standard error is zero; a finite result does not validate the experimental design. Repetition does not guarantee nominal significance or interval coverage.
- A replay requires the same complete input and implementation. The stated sampling convention applies within its eligible space. A finite draw does not exhaust all possible outputs; repeats remain possible unless a distinct-selection rule is explicit.
- The field tables state structural bounds; cross-field constraints and bounded search or output budgets also apply. Unknown keys are rejected where the object contract disallows them. Input errors are distinct from null fields and deliberate invalid controls in a successful output.
Alternative or Complementary methods
For a supplied example, inspect the returned construction or delegated check directly. To evaluate a consumer, also test cases chosen for its own boundaries; a generated finite corpus does not establish exhaustive coverage.
References
Similar or alternative tools
- Random Bootstrap Interval Experiment Generator
Generate bounded normal datasets and delegate mean/median percentile-bootstrap intervals with equal two-sided tail probabilities, adapting the older core level convention; retain nearest-rank endpoints and 12-significant-digit summary rounding, not BCa intervals or guaranteed nominal coverage.
- Random Confidence Interval Coverage Experiment Generator
Generate repeated normal samples and compare observed coverage of delegated Student t and known-sigma z mean intervals at supported confidence levels; normalize moment/interval calculations for scale and retain full-precision summaries and endpoints, not a nominal-coverage guarantee.