A/B Test Result Analyzer

Compare two variant conversion rates with a two-proportion z-test.

Description

Compare two variant conversion rates with a two-proportion z-test.

A/B Test Result Analyzer is a focused tool for the following task. Compare two variant conversion rates with a two-proportion z-test. It reports Rate (A), Rate (B), Difference, Standard error, z-score, p-value, Significant, CI lower bound, CI upper bound from the values you provide rather than inventing measurements, coefficients, or professional judgment that are not part of the input.

When to use A/B Test Result Analyzer

Use this calculation to reproduce a defined quantitative method when the observations, units, sampling process, and assumptions match the method shown here.

Visitors (A): Required integer. Visitors shown variant A.
Conversions (A): Required integer. Conversions achieved by variant A.
Visitors (B): Required integer. Visitors shown variant B.
Conversions (B): Required integer. Conversions achieved by variant B.
Confidence level: Optional number. Confidence level for the significance decision and difference interval.

The cited overview of Statistics supplies background for the terminology and domain context used by this tool.^[1]

How A/B Test Result Analyzer works

Compare two variant conversion rates with a two-proportion z-test. Inputs are interpreted exactly in the displayed units and the calculation returns the following fields without presentation rounding.

Rate (A): Returned number. Conversion rate of variant A.
Rate (B): Returned number. Conversion rate of variant B.
Difference: Returned number. Difference in rates, B minus A.
Standard error: Returned number. Pooled standard error of the difference.
z-score: Returned number. z-score of the observed difference.
p-value: Returned number. Two-sided p-value of the z-test.
Significant: Returned boolean. Whether the difference is significant at the level.
CI lower bound: Returned number. Lower bound of the difference confidence interval.
CI upper bound: Returned number. Upper bound of the difference confidence interval.

Limitations and assumptions

A numerical result does not by itself establish data quality, causation, representativeness, independence, distributional fit, or practical significance.
Visitors (A) must be at least 1.
Conversions (A) must be at least 0.
Visitors (B) must be at least 1.
Conversions (B) must be at least 0.
Use finite inputs in the displayed units, preserve source measurements and assumptions, and independently verify consequential decisions.

Alternative or Complementary approaches

Inspect the underlying data, visualize its distribution, report uncertainty and sample size, and compare the result with a robust or domain-specific method where appropriate.

References

Statistics — Wikipedia contributors

Similar or alternative tools

Fisher's Exact Test
Compute the two-sided Fisher exact p-value for a 2×2 contingency table with fixed margins. Tables above 10,000 total observations are rejected to bound exact enumeration.
Mann-Whitney U Test Calculator
Calculate the Mann-Whitney U statistic for two independent samples using average ranks for ties.
One-Way ANOVA F-Ratio Calculator
Compare variation between two or more independent groups with a one-way ANOVA F-ratio.