A/B Test Significance Calculator

It runs a two-sided two-proportion z-test. From the visitors and conversions of each variant it works out both conversion rates, then computes a z-score using the pooled standard error and converts that to a p-value. If the p-value is below your chosen significance threshold (for example 0.05 at 95% confidence), the difference is flagged as statistically significant. Everything is computed in your browser.

The p-value is the probability of seeing a difference at least as large as yours if the two variants actually performed the same. A small p-value means the gap is unlikely to be down to chance. At 95% confidence you treat a result as significant when the p-value is below 0.05; at 99% confidence the bar is 0.01.

There is no single number, but very small samples produce noisy p-values that can flip with a handful of extra conversions. As a rule of thumb, wait until each variant has at least a few hundred conversions and the test has run long enough to cover normal day-of-week variation before you trust the verdict.

No. Significance tells you a difference is probably real, not that it is large enough to matter. Always look at the relative lift alongside the p-value: a tiny lift can be statistically significant with a huge sample, while a promising lift may not yet be significant if the sample is small.