How it works
p̄ = (xA+xB)/(nA+nB). z = (pB−pA)/√[p̄(1−p̄)(1/nA+1/nB)]. Two-sided p-value = 2 × (1−Φ(|z|)).
Worked example
500/10,000 versus 600/10,000 give 5% and 6%, 20% lift and an approximate p-value of 0.0019.
Assumptions and limits
Requires at least 10 conversions and 10 non-conversions in each group. Repeated checking and stopping at significance increase false positives.
Method reference: NIST: difference of proportions
See it in real data
ASO
Keywords and search rank in the app stores
Frequently asked questions
How do I use the a/b test significance calculator?
Compare two proportions with a two-sided pooled z-test and inspect the difference, lift, and approximate p-value.
Does significance guarantee B is better?
No. Check direction, effect size, randomization integrity, and the planned horizon. A p-value is not the probability that a variant is correct.
Do I need an account and are my inputs uploaded?
No account is required. Calculations run in your browser. Entered numbers and text are not included in tool analytics events; only the tool ID, language, and action are recorded.