HesapHanem

A/B Test Significance Calculator

Check if your A/B test result is statistically significant.

How it is calculated

The tool runs a two-proportion z-test. Conversion rates are pA = cA / nA and pB = cB / nB, where n is visitors and c is conversions in each variant. The pooled rate is p = (cA + cB) / (nA + nB), the standard error is SE = √(p(1 − p)(1/nA + 1/nB)) and z = (pB − pA) / SE. The uplift shown is (pB − pA) / pA as a percentage. If A has no conversions (a 0% baseline) the relative uplift is undefined and is shown as "—".

The verdict is fixed: |z| ≥ 1.96, a two-sided 95% level, is reported as statistically significant, anything smaller as not significant. No p-value is displayed. Conversions cannot exceed visitors and visitor counts must be above zero. The test assumes the visitors were split randomly and the sample sizes were chosen in advance.

Worked example

Variant A: 5,000 visitors, 250 conversions, so pA = 5.00%. Variant B: 5,000 visitors, 300 conversions, so pB = 6.00%. The pooled rate is 550 / 10,000 = 5.5%, giving SE ≈ 0.00456. The difference of 0.01 divided by that is z ≈ 2.19. The uplift is 1 / 5 = 20.00%. Because 2.19 is above 1.96, the verdict is Statistically significant. With only 1,000 visitors per variant and the same rates, z falls to 0.98 and the result is not significant.

Try this example →

Common mistakes

  • Entering conversion rates in the conversions field. It needs counts (250), not percentages (5).
  • Stopping the test the moment the result turns significant. Checking repeatedly and stopping early inflates false positives; decide the sample size beforehand.
  • Reading the 20.00% uplift as percentage points. It is relative: 5% to 6% is a 1 point gain, which is a 20% relative lift.
  • Treating a significant result as proof the change is large or valuable. It only says the gap is unlikely to be noise at this threshold.
  • Pasting in totals from unequal or non-random traffic splits, or counting returning visitors twice, which breaks the independence the test assumes.

About this calculator

This calculator runs a two-proportion z-test to check whether the difference between two conversion rates is statistically significant, computing a pooled standard error from both sample sizes and conversion counts, then comparing the resulting z-score with 1.96, the cut-off for significance at the 95% confidence level (two-sided).

Product managers, growth marketers, and UX researchers use it after running an A/B test on a landing page, email subject line, or checkout flow to decide whether a lift in conversions is real or just random noise.

Was this helpful?

You can pick up to 3 reactions.

Comments (1)

  • Editorial team

    Decide the sample size and the stopping date before the test starts. Checking the result every day and stopping the moment the verdict flips to significant inflates false positives, because this one-shot z-test assumes a single look. The tool reports the 95% two-sided verdict only and says nothing about whether the uplift is large enough to matter. What do you do about peeking when you run your tests?

Popular calculators

Loan Calculator

Monthly payment, total interest and full amortization schedule.

Percentage Calculator

Percent change, percent of, and ratio calculations.

Discount Calculator

Calculate final price, savings, and discount amount.

BMI Calculator

Body mass index from height and weight (metric or imperial).

Age Calculator

Compute your exact age in years, months and days from a birth date.

Compound Interest Calculator

Calculate the final amount and interest earned with compounding.

All Calculators