Skip to content

AB Test Sample Size

BusinessOperations and HR

Current conversion rate of the control page.

Effect expressed as

Smallest change you want the test to detect reliably.

Confidence level
Statistical power
Visitors needed per variant —
Target conversion rate
—
Absolute difference (points)
—
Relative lift
—
Total visitors across variants
—

An A/B test needs n = (z for confidence + z for power)² × [p1(1−p1) + p2(1−p2)] ÷ (p2 − p1)² visitors per variant. Going from a 10% to a 12% conversion rate at 95% confidence and 80% power takes 3,839 visitors in each group, or 7,678 in total for a two-arm test.

About this tool

Product managers, growth marketers and analysts use this before launching an experiment to see whether their traffic can support it. Enter the control page's conversion rate, the smallest change worth detecting, and the confidence and power you want; the calculator returns visitors per variant, the total across all arms and the target rate implied by your effect. The effect can be set as a relative lift (20% of 5% is one point) or as percentage points. The working lists each z-value and the variance term so the figure can be checked by hand. One limit: this is a fixed-horizon calculation for a single two-sided comparison on a conversion metric. Peeking at results daily, revenue-per-visitor metrics and sequential designs all need different methods.

How to use it

  1. Enter the baseline rate

    Type the control's current conversion rate as a percentage, from recent analytics data.

  2. Set the effect

    Pick relative lift or percentage points, then enter the smallest effect that would change your decision.

  3. Choose confidence and power

    95% confidence with 80% power is the usual default; raise power when missing a real win is costly.

  4. Set the variant count

    Use the stepper for the number of arms, control included, to get total traffic.

Examples

10% to 12% checkout rate

Baseline conversion rate
10
Effect expressed as
Percentage points
Minimum detectable effect
2
Confidence level
95%
Statistical power
80%
Number of variants (incl. control)
2

Result Visitors needed per variant: 3,839
Target conversion rate: 12%
Absolute difference (points): 2
Relative lift: 20%
Total visitors across variants: 7,678

  1. Baseline p1 = 0.1, target p2 = 0.12
  2. z(α/2) = 1.959964 (95% confidence), z(β) = 0.841621 (80% power)
  3. p1(1−p1) + p2(1−p2) = 0.1956
  4. n = (2.801585)² × 0.1956 ÷ 0.02² = 3838.1 → 3839 per variant
  5. Total = 3839 × 2 variants = 7678

A two-point gain on a 10% baseline at 95% confidence and 80% power needs about 3,839 visitors in each arm.

5% sign-up rate, 20% relative lift

Baseline conversion rate
5
Effect expressed as
Relative lift
Minimum detectable effect
20
Confidence level
95%
Statistical power
80%
Number of variants (incl. control)
2

Result Visitors needed per variant: 8,155
Target conversion rate: 6%
Absolute difference (points): 1
Relative lift: 20%
Total visitors across variants: 16,310

  1. Baseline p1 = 0.05, target p2 = 0.06
  2. z(α/2) = 1.959964 (95% confidence), z(β) = 0.841621 (80% power)
  3. p1(1−p1) + p2(1−p2) = 0.1039
  4. n = (2.801585)² × 0.1039 ÷ 0.01² = 8154.98 → 8155 per variant
  5. Total = 8155 × 2 variants = 16310

Detecting a move from 5% to 6% takes more than twice the traffic of the first example because the gap is half as wide.

How it is calculated

n = (z₁₋α/₂ + z₁₋β)² × [p1(1 − p1) + p2(1 − p2)] ÷ (p2 − p1)²

p1
baseline conversion rate as a proportion
p2
target rate: p1 plus the absolute effect, or p1 × (1 + relative lift)
z₁₋α/₂
two-sided critical value: 1.645, 1.960 or 2.576 for 90, 95 or 99% confidence
z₁₋β
power quantile: 0.842, 1.282 or 1.645 for 80, 90 or 95% power

This is the normal-approximation formula for comparing two independent proportions with unpooled variance, as given in standard sample-size references. The result is rounded up to the next whole visitor and applies to each arm; total traffic is that figure times the number of variants. It assumes equal allocation and independent visitors.

When not to use it

  • Metrics such as revenue per visitor or time on page need a means-based formula.
  • Sequential or Bayesian tests that stop early use different stopping rules.
  • Very low baseline rates with tiny samples break the normal approximation.
  • Unequal traffic splits need an adjusted allocation formula.

Common mistakes

  • Confusing a 20% relative lift with a 20-point absolute jump.
  • Stopping the test as soon as it looks significant, before reaching the planned sample.
  • Quoting per-variant numbers as the total traffic requirement.
  • Setting an effect so small the test would run for months.

Frequently asked questions

What is a minimum detectable effect?

It is the smallest true difference between control and variant that the test is designed to detect with the chosen power. Smaller effects need far more visitors: halving the effect roughly quadruples the sample, because the difference is squared in the denominator.

Why does 80% power matter?

Power is the chance of detecting the effect if it truly exists. At 80%, one in five real improvements of that size would be missed. Moving to 90% power raises the sample by about a third, which is often worth it for high-stakes changes.

Should I use relative or absolute effect?

Both describe the same target. Relative lift is how most teams talk about wins (a 10% improvement), while percentage points are clearer for low rates. On a 5% baseline, a 20% relative lift and a 1-point absolute lift give identical results.

How long should my test run?

Divide the total visitors by the daily traffic entering the test, then round up to whole weeks so weekday and weekend behaviour are both covered. Running at least one or two full business cycles reduces bias from day-of-week effects.

Does adding more variants change the per-variant number?

In this calculation it does not; each variant still needs the same count, so total traffic grows with every arm. Strictly, multiple comparisons against control raise the false-positive risk, and a Bonferroni-style correction would increase each group further.

Why is the answer different from other calculators?

Some calculators use a pooled variance under the null hypothesis, one-sided tests, or a continuity correction. Those choices shift the answer by a few percent. This one uses the two-sided unpooled formula, which is the common textbook default.

Cite this tool

Using this tool in an assignment, article or report? Copy a citation, or a link for your website.

APA

Tools Directory Editorial Team. (2026, October 23). AB Test Sample Size. Tools Directory. Retrieved October 26, 2026, from https://tools.milunir.com/business/operations-and-hr/ab-test-sample-size/

MLA

Tools Directory Editorial Team. “AB Test Sample Size.” Tools Directory, 23 Oct. 2026, https://tools.milunir.com/business/operations-and-hr/ab-test-sample-size/. Accessed 26 Oct. 2026.

Chicago

Tools Directory Editorial Team. “AB Test Sample Size.” Tools Directory. Last modified October 23, 2026. Accessed October 26, 2026. https://tools.milunir.com/business/operations-and-hr/ab-test-sample-size/.

Link<a href="https://tools.milunir.com/business/operations-and-hr/ab-test-sample-size/">AB Test Sample Size – Tools Directory</a>