Skip to content

Split Test Duration

BusinessOperations and HR
Conversion and lift

Current conversion rate of the control page.

20% relative on a 10% baseline means detecting 12%.

Traffic
Statistical settings
Confidence level
Statistical power
Test duration (days) —
Visitors needed per variant
—
Total visitors needed
—
Duration in weeks
—
Conversion rate to detect
—

A split test runs until each variant collects the sample size needed to detect your chosen lift. With a 10% baseline, a 20% relative lift (10% to 12%), 95% confidence and 80% power, each arm needs 3,839 visitors; at 1,000 visitors a day split across two arms, that is 8 days.

About this tool

Growth marketers, product managers and CRO consultants use this to plan an experiment before launching it, so a test is neither stopped early on noise nor left running for months. Enter the control page's conversion rate, the smallest relative lift worth detecting, daily traffic, the share of traffic you send into the experiment and the number of arms. The tool returns the days required, visitors per variant and in total, and a table showing how the duration changes for lifts from 5% to 50%. It assumes a fixed-horizon two-sided test on a binary conversion; sequential or Bayesian methods and revenue-per-visitor metrics follow different maths.

How to use it

  1. Enter the baseline and lift

    Type the control's current conversion rate and the relative lift you want to be able to detect.

  2. Describe your traffic

    Add daily visitors, drag the slider to the share entering the test, and set the number of variants with the stepper.

  3. Pick confidence and power

    Choose 90, 95 or 99% confidence and 80, 90 or 95% power; 95% and 80% are the usual defaults.

  4. Read the duration

    Check the days, the per-variant sample and the lift table to see what a smaller or larger target effect would cost.

Examples

10% baseline, 20% lift, 1,000 visitors a day

Baseline conversion rate
10
Minimum detectable lift (relative)
20
Daily visitors
1000
Share of traffic in the test
100
Variants including control
2
Confidence level
95%
Statistical power
80%

Result Test duration (days): 8
Visitors needed per variant: 3,839
Total visitors needed: 7,678
Duration in weeks: 1.1
Conversion rate to detect: 12%

  1. Target rate p2 = 10% × (1 + 20%) = 12%
  2. z values: z(α/2) = 1.959964, z(β) = 0.841621
  3. n per variant = (1.959964 + 0.841621)² × [0.1×0.9 + 0.12×0.88] ÷ 0.02² = 3839 (rounded up)
  4. Total = 3839 × 2 variants = 7678
  5. Visitors in the test per day = 1000 × 100% = 1000
  6. Days = 7678 ÷ 1000, rounded up = 8

Detecting a move from 10% to 12% at 95% confidence and 80% power needs 3,839 visitors per arm, so a site with 1,000 daily visitors finishes in about eight days.

5% baseline, 10% lift, 99% confidence

Baseline conversion rate
5
Minimum detectable lift (relative)
10
Daily visitors
5000
Share of traffic in the test
50
Variants including control
2
Confidence level
99%
Statistical power
90%

Result Test duration (days): 48
Visitors needed per variant: 59,206
Total visitors needed: 118,412
Duration in weeks: 6.9
Conversion rate to detect: 5.5%

  1. Target rate p2 = 5% × (1 + 10%) = 5.5%
  2. z values: z(α/2) = 2.575829, z(β) = 1.281552
  3. n per variant = (2.575829 + 1.281552)² × [0.05×0.95 + 0.055×0.945] ÷ 0.005² = 59206 (rounded up)
  4. Total = 59206 × 2 variants = 118412
  5. Visitors in the test per day = 5000 × 50% = 2500
  6. Days = 118412 ÷ 2500, rounded up = 48

Small lifts on low baselines are expensive: with half of 5,000 daily visitors in the test, the run takes about seven weeks.

A/B/C test on the same traffic

Baseline conversion rate
10
Minimum detectable lift (relative)
20
Daily visitors
1000
Share of traffic in the test
100
Variants including control
3
Confidence level
95%
Statistical power
80%

Result Test duration (days): 12
Visitors needed per variant: 3,839
Total visitors needed: 11,517
Duration in weeks: 1.7
Conversion rate to detect: 12%

  1. Target rate p2 = 10% × (1 + 20%) = 12%
  2. z values: z(α/2) = 1.959964, z(β) = 0.841621
  3. n per variant = (1.959964 + 0.841621)² × [0.1×0.9 + 0.12×0.88] ÷ 0.02² = 3839 (rounded up)
  4. Total = 3839 × 3 variants = 11517
  5. Visitors in the test per day = 1000 × 100% = 1000
  6. Days = 11517 ÷ 1000, rounded up = 12

Adding a third arm adds another 3,839 visitors, stretching the test from 8 to 12 days.

How it is calculated

n = (z(α/2) + z(β))² × [p1(1 − p1) + p2(1 − p2)] ÷ (p2 − p1)²; days = ⌈n × k ÷ (V × s)⌉

p1
baseline conversion rate
p2
target rate, p1 × (1 + relative lift)
z(α/2), z(β)
normal quantiles for two-sided confidence and for power
k
number of variants including control
V, s
daily visitors and share of traffic in the test

The per-variant sample size uses the unpooled normal approximation for comparing two proportions, rounded up to a whole visitor. Multiplying by the number of arms gives the total traffic required, and dividing by the visitors entering the test each day gives the run length, rounded up to whole days. No correction for multiple comparisons is applied when there are more than two arms.

When not to use it

  • Do not use it for metrics like average order value, which need a variance-based formula.
  • It does not plan sequential or Bayesian tests that allow peeking.
  • Skip it when traffic is so low that the result exceeds a few months.

Common mistakes

  • Entering an absolute lift (2 points) where a relative lift (20%) is expected.
  • Stopping the test the first day the dashboard shows significance.
  • Ending mid-week, so weekday and weekend visitors are unevenly represented.
  • Counting all site visitors when only one page's traffic sees the test.

Frequently asked questions

Why does a smaller lift take so much longer?

Sample size scales with one over the squared difference between rates. Halving the detectable lift roughly quadruples the visitors needed, so a test designed for a 10% relative lift takes about four times as long as one designed for 20%.

What do confidence and power mean here?

Confidence (1 − α) controls false positives: at 95%, a variant with no real effect is declared a winner about 5% of the time. Power (1 − β) controls false negatives: at 80%, a true lift of the chosen size is detected four times out of five.

Should a test run for whole weeks?

Many teams round the duration up to full weeks, because conversion often differs between weekdays and weekends. If the tool says 8 days, running 14 keeps both weekly cycles balanced and slightly raises power.

How are extra variants handled?

Each arm needs the same per-variant sample, so total traffic grows linearly with the number of arms. The tool does not apply a Bonferroni or similar correction, so the real false-positive rate across several comparisons is higher than the level you pick.

Is the lift relative or absolute?

Relative. A 20% lift on a 10% baseline targets 12%, not 30%. To plan for an absolute change of two percentage points on a 10% baseline, enter 20%.

Why might my testing platform give a different number?

Platforms use pooled variance, one-sided tests, continuity corrections or sequential boundaries. For 10% to 12% at 95% and 80%, pooled formulas give around 3,840 per arm and some online planners show 3,600 to 3,900, so differences of a few percent are normal.

Cite this tool

Using this tool in an assignment, article or report? Copy a citation, or a link for your website.

APA

Tools Directory Editorial Team. (2026, October 23). Split Test Duration. Tools Directory. Retrieved October 27, 2026, from https://tools.milunir.com/business/operations-and-hr/split-test-duration/

MLA

Tools Directory Editorial Team. “Split Test Duration.” Tools Directory, 23 Oct. 2026, https://tools.milunir.com/business/operations-and-hr/split-test-duration/. Accessed 27 Oct. 2026.

Chicago

Tools Directory Editorial Team. “Split Test Duration.” Tools Directory. Last modified October 23, 2026. Accessed October 27, 2026. https://tools.milunir.com/business/operations-and-hr/split-test-duration/.

Link<a href="https://tools.milunir.com/business/operations-and-hr/split-test-duration/">Split Test Duration – Tools Directory</a>