How to calculate ab testing sample size

Question

Do you know formul for calculating ab testing sample size per variation base on:

1. Baseline Conversion Rate
2. Minimum Detectable Effect
3. Statistical Significance
4. Statistical Power

It is a few online tool to calculate the sample size:

• https://www.optimizely.com/resources/sample-size-calculator

• http://www.evanmiller.org/ab-testing/sample-size.html

Stetson

Answer 1

For a given power, false positive rate, minimum detectable effect (MDE) and baseline conversion rate the formula can be determined by comparing the graphs under the null and alternative hypothesis.

Gerald Van Belle wrote a book called "Statistical Rules of Thumb" that reviews this approach. Below are snapshots from Chapter 2 which covers Sample Size.

The formula can be derived when you equate the critical values for alternative and null hypothesis which yields this equation:

These are all screenshots from Van Belle's textbook.

Carew