What the Sample Size Calculator Does
The Sample Size Calculator helps determine how many observations are required to obtain statistically reliable results. It calculates the minimum sample size needed for experiments and studies based on the baseline conversion rate, minimum detectable effect (MDE), statistical significance level, and statistical power.
The tool helps you:
calculate the required sample size for A/B tests;
balance experiment speed and result accuracy;
estimate how the MDE affects test duration;
account for the significance level (α) and statistical power (1−β);
save calculation parameters as a shareable link for future use.
It is suitable for product analysts, marketers, CRO specialists, researchers, and data analysts.
Parameters Used
Parameter | Description |
|---|
Baseline Conversion Rate | The current conversion rate of the control group |
MDE (Minimum Detectable Effect) | The smallest conversion change you want to be able to detect |
Confidence Level | The statistical confidence level (for example, 95%) |
Statistical Power | The probability of detecting a real effect |
Effect Type | Absolute or relative change in the conversion rate |
Enter the current conversion rate.
Select the minimum effect that is practically meaningful.
Choose the confidence level.
Adjust the statistical power if necessary.
Select the effect type — absolute or relative.
View the minimum required sample size for each experiment variant.
The calculated value represents the number of users (or other observational units) required for each group, not the total sample size for the entire experiment.
What Affects Sample Size
Factor | Impact |
|---|
Smaller MDE | Requires more participants |
Higher statistical power | Requires a larger sample |
Higher confidence level | Increases the required sample size |
Higher baseline conversion rate | The impact depends on the expected effect size |
How to Choose an MDE
The Minimum Detectable Effect (MDE) is the smallest change that has practical value for the business.
For example:
Baseline Conversion | MDE | Interpretation |
|---|
5% | +0.5% | A very small effect |
5% | +1% | Commonly used in product experiments |
5% | +2% | Allows the experiment to finish more quickly |
Choosing an MDE that is too small significantly increases both the required sample size and the duration of the experiment.
Practical Recommendations
Choose the MDE based on business impact rather than the desire to detect the smallest possible change.
Do not reduce the significance level simply to decrease the required sample size.
Use a statistical power of 80–90%, which is the standard for most product experiments.
Calculate the required sample size before starting the experiment, not after seeing the initial results.
Do not stop an experiment before reaching the required sample size unless you are using specialized sequential analysis methods.
An insufficient sample size increases the risk of incorrect conclusions and may lead to implementing ineffective changes.
Common Mistakes
Starting an experiment without calculating the required sample size.
Choosing an unrealistically small MDE to detect tiny differences.
Ending the experiment immediately after obtaining the first "statistically significant" result.
Changing experiment parameters while the experiment is running.
Treating statistical significance as proof that the effect has substantial practical value.
The calculation assumes:
independent observations;
random assignment of users to experiment variants;
accurate data collection;
no systematic measurement errors.
If these assumptions are violated, the actual reliability of the experiment may differ from the calculated estimates.
Conclusion
Sample size estimation is the first step in preparing any statistical experiment. It helps determine the required amount of data in advance and prevents experiments from ending too early or running longer than necessary.
After determining the required sample size, it is recommended to use additional tools for a complete experiment analysis workflow:
Chi-Square (χ²) Calculator — for analyzing categorical data and comparing conversion rates when this statistical model is appropriate.
Two-Sample T-Test Calculator — for evaluating the statistical significance of differences between experiment variants after an A/B test has been completed.
If you also want to verify that users were assigned correctly across experiment variants, use the Sample Ratio Mismatch (SRM) Calculator.
