Two-proportion Z-test
The two-proportion Z-test (also called the two-sample proportion Z-test) is a statistical hypothesis test for assessing whether two groups differ in the proportion of a binary outcome, in such a significant way that is beyond chance.
≈ 11 min read
Names in other languages
No names in other languages are held for this entry yet.
Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.
The two-proportion Z-test (also called the two-sample proportion Z-test) is a statistical hypothesis test for assessing whether two groups differ in the proportion of a binary outcome, in such a significant way that is beyond chance. For example, the proportion of patients responding positively to a treatment in a clinical trial versus control, the defect rate in quality control for two production lines, or the click-through rate in an A/B test of two alternative webpage designs.
The test is appropriate when each observation is independent from another, can be classified as a success or failure (i.e., a Bernoulli trial) and the sample sizes are large enough that the sampling distribution of each sample proportion is well approximated by the central limit theorem. Under those conditions the observed difference of sample proportions can be converted to a standardized z-statistic (using a pooled standard error) and compared to the standard normal distribution to obtain p-values or form confidence intervals for the difference in proportions (derived slightly differently).
This also allows to determine required sample-size for a minimum-detectable-effect calculations. The test is related to other well known tests such as Pearson's chi-squared test, Fisher's exact test for small samples, and McNemar's test for paired binary data. null hypothesis (H 0 ): The proportions in the two populations are equal, i.e., p_1 = p_2 . alternative hypothesis (H 1 ): The proportions in the two populations are not equal, i.e., p_1 \neq p_2 (two-tailed) or p_1 > p_2 / p_1 (one-tailed).
The z-statistic for comparing two proportions is computed using: z = \frac{\hat{p}_1 - \hat{p}_2}{\sqrt{\hat{p}(1-\hat{p})\left(\frac{1}{n_1} + \frac{1}{n_2}\right)}}, where \hat{p}_1 and \hat{p}_2 are the sample proportion in the first and second sample, n_1 and n_2 are the size of the first and second sample, respectively, \hat{p} is pooled proportion, calculated as \hat{p} = \frac{x_1 + x_2}{n_1 + n_2} , where x_1 and x_2 are the counts of successes in the two samples The pooled proportion is used to estimate the shared probability of success under the null hypothesis, and the standard error accounts for variability across the two samples. The z-test determines statistical significance by comparing the calculated z-statistic to a critical value.
E.g., for a significance level of \alpha = 0.05 we reject the null hypothesis if |z| > 1.96 (for a two-tailed test). Or, alternatively, by computing the p-value and rejecting the null hypothesis if p . The confidence interval for the difference between two proportions, based on the definitions above, is: (\hat{p}_1 - \hat{p}_2) \pm z_{\alpha/2} \sqrt{\frac{\hat{p}_1(1-\hat{p}_1)}{n_1} + \frac{\hat{p}_2(1-\hat{p}_2)}{n_2}}, where z_{\alpha/2} is the critical value of the standard normal distribution (e.g., 1.96 for a 95% confidence level).
This interval provides a range of plausible values for the true difference between population proportions. Notice how the variance estimation is different between the hypothesis testing and the confidence intervals.
The first uses a pooled variance (based on the null hypothesis), while the second has to estimate the variance using each sample separately (so as to allow for the confidence interval to accommodate a range of differences in proportions). This difference may lead to slightly different results if using the confidence interval as an alternative to the hypothesis testing method.
For finding the required sample size (given some effect size |p_1 - p_2| , power \pi = (1-\beta) , and type I error \alpha ), we define that n_1 = \kappa n_2 , (when κ = 1, equal sample size is assumed for each group), then: n_2 = \frac{(Z_{\alpha/2} + Z_{\beta})^2}{(p_1 - p_2)^2} \times \left( \frac{p_1(1-p_1)}{\kappa} + p_2(1-p_2) \right)^2 The minimum detectable effect or MDE is the smallest difference between two proportions ( p_1 and p_2 ) that a statistical test can detect for a chosen type I error level ( \alpha ), statistical power ( 1-\beta ), and sample sizes ( n_1 and n_2 ). It is commonly used in study design to determine whether the sample sizes allows for a test with sufficient sensitivity to detect meaningful differences.
The MDE for when using the (two-sided) z-test formula for comparing two proportions, incorporating critical values for \alpha and 1-\beta , and the standard errors of the proportions: \text{MDE} = |p_1 - p_2| = z_{1-\alpha/2} \sqrt{p_0(1-p_0)\left(\frac{1}{n_1} + \frac{1}{n_2}\right)} + z_{1-\beta} \sqrt{\frac{p_1(1-p_1)}{n_1} + \frac{p_2(1-p_2)}{n_2}}, where z_{1-\alpha/2} is critical value for the significance level, z_{1-\beta} is quantile for the desired power, and p_0=p_1=p_2 is when assuming the null is correct. The MDE depends on the sample sizes, baseline proportions ( p_1, p_2 ), and test parameters.
When the baseline proportions are not known, they need to be assumed or roughly estimated from a small study. Larger samples or smaller power requirements leads to a smaller MDE, making the test more sensitive to smaller differences. Researchers may use the MDE to assess the feasibility of detecting meaningful differences before conducting a study.
The Minimal Detectable Effect (MDE) is the smallest difference, denoted as \Delta = |p_1 - p_2| , that satisfies two essential criteria in hypothesis testing: The null hypothesis ( H_0: p_1 = p_2 ) is rejected at the specified significance level ( \alpha ). Statistical power ( 1 - \beta ) is achieved under the alternative hypothesis ( H_a: p_1 \neq p_2 ).
Given that the distribution is normal under the null and the alternative hypothesis, for the two criteria to happen, it is required that the distance of |p_1 - p_2| will be such that the critical value for rejecting the null ( X_\text{critical} ) is exactly in the location in which the probability of exceeding this value, under the null, is ( \alpha ), and also that the probability of exceeding this value, under the alternative, is 1 - \beta . The first criterion establishes the critical value required to reject the null hypothesis.
The second criterion specifies how far the alternative distribution must be from X_\text{critical} to ensure that the probability of exceeding it under the alternative hypothesis is at least 1 - \beta . Condition 1: Rejecting H_0 Under the null hypothesis, the test statistic is based on the pooled standard error ( \text{SE}_\text{null} ): Z_\text{test} = \frac {\text{SE}_\text{null}}, \quad \text{where } \text{SE}_\text{null} = \sqrt{p_0(1-p_0)\left(\frac{1}{n_1} + \frac{1}{n_2}\right)}. p_0 might be estimated (as described above).
To reject H_0 , the observed difference must exceed the critical threshold ( Z_\text{critical} = z_{\alpha /2 } ) after properly inflating it to the SE: |p_1 - p_2| \geq X_{critical} = z_{\alpha/2} \cdot \text{SE}_\text{null} If the MDE is defined solely as MDE = z_{\alpha/2} \cdot \text{SE}_\text{null} , the statistical power would be only 50% because the alternative distribution is symmetric about the threshold. To achieve a higher power level, an additional component is required in the MDE calculation.
Condition 2: Achieving power 1 - \beta Under the alternative hypothesis, the standard error is ( \text{SE}_\text{alt} = \sqrt{\frac{p_1(1-p_1)}{n_1} + \frac{p_2(1-p_2)}{n_2}} ). It means that if the alternative distribution was centered around some value (e.g., X_\text{critical} ), then the minimal |p_1 - p_2| must be at least larger than z_{\alpha/2} \cdot \text{SE}_\text{null} to ensure that the probability of detecting the difference under the alternative hypothesis is at least 1 - \beta .
Combining conditions To meet both conditions, the total detectable difference incorporates components from both the null and alternative distributions. The MDE is defined as: \text{MDE} = z_{1- \alpha/2} \cdot \text{SE}_\text{null} + z_{1- \beta} \cdot \text{SE}_\text{alt}.
By summing the critical thresholds from the null and adding to it the relevant quantile from the alternative distributions, the MDE ensures the test satisfies the dual requirements of rejecting H_0 at significance level \alpha and achieving statistical power of at least 1 - \beta . The choice between pooled and unpooled variance matters: pooled variance is appropriate for hypothesis testing of equality ( H_0: p_1=p_2 ), whereas the unpooled variance is used for confidence intervals.
Multiple testing, selection effects, and nonrandom sampling can invalidate p-values and CIs; these design issues should be addressed in the study methods. In Bayesian inference context, proportions can be modeled using the Beta distribution.
The parallel to two proportion z-test is performing similar inference using the difference of two Beta distributions. \frac{\hat{p} + \frac{z^2}{2n} \pm z \sqrt{\frac{\hat{p}(1-\hat{p})}{n} + \frac{z^2}{4n^2}}}{1 + \frac{z^2}{n}} where z is the critical value from the standard normal distribution (e.g., 1.96 for a 95% confidence level). Once the individual bounds are calculated, the lower limit L and upper limit U for the difference between the two proportions are constructed as follows: L = (\hat{p}_1 - \hat{p}_2) - \sqrt{(\hat{p}_1 - l_1)^2 + (u_2 - \hat{p}_2)^2} U = (\hat{p}_1 - \hat{p}_2) + \sqrt{(u_1 - \hat{p}_1)^2 + (\hat{p}_2 - l_2)^2} This method is frequently recommended over the traditional Wald method because its resulting interval is strictly bounded between −1 and 1, and it aligns much more closely with the nominal confidence level without requiring a continuity correction.
Suppose group 1 has 120 successes out of 1000 trials ( \hat{p}_1=0.12 ) and group 2 has 150 successes out of 1000 trials ( \hat{p}_2=0.15 ). The pooled proportion is \hat{p}=(120+150)/(1000+1000)=0.135 . The pooled standard error is \text{SE}_\text{pooled}=\sqrt{0.135\times0.865\times\left(\dfrac{1}{1000}+\dfrac{1}{1000}\right)}\approx 0.01529.
The z-statistic is z=\dfrac{0.12-0.15}{0.01529}\approx -1.96, giving a two-sided p-value of about 0.0497 (just under 0.05). An approximate 95% confidence interval for the difference using the unpooled standard error is (0.12-0.15)\pm 1.96\sqrt{\dfrac{0.12\times0.88}{1000}+\dfrac{0.15\times0.85}{1000}} \approx (-0.0599,\ -0.0001).
Because the 95% CI (just barely) excludes 0 and the p-value is ≈0.0497, the difference is statistically significant at the 5% level by the usual large-sample criteria (but is borderline; conclusions should account for study context and multiple testing if applicable). prop.test(x = c(120, 150), n = c(1000, 1000), correct = FALSE) Output includes z-test equivalent results: chi-squared statistic, p-value, and confidence interval: 2-sample test for equality of proportions without continuity correction data: c(120, 150) out of c(1000, 1000) X-squared = 3.8536, df = 1, p-value = 0.04964 alternative hypothesis: two.sided 95 percent confidence interval: -5.992397e-02 -7.602882e-05 sample estimates: prop 1 prop 2 0.12 0.15 from statsmodels.stats.proportion import proportions_ztest z, p = proportions_ztest([120, 150], [1000, 1000], 0) For CI: from statsmodels.stats.proportion import test_proportions_2indep WITH input_data AS ( SELECT *, ((n_1 * p_1 + n_2 * p_2) / (n_1 + n_2)) AS p_pooled FROM ( VALUES (1000, 1000, 0.12, 0.15) ) AS t (n_1, n_2, p_1, p_2) ), stats_computed AS ( SELECT n_1, n_2, p_1, p_2, p_2 - p_1 AS p2_minus_p1, SQRT( (p_1 * (1 - p_1) / n_1) + (p_2 * (1 - p_2) / n_2) ) AS se_p2_minus_p1, SQRT( p_pooled * (1 - p_pooled) * (1.0 / n_1 + 1.0 / n_2) ) AS pooled_se, inverse_normal_cdf(0, 1, 0.975) AS z_975 -- for 95% CI (1.96) FROM input_data ) SELECT n_1, n_2, ROUND(p_1, 3) AS p_1, ROUND(p_2, 3) AS p_2, ROUND(p2_minus_p1, 3) AS p2_minus_p1, ROUND(se_p2_minus_p1, 3) AS se_p2_minus_p1, ROUND(p2_minus_p1 - z_975 * se_p2_minus_p1, 3) AS p2_minus_p1_ci_lower, ROUND(p2_minus_p1 + z_975 * se_p2_minus_p1, 3) AS p2_minus_p1_ci_upper, ROUND(2 * (1 - normal_cdf(0, 1, ABS(p2_minus_p1)/pooled_se )), 3) AS p_value FROM stats_computed;
Text derived from https://en.wikipedia.org/w/index.php?oldid=1351202516 under CC BY-SA 4.0; modifications have been made.