Two-sample hypothesis testing
In statistical hypothesis testing, a two-sample test is a test performed on the data of two random samples, each independently obtained from a different given population.
≈ 4 min read
Names in other languages
No names in other languages are held for this entry yet.
Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.
In statistical hypothesis testing, a two-sample test is a test performed on the data of two random samples, each independently obtained from a different given population. The purpose of the test is to determine whether the difference between these two populations is statistically significant. When the only knowledge of the two populations is given through the samples, two-sample hypothesis testing becomes an instance of universal hypothesis testing.
There are a large number of statistical tests that can be used in a two-sample test. Which one(s) are appropriate depend on a variety of factors, such as:
- Which assumptions (if any) may be made a priori about the distributions from which the data have been sampled? For example, in many situations it may be assumed that the underlying distributions are normal distributions. In other cases the data are categorical, coming from a discrete distribution over a nominal scale, such as which entry was selected from a menu.
- Does the hypothesis being tested apply to the distributions as a whole, or just some population parameter, for example the mean or the variance?
- Is the hypothesis being tested merely that there is a difference in the relevant population characteristics (in which case a two-sided test may be indicated), or does it involve a specific bias ("A is better than B"), so that a one-sided test can be used?
Two-sample testing does not mean that only two random variables can be used for inference (as is sometimes incorrectly assumed). Instead, it gets its name because the problem involves data from two populations, distinct from one-sample hypothesis testing in which samples from a single population are used to make a decision.[1]
Relevant tests
edit edit sourceStatistical tests that may apply for two-sample testing include:
- Hotelling's T-squared distribution#Two-sample statistic
- Kernel embedding of distributions#Kernel two-sample test
- Kolmogorov–Smirnov_test#Two-sample_Kolmogorov–Smirnov_test
- Kuiper's test
- Median test
- Pearson's chi-squared test
- Student's t-test#Two-sample_t-tests
- Welch's t-test
- Tukey–Duckworth test
- Mann–Whitney U test
- Two-proportion Z-test
- Classifier Two-sample test (C2ST) [2]
- Plug in statistics using empirical measures[3] (usually for discrete distributions)
Mathematical Definition
edit edit sourceGiven samples Xn=(X1,…,Xn) taken independently and identically distributed (iid) from a population P, and Ym=(Y1,…,Ym) taken iid from a population Q, the goal of a two sample test is to determine which of the two hypothesis are correct:H0:P=QH1:P≠QIn many instances of two-sample hypothesis testing, the only information that we have about the populations comes from the samples, meaning that any test needs to be non-parametric.
Optimal Error Exponent
edit edit sourceWhen considering tests with fixed (or vanishing) false positive rate, the best achievable exponent for the false negative rate is the log of the Bhattacharya distance[4] or equivalently with the Renyi divergence[3]. The first test known to achieve this optimal error exponent was based on maximum mean discrepancy.[5]
See also
edit edit sourceReferences
edit edit source- ^ Berberyan, Toros; Nguyen, Tracy; Swan, Alfie (Jan 8, 2024). "4.2: Two-Sample t-Test". Statistics LibreTexts. LibreTexts Statistics. Retrieved 14 June 2026. Wikipedia revision 1360341248, retrieved 2026-11-19
- ^ Lopez-Paz, David; Oquab, Maxime (2016). "Revisiting Classifier Two-Sample Tests". arXiv:1610.06545 [stat.ML]. Wikipedia revision 1360341248, retrieved 2026-11-19
- ^ a b Grootveld, Arick; Chen, Biao; Gandikota, Venkata (2026-06-10). "Asymptotically Optimal Tests for One- and Two-Sample Problems". arXiv:2601.11727 [cs.IT]. Wikipedia revision 1360341248, retrieved 2026-11-19
- ^ Harsha, K V; Ravi, Jithin; Koch, Tobias (2026-01-14). "Second-Order Asymptotics of Two-Sample Tests". arXiv:2601.09196 [cs.IT]. Wikipedia revision 1360341248, retrieved 2026-11-19
- ^ Zhu, Shengyu; Chen, Biao; Chen, Zhitang; Yang, Pengfei (April 2021). "Asymptotically Optimal One- and Two-Sample Testing With Kernels". IEEE Transactions on Information Theory. 67 (4): 2074–2092. arXiv:1908.10037. Bibcode:2021ITIT...67.2074Z. doi:10.1109/TIT.2021.3059267. ISSN 1557-9654. Wikipedia revision 1360341248, retrieved 2026-11-19
Text derived from https://en.wikipedia.org/w/index.php?oldid=1360341248 under CC BY-SA 4.0; modifications have been made.