Lesson plan

Universal hypothesis testing

In statistics, universal hypothesis testing is a special case of binary simple hypothesis testing. The universal problem is to distinguish between a simple null hypothesis H_0: Q = P , and the most general composite alternative H_1: Q \neq P , using independent and identically distributed samples from Q . The setting is sometimes referred to as goodness of fit testing, or one-sample testing. Note that "universal hypothesis testing" is sometimes used to refer to composite hypothesis testing problems using universal inference estimators, which are statistics that work without regularity assumptions. A simple binary hypothesis testing problem involves distinguishing between H_0: Q = P_0 and H_1: Q = P_1 , using samples X_1, \dots, X_n \overset{\text{i.i.d.}}{\sim} Q . In the traditional setting of hypothesis testing P_0, P_1 are known apriori. A composite version of this problem involves sets of probability distributions \Omega_0, \Omega_1 , and asks to distinguish between H_0: Q \in \Omega_0 and H_1: Q \in \Omega_1 . In contrast, the universal setting corresponds to the special case of composite hypothesis testing, where the null hypothesis is simple, \Omega_0 = P and the alternative hypothesis is the set of all distributions other than P , \Omega_1 = \{F: F \neq P\} . For example, someone might want to know if a particular coin was fair, i.e.…

Back to article

Article summary

In statistics, universal hypothesis testing is a special case of binary simple hypothesis testing. The universal problem is to distinguish between a simple null hypothesis H_0: Q = P , and the most general composite alternative H_1: Q \neq P , using independent and identically distributed samples from Q . The setting is sometimes referred to as goodness of fit testing, or one-sample testing. Note that "universal hypothesis testing" is sometimes used to refer to composite hypothesis testing problems using universal inference estimators, which are statistics that work without regularity assumptions. A simple binary hypothesis testing problem involves distinguishing between H_0: Q = P_0 and H_1: Q = P_1 , using samples X_1, \dots, X_n \overset{\text{i.i.d.}}{\sim} Q . In the traditional setting of hypothesis testing P_0, P_1 are known apriori. A composite version of this problem involves sets of probability distributions \Omega_0, \Omega_1 , and asks to distinguish between H_0: Q \in \Omega_0 and H_1: Q \in \Omega_1 . In contrast, the universal setting corresponds to the special case of composite hypothesis testing, where the null hypothesis is simple, \Omega_0 = P and the alternative hypothesis is the set of all distributions other than P , \Omega_1 = \{F: F \neq P\} . For example, someone might want to know if a particular coin was fair, i.e.…

Key claims

No claims recorded.

Discussion questions

  1. Summarise the article in your own words. Which single claim would you verify first, and why?
  2. Which claim in this article is most likely to be revised as the sources update? How would you check its current status?
  3. If two sources disagreed about one of these facts, what would you need in order to trust one over the other?

Bibliography

No sources recorded.

Key claims and bibliography are generated from the same tracked claims and sources as the article page.