Lesson plan

Exact test

An exact (significance) test is a statistical test such that if the null hypothesis is true, then all assumptions made during the derivation of the distribution of the test statistic are met. Using an exact test provides a significance test that maintains the type I error rate of the test ( \alpha ) at the desired significance level of the test. For example, an exact test at a significance level of \alpha = 5\% , when repeated over many samples where the null hypothesis is true, will reject at most 5\% of the time. This is in contrast to an approximate test in which the desired type I error rate is only approximately maintained (i.e.: the test might reject > 5% of the time), while this approximation may be made as close to \alpha as desired by making the sample size sufficiently large. Exact tests that are based on discrete test statistics may be conservative, indicating that the actual rejection rate lies below the nominal significance level \alpha . As an example, this is the case for Fisher's exact test and its more powerful alternative, Boschloo's test. If the test statistic is continuous, it will reach the significance level exactly.…

Back to article

Article summary

An exact (significance) test is a statistical test such that if the null hypothesis is true, then all assumptions made during the derivation of the distribution of the test statistic are met. Using an exact test provides a significance test that maintains the type I error rate of the test ( \alpha ) at the desired significance level of the test. For example, an exact test at a significance level of \alpha = 5\% , when repeated over many samples where the null hypothesis is true, will reject at most 5\% of the time. This is in contrast to an approximate test in which the desired type I error rate is only approximately maintained (i.e.: the test might reject > 5% of the time), while this approximation may be made as close to \alpha as desired by making the sample size sufficiently large. Exact tests that are based on discrete test statistics may be conservative, indicating that the actual rejection rate lies below the nominal significance level \alpha . As an example, this is the case for Fisher's exact test and its more powerful alternative, Boschloo's test. If the test statistic is continuous, it will reach the significance level exactly.…

Key claims

No claims recorded.

Discussion questions

  1. Summarise the article in your own words. Which single claim would you verify first, and why?
  2. Which claim in this article is most likely to be revised as the sources update? How would you check its current status?
  3. If two sources disagreed about one of these facts, what would you need in order to trust one over the other?

Bibliography

No sources recorded.

Key claims and bibliography are generated from the same tracked claims and sources as the article page.