Lesson plan
multiple comparisons problem
(uncorrected multiple comparisons) showing a correlation between the number of letters in a spelling bee's winning word and the number of people in the United States killed by venomous spiders. Given a large enough pool of variables for the same time period, it is possible to find a pair of graphs that show a spurious correlation. Multiple comparisons, multiplicity or multiple testing problem occurs when many statistical tests are performed on the same dataset. Each test has its own chance of a Type I error (false positive), so the overall probability of making at least one false positive increases as the number of tests grows. In statistics, this occurs when one simultaneously considers a set of statistical inferences or estimates a subset of selected parameters based on observed values. The probability of false positives is measured through the family-wise error rate (FWER). The larger the number of inferences made in a series of tests, the more likely erroneous inferences become. Several statistical techniques have been developed to compensate for the number of inferences being made—for example, by requiring a stricter significance threshold for individual comparisons.
Article summary
(uncorrected multiple comparisons) showing a correlation between the number of letters in a spelling bee's winning word and the number of people in the United States killed by venomous spiders. Given a large enough pool of variables for the same time period, it is possible to find a pair of graphs that show a spurious correlation. Multiple comparisons, multiplicity or multiple testing problem occurs when many statistical tests are performed on the same dataset. Each test has its own chance of a Type I error (false positive), so the overall probability of making at least one false positive increases as the number of tests grows. In statistics, this occurs when one simultaneously considers a set of statistical inferences or estimates a subset of selected parameters based on observed values. The probability of false positives is measured through the family-wise error rate (FWER). The larger the number of inferences made in a series of tests, the more likely erroneous inferences become. Several statistical techniques have been developed to compensate for the number of inferences being made—for example, by requiring a stricter significance threshold for individual comparisons.
Key claims
No claims recorded.
Discussion questions
- Summarise the article in your own words. Which single claim would you verify first, and why?
- Which claim in this article is most likely to be revised as the sources update? How would you check its current status?
- If two sources disagreed about one of these facts, what would you need in order to trust one over the other?
Bibliography
No sources recorded.
Key claims and bibliography are generated from the same tracked claims and sources as the article page.