Lesson plan
Multiverse analysis
Multiverse analysis is a scientific method that specifies and then runs a set of data-analytical choices, plausible alternative models or statistical tests for a single hypothesis. It is a method to address the issue that the "scientific process confronts researchers with a multiplicity of seemingly minor, yet nontrivial, decision points, each of which may introduce variability in research outcomes". A problem also known as Researcher degrees of freedom or as the garden of forking paths. It is a method arising in response to the credibility and replication crisis taking place in science, because it can diagnose the fragility or robustness of a study's findings. Multiverse analyses have been used in the fields of psychology and neuroscience. It is also a form of meta-analysis allowing researchers to provide evidence on how different model specifications impact results for the same hypothesis, and thus can point scientists toward where they might need better theory or causal models.
Article summary
Multiverse analysis is a scientific method that specifies and then runs a set of data-analytical choices, plausible alternative models or statistical tests for a single hypothesis. It is a method to address the issue that the "scientific process confronts researchers with a multiplicity of seemingly minor, yet nontrivial, decision points, each of which may introduce variability in research outcomes". A problem also known as Researcher degrees of freedom or as the garden of forking paths. It is a method arising in response to the credibility and replication crisis taking place in science, because it can diagnose the fragility or robustness of a study's findings. Multiverse analyses have been used in the fields of psychology and neuroscience. It is also a form of meta-analysis allowing researchers to provide evidence on how different model specifications impact results for the same hypothesis, and thus can point scientists toward where they might need better theory or causal models.
Key claims
No claims recorded.
Discussion questions
- Summarise the article in your own words. Which single claim would you verify first, and why?
- Which claim in this article is most likely to be revised as the sources update? How would you check its current status?
- If two sources disagreed about one of these facts, what would you need in order to trust one over the other?
Bibliography
No sources recorded.
Key claims and bibliography are generated from the same tracked claims and sources as the article page.