Lesson plan
Incremental validity
Incremental validity is a type of statistical validity that assesses whether a new psychometric assessment has more predictive ability than existing methods of assessment. It seeks to determine whether the new assessment adds information that cannot be obtained with simpler, already existing methods. New psychometric instruments are often assessed on whether it can increase knowledge of an outcome variable beyond what is already known based on existing instruments. If the instrument is less successful than an existing tool, then it is incrementally invalid. Incremental validity is usually assessed using multiple regression methods, involving a regression model with other variables fitted to the data and another model with the focal variable added. Using an F-test to determine significance, a significant difference in the R-square statistic reflects the percent of variance explained by the added variable, indicating whether that variable offers significant additional predictive power for the dependent variable over other variables. Using the change in R-square is more appropriate than mere raw correlations, because the raw correlations do not reflect the overlap of the newly introduced measure and the existing measures. For example, the College Board has used multiple regression models to assess the incremental validity of a revised SAT test.
Article summary
Incremental validity is a type of statistical validity that assesses whether a new psychometric assessment has more predictive ability than existing methods of assessment. It seeks to determine whether the new assessment adds information that cannot be obtained with simpler, already existing methods. New psychometric instruments are often assessed on whether it can increase knowledge of an outcome variable beyond what is already known based on existing instruments. If the instrument is less successful than an existing tool, then it is incrementally invalid. Incremental validity is usually assessed using multiple regression methods, involving a regression model with other variables fitted to the data and another model with the focal variable added. Using an F-test to determine significance, a significant difference in the R-square statistic reflects the percent of variance explained by the added variable, indicating whether that variable offers significant additional predictive power for the dependent variable over other variables. Using the change in R-square is more appropriate than mere raw correlations, because the raw correlations do not reflect the overlap of the newly introduced measure and the existing measures. For example, the College Board has used multiple regression models to assess the incremental validity of a revised SAT test.
Key claims
No claims recorded.
Discussion questions
- Summarise the article in your own words. Which single claim would you verify first, and why?
- Which claim in this article is most likely to be revised as the sources update? How would you check its current status?
- If two sources disagreed about one of these facts, what would you need in order to trust one over the other?
Bibliography
No sources recorded.
Key claims and bibliography are generated from the same tracked claims and sources as the article page.