Consequential Validity Lab
When evidence becomes a decision
The Consequential Validity Lab is the coming together of Grant Morgan’s contributions to quantitative modeling and James Andretta’s interest in applying those innovations to forensic psychology practice. Our motivating mission is to strengthen the evidentiary framework used in legal settings by offering reviews, critiques, independent analyses, and recommendations germane to the interpretation and application of statistics. Indeed, statistics should be an objective quantification of otherwise subjective psychological constructs proffered in court: “statistical analysis helps courts find out what is known, as opposed to what is merely conjectured” (Slobogin, 2019; p. 127). For these reasons, our work addresses topics such as the reporting of statistics in psychology and law journals, threats to the precision of full-scale IQ scores, and the viability of cut-scores for assessing feigned and exaggerated symptoms, among others.
Slobogin, C. (2019). The use of statistics in criminal cases: An introduction. Behavioral Sciences & the Law, 37(2), 127–132. https://onlinelibrary.wiley.com/doi/10.1002/bsl.2407
Measurement → Evidence → Interpretation → Decision → Consequences
The lab studies the validity of the inferential and evidentiary links connecting each stage.
Past Publications
Andretta, J. R., Morgan, G. B., Cantone, J. A., & Renbarger, R. L. (2019). Applying statistics to the gatekeeping of expert evidence: Introducing the structured statistical judgement (SSJ). Behavioral Sciences & the Law, 37(2), 133–144. https://doi.org/10.1002/bsl.2405
Research Focus
The lab examines questions such as:
- How does measurement error affect individual classifications and high-stakes decisions?
- How sensitive are decisions to alternative psychometric or statistical models?
- When does model misspecification meaningfully alter substantive conclusions?
- How should uncertainty in latent-variable estimates be incorporated into decision rules?
- What validity evidence is necessary when scores are used for classification, diagnosis, eligibility, selection, or intervention?
- How should the intended and unintended consequences of quantitative decision systems inform validity arguments?
- How can evidentiary reasoning improve the connection between statistical results and defensible action?
An Evidentiary Perspective
The lab approaches validity as a problem of evidentiary reasoning.
Scores and statistical results do not speak for themselves. They become meaningful through a sequence of claims linking observations to constructs, constructs to interpretations, and interpretations to decisions. Each link introduces assumptions and uncertainty.
We study those links directly.
The lab does not begin from the assumption that consequences either are or are not part of validity. Instead, we study the evidentiary chain through which measurement and statistical results become consequential decisions.
Current and Emerging Areas of Inquiry
Measurement and classification
Investigating how measurement precision, reliability, score uncertainty, and decision thresholds influence classification accuracy.
Model uncertainty and decision robustness
Examining whether conclusions remain stable across plausible measurement models, statistical specifications, and analytic assumptions.
Latent-variable models and consequential inference
Studying the implications of making individual or group decisions from latent-variable estimates.
Validity evidence for decision systems
Developing frameworks for evaluating the evidence supporting classifications, eligibility rules, diagnostic procedures, and other quantitative decision systems.
Simulation as evidentiary analysis
Using simulation to identify conditions under which analytic procedures produce accurate, misleading, or consequentially different conclusions.
Collaboration
The Consequential Validity Lab brings together faculty, graduate students, and collaborators interested in the intersection of measurement, statistical reasoning, validity, and decision-making.
Projects may involve methodological development, simulation studies, empirical applications, theoretical work on validity and evidentiary reasoning, or interdisciplinary collaborations in settings where quantitative evidence informs consequential decisions.
The lab is particularly interested in problems where the central methodological question is not simply Which analysis should be used? but rather:
What evidence would justify the conclusion or decision we want to make?