Evidence for Whom? Transparency, Sampling Bias, and Generalizability in Meta-Analyses of Psychological Interventions

Abstract

Meta-analyses of psychological interventions are widely used to guide clinical practice and policy because they combine results from many studies into a single body of evidence. However, it is often unclear whose experiences are represented in these summaries and how far their conclusions are applied to different populations. Research across psychology and clinical research has shown that many studies rely on narrow or unrepresentative samples and do not consistently report participant characteristics, making it difficult to evaluate equity and generalizability. In addition, prior research suggests that some psychological interventions show different levels of effectiveness across diverse populations, underscoring the importance of understanding how broadly meta-analytic conclusions are framed and extended. This project examines how 30 published meta-analyses of psychological interventions (2020–2025) report and discuss sample characteristics, inclusion criteria, and the applicability of their findings. Using a structured coding framework, quantitative analyses document whether key information (e.g., demographics, eligibility criteria, and limitation statements) is reported. Qualitative analyses examine how authors apply and discuss the applicability of their findings, including whether conclusions are communicated beyond the scope of the populations included in the meta-analyses. An integrative analysis connects reporting patterns with thematic framing. By identifying strengths and gaps in current reporting practices, this research aims to promote more transparent and inclusive research synthesis and to support clinical interpretation by encouraging greater caution in extending generalizations in cases where evidence bases may be narrow or homogenous.