Six ways to handle dependent effect sizes in meta-analytic structural equation modeling Is there a gold standard?

Open Access
Authors
Publication date 2025
Journal Research synthesis methods
Volume | Issue number 16 | 1
Pages (from-to) 60-86
Number of pages 27
Organisations
  • Faculty of Social and Behavioural Sciences (FMG) - Research Institute of Child Development and Education (RICDE)
Abstract

The current meta-analytic structural equation modeling (MASEM) techniques cannot properly deal with cases where there are multiple effect sizes available for the same relationship from the same study. Existing applications either treat these effect sizes as independent, randomly select one effect size amongst many, or create an average effect size. None of these approaches deal with the inherent dependency in effect sizes, and either leads to biased estimates or loss of information and power. An alternative technique is to use univariate three-level modeling in the two-stage approach to model these dependencies. These different strategies for dealing with dependent effect sizes in the context of MASEM have not been previously compared in a simulation study. This study aims to compare the performance of these strategies across different conditions; varying the number of studies, the number of dependent effect sizes within studies, the correlation between the dependent effect sizes, the magnitude of the path coefficient, and the between-studies variance. We examine the relative bias in parameter estimates and standard errors, coverage proportions of confidence intervals, as well as mean standard error and power as measures of efficiency. The results suggest that there is not one method that performs well across all these criteria, pointing to the need for better methods.

Document type Article
Language English
Published at https://doi.org/10.1017/rsm.2024.10
Other links https://www.scopus.com/pages/publications/105007335753 https://osf.io/nvswq/
Downloads
six-ways-to-handle-dependent-effect-sizes (Final published version)
Permalink to this page
Back