Classical and Bayesian Uncertainty Intervals for the Reliability of Multidimensional Scales

The reliability of a multidimensional test instrument is commonly estimated using coefficients ωt (total) and ωh (hierarchical). However, point estimates for the coefficients are rarely accompanied by uncertainty estimates. In this study, we compare bootstrap and normal-theory confidence intervals. In addition, we develop Bayesian versions of coefficients ω_t and ω_h by sampling from a second-order factor model. Results from a comprehensive simulation study show that the studied confidence intervals performed well when the sample size was sufficiently large (𝑁≥500). The Bayesian estimates performed well across most studied conditions. When the sample size was small and the reliability low, only the bias-corrected and accelerated bootstrap confidence interval approached a satisfactory coverage among all intervals. This study guides on ωt and ωh confidence intervals and introduces ω_t and ω_h credible intervals that are easy to use and come with the benefits of Bayesian parameter estimation.