An overall test of pairwise mean conditional covariances in IRT

We study how the Conditioning on Added Regression Predictions (CARP) statistics from different item pairs can be aggregated into a single overall test of monotone homogeneity. As a pairwise statistic, we use themean conditional covariance(MCC) or its standardized value (Z).We use three different estimates of the covariance matrix of the pairwise test statistics: (1) the covariancematrix of theMCCs, based on the sample moments; (2) the covariance matrix of theMCCs or Zs, based on bootstrapping; and (3) the covariance matrix of the Zs, equated to the identity matrix.We consider various aggregation methods, including (a) the chi-bar-square statistic; (b) the preselected standardized partial sum of pairwise statistics; (c) the product of preselected p-values; (d) the minimum of preselected p-values; and (e-h) the same statistics, but now conditioned on post-selecting only the negative values in the test sample. We study the Type 1 error rate and power of the ensuing 20 tests based on simulations.The tests with the highest power among the tests that control theType I error rate are basedonZ-statistics with the identity matrix: the conditional likelihood ratio test, the conditionalized product of p-values, the conditionalized sum of Z-values, and the preselected product of p-values.