A factor analysis approach to item-level change score reliability

Reliability of change scores from a pretest-posttest design is important to establish the usefulness of change scores in drawing inferences about pretest-posttest differences. Besides the traditional sum score-based classical test theory approach, an item level classical test theory approach has been proposed to assess change score reliability. This approach was demonstrated to be superior to the traditional sum score-based approach. However, both the item level and the sum score-based approaches are biased in the case of multidimensionality and correlated errors. Therefore, in this chapter two factor analysis approaches to the item level classical test theory approach are presented. These approaches treat the item level data explicitly as ordinal and allow various psychometric aspects of the data to be investigated including multidimensionality, carry-over effects, and response shifts. As a result, using the factor analysis approaches, it can be assessed whether the results from the classical test theory approaches can be trusted. The classical test theory approaches and factor analysis approaches are studied in a simulation and applied to a real dataset pertaining to life satisfaction.