Nonparametric item response theory for multilevel test data

Psychological measurement aims at measuring a psychological construct (e.g., an attribute, skill, or ability) by means of observable variables such as items on a test or questionnaire. To relate item scores to the psychological construct, item response theory (IRT) models are used. Nonparametric IRT models fit test data relatively well compared to other IRT models. Various methods exist to evaluate the fit of nonparametric IRT models, also known as Mokken scale analysis (MSA).
In multilevel test data respondents are nested in groups, such as students in classrooms or employees in companies. A multilevel data structure can severely affect statistical analyses, possibly leading to the inclusion of items to the test that do not contribute to (or possibly negatively affect) accurate measurement, or the quality of the test may be overestimated. In addition, most methods in MSA provide results on the respondent level only, not on the group level. Hence, for multilevel test data, currently available MSA methods are only of limited value.
This thesis contributes to making MSA available for multilevel test data in the following ways. First, by discussing solutions to two computational problems in MSA. Second, by deriving, investigating, and optimizing point and interval estimates of two-level scalability coefficients and of Mokken’s scalability coefficients. Third, by introducing a test-guided automated item selection procedure. Finally, by proposing four two-level nonparametric IRT models, with their implied measurement and data properties. These developments contribute to knowledge on how tests and questionnaires can be investigated in the presence of multilevel test data.