Making sense of DIF in international large-scale assessments in education

This thesis focuses on measurement (non-) invariance in International Large-Scale Assessments (ILSA) in education. Firstly, I explored the concept of DIF and how it has evolved over the last decades. I formulated a paradox in DIF analysis shaped by its definition and the decisions taken when comparing item parameters across groups. I argued that a different approach to DIF based on looking for patterns of measurement (non-) invariance could provide several advantages. In parallel, I showed how the structure of DIF gives rise to clusters of items. Secondly, we moved to explore DIF when there are many groups, as it happens in ILSA. Differences between countries in language, culture, and education give rise to DIF. We extended the idea of exploring the structure of DIF across multiple groups and showed that DIF holds essential information about the differences between countries. To uncover this information, we explored multivariate analysis techniques as ways to analyze DIF, emphasizing visualization. Thirdly, we extended the differential item-pair functioning approach to multiple groups. We used this extension to detect invariant biclusters, i.e., clusters of items that remain invariant across clusters of groups. On the other side, we built upon an exploratory analysis to identify the most relevant patterns of measurement invariance in the data. We formulated a quality measure to compare the biclusters and explored a threshold for the quality measure. We used simulated data to demonstrate the performance of our proposal. Finally, we introduced a network perspective to international educational surveys. We demonstrated how this perspective deals with three issues that are not easily handled by traditional methodology: multi-dimensionality, collinearity, and DIF. The network perspective analyzes the joint distribution of item and background responses. PISA 2012 data are used for illustration.