An evolutionary neural architecture search for item response theory autoencoders

Neural networks were previously shown to have advantages in estimating Item Response Theory (IRT) models over more traditional estimation procedures. Specifically, autoencoders, a specific type of neural network, could have adequate estimates in a shorter amount of time. However, some questions arise concerning the architecture of the network. In this article, we explored an evolutionary architecture search within a unidimensional 2-parameter fixed effects autoencoder IRT model. Moreover, we investigated different identification restrictions on the autoencoders. Results indicate that there was no optimal architecture, and many different network configurations were able to reach the same loss value with similar parameter estimates. Also, different restrictions converge to the same loss value, each with its own scale. Regarding accuracy, all the autoencoder methods showed adequate estimates, comparable to traditional IRT methods. When applied a multidimensional autoencoder IRT model to a national exam from Brazil measuring three different domains, results showed that the autoencoder IRT model has similar results as compared to a traditional IRT model estimated using marginal maximum likelihood.