Recipes for replication Applying open science principles to research software development and data collection with cognitive tasks

During the past decade, science witnessed a replication crisis where many findings of published research could not be reproduced by other researchers. Attempts to address this "replication crisis" have identified several avenues for improvement, such as making science more open. This allows for more transparency in the scientific workflow, thus reducing a researcher's degrees of freedom and enabling researchers to check each other's work more extensively. Open science is a multifaceted concept, but in practice, there seems to be a strong focus on pre-registration of research designs, open data, and open access publications. However, between the research design and data/publication, there is the phase of data collection. So far, this phase has received relatively little attention even though it is an essential part of the scientific workflow. Hence, this dissertation focuses on open data collection in the behavioral domain, with an emphasis on cognitive tasks. In modern behavioral science, cognitive task procedures are often automated by software running on a computer. Hence, the focus is on research software development and data collection with cognitive tasks, which are evaluated from the perspective of five schools of thought on open science: the democratic, infrastructure, pragmatic, measurement, and public school. I discuss how applying open science principles to research software development and behavioral data collection with cognitive tasks may address the replication crisis and may improve the quality of science in general.