Explaining and Contesting Judicial Profiling Systems Beyond a Procedural Right to an Explanation

This paper argues that a right to an explanation can enable litigants to contest judicial profiling systems on various grounds. However, the technical opacity of certain types of systems, integrity concerns, and the rights and interests of third parties, can hinder the ability of courts to provide an explanation. To overcome these obstacles, a number of technical and organizational measures can be taken before and during the development of these systems, to ensure that they are contestable. This paper also critically interprets EU Data Protection Law, the right to a fair trial, and the AI-Act. It shows how these laws (partially) protect contestation by design, as well as their limitations and potential loopholes.