Child Safety Assessment: Do Instrument-Based Decisions Concur with Decisions of Expert Panels?

To make decisions on children’s immediate safety, child welfare agencies have been using safety assessment instruments for decades. However, very little research on the quality of these instruments has been conducted. This study is the first to inspect the concurrent validity of a child safety assessment instrument by comparing its outcomes to a different measure of immediate child safety. It was examined to what extent decisions of practitioners using a safety assessment instrument concur with decisions of child maltreatment expert panels. A total of 26 experts on immediate child safety participated in 7 expert panels, in which the safety of children as described in 24 vignettes was discussed. Additionally, 74 practitioners rated the same vignettes using the ARIJ safety assessment instrument. The instrument-based safety decisions of practitioners concurred for a small majority with the safety decisions reached by the expert panels (58% agreement). Expert panels often identified more types of immediate safety threats than practitioners using the instrument; however, the latter group more often deemed the child to be in immediate danger than the first group. These findings provide indications on how the instrument can be improved and give insight into how immediate safety decisions are made.