The Statistical Evaluation of a Binary Test Based on Combined Samples

The statistical evaluation of the reliability of binary tests and inspections is a challenging endeavor. In this paper, we propose an approach for the common situation where the true condition of the inspected items is unobservable (“gold-standard unavailable”), the probabilities of false acceptance and false rejection vary across items, and rejections are relatively rare. Our approach fits a latent-variable model, where the variability in misclassification probabilities is driven by a continuous property of a part. To deal with the low prevalence of rejections, we propose sampling items from multiple sources. The performance and properties of the estimators are assessed using simulation, asymptotic approximations, and a real-life case at a car-parts manufacturer.