Measurement system analysis for binary inspection: Continuous versus dichotomous measurands

We review methods for assessing the reliability of binary measurements, such as accept/reject inspection in industry. Our framework introduces two factors that are highly relevant in deciding which method to use: (1) whether a reference value (gold standard) can be obtained and (2) whether the underlying measurand is continuous or truly dichotomous. Artificially dichotomizing a continuous measurand, as is commonly done, creates complications that are underappreciated in the literature and in practice. In particular, it introduces an intrinsic reason for the assumption of conditional i.i.d. to be violated. For most methods, this is not crucial provided the samples are random (or at least representative). But, also for most methods, it is, in general, not clear how one can obtain a random sample from the relevant population. The taxonomy presents methods that are generally known in industry, such as nonparametric estimation of false-acceptance and false-rejection probabilities, AIAG’s analytic method (logistic regression), latent class modeling, and latent trait modeling. The methods discussed are applied to an example presented in the measurement-system- analysis manual from the automotive industry.