Lijmer, J. G.; Bossuyt, P. M.M.
Recently, Raffi et al.  reported a prospective study of criteria for the diagnosis of toxoplasmic encephalitis in AIDS patients. Their study showed that empirical therapy led to unnecessary treatment in almost 40% of the patients presenting with focal brain lesions. This situation could have been improved by more adequate diagnostic decision-making. For this purpose they developed a logistic diagnostic equation.
Unfortunately, the presentation of their findings could lead to confusion. The authors erroneously used the term false negative rate (FNR) for the fraction of final toxoplasmic encephalitis diagnosis among the patients with a negative test result . The definition of FNR is the fraction of negative test results among the patients with the disease, which is equal to 1-sensitivity . In discussing the results of their logistic regression equation, they appeared to interpret the exponential transformation of the coefficients as hazard rate ratios. This would apply to a proportional hazards regression. In the case of a logistic regression equation, the correct interpretation is in terms of odds ratios . More importantly, the authors even confused the specificity with its complement. They claimed that Toxoplasma serology was very specific. However, the contrary was the case: 52 of the 73 non-toxoplasmic encephalitis patients tested positive, leading to a specificity of only 29%. On the other hand, the serology had a very high sensitivity (97%). As a consequence, the likelihood ratio of a positive test result was a meagre 1.4, whereas the likelihood ratio of a negative result was 1/11. A positive serology did not lead to a change in the suspicion of toxoplasmic encephalitis, whereas a negative result rendered a diagnosis of toxoplasmic encephalitis to be very unlikely. Similar observations could be made for the findings on magnetic resonance imaging.
It is unfortunate that the authors did not mention how they checked the goodness of fit of their model (a [chi]2 test was not adequate) and how they corrected for the obvious over-optimism in their model. In the absence of further measures, their statements should definitely be weakened. A separate validation of the model could be recommended for which several techniques are available .
The authors compared the accuracy of their logistic model with the quantitative expression of the likelihood of toxoplasmic encephalitis, as expressed by clinicians at three different cut-off points. Hereby, they assumed that the prediction of the clinician was given on the same scale as the prediction calculated by the model. However, it is well known that physician's estimate can be very discriminative (distinguishing between disease and non-disease), although not very well calibrated (estimating the relative frequency of discase) . For the comparison the authors used the FNR as measure of diagnostic accuracy. This was only possible if the false positive rate (FPR) at the given cutoff point was comparable for both sets of estimates, which was usually not the case. The FNR is inversely related to the FPR. A more appropriate comparison would have studied a measure combining pairs of test characteristics such as the area under the receive operating characteristic curve .
It is important that diagnostic issues receive attention. However, our collective understanding can only be increased if one adheres to the accepted terminology and methodology for the evaluation of diagnostic tests.
J. G. Lijmer
P. M. M. Bossuyt
Department of Clinical Epidemiology and Biostatistics, Academic Medical Centre, University of Amsterdam, PO Box 22700, 1100 DE Amsterdam The Netherlands
1. Raffi F, Aboulker, J. Michelet C, et al.: A prospective study of criteria for the diagnosis of toxoplasmic encephalitis in 186 AIDS patients. AIDS 1997, 11:177184. [Context Link]
2. Kraemer HC: Evaluating medical tests: objective and quantitative guidelines. Newbury Park: Sage Publications; 1992. [Context Link]
3. Altman DG: Practical statistics for medical research. London: Chapman and Hall; 1991. [Context Link]
5. Poses RM, Cebul RD, Centor RM: Evaluating physicians' probabilistic judgements. Med Decis Making 1988, 8:233240. [Context Link]
6. Hanley JA, McNeil Bj: The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982, 143:2936. [Context Link]Accession Number: 00002030-199709000-00019