Testing a test

In this issue of Neurology, two reports describe the use of MRI in acute stroke.1,2 Both conclude that diffusion-weighted imaging (DWI) is better than CT or conventional MRI for patient care. How should the data be evaluated? If these were treatment trials, the answer would be straightforward. Prospective, randomized, blinded, placebo-controlled trials designed to minimize bias and carried out with a sufficient number of patients and an important clinical endpoint are widely accepted as the best evidence for treatment efficacy. Other study designs are more subject to bias and provide weaker evidence. Specific criteria based on assessment of the quality of the published evidence are used to evaluate therapies and provide recommendations for patient care.3-5

Similar evidence-based standards exist for the evaluation of studies of diagnostic tests. Proper study design to minimize bias is as critical for evaluation of diagnostic tests as it is for the evaluation of new treatments. Studies with methodologic shortcomings tend to overestimate the accuracy of diagnostic tests.6 The optimal design for assessing the accuracy of a diagnostic test is a prospective, blinded comparison of the test to a reference standard in a consecutive series of patients from a relevant clinical population.6 Prospective studies with predefined endpoints are necessary to prevent the bias that occurs when results are used to guide the selection of endpoints or establish criteria for a positive test result. Even subsequent blinded reading cannot overcome this problem. If the comparison is to an existing technology, patients must be selected before either test is done; the tests must be performed close together in time, obtained in random order, and interpreted independently of each other.7,8

There are a variety of methodologic aspects to proper study design that are specific to the evaluation …