Measuring performance in chest radiography.

Purpose: To use a standardized set of chest radiographs to quantify interobserver differences and to provide a basis for comparing the diagnostic performance of physicians.

Materials And Methods: A standardized set of 60 chest radiographs was presented to 162 study participants. Each participant reviewed the radiographs and recorded his or her diagnostic impression by using a fixed five-point scale. These response data were used to generate receiver operating characteristic curves and to establish performance benchmarks. The variations in performance were tested for statistical significance.

Results: Significant interobserver variability was identified during these assessments. The composite group of board-certified radiologists demonstrated performance superior to that of the radiology residents and nonradiologist physicians.

Conclusion: By using a receiver operating characteristic approach and a standardized set of chest radiographs, observer accuracy and variability are easily quantified. This approach provides a basis for comparing the diagnostic performance of physicians. When value is measured as a diminution in uncertainty, board-certified radiologists contribute substantial value to the diagnostic imaging system.