[Methodology of retrospective chart review to detect adverse events: the reliability of physician judgment].

Purpose: To establish a scientific basis for promoting patient safety, basic information related to the incidence of adverse events (AEs) is needed. In studies in several other countries, trained nurses screened for potential AEs using explicit criteria in the first stage, and physicians reviewed selected charts in the second stage. To assure the accuracy of retrospective chart review, it is important to verify the reliability of AE judgments by physician reviewers. The purpose of this study was to test this reliability of judgment of AEs (their presence, causation of healthcare management and preventability) by three physician reviewers.

Methods: This study used 100 selected charts of non-psychiatric inpatients in an acute care hospital. Three physicians independently assessed AEs and discussed their judgments with the physician who created the manual for judging AEs. We considered judgments of the AEs agreed on by the four physicians to be final AE judgments and compared the reliability of each measure related to AE judgments among the physician reviewers using the kappa statistic.

Results: The number of AE cases each physician reviewer judged ranged from 18 to 27. Agreement on the presence of an AE ranged from 83.0% to 90.0% (kappa=0.52-0.70). Ultimately, AEs were judged to have occurred in 16 cases while 7 cases were deferred. The agreement on the presence of an AE between the physician's and the final judgment ranged from 86.0% to 96.8% (kappa = 0.56-0.88). However, agreement on the causation of healthcare management and preventability between the physician's and the final judgment was not in the acceptable range.

Conclusion: The reliability of each physician's judgments regarding the presence of an AE was satisfactory. However, the reliability of judgments related to the causation of health care management and preventability was not necessarily satisfactory. Therefore, it is considered important to judge causation and preventability based on discussion with clinical experts in the relevant field.