Agreement between pharmacists for problem identification: an initial quality measurement of cognitive services.

Background: Recent changes in national reimbursement policies expand the ability of pharmacists to seek reimbursement for cognitive services. The quality of pharmacist-provided cognitive services has, until now, remained unassessed. Pharmacists should demonstrate the quality and value of their work to ensure the continued and expanded acceptance of reimbursement for their services. A preliminary step in assessing quality is to compare agreement between pharmacists for basic problem identification.

Objective: To quantify agreement between pharmacist reviewers for problem identification among Utah Medicaid recipients.

Methods: Five pharmacists retrospectively reviewed drug regimens, patient characteristics, diagnosis codes, and procedures for 80 Medicaid patients in September 2008 and identified drug-related problems (DRPs) in 15 predetermined categories. Data for each patient were reviewed twice, and each combination of 2 pharmacists reviewed the same 8 patients' information. We calculated a reliability coefficient to compare the number of DRPs identified and used prevalence and bias adjusted kappa (PABAK) to determine interrater reliability for the presence of a specific DRP.

Results: Of the 15 DRPs categorized by pharmacist reviewers, 1 (untreated indications) had a PABAK coefficient of 0.20, indicating a relatively low level of agreement between reviewers. All other DRP categories had good to excellent agreement, with PABAK coefficients ranging between 0.43 and 0.98.

Conclusions: Pharmacist reviewers exhibited less variability in DRP identification or categorization than had been expected for most categories. This work supports the conclusion that pharmacists in our center provide a basic and necessary level of quality for problem assessment. Future work is needed to document the impact of this quality on patient outcomes.