Inter-rater reliability of the PCL-R total and factor scores among psychopathic sex offenders: are personality features more prone to disagreement than behavioral features?

Despite considerable support for the inter-rater reliability of the Hare Psychopathy Checklist--Revised (PCL-R; Hare, 1991, 2003) in research contexts, there is increasing concern that scores from this instrument may be considerably less stable across examiners in applied contexts, particularly when scoring is based on separate interviews. The present study examines archival data from a sample of imprisoned sex offenders (n = 20) who obtained relatively high PCL-R total scores (> or =25) and were administered this instrument on a second occasion by a different examiner. Intraclass correlations for the total and Factor 2 score were lower than those generally reported in research studies. Of greater concern, Factor 1 scores were only negligibly related to each other (ICC(A,1) = .16). Correcting for potential range restriction among these high scoring individuals resulted in total and Factor 2 score measures of agreement that were somewhat more consistent with published research, but Factor 1 continued to display exceedingly poor agreement across examiners.