Lack of Concordance in Interrater Scoring of the Provider's Global Assessment of Children With Juvenile Idiopathic Arthritis With Low Disease Activity.

Objective: To measure agreement among raters when scoring the physician/provider global assessment (PGA) of disease activity in patients with juvenile idiopathic arthritis (JIA) with no apparent disease activity, and to identify clinical and laboratory parameters that most strongly influence provider scoring of the PGA.

Methods: Profiles of clinical and laboratory findings from 20 patients with JIA with no apparent disease activity were given to 51 providers, who were asked to score the PGA using a 21-circle visual analog scale (range 0-10). Following initial scoring, providers discussed each profile and reasons for assigning the score given, and then were asked to rescore each profile. Providers were asked to list variables that influenced their scoring most strongly. Using a mixed-model approach, the intraclass correlation coefficient (ICC) of the final scores served as the measure of concordance.

Results: A total of 504 PGA scores were obtained. The overall ICC of the initial scores was 0.18. Thus, 18% of nonconcordance of the scores was attributable to patient differences, while 82% was due to provider variation. Variables that influenced scoring most strongly were (in order of frequency) presence of pain, questionable temporomandibular joint involvement, loss of joint motion, presence of any morning stiffness, psoriasis, and past history of uveitis.

Conclusion: The low ICC suggests poor agreement among providers scoring the PGA in JIA patients with low or no disease activity. Given the ubiquitous use of the PGA in classification and response criteria for JIA and other pediatric rheumatic diseases, substantive efforts are needed to bring about greater uniformity in scoring of global disease activity by providers.