Observer reliability as a function of circumstances of assessment.

THREE FACTORS CHARACTERISTIC OF EXPERIMENTAL SETTINGS WERE HYPOTHESIZED TO INFLATE ARTIFACTUALLY THE RELIABILITY OF OBSERVATIONAL RECORDINGS: (a) knowledge by observers of when and by whom their reliability is being assessed, (b) the absence of the experimenter or a monitor to prevent cheating, and (c) computation of reliability within- (versus between-) observer group. Three groups of four observers used a standard nine-category observational code for disruptive behavior in recording from videotapes of a classroom for 22 days. Analyses revealed considerable increases in average occurrence reliability as a function of the main effects of each of the experimental factors.