Central tendency and matched difference approaches for assessing interrater agreement.

In Study 1 of this two-part investigation, we present a "central tendency approach" and procedures for assessing overall interrater agreement across multiple groups. We define parameters for mean group agreement and construct bootstrapped confidence intervals around the mean population parameters for r, AD, and ICC(1). In Study 2, we extend assessments of overall interrater agreement by developing a "matched difference approach" and procedures for assessing real versus pseudo agreement in a sample of groups.

Statistical significance criteria for the rWG and average deviation interrater agreement indices.

Despite the widespread use of interrater agreement statistics for multilevel modeling and other types of research, the existing guidelines for inferring the statistical significance of interrater agreement are quite limited. They are largely relevant only under conditions that numerous researchers have argued rarely exist. Here we address this problem by generating guidelines for inferring statistical significance under a number of conditions via a computer simulation.

The dread factor: how hazards and safety training influence learning and performance.

On the basis of hypotheses derived from social and experiential learning theories, we meta-analytically investigated how safety training and workplace hazards impact the development of safety knowledge and safety performance. The results were consistent with an expected interaction between the level of engagement of safety training and hazardous event/exposure severity in the promotion of safety knowledge and performance. For safety knowledge and safety performance, highly engaging training was considerably more effective than less engaging training when hazardous event/exposure severity was high, whereas highly and less engaging training had comparable levels of effectiveness when hazardous event/exposure severity was low.

When birds of a feather flock together and when they do not: status composition, social dominance orientation, and organizational attractiveness.

Although similarity-attraction notions suggest that similarity--for example, in terms of values, personality, and demography--attracts, the authors found that sometimes demographic similarity attracts and sometimes it repels. Consistent with social dominance theory (J. Sidanius & F.

Relative effectiveness of worker safety and health training methods.

Objectives: We sought to determine the relative effectiveness of different methods of worker safety and health training aimed at improving safety knowledge and performance and reducing negative outcomes (accidents, illnesses, and injuries).

Methods: Ninety-five quasi-experimental studies (n=20991) were included in the analysis. Three types of intervention methods were distinguished on the basis of learners' participation in the training process: least engaging (lecture, pamphlets, videos), moderately engaging (programmed instruction, feedback interventions), and most engaging (training in behavioral modeling, hands-on training).

Accurate tests of statistical significance for r(WG) and average deviation interrater agreement indexes.

The authors demonstrated that the most common statistical significance test used with r(WG)-type interrater agreement indexes in applied psychology, based on the chi-square distribution, is flawed and inaccurate. The chi-square test is shown to be extremely conservative even for modest, standard significance levels (e.g.