Meta-analytical estimates of interrater reliability for direct supervisor performance ratings: Optimism under optimal measurement designs.

Performance appraisal (PA) is used for various organizational purposes and is vital to human resources practices. Despite this, current estimates of PA reliability are low, leading to decades of criticism regarding the use of PA in organizational contexts. In this article, we argue that current meta-analytical interrater reliability (IRR) coefficients are underestimates and do not reflect the reliability of interest to most practitioners and researchers-the reliability of an employee's direct supervisor.

Comparing forced-choice and single-stimulus personality scores on a level playing field: A meta-analysis of psychometric properties and susceptibility to faking.

Forced-choice (FC) personality assessments have shown potential in mitigating the effects of faking. Yet despite increased attention and usage, there exist gaps in understanding the psychometric properties of FC assessments, and particularly when compared to traditional single-stimulus (SS) measures. The present study conducted a series of meta-analyses comparing the psychometric properties of FC and SS assessments after placing them on an equal playing field-by restricting to only studies that examined matched assessments of each format, and thus, avoiding the extraneous confound of using comparisons from different contexts (Sackett, 2021).

Turning words into numbers: Assessing work attitudes using natural language processing.

Researchers and practitioners are often interested in assessing employee attitudes and work perceptions. Although such perceptions are typically measured using Likert surveys or some other closed-end numerical rating format, many organizations also have access to large amounts of qualitative employee data. For example, open-ended comments from employee surveys allow workers to provide rich and contextualized perspectives about work.

Meta-analysis of biodata in employment settings: Providing clarity to criterion and construct-related validity estimates.

Although biodata inventories have long been used to hire job applicants, there are limitations to current biodata knowledge and little in the way of contemporary biodata meta-analytic reviews. This study establishes a precise understanding of biodata validity by conducting an updated meta-analysis that differentiates biodata validity in terms of two important defining features: construct domain and scoring method (rational, hybrid, empirical). Evidence was established in terms of criterion-related validity with job performance and additional work outcomes, as well as convergent validity with common external hiring measures.