From alpha to omega and beyond! A look at the past, present, and (possible) future of psychometric soundness in the Journal of Applied Psychology.

The psychometric soundness of measures has been a central concern of articles published in the Journal of Applied Psychology (JAP) since the inception of the journal. At the same time, it isn't clear that investigators and reviewers prioritize psychometric soundness to a degree that would allow one to have sufficient confidence in conclusions regarding constructs. The purposes of the present article are to (a) examine current scale development and evaluation practices in JAP; (b) compare these practices to recommended practices, previous practices, and practices in other journals; and (c) use these comparisons to make recommendations for reviewers, editors, and investigators regarding the creation and evaluation of measures including Excel-based calculators for various indices.

Using the Nursing Culture Assessment Tool (NCAT) in Long-Term Care: An Update on Psychometrics and Scoring Standardization.

An effective workforce performing within the context of a positive cultural environment is central to a healthcare organization's ability to achieve quality outcomes. The Nursing Culture Assessment Tool (NCAT) provides nurses with a valid and reliable tool that captures the general aspects of nursing culture. This study extends earlier work confirming the tool's construct validity and dimensionality by standardizing the scoring approach and establishing norm-referenced scoring.

Predictive validity of the MMPI-2 among female offenders in a residential treatment program.

This study investigated the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) validity, clinical, restructured clinical, supplementary, and content scales in a female correctional population. The study used archival data for a final sample of 116 women who had been evaluated for acceptance into a residential rehabilitative treatment program for nonviolent female offenders in North Carolina. MMPI-2 scale elevations are reported and assessed for predictive validity in relation to treatment success, as measured by treatment attendance and graduation status.

Personality retesting for managing intentional distortion.

Self-report personality questionnaires often contain validity scales designed to flag individuals who intentionally distort their responses toward a more favorable characterization of themselves. Yet, there are no clear directives on how scores on these scales should be used by administrators when making high-stakes decisions about respondents. Two studies were conducted to investigate whether administrator-initiated retesting of flagged individuals represents a viable response to managing intentional distortion on personality questionnaires.

An inductive exploration of the social effectiveness construct space.

There is no agreement regarding the nature or number of dimensions that make up the social effectiveness domain. We inductively explore the relationships between a set of social effectiveness measures with the intention of identifying an initial set of dimensions. An exploratory factor analysis of the Social Competence Inventory (SCI, Schneider, 2001) resulted in the identification of four factors: Social Potency, Social Appropriateness, Social Emotional Expression, and Social Reputation.

An examination of psychometric bias due to retesting on cognitive ability tests in selection settings.

Using a latent variable approach, the authors examined whether retesting on a cognitive ability measure resulted in measurement and predictive bias. A sample of 941 candidates completed a cognitive ability test in a high-stakes context. Results of both the within-group between-occasions comparison and the between-groups within-occasion comparison indicated that no measurement bias existed during the initial testing but that retesting induced both measurement and predictive bias.

Forced-choice assessments of personality for selection: evaluating issues of normative assessment and faking resistance.

Recent research suggests multidimensional forced-choice (MFC) response formats may provide resistance to purposeful response distortion on personality assessments. It remains unclear, however, whether these formats provide normative trait information required for selection contexts. The current research evaluated score correspondences between an MFC format measure and 2 Likert-type measures in honest and instructed-faking conditions.

The predictive validity of self-efficacy in training performance: little more than past performance.

Past research on the influence of self-efficacy in training has provided mixed results. Key differences between studies pertain to whether past performance is operationalized as a residual variable or as an unadjusted variable and to the type of task used. In this study, the authors conducted and performed a reanalysis to examine the influence of self-efficacy using both operationalizations of past performance in 2 experimental tasks.

Interruption management: the use of attention-directing tactile cues.

Previous research has suggested that providing informative cues about interrupting stimuli aids management of multiple tasks. However, auditory and visual cues can be ineffective in certain situations. The objective of the present study was to explore whether attention-directing tactile cues aid or interfere with performance.

Faking and selection: considering the use of personality from select-in and select-out perspectives.

The effects of faking on criterion-related validity and the quality of selection decisions are examined in the present study by combining the control of an experiment with the realism of an applicant setting. Participants completed an achievement motivation measure in either a control group or an incentive group and then completed a performance task. With respect to validity, greater prediction error was found in the incentive condition among those with scores at the high end of the predictor distribution.