A Confidence Interval for the Difference Between Standardized Regression Coefficients.

Researchers are often interested in comparing predictors, a practice commonly done informal comparisons of standardized regression slopes. However, formal interval-based approaches offer advantages over informal comparison. Specifically, this article examines a delta-method-based confidence interval for the difference between two standardized regression coefficients, building upon previous work on confidence intervals for single coefficients.

Evaluating avoidable heterogeneity in exploratory factor analysis results.

Meaningful interpretations of scores derived from psychological scales depend on the replicability of psychometric properties. Despite this, and unexpected inconsistencies in psychometric results across studies, psychometrics has often been overlooked in the replication literature. In this article, we begin to address replication issues in exploratory factor analysis (EFA).

Questionable research practices and cumulative science: The consequences of selective reporting on effect size bias and heterogeneity.

Despite increased attention to open science and transparency, questionable research practices (QRPs) remain common, and studies published using QRPs will remain a part of the published record for some time. A particularly common type of QRP involves multiple testing, and in some forms of this, researchers report only a selection of the tests conducted. Methodological investigations of multiple testing and QRPs have often focused on implications for a single study, as well as how these practices can increase the likelihood of false positive results.

The effects of the 4/20 cannabis holiday and adult-use cannabis legalization on medical cannabis sales and medical cannabis patient registration in Arizona.

Background: This study examined whether the 4/20 cannabis holiday was associated with increases in medical cannabis sales from licensed dispensaries in Arizona from 2018-2021, and whether adult-use cannabis legalization (the vote in November 2020 and retail sales in January 2021) was associated with declines in medical cannabis sales and in the number of registered medical patients.

Methods: Data came from the Arizona Medical Marijuana Program monthly reports from January 2018-December 2021. The reports show daily sales from licensed medical cannabis dispensaries (i.

Validity and reliability of the Body-Esteem Scale among a diverse sample of preadolescent youth.

Researchers have called for more psychometric research on body image assessment measures in children. The Body-Esteem Scale (BES) is a commonly used 24-item measure of self-evaluation of body image in youth, yet only one study has evaluated the psychometric properties of this measure in preadolescent children. The goal of the present study was to replicate and extend the original psychometric work on the BES by examining the factor structure, reliability, and convergent validity in a diverse sample of children ages 9-11 years old.

Appropriately estimating the standardized average treatment effect with missing data: A simulation and primer.

Reporting standardized effects in randomized treatment studies aids interpretation and facilitates future meta-analyses and policy considerations. However, when outcome data are missing, achieving an unbiased, accurate estimate of the standardized average treatment effect, sATE, can pose challenges even for those with general knowledge of missing data handling, given that the sATE is a ratio of a mean difference to a (within-group) standard deviation. Under both homogeneity and heterogeneity of variance, a Monte Carlo simulation study was conducted to compare missing data handling strategies in terms of bias and accuracy in the sATE, under specific missingness patterns plausible for randomized pretest posttest studies.

Sample size planning for replication studies: The devil is in the design.

Replication is central to scientific progress. Because of widely reported replication failures, replication has received increased attention in psychology, sociology, education, management, and related fields in recent years. Replication studies have generally been assessed dichotomously, designated either a "success" or "failure" based entirely on the outcome of a null hypothesis significance test (i.

Multiplicity in multiple regression: Defining the issue, evaluating solutions, and integrating perspectives.

When multiple hypothesis tests are conducted, the familywise Type I error probability correspondingly increases. Various multiple test procedures (MTPs) have been developed, which generally aim to control the familywise Type I error rate at the desired level. However, although multiplicity is frequently discussed in the ANOVA literature and MTPs are correspondingly employed, the issue has received considerably little attention in the regression literature and it is rare to see MTPs employed empirically.

A revised and expanded taxonomy for understanding heterogeneity in research and reporting practices.

Concerns about replication failures can be partially recast as concerns about excessive heterogeneity in research results. Although this heterogeneity is an inherent part of science (e.g.

Discrimination and depressive symptoms in sexual minority Latinx youth: Moderation by religious importance and attendance.

Relative to heterosexual peers, sexual minority youth experience significant mental health disparities. This is partly due to prejudicial encounters (e.g.

Power(ful) myths: misconceptions regarding sample size in quality of life research.

Purpose: Carefully selecting the sample size for a research study is one of the most fundamental ways to utilize resources in an ethical manner, maximize impact and replicability, and minimize research waste when investigating questions relevant to health-related quality of life (HRQOL). Despite an increased focus on sample size in the methodological literature, the topic has received limited attention in the HRQOL field, and there are still misconceptions that can weaken even well-intentioned sample size planning. This article aims to highlight common misconceptions, provide accessible and non-technical corrections to these misconceptions, and show how HRQOL researchers can benefit from a more nuanced understanding of sample size planning.

The click is not the trick: the efficacy of clickers and other reinforcement methods in training naïve dogs to perform new tasks.

Background: A handheld metal noisemaker known as a "clicker" is widely used to train new behaviors in dogs; however, evidence for their superior efficacy compared to providing solely primary reinforcement or other secondary reinforcers in the acquisition of novel behavior in dogs is largely anecdotal.

Methods: Three experiments were conducted to determine under what circumstances a clicker secondary reinforcer may result in acquisition of a novel behavior more rapidly or to a higher level compared to other readily available reinforcement methods. In Experiment 1, three groups of 30 dogs each were shaped to emit a novel sit and stay behavior of increasing duration with either the delivery of food alone, a verbal stimulus paired with food, or a clicker with food.

The prospective interactive effects of alcohol expectancies and subjective response on future drinking behavior.

Although alcohol expectancies and subjective response are independent predictors of drinking, social-cognitive theory suggests that expectancies may distort one's subjective response, creating discrepancies between expected and actual alcohol effects. A recent cross-sectional study found that unmet expectancies (using difference scores) were associated with heavier drinking. However, cross-sectional data cannot establish temporal precedence, and using difference scores ignores important conditional main effects.

Using prior information to plan appropriately powered regression studies: A tutorial using BUCSS.

Despite increased attention to the role of statistical power in psychological studies, navigating the process of sample size planning for linear regression designs can be challenging. In particular, it can be difficult to decide upon an appropriate value for the effect size, owing to a variety of factors, including the influence of the correlations among the predictors and between the other predictors and the outcome, in addition to the correlation between the particular predictor(s) in question and the outcome, on statistical power. One approach that addresses these concerns is to use available prior sample information but adjust the sample effect size appropriately for publication bias and/or uncertainty.

Model specification for nonlinearity and heterogeneity of regression in randomized pretest posttest studies: Practical solutions for missing data.

The randomized pretest posttest design is common in psychology, as is the corresponding missing data concern. Although missing data handling has seen advances over the past several decades, effective and practical solutions for handling missing data in randomized pretest posttest designs are lacking, particularly when assumptions of commonly used statistical models are violated. Although analysis of covariance can capture the average treatment effect with complete data, even when assumptions are tenuous, this becomes more difficult with missing data.

Negative affect reactivity to stress and internalizing symptoms over the transition to college for Latinx adolescents: Buffering role of family support.

Affect reactivity to stress may play a role in the development of internalizing symptoms during the college transition, a critical developmental juncture for Latinx adolescents, the largest ethnic minority group on college campuses. This study examined whether affect reactivity during high school is associated with internalizing symptoms in college and explored two potential protective factors, perceived family and peer support. Participants were 209 Latinx adolescents (Mage = 18.

Misinterpreting p: The discrepancy between p values and the probability the null hypothesis is true, the influence of multiple testing, and implications for the replication crisis.

The value is still misinterpreted as the probability that the null hypothesis is true. Even psychologists who correctly understand that values do not provide this probability may not realize the degree to which values differ from the probability that the null hypothesis is true. Importantly, previous research on this topic has not addressed the influence of multiple testing, often a reality in psychological studies, and has not extensively considered the influence of different prior probabilities favoring the null and alternative hypotheses.

Life stress and major depression: The mysteries of recurrences.

Approximately half of the people who suffer a major depressive episode for the first time experience recurrences, while the other half do not. Among the initially depressed, however, who will have recurrences remains a mystery, and cannot be forecasted with any statistical or clinical confidence. It is well documented, though, that highly stressful life events commonly precede first episodes of major depression, and that these experiences become progressively less common prior to recurrences.

Best (but oft forgotten) practices: sample size planning for powerful studies.

Given recent concerns regarding replicability and trustworthiness in several areas of science, it is vital to encourage researchers to conduct statistically rigorous studies. Achieving a high level of statistical power is one particularly important domain in which researchers can improve the quality and reproducibility of their studies. Although several factors influence statistical power, appropriate sample size planning is often under the control of the researcher and can result in powerful studies.

Sample-Size Planning for More Accurate Statistical Power: A Method Adjusting Sample Effect Sizes for Publication Bias and Uncertainty.

The sample size necessary to obtain a desired level of statistical power depends in part on the population value of the effect size, which is, by definition, unknown. A common approach to sample-size planning uses the sample effect size from a prior study as an estimate of the population value of the effect to be detected in the future study. Although this strategy is intuitively appealing, effect-size estimates, taken at face value, are typically not accurate estimates of the population effect size because of publication bias and uncertainty.

Addressing the "Replication Crisis": Using Original Studies to Design Replication Studies with Appropriate Statistical Power.

Psychology is undergoing a replication crisis. The discussion surrounding this crisis has centered on mistrust of previous findings. Researchers planning replication studies often use the original study sample effect size as the basis for sample size planning.

Questioning kindling: An analysis of cycle acceleration in unipolar depression.

The kindling hypothesis for depression predicts that with more recurrences, the interval between successive recurrences decreases. Studies with unipolar and bipolar samples generally have been consistent with this premise. However, previous research is subject to a statistical artifact.

There's more than one way to conduct a replication study: Beyond statistical significance.

As the field of psychology struggles to trust published findings, replication research has begun to become more of a priority to both scientists and journals. With this increasing emphasis placed on reproducibility, it is essential that replication studies be capable of advancing the field. However, we argue that many researchers have been only narrowly interpreting the meaning of replication, with studies being designed with a simple statistically significant or nonsignificant results framework in mind.