The dire disregard of measurement invariance testing in psychological science.

Self-report scales are widely used in psychology to compare means in latent constructs across groups, experimental conditions, or time points. However, for these comparisons to be meaningful and unbiased, the scales must demonstrate measurement invariance (MI) across compared time points or (experimental) groups. MI testing determines whether the latent constructs are measured equivalently across groups or time, which is essential for meaningful comparisons.

Associations of individual chronic diseases and multimorbidity with multidimensional frailty.

Objective: To examine the associations between individual chronic diseases and multidimensional frailty comprising physical, psychological, and social frailty.

Methods: Dutch individuals (N = 47,768) age ≥ 65 years completed a general health questionnaire sent by the Public Health Services (response rate of 58.5 %), including data concerning self-reported chronic diseases, multidimensional frailty, and sociodemographic characteristics.

Preregistration in practice: A comparison of preregistered and non-preregistered studies in psychology.

Preregistration has gained traction as one of the most promising solutions to improve the replicability of scientific effects. In this project, we compared 193 psychology studies that earned a Preregistration Challenge prize or preregistration badge to 193 related studies that were not preregistered. In contrast to our theoretical expectations and prior research, we did not find that preregistered studies had a lower proportion of positive results (Hypothesis 1), smaller effect sizes (Hypothesis 2), or fewer statistical errors (Hypothesis 3) than non-preregistered studies.

How do psychology researchers interpret the results of multiple replication studies?

Employing two vignette studies, we examined how psychology researchers interpret the results of a set of four experiments that all test a given theory. In both studies, we found that participants' belief in the theory increased with the number of statistically significant results, and that the result of a direct replication had a stronger effect on belief in the theory than the result of a conceptual replication. In Study 2, we additionally found that participants' belief in the theory was lower when they assumed the presence of p-hacking, but that belief in the theory did not differ between preregistered and non-preregistered replication studies.

Associations between lifestyle factors and multidimensional frailty: a cross-sectional study among community-dwelling older people.

Background: Multidimensional frailty, including physical, psychological, and social components, is associated to disability, lower quality of life, increased healthcare utilization, and mortality. In order to prevent or delay frailty, more knowledge of its determinants is necessary; one of these determinants is lifestyle. The aim of this study is to determine the association between lifestyle factors smoking, alcohol use, nutrition, physical activity, and multidimensional frailty.

Group autonomy enhancing treatment versus cognitive behavioral therapy for anxiety disorders: A cluster-randomized clinical trial.

Background: Although cognitive behavioral therapy (CBT) is effective in the treatment of anxiety disorders, few evidence-based alternatives exist. Autonomy enhancing treatment (AET) aims to decrease the vulnerability for anxiety disorders by targeting underlying autonomy deficits and may therefore have similar effects on anxiety as CBT, but yield broader effects.

Methods: A multicenter cluster-randomized clinical trial was conducted including 129 patients with DSM-5 anxiety disorders, on average 33.

Ensuring the quality and specificity of preregistrations.

Researchers face many, often seemingly arbitrary, choices in formulating hypotheses, designing protocols, collecting data, analyzing data, and reporting results. Opportunistic use of "researcher degrees of freedom" aimed at obtaining statistical significance increases the likelihood of obtaining and publishing false-positive results and overestimated effect sizes. Preregistration is a mechanism for reducing such degrees of freedom by specifying designs and analysis plans before observing the research outcomes.

Effect Sizes, Power, and Biases in Intelligence Research: A Meta-Meta-Analysis.

In this meta-study, we analyzed 2442 effect sizes from 131 meta-analyses in intelligence research, published from 1984 to 2014, to estimate the average effect size, median power, and evidence for bias. We found that the average effect size in intelligence research was a Pearson's correlation of 0.26, and the median sample size was 60.

Prediction of Mortality by the Tilburg Frailty Indicator (TFI).

Objective: To predict mortality with the Tilburg Frailty Indicator (TFI) in a sample of community-dwelling older people, using a follow-up of 7 years.

Design: Longitudinal.

Setting And Participants: 479 Dutch community-dwelling people aged 75 years or older.

Recommendations in pre-registrations and internal review board proposals promote formal power analyses but do not increase sample size.

In this preregistered study, we investigated whether the statistical power of a study is higher when researchers are asked to make a formal power analysis before collecting data. We compared the sample size descriptions from two sources: (i) a sample of pre-registrations created according to the guidelines for the Center for Open Science Preregistration Challenge (PCRs) and a sample of institutional review board (IRB) proposals from Tilburg School of Behavior and Social Sciences, which both include a recommendation to do a formal power analysis, and (ii) a sample of pre-registrations created according to the guidelines for Open Science Framework Standard Pre-Data Collection Registrations (SPRs) in which no guidance on sample size planning is given. We found that PCRs and IRBs (72%) more often included sample size decisions based on power analyses than the SPRs (45%).

Heterogeneity in direct replications in psychology and its association with effect size.

We examined the evidence for heterogeneity (of effect sizes) when only minor changes to sample population and settings were made between studies and explored the association between heterogeneity and average effect size in a sample of 68 meta-analyses from 13 preregistered multilab direct replication projects in social and cognitive psychology. Among the many examined effects, examples include the Stroop effect, the "verbal overshadowing" effect, and various priming effects such as "anchoring" effects. We found limited heterogeneity; 48/68 (71%) meta-analyses had nonsignificant heterogeneity, and most (49/68; 72%) were most likely to have zero to small heterogeneity.

Reproducibility of individual effect sizes in meta-analyses in psychology.

To determine the reproducibility of psychological meta-analyses, we investigated whether we could reproduce 500 primary study effect sizes drawn from 33 published meta-analyses based on the information given in the meta-analyses, and whether recomputations of primary study effect sizes altered the overall results of the meta-analysis. Results showed that almost half (k = 224) of all sampled primary effect sizes could not be reproduced based on the reported information in the meta-analysis, mostly because of incomplete or missing information on how effect sizes from primary studies were selected and computed. Overall, this led to small discrepancies in the computation of mean effect sizes, confidence intervals and heterogeneity estimates in 13 out of 33 meta-analyses.

Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis.

Publication bias is a substantial problem for the credibility of research in general and of meta-analyses in particular, as it yields overestimated effects and may suggest the existence of non-existing effects. Although there is consensus that publication bias exists, how strongly it affects different scientific literatures is currently less well-known. We examined evidence of publication bias in a large-scale data set of primary studies that were included in 83 meta-analyses published in Psychological Bulletin (representing meta-analyses from psychology) and 499 systematic reviews from the Cochrane Database of Systematic Reviews (CDSR; representing meta-analyses from medicine).

The Prevalence of Marginally Significant Results in Psychology Over Time.

We examined the percentage of p values (.05 < p ≤ .10) reported as marginally significant in 44,200 articles, across nine psychology disciplines, published in 70 journals belonging to the American Psychological Association between 1985 and 2016.

Statistical properties of methods based on the Q-statistic for constructing a confidence interval for the between-study variance in meta-analysis.

The effect sizes of studies included in a meta-analysis do often not share a common true effect size due to differences in for instance the design of the studies. Estimates of this so-called between-study variance are usually imprecise. Hence, reporting a confidence interval together with a point estimate of the amount of between-study variance facilitates interpretation of the meta-analytic results.

The effect of publication bias on the Q test and assessment of heterogeneity.

One of the main goals of meta-analysis is to test for and estimate the heterogeneity of effect sizes. We examined the effect of publication bias on the test and assessments of heterogeneity as a function of true heterogeneity, publication bias, true effect size, number of studies, and variation of sample sizes. The present study has two main contributions and is relevant to all researchers conducting meta-analysis.

Modeling the micro-macro link: Understanding macro-level outcomes using randomization tests on micro-level data.

Analytical sociology explains macro-level outcomes by referring to micro-level behaviors, and its hypotheses thus take macro-level entities (e.g. groups) as their units of analysis.

Associations between maternal psychological distress and salivary cortisol during pregnancy: A mixed-models approach.

Background: Maternal psychological distress during pregnancy is related to adverse child behavioral and emotional outcomes later in life, such as ADHD and anxiety/depression. The underlying mechanisms for this, however, are still largely unknown. The hypothalamic-pituitary-adrenal (HPA)-axis, with its most important effector hormone cortisol, has been proposed as a mechanism, but results have been inconsistent.

Predictive validity of the Brazilian version of the Tilburg Frailty Indicator for adverse health outcomes in older adults.

Purpose: This study aimed to determine the predictive value of the Brazilian Tilburg Frailty Indicator (TFI) for adverse health outcomes (falls, hospitalization, disability and death), in a follow-up period of twelve months.

Methods: This longitudinal study was carried out with a sample of people using primary health care services in Rio de Janeiro, Brazil. At baseline the sample consisted of 963 people aged 60 years and older.

Associations Between Secret-Keeping and Quality of Life in Older Adults.

This study examined the effects of secrecy on quality of life in a sample consisting of older adults (>50 years; N = 301). Three key components of secrecy were examined with the Tilburg Secrecy Scale-25 (TSS25; possession of a secret, self-concealment, and cognitive preoccupation). The TSS25 distinguishes between the tendency to conceal personal information (self-concealment) and the tendency to worry or ruminate about the secret (cognitive preoccupation), thereby enabling investigation of the effects of secrecy on quality of life in detail.

Examining reproducibility in psychology: A hybrid method for combining a statistically significant original study and a replication.

The unrealistically high rate of positive results within psychology has increased the attention to replication research. However, researchers who conduct a replication and want to statistically combine the results of their replication with a statistically significant original study encounter problems when using traditional meta-analysis techniques. The original study's effect size is most probably overestimated because it is statistically significant, and this bias is not taken into consideration in traditional meta-analysis.

Associations between multidimensional frailty and quality of life among Dutch older people.

Purpose: To examine the associations between components of physical, psychological and social frailty with quality of life among older people.

Methods: This cross-sectional study was carried out in a sample of Dutch citizens. A total of 671 people aged 70 years or older completed a web-based questionnaire ('the Senioren Barometer').

Autonomy-connectedness mediates sex differences in symptoms of psychopathology.

Objectives: This study aimed to examine if autonomy-connectedness, capacity for self-governance under the condition of connectedness, would mediate sex differences in symptoms of various mental disorders (depression, anxiety, eating disorders, antisocial personality disorder).

Method: Participants (N = 5,525) from a representative community sample in the Netherlands filled out questionnaires regarding the variables under study.

Results: Autonomy-connectedness (self-awareness, SA; sensitivity to others, SO; capacity for managing new situations, CMNS) fully mediated the sex differences in depression and anxiety, and partly in eating disorder -(drive for thinness, bulimia, and body dissatisfaction) and anti-social personality disorder characteristics.

Bayesian evaluation of constrained hypotheses on variances of multiple independent groups.

Research has shown that independent groups often differ not only in their means, but also in their variances. Comparing and testing variances is therefore of crucial importance to understand the effect of a grouping variable on an outcome variable. Researchers may have specific expectations concerning the relations between the variances of multiple groups.