AI Article Synopsis
- - Replicated 21 experimental studies from reputable journals (Nature and Science, 2010-2015) using rigorous methods, including pre-registered analysis plans and larger sample sizes, averaging five times bigger than the originals.
- - Out of the replications, 62% showed effects consistent with the original studies, but the average effect size was about 50% of what was originally reported.
- - The study found that researchers’ beliefs about replicability were linked to actual outcomes, indicating that the scientific community can often predict which results are likely to replicate, suggesting that failures are not purely due to random chance.
Article Abstract
Being able to replicate scientific findings is crucial for scientific progress. We replicate 21 systematically selected experimental studies in the social sciences published in Nature and Science between 2010 and 2015. The replications follow analysis plans reviewed by the original authors and pre-registered prior to the replications. The replications are high powered, with sample sizes on average about five times higher than in the original studies. We find a significant effect in the same direction as the original study for 13 (62%) studies, and the effect size of the replications is on average about 50% of the original effect size. Replicability varies between 12 (57%) and 14 (67%) studies for complementary replicability indicators. Consistent with these results, the estimated true-positive rate is 67% in a Bayesian analysis. The relative effect size of true positives is estimated to be 71%, suggesting that both false positives and inflated effect sizes of true positives contribute to imperfect reproducibility. Furthermore, we find that peer beliefs of replicability are strongly related to replicability, suggesting that the research community could predict which results would replicate and that failures to replicate were not the result of chance alone.
Download full-text PDF |
Source |
|---|---|
| http://dx.doi.org/10.1038/s41562-018-0399-z | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
The misalignment of incentives in academic publishing and implications for journal reform.
Proc Natl Acad Sci U S A
February 2025
Department of Data and Decision Science, Technion-Israel Institute of Technology, Haifa 3200003, Israel.
For most researchers, academic publishing serves two goals that are often misaligned-knowledge dissemination and establishing scientific credentials. While both goals can encourage research with significant depth and scope, the latter can also pressure scholars to maximize publication metrics. Commercial publishing companies have capitalized on the centrality of publishing to the scientific enterprises of knowledge dissemination and academic recognition to extract large profits from academia by leveraging unpaid services from reviewers, creating financial barriers to research dissemination, and imposing substantial fees for open access.
View Article and Find Full Text PDFExamining Mammalian facial behavior using Facial Action Coding Systems (FACS) and combinatorics.
PLoS One
January 2025
Department of Psychology, Lyon College, Batesville, Arkansas, United States of America.
There has been an increased interest in standardized approaches to coding facial movement in mammals. Such approaches include Facial Action Coding Systems (FACS), where individuals are trained to identify discrete facial muscle movements that combine to create a facial configuration. Some studies have utilized FACS to analyze facial signaling, recording the quantity of morphologically distinct facial signals a species can generate.
View Article and Find Full Text PDFFrom impact metrics and open science to communicating research: Journalists' awareness of academic controversies.
PLoS One
January 2025
College of Medicine, University of Illinois Chicago, Chicago, IL, United States of America.
This study sheds light on how journalists respond to evolving debates within academia around topics including research integrity, improper use of metrics to measure research quality and impact, and the risks and benefits of the open science movement. It does so through a codebook thematic analysis of semi-structured interviews with 19 health and science journalists from the Global North. We find that journalists' perceptions of these academic controversies vary widely, with some displaying a highly critical and nuanced understanding and others presenting a more limited awareness.
View Article and Find Full Text PDFSite-selective photo-crosslinking for the characterisation of transient ubiquitin-like protein-protein interactions.
PLoS One
January 2025
Manchester Cancer Research Centre, Division of Cancer Sciences, School of Medical Sciences, Faculty of Biology, Medicine and Health, University of Manchester, Manchester, United Kingdom.
Non-covalent protein-protein interactions are one of the most fundamental building blocks in cellular signalling pathways. Despite this, they have been historically hard to identify using conventional methods due to their often weak and transient nature. Using genetic code expansion and incorporation of commercially available unnatural amino acids, we have developed a highly accessible method whereby interactions between biotinylated ubiquitin-like protein (UBL) probes and their binding partners can be stabilised using ultraviolet (UV) light-induced crosslinks.
View Article and Find Full Text PDFIntramolecular Redox-driven Synthesis of Mono- and Diradical Iridium(III) Complexes: Insights into Molecular and Electronic Structure, Electrochemistry, and Spin-Communication.
Chemistry
January 2025
Jadavpur University, Chemistry, 188 Raja S. C. Mallick Road, 700032, Kolkata, INDIA.
Two π-radical complexes containing bisazo-aromatic-centered radical anion (1•-) were synthesized through in-situ electron transfer from metal-to-ligand using [IrI] and 2-(2-Pyridylazo)azobenzene (1) in inert hydrocarbon solvent. These are characterized as diradical [IrIII(1•-)2]+[2]+ and monoradical [IrIII(1•-)Cl2(PPh3)] 3. In contrast, a rare metal-mediated hydrolytic cleavage of the C(sp2)-N bond occurred in protic solvent resulting in quaternary radical complex [IrIII(1•-)(1')(PPh3)]+(4)+.
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!