Background: Vast sums are distributed based on grant peer review, but studies show that interrater reliability is often low. In this study, we tested the effect of receiving two short individual feedback reports compared to one short general feedback report on the agreement between reviewers.
Methods: A total of 42 reviewers at the Norwegian Foundation Dam were randomly assigned to receive either a general feedback report or an individual feedback report. The general feedback group received one report before the start of the reviews that contained general information about the previous call in which the reviewers participated. In the individual feedback group, the reviewers received two reports, one before the review period (based on the previous call) and one during the period (based on the current call). In the individual feedback group, the reviewers were presented with detailed information on their scoring compared with the review committee as a whole, both before and during the review period. The main outcomes were the proportion of agreement in the eligibility assessment and the average difference in scores between pairs of reviewers assessing the same proposal. The outcomes were measured in 2017 and after the feedback was provided in 2018.
Results: A total of 2398 paired reviews were included in the analysis. There was a significant difference between the two groups in the proportion of absolute agreement on whether the proposal was eligible for the funding programme, with the general feedback group demonstrating a higher rate of agreement. There was no difference between the two groups in terms of the average score difference. However, the agreement regarding the proposal score remained critically low for both groups.
Conclusions: We did not observe changes in proposal score agreement between 2017 and 2018 in reviewers receiving different feedback. The low levels of agreement remain a major concern in grant peer review, and research to identify contributing factors as well as the development and testing of interventions to increase agreement rates are still needed.
Trial Registration: The study was preregistered at OSF.io/n4fq3 .
Download full-text PDF |
Source |
|---|---|
| http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8485516 | PMC |
| http://dx.doi.org/10.1186/s41073-021-00115-5 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
Prevalence of body image dissatisfaction and associated factors among high school adolescent in Hawassa city, Ethiopia.
Heliyon
January 2025
School of Nursing, College of Medicine and Health Sciences, Arba Minch University, Arba Minch, Ethiopia.
Background: Adolescence is a period in which individuals are more concerned with their body image. However, little is known about the prevalence of body image dissatisfaction and its associated factors. Thus, this study aimed to assess the prevalence of body image dissatisfaction and associated factors among high school adolescents in Hawassa city in 2023.
View Article and Find Full Text PDFEffects of Altered Haptic Feedback Gain Upon Balance Are Explained by Sensory Conflict Estimation.
Eur J Neurosci
January 2025
Human Performance Research Centre, University of Konstanz, Constance, Germany.
Lightly touching a solid object reduces postural sway. Here, we determine the effect of artificially modifying haptic feedback for balance. Participants stood with their eyes closed, lightly gripping a manipulandum that moved synchronously with body sway to systematically enhance or attenuate feedback gain between +2 and -2, corresponding to motion in the same or opposite direction to the body, respectively.
View Article and Find Full Text PDFHow Delphi studies in the health sciences find consensus: a scoping review.
Syst Rev
January 2025
Department of Research Methods in Health Promotion and Prevention, Institute for Health Sciences, University of Education Schwäbisch Gmünd, Oberbettringer Straße 200, Schwäbisch Gmünd, 73525, Germany.
Background: Delphi studies are primarily used in the health sciences to find consensus. They inform clinical practice and influence structures, processes, and framework conditions of healthcare. The practical research-how Delphi studies are conducted-has seldom been discussed methodologically or documented systematically.
View Article and Find Full Text PDFHead-mounted camera as a debriefing tool for surgical simulation-based training: a randomized controlled study in Senegal.
BMC Med Educ
January 2025
Department of Surgery, Saint-Louis Regional Hospital, Gaston Berger University, Road of Ngallelle, 234, Saint-Louis, Senegal.
Introduction: Video feedback, particularly with a head-mounted camera, has previously been described as a useful debriefing tool in well-funded health systems but has never been performed in a low-resource environment. The purpose of this randomized, intervention-controlled study is to evaluate the feasibility of using video feedback with a head-mounted camera during intestinal anastomosis simulation training in a low-resource setting.
Methodology: This study recruited 14 first-year surgery residents in Senegal, who were randomized into control and camera groups.
Effect of feedback-integrated reflection, on deep learning of undergraduate medical students in a clinical setting.
BMC Med Educ
January 2025
Riphah international university, Rawalpindi, Pakistan.
Background: Reflection fosters self-regulated learning by enabling learners to critically evaluate their performance, identify gaps, and make plans to improve. Feedback, in turn, provides external insights that complement reflection, helping learners recognize their strengths and weaknesses, adjust their learning strategies, and enhance clinical reasoning and decision-making skills. However, reflection alone may not produce the desirable effects unless coupled with feedback.
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!