Background: The establishment of assessment reliability at the level of the individual trainee is an important attribute of assessment methodologies, particularly for doctors who have been failed. This issue is of particular importance for the process of competence assessment in the USA, UK, Australia and New Zealand.
Methods: We use data from 19 applicants for higher surgical training in 2008 at the Royal College of Surgeons in Ireland to compare: (i) the objective structured assessment of technical skills (OSATS) method; and (ii) a procedure-specific checklist to assess surgical technical skills in the excision of a sebaceous cyst task by two experienced senior surgeons.
Results: The overall interrater reliability (IRR) of the OSATS assessment as determined by a correlation coefficient was 0.507 (P < 0.03) and 0.67 with coefficient alpha, considerably below the accepted 0.8 level of IRR. The checklist's overall IRR was 0.89. Individually, only five (26%) of the OSATS assessments reached the 0.8 level of IRR in contrast to 18 (95%) of the checklist assessments.
Discussion: We propose binary procedure-based assessment checklists as more reliable assessment instruments with more robust reproducibility.
Download full-text PDF |
Source |
|---|---|
| http://dx.doi.org/10.1111/j.1445-2197.2012.06236.x | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
A prospective ecological momentary assessment study of an ayahuasca retreat: exploring the salutary impact of acute psychedelic experiences on subacute affect and mindfulness skills in daily life.
Psychopharmacology (Berl)
January 2025
Observing Minds Lab, Department of Psychology, School of Psychological Sciences, University of Haifa, Haifa, Israel.
Rationale: To examine the acute effects of ayahuasca use and their relationship to sub-acute changes in affect and mindfulness in a non-clinical sample, addressing the need for a better understanding of ayahuasca's immediate and short-term impacts as interest in its use grows.
Objectives: Using prospective ecological assessment, this study investigates how ayahuasca used at a 4-day retreat affects positive/negative affect and mindfulness skills in daily living compared to pre-retreat. Additionally, we explore acute psychedelic experiences during the ayahuasca retreat, assessed retrospectively 1-2 days post-retreat, as potential mechanisms for theorized effects in daily living post-retreat.
Interprofessional teaching rounds in medical education: improving clinical problem-solving ability and interprofessional collaboration skills.
Med Educ Online
December 2025
Department of Gynecology and Obstetrics, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China.
Interprofessional teaching rounds are a practical application of interprofessional education in bedside teaching, yet there is a lack of research on how interprofessional teaching rounds should be implemented into medical education. This study aimed to describe our experience in developing and implementing interprofessional teaching rounds during a clerkship rotation for medical students, and compares its strengths and weaknesses relative to traditional teaching rounds. Medical students were assigned to either the interprofessional teaching round group ( = 24) or the traditional teaching round group ( = 25), and each group participated in their assigned type of teaching round.
View Article and Find Full Text PDFRubric for peer evaluation of oral presentations: Use and perceptions among experienced and non-experienced students.
J Dent Educ
January 2025
Department of Conservative and Prosthetic Dentistry, Complutense University, Madrid, Spain.
Purpose: A properly designed rubric for oral presentations should be useful both to assess students' performance and to help them prepare for the task. However, its use and perceptions might be influenced by scholars' previous familiarization with rubrics during pre-university courses. The aim of this study was to evaluate how the previous experience of students in the use of rubrics can influence their assessment of oral presentations and to compare their ratings with those assigned by educators.
View Article and Find Full Text PDFBackground: In the United States, complete abstinence persists as the standard for demonstrating recovery success from substance use disorders (SUDs), apart from alcohol use disorder (AUD). Although the FDA has recently indicated openness for non-abstinence outcomes as treatment targets, the traditional benchmark of complete abstinence for new medications to treat SUDs remains a hurdle and overshadows other non-abstinent outcomes desired by people with SUDs (e.g.
View Article and Find Full Text PDFImplementation effectiveness of 'Problem solving for better health' training in Lesotho using mixed methods and the reach, effectiveness, adoption, implementation and maintenance (RE-AIM) framework.
BMC Med Educ
January 2025
Boston University School of Public Health, 715 Albany St, Boston, MA, 02118, USA.
Background: Healthcare resources have been concentrated in urban areas, leaving rural regions vulnerable to poorer health outcomes. The Problem Solving for Better Health (PSBH) program was implemented to enhance healthcare systems in resource-limited regions by training personnel to maximize existing resources in problem-solving. This study evaluated the implementation effectiveness of PSBH-Nursing (PSBHN), a nationally led initiative to train nurses in PSBH in Lesotho.
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!