Objective: To evaluate the use of high-fidelity simulation for summative high-stakes assessment of intensive care trainees, focusing on non-technical skills (NTS), testing feasibility and acceptability of simulation assessment, and the reliability of two NTS rating scales.
Design, Setting And Participants: Prospective observational study of senior intensive care trainees in a simulated specialist examination.
Methods: Participants undertook a simulated patient management scenario and were assessed using two rating scales: the Anaesthesia Non-technical Skills (ANTS) scale and the Ottawa Global Rating Scale (GRS). Assessors were trained, currently active, high-stakes examiners. Participants also completed a survey on simulation-based summative assessment.
Outcome Measures: The inter-rater reliability of two rating scales for NTS assessment. We evaluated the feasibility of simulation-based assessment, and used survey results to assess acceptability to participants.
Results: Simulation assessment was feasible. Participants considered simulation-based high-stakes assessment to be acceptable and felt their scenario performance was reflective of real-world performance. Participants identified a need for debriefing following scenario-based assessment. Inter-rater reliability was fair for the ANTS and Ottawa GRS scores (intra-class correlation coefficient, 0.39 and 0.42, respectively). There was only fair agreement between raters for an NTS pass or fail (weighted kappa, 0.32) and for a technical skills pass or fail (weighted kappa, 0.36).
Conclusions: Summative high-stakes assessment using a single simulated scenario was feasible and acceptable to senior intensive care trainees. The low inter-rater reliability for the ANTS and Ottawa GRS rating scales and for pass or fail discrimination may limit its incorporation into an existing examination format.
Download full-text PDF |
Source |
|---|
Publication Analysis
Top Keywords
Similar Publications
Self-report personality tests used in high-stakes assessments hold the risk that test-takers engage in faking. In this article, we demonstrate an extension of the multidimensional nominal response model (MNRM) to account for the response bias of faking. The MNRM is a flexible item response theory (IRT) model that allows modeling response biases whose effect patterns vary between items.
View Article and Find Full Text PDFOptimizing autonomous artificial intelligence diagnostics for neuro-ocular health in space missions.
Life Sci Space Res (Amst)
February 2025
Human-Machine Perception Laboratory, Department of Computer Science and Engineering, University of Nevada, Reno, Reno, NV, United States.
Spaceflight-Associated Neuro-Ocular Syndrome (SANS) presents a critical risk in long-duration missions, with microgravity-induced changes that threaten astronaut vision and mission outcomes. Current SANS monitoring, limited to pre- and post-flight exams, lacks in-flight diagnostics, highlighting an urgent need for autonomous tools capable of real-time assessment. Grok, an AI platform by xAI, offers promising potential as an advanced diagnostic tool for space-based health monitoring.
View Article and Find Full Text PDFEmergency Airway Management: A Systematic Review on the Effectiveness of Cognitive Aids in Improving Outcomes and Provider Performance.
Clin Pract
January 2025
Department of Otolaryngology-Head and Neck Surgery, McGill University, Montreal, QC H4A 3J1, Canada.
Emergency airway management is a critical skill for healthcare professionals, particularly in life-threatening situations like "cannot intubate, cannot oxygenate" (CICO) scenarios. Errors and delays in airway management can lead to adverse outcomes, including hypoxia and death. Cognitive aids, such as checklists and algorithms, have been proposed as tools to improve decision-making, procedural competency, and non-technical skills in these high-stakes environments.
View Article and Find Full Text PDFIntegrative and syntactic complexity's role in decision-making under uncertainty.
Front Psychol
January 2025
School of Government and Public Transformation, Tecnologico de Monterrey, Monterrey, Mexico.
This study explores the impact of integrative complexity (IC) and syntactic complexity (SC) on decision-making under uncertainty. The research addresses how cognitive structures contribute to decision quality in ambiguous situations. A modified Ellsberg experiment was conducted using an online platform.
View Article and Find Full Text PDFTeam resuscitation for paediatrics (TRAP); application and validation of a paediatric resuscitation quality instrument in non-simulated resuscitations.
Resusc Plus
January 2025
University of Colorado, Children's Hospital Colorado, Aurora, CO, USA.
Background: Resuscitation of paediatric cardiac and respiratory arrest is a high-stakes and low frequency event in the paediatric emergency department. Resuscitation team performance assessment tools have been developed and validated for use in the simulation environment, but no tool currently exists to evaluate clinical performance in non-simulated, live paediatric resuscitations.
Methods: This is a validation study assessing inter-rater reliability of a novel assessment tool of clinical performance of non-simulated resuscitations, the Team Resuscitation for Paediatrics tool.
Want AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!