Background: High stakes examinations used to credential trainees for independent specialist practice should be evaluated periodically to ensure defensible decisions are made. This study aims to quantify the College of Intensive Care Medicine of Australia and New Zealand (CICM) Hot Case reliability coefficient and evaluate contributions to variance from candidates, cases and examiners.
Methods: This retrospective, de-identified analysis of CICM examination data used descriptive statistics and generalisability theory to evaluate the reliability of the Hot Case examination component. Decision studies were used to project generalisability coefficients for alternate examination designs.
Results: Examination results from 2019 to 2022 included 592 Hot Cases, totalling 1184 individual examiner scores. The mean examiner Hot Case score was 5.17 (standard deviation 1.65). The correlation between candidates' two Hot Case scores was low (0.30). The overall reliability coefficient for the Hot Case component consisting of two cases observed by two separate pairs of examiners was 0.42. Sources of variance included candidate proficiency (25%), case difficulty and case specificity (63.4%), examiner stringency (3.5%) and other error (8.2%). To achieve a reliability coefficient of > 0.8 a candidate would need to perform 11 Hot Cases observed by two examiners.
Conclusion: The reliability coefficient for the Hot Case component of the CICM second part examination is below the generally accepted value for a high stakes examination. Modifications to case selection and introduction of a clear scoring rubric to mitigate the effects of variation in case difficulty may be helpful. Increasing the number of cases and overall assessment time appears to be the best way to increase the overall reliability. Further research is required to assess the combined reliability of the Hot Case and viva components.
Download full-text PDF |
Source |
|---|---|
| http://www.ncbi.nlm.nih.gov/pmc/articles/PMC11088756 | PMC |
| http://dx.doi.org/10.1186/s12909-024-05516-w | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
Emergence of West Nile virus infection in Kerala, South India: A case of concern.
J Vector Borne Dis
January 2025
ICMR-Vector Control Research Centre Field Station, Kottayam, Kerala, India.
West Nile virus (WNV) infection is emerging as a disease of public health concern in Kerala, India with recurring outbreaks since 2011. With its tropical climate, biodiversity hot spots of Western ghats, forest cover, plenty of water bodies and bird sanctuaries, Kerala provides an ideal ecological niche for vector breeding and transmission of WNV. In this article, we reflect on the peculiar features of the outbreaks of WNV fever in Kerala and highlight the knowledge gaps, the research priorities and the need for effective control measures.
View Article and Find Full Text PDFRecurrent NTM pulmonary disease despite avoidance of hot spring exposure in a plaque psoriasis patient treated with Secukinumab: a case report.
BMC Infect Dis
January 2025
National Clinical Research Center of Respiratory Diseases, Center for Respiratory Diseases, China-Japan Friendship Hospital, Beijing, 100029, China.
Background: Recurrent Non-Tuberculous Mycobacterial Pulmonary Disease (NTM-PD) related to treatment with Secukinumab has not been previously documented.
Case Presentation: Despite adherence to treatment and avoiding hot springs, a plaque psoriasis patient experienced persistent NTM-PD relapses.
Conclusions: There is potential association between Secukinumab, an IL-17A inhibitor, and NTM disease, echoing anti-TNF biologics' NTM risk, indicating the urgent need for further research on pathogenic mechanisms and risk factors.
Epidemiology of Yellow Fever in Nigeria: Analysis of Climatic, Ecological, Socio-Demographic, and Clinical Factors Associated with Viral Positivity Among Suspected Cases Using National Surveillance Data, 2017-2023.
J Epidemiol Glob Health
January 2025
Department of Eco-Epidemiology, Institute of Tropical Medicine, Nagasaki University, Nagasaki, Japan.
Background: Since its resurgence in 2017, Yellow fever (YF) outbreaks have continued to occur in Nigeria despite routine immunization and the implementation of several reactive mass vaccinations. Nigeria, Africa's most populous endemic country, is considered a high-priority country for implementing the End Yellow fever Epidemics strategy.
Methods: This retrospective analysis described the epidemiological profile, trends, and factors associated with Yellow fever viral positivity in Nigeria.
Scoping review of the societal impacts of compound climate events.
Discov Environ
January 2025
Department of Geography and Planning, Appalachian State University, Boone, NC USA.
Unlabelled: Climatic extremes have historically been seen as univariate; however, recent international reports have highlighted the potential for an increase in compound climate events (e.g., hot and dry events, recurrent flooding).
View Article and Find Full Text PDFCharacteristics of Harmful Hot Gas Group Migration and Ventilation Control after Mine Explosion.
ACS Omega
January 2025
College of Safety Science and Engineering, Liaoning Technical University, Fuxin 123000, Liaoning, China.
To minimize the number of casualties due to poisoning in coal mine gas explosions, the migration characteristics of harmful hot gas groups in the mine ventilation system following an explosion were analyzed. Through pipeline experiments, initial CO concentrations and residual temperatures after gas explosions were determined and used as key simulation parameters. Simulations were performed using the TF1M(3D) software based on the case of the gas explosion at the Tunlan Mine.
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!