Objectives: Case specificity implies that success on any case is specific to that case. In examining the sources of error variance in performance on case-based examinations, how much error variance results from differences between cases compared with differences between items within cases? What is the optimal number of cases and questions within cases to maximise test reliability given some fixed period of examination time?
Methods: G and D generalisability studies were conducted to identify variance components and reliability for each examination analysed, and to optimise the reliability of the given test composition (1, 1.5, 2, 3, 4 and 5 questions per case), using data from 3 key features examinations of the Medical Council of Canada (n = 6342 graduating medical students), each of which consisted of about 35 written cases followed by 1- questions regarding specific key elements of data gathering, diagnosis and/or management.
Results: The smallest variance component was due to subjects; the variance due to subject-item interaction was over 5 times the interaction with cases (on average, 0.1106 compared with 0.0195). Relatively little variance was due to differences between cases; about 80% of the error variance was due to variability in performance among items within cases. The D study showed that reliability varied between 0.541 and 0.579, was least with 1 item per case and highest at 2 and 3 items per case.
Conclusions: The main source of error variance was items within cases, not cases, and the optimal strategy in terms of enhancing reliability would use cases with 2-3 items per case.
Download full-text PDF |
Source |
|---|---|
| http://dx.doi.org/10.1111/j.1365-2929.2006.02511.x | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
South African Dietitians' Knowledge and Perceptions of Food-Drug Interactions and Factors Affecting It.
J Hum Nutr Diet
February 2025
Department of Pharmacology, School of Medicine, Faculty of Health Sciences, University of Pretoria, Pretoria, Gauteng, South Africa.
Background: Dietitians ensure that patients receive tailored medical nutrition therapy to integrate with pharmacotherapy safely. Dietitians require a pharmacological understanding to prevent detrimental food-drug interactions (FDIs). The study investigated dietitians' knowledge of FDIs and their information sourcing.
View Article and Find Full Text PDFMultiple Contrast Tests in the Presence of Partial Heteroskedasticity.
Biom J
February 2025
Retired.
This paper proposes a general approach for handling multiple contrast tests for normally distributed data in the presence of partial heteroskedasticity. In contrast to the usual case of complete heteroskedasticity, the treatments belong to subgroups according to their variances. Treatments within these subgroups are homoskedastic, whereas treatments of different subgroups are heteroskedastic.
View Article and Find Full Text PDF%diag_test: a generic SAS macro for evaluating diagnostic accuracy measures for multiple diagnostic tests.
BMC Med Inform Decis Mak
January 2025
Institute of Mathematical Sciences Centre for Health Analytics and Modelling (CHaM), Strathmore University, Nairobi, Kenya.
Background: Measures of diagnostic test accuracy provide evidence of how well a test correctly identifies or rules-out disease. Commonly used diagnostic accuracy measures (DAMs) include sensitivity and specificity, predictive values, likelihood ratios, area under the receiver operator characteristic curve (AUROC), area under precision-recall curves (AUPRC), diagnostic effectiveness (accuracy), disease prevalence, and diagnostic odds ratio (DOR) etc. Most available analysis tools perform accuracy testing for a single diagnostic test using summarized data.
View Article and Find Full Text PDFThe impact of co-housing on murine aging studies.
Geroscience
January 2025
The Jackson Laboratory, Bar Harbor, ME, USA.
Analysis of preclinical lifespan studies often assume that outcome data from co-housed animals are independent. In practice, treatments, such as controlled feeding or putative life-extending compounds, are applied to whole housing units, and as a result, the outcomes are potentially correlated within housing units. We consider intra-class (here, intra-cage) correlation in three published and two unpublished lifespan studies of aged mice encompassing more than 20,000 observations.
View Article and Find Full Text PDFEffect of scan path on accuracy of complete arch intraoral scan.
J Adv Prosthodont
December 2024
Department of Prosthodontics and Research Institute of Oral Science, College of Dentistry, Gangneung-Wonju National University, Gangneung, Republic of Korea.
Purpose: This study aimed to compare the accuracy of an alternative scan path with that of traditional scan paths to obtain a more accurate method for complete arch scans.
Materials And Methods: A mandibular stone cast, including tooth preparations for the inlay, crown, and fixed prosthesis, was scanned 10 times using four different scan paths (A, B, C, and D). The scans were converted into stereolithography files, resized, and superimposed onto a control file obtained from a desktop scanner.
Want AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!