Background: Understanding differences between types of study design (SD) and level of evidence (LOE) are important when selecting research for presentation or publication and determining its potential clinical impact. The purpose of this study was to evaluate interobserver and intraobserver reliability when assigning LOE and SD as well as quantify the impact of a commonly used reference aid on these assessments.
Methods: Thirty-six accepted abstracts from the Pediatric Orthopaedic Society of North America (POSNA) 2021 annual meeting were selected for this study. Thirteen reviewers from the POSNA Evidence-Based Practice Committee were asked to determine LOE and SD for each abstract, first without any assistance or resources. Four weeks later, abstracts were reviewed again with the guidance of the Journal of Bone and Joint Surgery (JBJS) LOE chart, which is adapted from the Oxford Centre for Evidence-Based Medicine. Interobserver and intraobserver reliability were calculated using Fleiss' kappa statistic (k). χ2 analysis was used to compare the rate of SD-LOE mismatch between the first and second round of reviews.
Results: Interobserver reliability for LOE improved slightly from fair (k=0.28) to moderate (k=0.43) with use of the JBJS chart. There was better agreement with increasing LOE, with the most frequent disagreement between levels 3 and 4. Interobserver reliability for SD was fair for both rounds 1 (k=0.29) and 2 (k=0.37). Similar to LOE, there was better agreement with stronger SD. Intraobserver reliability was widely variable for both LOE and SD (k=0.10 to 0.92 for both). When matching a selected SD to its associated LOE, the overall rate of correct concordance was 82% in round 1 and 92% in round 2 (P<0.001).
Conclusion: Interobserver reliability for LOE and SD was fair to moderate at best, even among experienced reviewers. Use of the JBJS/Oxford chart mildly improved agreement on LOE and resulted in less SD-LOE mismatch, but did not affect agreement on SD.
Level Of Evidence: Level II.
Download full-text PDF |
Source |
|---|---|
| http://dx.doi.org/10.1097/BPO.0000000000002136 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
New Classification System for Proximal Fifth Metatarsal Fractures: Intraobserver and Interobserver Reliability Assessment.
Foot Ankle Int
January 2025
University Hospital October 12, Madrid, Spain.
Background: The most commonly used classification for proximal fifth metatarsal fractures has not shown good reproducibility. The aim of this study was to evaluate the intraobserver and interobserver agreement of a new classification system for such fractures.
Methods: The study involved the development of a novel classification system that categorized these fractures into 2 main types and 2 subtypes.
Validation of a novel tool for automated tooth modelling by fusion of CBCT-derived roots with the respective IOS-derived crowns.
J Dent
December 2024
OMFS-IMPATH Research Group, Department of Imaging and Pathology, Faculty of Medicine, KU Leuven, Leuven, Belgium; Department of Dental Medicine, Karolinska Institute, Stockholm, Sweden. Electronic address:
Objectives: To validate a novel artificial intelligence (AI)-based tool for automated tooth modelling by fusing cone beam computed tomography (CBCT)-derived roots with corresponding intraoral scanner (IOS)-derived crowns.
Methods: A retrospective dataset of 30 patients, comprising 30 CBCT scans and 55 IOS dental arches, was used to evaluate the fusion model at full arch and single tooth levels. AI-fused models were compared with CBCT tooth segmentation using point-to-point surface distances-reported as median surface distance (MSD), root mean square distance (RMSD), and Hausdorff distance (HD)- alongside visual assessments.
Inter- and Intra-observer Reproducibility of Thyroid Fine Needle Aspiration Cytology: An investigation of Bethesda 2023 Using Immunohistochemical BRAFV600E Antibody.
J Cytol
November 2024
Department of Pathology, Mardin Training and Research Hospital, Mardin, Turkey.
Background: The Bethesda System for Reporting Thyroid Cytology (TBSRTC) recommended for the interpretation of needle aspiration cytology of the thyroid, is the most widely used worldwide. Studies have shown that the disagreement between observers, especially in the Bethesda III and IV diagnostic categories, is not insignificant at 10%-40%. In the TBSRTC 2023 version, some definitions were removed and simplified, and molecular pathology was proposed as a complement to cytopathology.
View Article and Find Full Text PDFA modified Radiographic Union Score for Tibia (RUST) scoring system for patella fractures treated with osteosynthesis shows excellent intra und interobserver reliability.
Arch Orthop Trauma Surg
December 2024
University Hospital Regensburg, Regensburg, Germany.
Introduction: Patellar fractures are rare at 1% incidence of all fractures. However, they can cause significant functional impairments due to the patella's role in knee joint extension. Current scoring systems lack objectivity in assessing patellar healing.
View Article and Find Full Text PDFSpinopelvic Motion Evaluation in Patients Undergoing Total Hip Arthroplasty and Patient-Specific Target for Acetabular Cup Placement.
J Pers Med
December 2024
Orthopaedic Department, School of Health Sciences, Faculty of Medicine, University of Thessaly, 41110 Larissa, Greece.
: Instability is a major reason for revision after total hip arthroplasty (THA), and acetabular cup placement in the "traditional" safe zone does not protect against dislocations. Spinopelvic mobility may play a role in impingement and dislocation after THA. Personalized acetabular cup placement that incorporates spinopelvic mobility is currently lacking in the literature.
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!