A comprehensive guide to study the agreement and reliability of multi-observer ordinal data.

Background: A recent systematic review revealed issues in regard to performing and reporting agreement and reliability studies for ordinal scales, especially in the presence of more than two observers. This paper therefore aims to provide all necessary information in regard to the choice among the most meaningful and most used measures and the planning of agreement and reliability studies for ordinal outcomes.

Methods: This paper considers the generalisation of the proportion of (dis)agreement, the mean absolute deviation, the mean squared deviation and weighted kappa coefficients to more than two observers in the presence of an ordinal outcome.

Underpowered studies in muscle metabolism research: Determinants and considerations.

Biomedical research frequently employs null hypothesis testing to determine whether an observed difference in a sample is likely to exist in the broader population. Null hypothesis testing generally assumes that differences between groups or interventions are non-existent, unless proven otherwise. Because biomedical studies with human subjects are often limited by financial and logistical resources, they tend to have low statistical power, i.

How well can the fetal heart rate baseline be assessed by intrapartum intermittent auscultation? An interrater reliability and agreement study.

Background: We aimed to examine the inter-reliability and agreement among midwives when assessing the fetal heart rate (FHR) using the handheld Doppler. The primary aim was to measure the reliability and agreement of FHR baseline (baseline) as beats per minute (bpm). The secondary aims were to measure fluctuations from the baseline, defined as increases and decreases, and classifications (normal or abnormal) of FHR soundtracks.

Review of sample size determination methods for the intraclass correlation coefficient in the one-way analysis of variance model.

Reliability of measurement instruments providing quantitative outcomes is usually assessed by an intraclass correlation coefficient. When participants are repeatedly measured by a single rater or device, or, are each rated by a different group of raters, the intraclass correlation coefficient is based on a one-way analysis of variance model. When planning a reliability study, it is essential to determine the number of participants and measurements per participant (i.

Statistical inference for agreement between multiple raters on a binary scale.

Agreement studies often involve more than two raters or repeated measurements. In the presence of two raters, the proportion of agreement and of positive agreement are simple and popular agreement measures for binary scales. These measures were generalized to agreement studies involving more than two raters with statistical inference procedures proposed on an empirical basis.

Chasing consistency: On the measurement error in self-reported affect in experiments.

How feelings change over time is a central topic in emotion research. To study these affective fluctuations, researchers often ask participants to repeatedly indicate how they feel on a self-report rating scale. Despite widespread recognition that this kind of data is subject to measurement error, the extent of this error remains an open question.

Reliability and agreement in intrapartum fetal heart rate monitoring interpretation: A systematic review.

Introduction: Fetal heart rate (FHR) monitoring is routine in intrapartum care worldwide and one of the most common obstetrical procedures. Intrapartum FHR monitoring helps assess fetal wellbeing and interpretation of the FHR help form decisions for clinical management and intervention. It relies on the observers' subjective assessments, with variation in interpretations leading to variations in intrapartum care.

Affective Symptoms and Oropharyngeal Dysphagia in Head-and-Neck Cancer Patients: A Systematic Review.

Oropharyngeal dysphagia (OD) is a high impact morbidity in head-and-neck cancer (HNC) patients. A wide variety of instruments are developed to screen for affective symptoms and OD. The current paper aims to systematically review and appraise the literature to obtain insight into the prevalence, strength, and causal direction of the relationship between affective symptoms and OD in HNC patients.

Impact of the biopsy forceps size on histological analysis and performances of the histological scoring systems.

To improve the reliability of the quantitative scorings of the synovial biopsies, we evaluate whether diameter of arthroscopic forceps influences histological quality of synovial tissue and/or histological scores and we compare the intra- and inter-observer performances of the main histological scoring systems. Synovial biopsies were retrieved in the same part of the joint using 1, 2 and 4 mm diameters grasping forceps. After standard staining and immunohistochemistry with anti-CD68 antibody, slides were scored blindly by 2 independent experienced operators for tissue quality and with Krenn score, de Bois-Tak score and CD68 semi-quantitative score.

Punch Trackers: Correct Recognition Depends on Punch Type and Training Experience.

To determine the ability of different punch trackers (PT) (Corner (CPT), Everlast (EPT), and Hykso (HPT)) to recognize specific punch types (lead and rear straight punches, lead and rear hooks, and lead and rear uppercuts) thrown by trained (TR, n = 10) and untrained punchers (UNTR, n = 11), subjects performed different punch combinations, and PT data were compared to data from video recordings to determine how well each PT recognized the punches that were actually thrown. Descriptive statistics and multilevel modelling were used to analyze the data. The CPT, EPT and HPT detected punches more accurately in TR than UNTR, evidenced by a lower percentage error in TR ( = 0.

Affective symptoms and swallow-specific quality of life in total laryngectomy patients.

Background: The aim of this study is to determine the prevalence of clinically relevant affective symptoms and level of swallow-specific quality of life (QoL) in dysphagic patients with total laryngectomy (TL) and to explore the relationship between affective symptoms and swallow-specific QoL.

Methods: Thirty-five TL patients completed the Hospital Anxiety and Depression Scale (HADS) and the MD Anderson Dysphagia Inventory (MDADI). Student's t test and linear regression were used.

The use of spectrograms improves the classification of wheezes and crackles in an educational setting.

Chest auscultation is a widely used method in the diagnosis of lung diseases. However, the interpretation of lung sounds is a subjective task and disagreements arise. New technological developments like the use of visSual representation of sounds through spectrograms could improve the agreement when classifying lung sounds, but this is not yet known.

Evaluating Electrocardiography-Based Identification of Cardiac Resynchronization Therapy Responders Beyond Current Left Bundle Branch Block Definitions.

Objectives: This study aimed to evaluate the association of 4 left bundle branch block (LBBB) definitions and their individual ECG characteristics with clinical outcome. Furthermore, it aimed to combine relevant outcome-associated electrocardiographic (ECG) characteristics into a novel outcome-based definition.

Background: LBBB morphology is associated with positive response to cardiac resynchronization therapy.

Voice and Vocal Fold Condition Following Short-Term General Anesthesia: A Prospective Study.

Background: Dysphonia, with or without laryngeal changes, has been reported as a complication following prolonged intubation. In contrast, it is unknown if laryngeal changes also occur following short-term airway instrumentation. The objectives of this study were to determine the prevalence of laryngeal changes in patients undergoing short-term routine general anesthesia using an endotracheal tube (ETT) or supraglottic airway (SGA), and to identify predictors to these changes.

Prevalence and clinical associations of wheezes and crackles in the general population: the Tromsø study.

Background: Wheezes and crackles are well-known signs of lung diseases, but can also be heard in apparently healthy adults. However, their prevalence in a general population has been sparsely described. The objective of this study was to determine the prevalence of wheezes and crackles in a large general adult population and explore associations with self-reported disease, smoking status and lung function.

Large variability in clinical judgement and definitions of left bundle branch block to identify candidates for cardiac resynchronisation therapy.

Background: Left bundle branch block (LBBB) morphology is associated with improved outcome of cardiac resynchronisation therapy (CRT) and is an important criterion for patient selection. There are, however, multiple definitions for LBBB. Moreover, applying these definitions seems subjective.

Asymptotic variability of (multilevel) multirater kappa coefficients.

Agreement studies are of paramount importance in various scientific domains. When several observers classify objects on categorical scales, agreement can be quantified through multirater kappa coefficients. In most statistical packages, the standard error of these coefficients is only available under the null hypothesis that the coefficient is equal to zero, preventing the construction of confidence intervals in the general case.

Medically Unexplained Oropharyngeal Dysphagia at the University Hospital ENT Outpatient Clinic for Dysphagia: A Cross-Sectional Cohort Study.

Medically unexplained oropharyngeal dysphagia (MUNOD) is a rare condition. It presents without demonstrable abnormalities in the anatomy of the upper aero-digestive tract and/or swallowing physiology. This study investigates whether MUNOD is related to affective or other psychiatric conditions.

International perception of lung sounds: a comparison of classification across some European borders.

Introduction: Lung auscultation is helpful in the diagnosis of lung and heart diseases; however, the diagnostic value of lung sounds may be questioned due to interobserver variation. This situation may also impair clinical research in this area to generate evidence-based knowledge about the role that chest auscultation has in a modern clinical setting. The recording and visual display of lung sounds is a method that is both repeatable and feasible to use in large samples, and the aim of this study was to evaluate interobserver agreement using this method.

Observer Agreement for Measurements in Videolaryngostroboscopy.

Objective: This study evaluated the levels of intraobserver and interobserver agreement for measurements of visuoperceptual variables in videolaryngostroboscopic examinations and compared the observers' behavior during independent versus consensus panel rating.

Study Design: This is a retrospective study.

Setting: This study was conducted in a single-center tertiary care facility.

Affective symptoms in patients with oropharyngeal dysphagia: A systematic review.

Objective: Affective disorders are prevalent in different somatic conditions and influence somatic symptom bother and quality of life. Mood and anxiety disorders impact patients' compliance and adherence to treatment. This systematic review summarizes published studies on affective complaints in patients with oropharyngeal dysphagia (OD) in order to determine the quality of studies concerning any association of OD with symptoms of depression and/or anxiety.

Modeling agreement on bounded scales.

Agreement is an important concept in medical and behavioral sciences, in particular in clinical decision making where disagreements possibly imply a different patient management. The concordance correlation coefficient is an appropriate measure to quantify agreement between two scorers on a quantitative scale. However, this measure is based on the first two moments, which could poorly summarize the shape of the score distribution on bounded scales.

Comparing dependent kappa coefficients obtained on multilevel data.

Reliability and agreement are two notions of paramount importance in medical and behavioral sciences. They provide information about the quality of the measurements. When the scale is categorical, reliability and agreement can be quantified through different kappa coefficients.

Observers' Agreement on Measurements in Fiberoptic Endoscopic Evaluation of Swallowing.

This study analyzed the effect that dysphagia etiology, different observers, and bolus consistency might have on the level of agreement for measurements in FEES images reached by independent versus consensus panel rating. Sixty patients were included and divided into two groups according to dysphagia etiology: neurological or head and neck oncological. All patients underwent standardized FEES examination using thin and thick liquid consistencies.