The valid and reliable assessment of fidelity is critical at all stages of intervention research and is particularly germane to interpreting the results of efficacy and implementation trials. Ratings of protocol adherence typically are reliable, but ratings of therapist competence are plagued by low reliability. Because family context and case conceptualization guide the therapist's delivery of interventions, the reliability of fidelity ratings might be improved if the coder is privy to client context in the form of an ecological assessment. We conducted a randomized experiment to test this hypothesis. A subsample of 46 families with 5-year-old children from a multisite randomized trial who participated in the feedback session of the Family Check-Up (FCU) intervention were selected. We randomly assigned FCU feedback sessions to be rated for fidelity to the protocol using the COACH rating system either after the coder reviewed the results of a recent ecological assessment or had not. Inter-rater reliability estimates of fidelity ratings were meaningfully higher for the assessment information condition compared to the no-information condition. Importantly, the reliability of the COACH mean score was found to be statistically significantly higher in the information condition. These findings suggest that the reliability of observational ratings of fidelity, particularly when the competence or quality of delivery is considered, could be improved by providing assessment data to the coders. Our findings might be most applicable to assessment-driven interventions, where assessment data explicitly guides therapist's selection of intervention strategies tailored to the family's context and needs, but they could also apply to other intervention programs and observational coding of context-dependent therapy processes, such as the working alliance.

Download full-text PDF

Source
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4697869PMC
http://dx.doi.org/10.1007/s11121-015-0589-0DOI Listing

Publication Analysis

Top Keywords

ratings fidelity
8
fidelity ratings
8
ecological assessment
8
assessment data
8
ratings
6
fidelity
6
assessment
6
intervention
5
reliability
5
experimental study
4

Similar Publications

A Patient-Oriented Implementation Strategy for a Perioperative mHealth Intervention: Feasibility Cohort Study.

JMIR Perioper Med

January 2025

Societal Participation & Health, Amsterdam Public Health, Amsterdam, The Netherlands.

Background: Day surgery is being increasingly implemented across Europe, driven in part by capacity problems. Patients recovering at home could benefit from tools tailored to their new care setting to effectively manage their convalescence. The mHealth application ikHerstel is one such tool, but although it administers its functions in the home, its implementation hinges on health care professionals within the hospital.

View Article and Find Full Text PDF

Design and validation of a simulation-based training module for ileo-transverse intracorporeal anastomosis.

Surg Endosc

January 2025

Colorectal Surgery Unit, Department of Digestive Surgery, Pontificia Universidad Católica de Chile, Uc-Christus Health Network, Santiago, Chile.

Background: The benefits of the totally laparoscopic right hemicolectomy have been established, but its adoption has been limited by the challenges of intracorporeal suturing. While simulation is effective for training advanced surgical skills, no dedicated simulation-based course exists for intracorporeal ileo-transverse anastomosis (ICA). This study aimed to develop and validate a simulation module for training in ICA.

View Article and Find Full Text PDF

Effectiveness of Synchronous Postdischarge Contacts on Health Care Use and Patient Satisfaction : A Systematic Review and Meta-analysis.

Ann Intern Med

January 2025

Center of Innovation to Accelerate Discovery and Practice Transformation, Durham Veterans Affairs Health Care System; Department of Population Health Sciences, Duke University School of Medicine; and Durham Evidence Synthesis Program, Durham Veterans Affairs Health Care System, Durham, North Carolina (J.M.G.).

Background: Postdischarge contacts (PDCs) after hospitalization are common practice, but their effectiveness in reducing use of acute care after discharge remains unclear.

Purpose: To assess the effects of PDC on 30-day emergency department (ED) visits, 30-day hospital readmissions, and patient satisfaction.

Data Sources: MEDLINE, Embase, and CINAHL searched from 2012 to 25 May 2023.

View Article and Find Full Text PDF

Effect of telehealth yoga on loneliness and social isolation among rural older adults: a randomized controlled trial.

Aging Ment Health

January 2025

Integrative Health Service, San Francisco VA Health Care System, San Francisco, California, USA.

Objectives: Loneliness and social isolation are associated with a range of deleterious health outcomes. Yoga is a mind-body physical activity that is used in health care settings and increasingly in a telehealth group format. This trial aimed to determine the effect of a brief course of telehealth yoga on loneliness among rural older adults.

View Article and Find Full Text PDF

Moulage plays an important part in enhancing simulation-based learning experience fidelity; however, limited tools exist to measure the fidelity of moulage. The original Moulage Authenticity Rating Scale (MARS) is a possible tool to determine moulage's physical and conceptual fidelity in simulation. Although the MARS was proven reliable and valid in the original context, the authors sought to validate it in South Africa.

View Article and Find Full Text PDF

Want AI Summaries of new PubMed Abstracts delivered to your In-box?

Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!