Looking under the hood of a hybrid two-way texting intervention to improve early retention on antiretroviral therapy in Malawi: an implementation fidelity evaluation.

Background While key to interpreting findings and assessing generalizability, implementation fidelity is underreported in mobile health (mHealth) literature. We evaluated implementation fidelity of an opt-in, hybrid, two-way texting (2wT) intervention previously demonstrated to improve 12-month retention on antiretroviral therapy (ART) among people living with HIV (PLHIV) in a quasi-experimental study in Lilongwe, Malawi. Methods Short message service (SMS) data and ART refill visit records were used to evaluate adherence to 2wT content, frequency and duration through the lens of the Conceptual Framework for Implementation Fidelity. Message delivery and 2wT participant interactions were considered across four core 2wT components: 1) weekly motivational SMS messages; 2) proactive SMS appointment reminders; 3) SMS reminders after missed appointments; and 4) interactive messaging with 2wT staff about transfers and appointment rescheduling. In mixed-effects logistic regression, associations were assessed between 2wT messages and on-time appointment attendance and timely return to care after a missed appointment, respectively, presenting odds ratios (ORs) and 95% confidence intervals (CIs). Results The 468 2wT participants had a median of 52 study weeks (interquartile range [IQR] 34-52) with 6 ART appointments (IQR 4-7) of which 2 (IQR 1-3) were missed. On average, participants received a motivation message for 75% (IQR 56% - 83%) of enrolled weeks, a reminder before 83% (IQR 67% - 100%) of appointments, and after 67% (IQR 0% - 100%) of missed appointments. Participants reported 9 transfers and rescheduled 46 appointments through 2wT prompts; 196 appointments were changed via unprompted interaction. Participants with 10% higher expected motivation message delivery were more likely to attend clinic on time (OR: 1.13; 95%CI: 1.06-1.20, p < 0.001) and return to care within 14 days of a missed appointment (OR: 1.12; 95%CI: 1.03-1.22, p = 0.01). Receiving an appointment reminder increased on-time attendance (OR: 1.83; 95%CI: 1.44-2.33, p < 0.001). Conclusion Greater 2wT implementation fidelity was associated with improved care outcomes. Although implementation fidelity monitoring of mHealth interventions is complex, it should be integrated into study design.