Rethinking the gold standard - The feasibility of randomized controlled trials within health services effectiveness research.

Background: An evidence-based randomized controlled trial for a novel Pharmacy Asthma Service was tested in 3 Australian states. Positive asthma outcomes were achieved after the 12-month intervention, albeit in both the intervention and comparator arms. The current investigation uses a mixed methods approach to 1) qualitatively explore how comparator arm pharmacists implemented the trial protocol and 2) quantitatively examine how this may have impacted patient outcomes in this trial.

Methods: Post-intervention semi-structured qualitative interviews were conducted with 20 pharmacists, representing 21 of 37 (57%) comparator arm pharmacies that completed the trial. Based on these interviews, pharmacies were classified as 'adherent' to the trial protocol (reporting no interventions other than general practitioner referral) or 'non-adherent' (reporting at least one extra intervention to the trial protocol), or 'inconclusive'. These subgroups were compared descriptively in relation to patient outcomes.

Results: Overall, 33% (n = 8/24) of the comparator pharmacies who were interviewed (n = 21) or determined to have monitoring by a project officer to ensure adherence to the protocol (n = 3) were classified as adherent), 58% (n = 14/24) as non-adherent, 8% inconclusive (n = 2/24). While all patients commenced with uncontrolled asthma (Asthma Control Questionnaire score (ACQ) > 1.5), after 12 months the mean ACQ score for patients from adherent comparator pharmacies ('true control') was 1.8 (still uncontrolled asthma) compared to a score of 1.4 (controlled asthma) in the non-adherent comparator group. Quality of life significantly improved in the non-adherent comparator group over the 12 months of the trial.

Conclusion: The majority of pharmacists in the comparator arm who were interviewed, introduced their own interventions, which may have influenced the outcomes of the trial. The naturalistic setting of the study was not protective against these confounders. These findings question the feasibility of comparator arms within primary care settings and that alternative study designs should be considered when designing future intervention studies in pharmacy practice.