A scattered landscape: assessment of the evidence base for 71 patient decision aids developed in a hospital setting.

Background: Recent publications reveal shortcomings in evidence review and summarization methods for patient decision aids. In the large-scale "Share to Care (S2C)" Shared Decision Making (SDM) project at the University Hospital Kiel, Germany, one of 4 SDM interventions was to develop up to 80 decision aids for patients. Best available evidence on the treatments' impact on patient-relevant outcomes was systematically appraised to feed this information into the decision aids. Aims of this paper were to (1) describe how PtDAs are developed and how S2C evidence reviews for each PtDA are conducted, (2) appraise the quality of the best available evidence identified and (3) identify challenges associated with identified evidence.

Methods: The quality of the identified evidence was assessed based on GRADE quality criteria and categorized into high-, moderate-, low-, very low-quality evidence. Evidence appraisal was conducted across all outcomes assessed in an evidence review and for specific groups of outcomes, namely mortality, morbidity, quality of life, and treatment harms. Challenges in evidence interpretation and summarization resulting from the characteristics of decision aids and the type and quality of evidence are identified and discussed.

Results: Evidence reviews assessed on average 25 systematic reviews/guidelines/studies and took about 3 months to be completed. Despite rigorous review processes, nearly 70% of outcome-specific information derived for decision aids was based on low-quality and mostly on non-directly comparative evidence. Evidence on quality of life and harms was often not provided or not in sufficient form/detail. Challenges in evidence interpretation for use in decision aids resulted from, e.g., a lack of directly comparative evidence or the existence of very heterogeneous evidence for the diverse treatments being compared.

Conclusions: Evidence reviews in this project were carefully conducted and summarized. However, the evidence identified for our decision aids was indeed a "scattered landscape" and often poor quality. Facing a high prevalence of low-quality, non-directly comparative evidence for treatment alternatives doesn't mean it is not necessary to choose an evidence-based approach to inform patients. While there is an urgent need for high quality comparative trials, best available evidence nevertheless has to be appraised and transparently communicated to patients.