A critical appraisal of the quality of critical care pharmacotherapy clinical practice guidelines and their strength of recommendations.

Objective: Clinical practice guideline (CPG) quality assessment is important before applying their recommendations. Determining whether recommendation strength is consistent with supporting quality of evidence is also essential. We aimed to determine quality of critical care pharmacotherapy CPGs and to assess whether high quality evidence supports strong pharmacotherapy recommendations.

Methods: MEDLINE (1966-February 2008), EMBASE (1980-February 2008), National Guideline Clearinghouse (February 2008) and personal files were searched to identify CPGs. Four appraisers evaluated each guideline using the appraisal of guidelines, research and evaluation (AGREE) instrument. AGREE assesses 23 items in six domains that include scope/purpose, stakeholder involvement, rigor of development, clarity, applicability and editorial independence. Standardized domain scores (0-100%) were determined to decide whether to recommend a guideline for use. One appraiser extracted strong pharmacotherapy recommendations and supporting evidence quality.

Results: Twenty-four CPGs were included. Standardized domain scores were clarity [69% (95% confidence interval (CI) 62-76%)], scope/purpose [62% (95% CI 55-68%)], rigor of development [51% (95% CI 42-60%)], editorial independence [39% (95% CI 26-52%)], stakeholder involvement [32% (95% CI 26-37%)] and applicability [19% (95% CI 12-26%)]. The proportion of guidelines that could be strongly recommended, recommended with alterations and not recommended was 25, 37.5 and 37.5%, respectively. High quality evidence supported 36% of strong pharmacotherapy recommendations.

Conclusion: Variation in AGREE domain scores explain why one-third of critical care pharmacotherapy CPGs cannot be recommended. Only one-third of strong pharmacotherapy recommendations were supported by high quality evidence. We recommend appraisal of guideline quality and the caliber of supporting evidence prior to applying recommendations.