Systematic quality review of guidelines – feasible and useful?

Background: Many clinical guidelines are produced and used, but there is no consensus on how to evaluate their quality. The purpose of this study was to test the feasibility and the benefit of systematic quality review of clinical guidelines.

Methods: 127 Norwegian guidelines were evaluated with the AGREE (Appraisal of Guidelines Research and Evaluation) instrument. Each guideline was assessed by two certified reviewers.

Results: Each reviewer spent 2-5 hours on each guideline. The average cost for evaluating one guideline amounted to about €800. Fifteen guidelines received the conclusion «strongly agree», 98 «recommend with provisos or alterations», 8 «would not recommend», and 6 received «unsure».

Conclusions: Most Norwegian clinical guidelines do not fulfil the quality criteria in the AGREE instrument. Better guidance for rating the overall assessment is needed. Systematic quality review of guidelines is more structured than peer review of scientific articles, but has less consequence as it is done independently of publication. Guidelines should be reviewed by an independent body before publication and their evaluation should include novelty and relevance.