Unreliable summary

Unsystematic reviews

Look out for reviews or summaries of multiple studies comparing interventions that were not done systematically. Reviews of fair comparisons should be systematic.

Reviews that do not use systematic methods may result in biased or imprecise estimates of the effects of interventions because the selection of studies for inclusion may be biased, or the methods may result in some studies not being found. In addition, the appraisal of the quality of some studies may be biased, or the synthesis of the results of the selected studies may be inadequate or inappropriate. Even reviews that purport to be systematic may not be.

Whenever possible, use systematic reviews of fair comparisons rather than non-systematic reviews to inform decisions.

REMEMBER: Think about whether a summary of studies comparing effects of interventions was a systematic review (even if the authors claim it is).