Unreliable summary

Selective reporting

Look out for unpublished results of fair comparisons. All results of studies should be reported (even where they are unfavourable, or the effects are minimal)

Many fair comparisons are never published, and outcomes are sometimes left out. Those that are published are more likely to report favourable results. As a consequence, reliance on published reports alone sometimes results in the beneficial effects of interventions being overestimated and the harmful effects being underestimated.

Biased under-reporting of research is a major problem that is far from being solved. It is scientific and ethical malpractice, and wastes research resources. Selective reporting is an important reason why fair comparisons of interventions should have clear and available methods and accessible data, even if the studies are not published.

REMEMBER: Think about the possibility of biased underreporting, and whether or not the authors of systematic reviews (or other summaries of studies) have considered this risk.