Unreliable summary

Unsystematic summary

Look out for summaries of studies comparing nutrition interventions that were not done systematically.

There is often more than one study that has compared the same nutrition interventions. These studies can have different results. This can happen because some studies had fair comparisons and others did not. Or, it can happen because some studies were small (with few people), or some study outcomes had few events (e.g. in a big study comparing different diets for preventing stroke, only a few people had a stroke during the study, thus there were few events).

When the results of these studies are not summarised systematically, it can make you think that the effects of the nutrition interventions are larger or smaller than they are. For example, a summary including only studies that found big effects, instead of systematically looking at all studies that answered the same question regardless of their effects, might make the overall intervention effects in the summary seem bigger than they are.

Researchers protect against mistakes like these by being systematic. They start out with a plan for finding and summarising all the studies that compare the same nutrition interventions, and stick to that plan. Such summaries are called systematic reviews.

REMEMBER: Think about whether a summary of studies comparing nutrition interventions was a systematic review.