Misleading description

Statistically significant

Look out for results that are reported as “statistically significant” or “not statistically significant”.

In comparisons of interventions, statistical significance is the probability of the observed difference (the effect estimate), or a bigger difference having occurred by chance, if in reality there was no difference.

A “statistically significant” result means the difference in effects of a comparison of interventions, is unlikely to have happened by chance. The usual threshold for this judgement is a probability of less than 5% (0.05).

However, statistical significance does not tell us anything about how important an effect is. A small, unimportant effect can be “statistically significant”. Similarly, a large, important effect can be “statistically non-significant”. For example, say a nutrition intervention leads to obese adult patients losing 300 grams more weight over a month, compared to those who did not receive the nutrition intervention. The effect estimate of losing 300 grams more weight might be statistically significant, but is not important for the health of such patients.

REMEMBER: Claims that results of a study were significant or non-significant usually mean that they were statistically significant or statistically non-significant. This is not the same as important or not important. Do not be misled by such claims.