Misleading outcomes

Statistically significant

Carefully consider the claims made around results that are reported as “statistically significant”.

In treatment comparisons, statistical significance is the probability of the observed difference (the Effect estimate) or a bigger difference having occurred by chance if in reality there was no difference.

A “statistically significant” result is unlikely to have happened by chance. The usual threshold for this judgement is a probability of less than 5% (0.05).

However, statistical significance does not tell us anything about how important an effect is. A small, unimportant effect can be “statistically significant”. Similarly, a large, important effect can be “statistically non-significant”.

REMEMBER: Results that were significant or non-significant refer only to how likely you are to see the change observed in the people who took part in the study in the general population. This is not the same as saying the change was important or not important.

Do not be misled by such claims.