Misleading description

Few subjects or events

Look out for intervention effects that are based on small studies with few subjects.

Sometimes there are only a few subjects (e.g. plants, animals or farms) in studies of the effects of interventions, or there were few subjects that showed an improved performance. The results of studies like these are sometimes described as though we can be sure about the results. However, we cannot be sure that what seems to be a difference in outcomes was because of a real difference in the effects of the interventions that were compared: it may have occurred by chance.

In other words, some subjects would have had an improved performance anyway, regardless of the intervention. It is possible that there just happened to be more of those subjects in one group than the other.

It is like tossing coins. Imagine you have two coins that are exactly the same. You would expect to get similar results from both by tossing each coin in the air 100 times. But if you tossed each coin just 5 times, you might get “heads” 4 times with one coin and 1 time with the other coin, just by chance.

REMEMBER: You can’t be sure about intervention effects that are based on small studies with few subjects or few events.