Misleading description

Few people or events

Look out for treatment effects that are based on small studies with few people.

Sometimes there were few people in studies of the effects of interventions, or there were few people who got worse or got better. The results of studies like these are sometimes described as though we can be sure about the results. However, we cannot be sure that what seems to be a difference in outcomes was because of a real difference in the effects of the treatments that were compared: it may have occurred by chance.

In other words, some people would have got better or worse anyway, regardless of the treatment they received. And it is possible that there just happened to be more of those people in one group than the other.

It is like tossing coins. Imagine you have two coins that are exactly the same. You would expect to get the same result from tossing each coin in the air 100 times. But if you tossed each coin just 5 times, you might get “heads” 4 times with one coin and 1 time with the other coin, just by chance.

REMEMBER: You can’t be sure about intervention effects that are based on small studies with few people or few events.