Danger of assumptions

“Lots of data!”

Quality not quantity is better for the evidence base.

Claims based on lots of data can be misleading. Sometimes this is called “big data” (data from large databases) or “real world data” (routinely collected data). Routinely collected data often does not include data about “confounders”.

Confounders are factors other than the treatments being compared that can affect the outcomes. For example, a database may report on the language outcomes for a group of 2 year olds who had received SLT over the course of a year.

Given that a year is a long time for a 2 year old this would be a confounder since, most children will make language gains spontaneously in that time as part of growing up.

When using routinely collected data, it is only possible to control for confounders that were already known and were measured when the data was collected. So, we cannot be sure that an association between a treatment and an outcome means that the treatment caused the outcome, rather than confounders.

BEWARE of claims that a treatment has an effect based on an association found using “big data” or “real world data”.

REMEMBER: Think about whether you can be sure that there aren’t other reasons for the association.