Unfair comparison

Indirect comparisons

Be careful with comparisons of treatments between studies that are different.

For many conditions (e.g. Autism Spectrum Disorder or ASD) there are more than two possible interventions that may be suggested by your SLT- for example, different communication interventions including verbally-based and AAC-based interventions. All the possible interventions for a condition are very rarely compared in a single study, so it may be necessary to consider indirect comparisons among treatments.

For example, there may be comparisons of a verbally-based intervention and ‘usual care’ and comparisons of an AAC-based intervention and ‘usual care’ but no studies that directly compare verbally-based with AAC-based. In this case, people making a decision about what intervention will be beneficial might indirectly compare verbally-based with AAC-based by examining how each intervention compared to ‘usual care’.

However, there can be important differences between studies when treatments are indirectly compared. For example, the participants might have had more severe communication needs in verbally-based intervention or ‘usual care’ might have been more or less effective in those studies. These differences between the studies can make the difference in outcomes for verbally-based intervention and AAC-based intervention seems smaller or larger than it actually is.

REMEMBER: When indirect comparisons are needed to inform intervention choices, think about whether careful consideration was given to differences between studies.