Unfair comparison

Indirect comparisons

Look out for comparisons of interventions between studies that are different.

For many situations there are more than two interventions (for example, different herbicides and different tillage methods to deal with weed infestations). All the possible interventions for a condition are very rarely compared in a single study, so it may be necessary to consider indirect comparisons among interventions.

For example, there may be comparisons of a new herbicide (A) with a conventional method of weed control, and comparisons of a named tillage method (B) with the same conventional method of weed control, but no studies that directly compare herbicide A with tillage method B. In this case, people making a decision about which method to use might indirectly compare herbicide A with tillage method B by comparing how each method compared to no-intervention or to the conventional method that was used in both studies.

However, there can be important differences between studies when interventions are indirectly compared. For example, the weed infestation might have been more or less severe in the herbicide studies, or the conventional method might have been more or less effective in those studies. These differences between the studies can make the difference in outcomes for herbicide method A and tillage method B seem smaller or larger than it actually is.

REMEMBER: When indirect comparisons are needed to inform intervention choices, think about whether careful consideration was given to differences between studies.