Unfair comparison

Dissimilar measurement

Look out for intervention comparisons where what happened was measured differently in the comparison groups. Impacts should be assessed similarly.

If an effect of a human action is measured differently in two comparison groups, differences in that effect may be due to how the effect was measured rather than because of the interventions to which each group was exposed. For example, if assessors believe that a particular action has an effect and they know which subjects have been exposed to that action, they may be more likely to observe greater effects in those that have been exposed to the action. One way of preventing this is to keep assessors unaware of (“blind” to) which subjects have been allocated to which group. This precaution is less important for “objective” outcomes, like invasive species eradication, than for “subjective” outcomes like qualitative estimates of species abundance.

REMEMBER: Be cautious about relying on the results of comparisons if effects were not measured in the same way in the different comparison groups. The results of such comparisons can be misleading.