Unfair comparison

Dissimilar comparison groups

Look out for intervention comparisons where the comparison groups were not alike. Comparison groups need to be similar at the beginning of a comparison.

If subjects in the comparison groups differ in ways other than exposure to the interventions being compared, the apparent effects of the interventions might reflect those differences rather than actual intervention effects. Differences in the characteristics of the subjects in the groups at the beginning of the comparison might result in estimates of effects that appear either larger or smaller than they actually are. A method such as allocating subjects to different interventions by assigning them random numbers (the equivalent of flipping a coin) is the best way to ensure that the groups being compared are similar in terms of both measured and unmeasured characteristics.

Be cautious about relying on the results of non-randomized comparisons (for example, if the subjects being compared were or were not selected for exposure for other reasons). Be particularly cautious when you cannot be confident that the characteristics of the comparison groups were similar. If subjects were not randomly allocated to comparison groups, ask if there were important differences between the groups that might have resulted in the estimates of exposure effects appearing either larger or smaller than they actually are.

REMEMBER: If subjects were not randomly allocated to comparison groups, ask if there were important differences between the groups that might have resulted in the estimates of exposure effects appearing either larger or smaller than they actually are.