Unfair comparison

Reliable assessment of outcomes

Some outcomes are easy to assess, such as school attendance or GCSE grades. Others are more difficult, such as assessing students’ attitudes or motivation to learning. For intervention comparisons to be meaningful, outcomes should be assessed using methods that have been shown to be reliable.

For example, self-report of proficiency in a skill tends to be less reliable than testing all participants on a specific skill using the same test. As the test is the same for all students, conclusions based on this measure are more reliable than conclusions based on students’ self-reported proficiency as they might over, or under, estimate their abilities.

BEWARE of studies that use unreliable methods.

REMEMBER: Think about whether outcomes were assessed using methods that have been shown to be reliable.