What and how should we teach young people? There are lots of claims about what is and isn’t effective, many of which you will have come across. For example, there are claims that extending the school day improves pupils’ learning, increases attendance, and adds to teacher stress. How can you know which of these claims are trustworthy? And how should you decide when to act on claims like these?

An educational ‘intervention’ refers to any kind of change to the practices within a school that is intended to achieve better pupil outcomes – for example, restructuring the curriculum or introducing a new teaching method. An effect is something an intervention makes happen – – like improving pupil attainment or reducing the incidence of problem behaviours.

People make lots of claims about the effects of interventions. How can we tell which claims are right or wrong? To do this, you need to look at what supports their claim – its basis. For example, someone’s personal experience is not by itself a good basis for a claim about what is effective teaching. This is because we don’t know what would have happened if that person had done something else.

To know if a treatment (like extending the school day) causes an effect (like improved attainment), the intervention has to be compared to something else (like not extending the school day). Researchers compare an intervention given to people in one group with something else given to people in another group. Those comparisons provide evidence – facts to support a conclusion about whether a claim is right or wrong. For those comparisons to be fair, the only important difference between the groups should be the interventions they receive.

A good choice is one that uses the best information available at the time. For education choices, this includes using the best available evidence of intervention effects. Good choices don’t guarantee good outcomes, but they make good outcomes more likely.

Claims, comparisons, and choices

Claims: Many claims about the effects of interventions are not trustworthy. Often this is because the basis for the claim is not trustworthy. The first (pink) group of guides are things you should watch out for when you hear or read an education claim. BEWARE of claims that are too good to be true, that are based on faulty logic, or that are based on trust alone.

Comparisons: It is always a good idea to question what evidence there is to support a claim. Evidence about the effects of interventions comes from intervention comparisons. The second (yellow) group of guides can help you decide how trustworthy that evidence is. Make sure that intervention comparisons are FAIR, that summaries of studies comparing interventions are reliable, and that you are not misled by the way that interventions effects are described.

Choices: Knowing how trustworthy the evidence is can help you make good choices. But there are other things you need to think about when you decide what to do and what not to do in schools. The third (blue) group of guides can help you make good choices. TAKE CARE when you decide what to do. Think carefully about what your problem is and what your options are, whether the evidence is relevant, and whether the advantages are better than the disadvantages.

Using the guides

Claims about the effects of interventions that are not supported by trustworthy evidence often turn out to be wrong. Consequently, people who believe and act on these claims waste resources and suffer unnecessarily by doing things that do not help and might be harmful, and by not doing things that do help.

We developed the Assessing Claims in Education (ACE) Key Concepts as part of a research project to help people make informed choices across a range of fields, including education. The original Key Concepts were developed for health and have been adapted here for an educational context. The ACE Key Concepts are the starting point for developing learning resources, such as this website. Descriptions of all the concepts are available here, and summarise in this poster. An introduction to the Key Concepts, with school-based examples, is available here.

The website, poster and introductory summary can be used in different ways. For example, by finding examples of educational claims in the media and – using the guides – thinking critically and discussing how trustworthy those claims are and what you would do. The resources could feed into a training session for staff, support the decision making of a Senior Leadership Team, or be used by the governing body.

Please contact us and share your experience using this website and any suggestions you have.