BEWARE of claims about interventions that have an untrustworthy basis.
Claims about the effects and impact of particular interventions may not be trustworthy. Often this is because the reason (the basis) for the claim is not trustworthy. You should be careful when you hear claims that:
• Sound 'Too good to be true' (like 'miracle cures')
• Follow faulty logic or are purely theoretical
• Are based on trust or experience alone
People often think about the benefits of speech & language therapy, but ignore possible problems. But all assessments and interventions can cause problems as well as benefits.
Most claims that a treatment or intervention will ‘make’ a service user 100% better, 'cure' multiple conditions, or that it will 'work for everyone', turn out to be wrong.
There is no way of being completely certain about the effects of therapy
fair comparisons of therapies provide the best basis for being confident about the effects of therapies.
Some people who have speech, language, communication or swallowing difficulties may get better spontaneously. Sometimes a treatment will not help and in some cases may have a negative effect.
Therapy that works in theory may not work in practice.
Just because a therapy is associated with people getting better or worse, that doesn’t mean that the treatment made them better or worse.
Quality not quantity is better for the evidence base.
Unless a therapy is compared to something else, it is not possible to know what would happen without it.
If a single intervention comparison study shows that people who got one kind of therapy did better or worse than people who got something else, it does not mean that is the final answer.
Just because an approach has been used by speech and language therapists for a long time or by many people, it does not mean that it helps or that it is safe.
Just because a therapy is new, expensive, technologically impressive, or brand-named does not mean that it is better or safer than other treatments.
Delivering more therapy sessions than necessary can lead to problems without increasing how much it helps.
Earlier detection of a speech, language, communication or swallowing difficulties is not necessarily better.
It is rarely possible to know in advance who will benefit, who will not, and who will have problems because of a specific therapy.
Someone with an interest in getting people to use a therapy, such as making money, may overstate benefits and ignore possible problems.
If someone believes they helped a service user get better after delivering a certain therapy it does not necessarily mean that the therapy made them better.
Just because a therapy claim is made by an expert or authority, you cannot be sure that it is trustworthy.
“Peer-reviewed” and published studies may not be fair comparisons.
THINK 'FAIR' and check the evidence behind comparisons of interventions
Evidence from comparisons of treatments or interventions can fool you. You should think carefully about the evidence that is used to support claims about the effects of specific speech and language therapy approaches. Look out for:
• Unfair comparisons of treatments or interventions
• Unreliable summaries of comparisons from research
• Use of misleading outcomes and inappropriate measures
Be careful with intervention comparisons where the comparison groups were too different.
Be careful with comparisons of treatments between studies that are different.
Be careful with intervention comparisons where people knew which treatment they received, and knowing that could have changed how they felt or behaved.
Look out for intervention comparisons where what happened was measured differently in the comparison groups.
Be careful with outcomes that were not assessed reliably in treatment comparisons.
Be careful with treatment comparisons where what happened was not measured in lots of people.
Be careful with intervention comparisons where people’s outcomes were not counted in the group to which they were assigned.
Be careful with summaries of studies comparing treatments that were not done systematically.
Be careful with unpublished results of fair comparisons.
Be careful with intervention comparisons that are sensitive to assumptions that are made.
Look out for treatment effects that are described just using words, or use more emotive vocabulary.
Look out for treatment effects that are described as relative effects.
Look out for treatment effects that are described as average differences.
Look out for treatment effects that are based on small studies with few people.
Look out for results that are reported for a selected group of people within a study or systematic review.
Look out for results that are reported using p-values only and do not report confidence intervals (CI).
Carefully consider the claims made around results that are reported as “statistically significant”.
Look out for a “lack of evidence” being described as evidence of “no difference”.
TAKE CARE and make good choices
Good intervention choices depend on thinking carefully about what to do. Think carefully about:
• What the goals and therapy options are
• The relevance of the evidence behind each intervention
• The risks and benefits of providing an intervention, or choosing one intervention over another
Always ask yourself whether the treatment outcomes that are important to your service user have been measured in fair comparisons in studies.
Always ask yourself if the study targeted individuals with similar presentations to your target population
Always ask yourself if the interventions evaluated in fair comparisons are relevant.
Always ask yourself if the setting, population and intervention methods used in the study are able to be replicated in your clinical setting.
Always ask yourself whether there are risks associated with treatment, and whether the benefits of a treatment outweigh potential risk.
Always ask yourself how sure you are that the possible benefits of a treatment are better than and the possible risks of the treatment.