BEWARE of claims about new, succesful interventions
Many claims about the effects of interventions, models or techniques are not trustworthy. Especially claims about new, successful management techniques are often overstated and sometimes not true.
• Large effects are rare
• New is not necessarily better
• Interventions can harm
Claims often suggest large effects. But in management large effects are rare.
New interventions may not be better than (old) available alternatives
Popular management interventions that are widely used are not necessarily effective.
Management interventions may have negative side effects.
ALWAYS ASK what is the evidence for this claim?
Many claims in management are not based on trustworthy, reliable evidence. You should therefore determine whether there is evidence supporting a claim. Always ask:
• What is the problem to solve?
• How would the intervention solve this problem?
• What is the evidence?
• Do the advantages outweigh the risks?
Implementing an intervention when there is no evident problem makes little sense
What is the evidence supporting the claim?
Always consider the balance between the costs and benefits of interventions
Is the evidence applicable to your organizational context?
BE CRITICAL how trustworthy is the evidence?
Evidence is never perfect and can be misleading. Therefore evidence should be critically evaluated by systematically assess its trustworthiness and relevance. Keep in mind:
• Best practice is seldom evidence based
• Anecdotes are unreliable evidence
• Professional experience is often not reliable
• Association is not the same as causation
• A reliable measurement method is needed to assess an effect
• A fair comparison is needed to assess an effect
Anecdotes are an unreliable source for assessing the effects of an intervention
Personal experience alone is an unreliable source for assessing the effects of an intervention
Just because a claim is made by an expert or authority, does not mean the claim is trustworthy.
Conflicting interests may result in misleading claims about the effects of interventions
No single study can be considered to be strong evidence – it is merely indicative.
The fact that an organizational outcome is associated with an intervention does not mean that the intervention caused the outcome.
A fair comparisons of interventions provides the most trustworthy evidence about the effect of an intervention.
Whether a claim is trustworthy depends on how the evidence for that claim was obtained and measured
Saying that an effect or difference is significant can be misleading.
Always ask yourself how likely a claim about the advantages of an intervention is likely to be true given the available evidence.