Bradford Hill – assess the assessors!

Suppose a responsible body demanded high quality evidence of accreditation’s efficacy and safety.  Then others might consider applying to accreditation the criteria proposed by Bradford Hill for assessing the cause of an illness.  If these criteria can test the cause and effect of a medical diagnosis, they should also be helpful in detecting any ability of accreditation of bring quality.

 The Bradford Hill Criteria

In 1965 Austin Bradford Hill described the minimal conditions establishing cause and effect in medical diagnosis.

There is a time relationship between cause and effect in that the effect occurs after the cause. Also, if it is to be expected that there is some delay between cause and effect then that delay should also be observed.

Inspection will be tighter next year

Accreditation ensures that work is always changing, always improving, yet is always assured of quality.  But “quality” only means inspectability.  So how does the validity of a lab’s results change over time as the tourniquet of inspection is tightened?

Strength and association
Cause-and-effect may be observed by statistical correlation between these in repeated events or experiments. Full strength correlation has a coefficient of 1. A weaker association between cause and effect will see greater variation.

What is quality apart from the ability to inspect?  How would one measure it? If it hasn’t been measured how would one know it had an association with accreditation?  Would quality correlate positively or negatively with inspection?

Why have you accepted accreditation if your can’t answer these questions?

Biological gradient (dose-response)
In treatment, there might be expected to be a relationship between the dose given and the reaction of the patient. This may not be a simple linear relationship and may have minimum and maximum thresholds.

Accreditation is always heavy.  It is also always changing.  If labs truly gained quality by recording everything they do, work for inspectors would diminish.  That’s why they plan and tighten the screw each year.  The workload distracts everyone from the fact no good is being done.

One apparent success does not prove a general cause and effect in wider contexts. To prove a treatment is useful, it must give consistent results in a wide range of circumstances.

No randomised double blind placebo controlled trials exist to test accreditation.  Just a few anecdotal reports saying “we worked very hard and got accredited and now everything’s better.”  Yeah, sure it is.  Speak to the staff; don’t just read their papers.

The apparent cause and effect must make sense in the light of current theories and results. If a causal relationship appears to be outside of current science then significant additional hypothesizing and testing will be required before a true cause and effect can be found.

It must be plausible.  So many people believe the accreditation cartel’s claims that quality arises from inspection.  The deeply inquiring minds of science have been stupefied by the dogma.  They are happy for their employees to comply.  Maybe one day they will realise that there may be other explanations worth considering.  Plausibility is not proof.

A specific relationship is found if there is no other plausible explanation. This is not always the case in medicine where any given symptoms may have a range of possible causing conditions.

Accreditation has no measure outside itself.  The specificity of nothing else gets tested.  The ISO stuffs many procedural tools into its standards and claims results arise from its own standard, not the component parts.   Compliance equals quality.  This dogma prevents any competing or contradictory views being considered.  Accreditation is constructed to specifically exclude competition with itself.  Nobody even looks for evidence.  They are happy to comply with inspection bodies because they are more obscure than pharma companies. They don’t think that plenty of developed countries get satisfactory quality without paying the ISO tax.

A very strong proof of cause and effect comes from the results of experiments, where many significant variables are held stable to prevent them interfering with the results. Other evidence is also useful but can be more difficult to isolate cause and effect.

The ISO standard allows no measure outside compliance with itself.  Every year’s assessment is different.  Each decade has two revisions of the standard.  Laboratories exist in a state of continual management change anyway.  This makes experimental testing very difficult.

When something is suspected of causing an effect, then other factors similar or analogous to the supposed cause should also be considered and identified as a possible cause or otherwise eliminated from the investigation.

Maybe the obsessive compulsive characteristics of the ISO standards are transmitted to its victims and changes arise from their obsessiveness rather than from the standard itself.

Maybe some components subsumed into the standards – such as EQA or IQC or calibration  – account for any improvement in genuine quality that exists apart from compliance with the standards.  

Maybe improvements in the minds of some staff are due to the placebo effect – based on being unable to accept that all the hard work adds nothing?

Can you think of other reasons that might confound the cartel’s view?

Go and test them!  Can you remember real science before the age of compliance?

If laboratory experiments in which variables are controlled and external everyday evidence are in alignment, then it is said that there is coherence.

Customers don’t notice the assurance of compliance as an improvement in quality.  This is not what they care about.  The variables of accreditation are in ongoing flux and therefore difficult to control.  That’s why the inspectors have to assure you that, whatever changes are occurring, there is always quality you can have confidence in.

The assessment tools are out there.  Test accreditation for effectiveness before funding the inspection cartel.  

This entry was posted in Cartel, Laboratory medicine, Management, Medicine, Philosophy, Questions, Science and tagged , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s