Validation and verification guesswork

Dr Bignardi has done a lot of work with validation and verification required by ISO 15189 for just one laboratory device.

Validation and verification of automated urine particle analysers.   http://jcp.bmj.com.sci-hub.io/lookup/doi/10.1136/jclinpath-2016-203958.

He reports it more simply in the Royal College of Pathologists Bulletin,

v&v

Note: the standards are lacking in detail to give free reign the imagination of UKAS inspectors.

As we have explained previously, the ISO managements standards are designed with a hypocritical vagueness because of the symbiotic relationship they have with the inspection side of the quality cartel.

Dr Bignardi has tried to make it more simple and sensible.  Professionals rise to the challenge of complying.  Will this approach work with all the non-standardised inspectors?

It remains to be seen whether the years of work in this one aspect of meeting the ISO standard will make much difference to patient outcomes.

 

Advertisements
Posted in Laboratory medicine, Medicine | Tagged , , | Leave a comment

Resistance from academia to Chartered Scientist registration popular in the NHS

Another bureaucratic body seeks to expand its membership by coercion rather than proven virtue:

Drive to register all UK scientists: benchmark or bureaucracy?

Critics call proposal for world-first professional recognition system ‘demented’

April 20, 2017 https://www.timeshighereducation.com/news/drive-register-all-uk-scientists-benchmark-or-bureaucracy

A drive has been launched to make the UK the world’s first country where all scientists are professionally recognised, to help benchmark standards in academia.

The Science Council, which has previously worked closely with researchers from industry, wants more academics and university technicians to become registered or achieve chartered status.

The membership organisation argues that, at a time when there is growing concern about the reproducibility of scientific evidence, professional registration will help to make academics’ research credible, as they will be signing up to a code of professional ethics.

But university-based researchers have spoken out against the idea. One said it was “utterly demented” and could “regulate creativity”, and others questioned the scheme’s usefulness and potential bureaucratic burden.

While other countries have forms of registration for specific disciplines, such as chemistry or geology, the council said that the UK is the first country to look at registering scientists as one group.

The Science Council, a membership organisation for professional bodies and learned societies in science, currently has no power to enforce registration upon scientists. It sets the standards for professional registration of scientists and science technicians and licenses its members to accredit their individual members.

However, the organisation is working with the research councils, which are exploring a registration requirement for science technicians. It is also working with the Government Science and Engineering profession – the professional community for scientists and engineers in the Civil Service, which provides evidence for policy – to roll out registration and raise the profile of the process with key ministers.

Becoming a registered or chartered scientist has previously been popular in industry, and the Science Council hopes to extend this to academia.

The council’s new chair, David Croisdale-Appleby, who is also a visiting professor at Durham University Business School, told Times Higher Education that the introduction of tuition fees and the advent of the teaching excellence framework in England mean that measures of scientific career progression are changing.

“It won’t be enough that an academic is able to get published; she or he will have to also demonstrate that they are an able professional, with skills and experience in applying them that registration publicly recognises,” he said.

He added that “registration is an external mark of competence awarded and recognised outside your institution”, which will help academics hoping to move between universities and industry. “Academic pedigree alone will not be enough to secure jobs in the future,” he said.

But Tony Barrett, director of the Wolfson Centre for Organic Chemistry in Medical Science at Imperial College London, was not convinced of the idea’s merits.

“This is an utterly demented idea by some functionary or functionaries who have far too much spare time, little imagination, and the desire to thwart the progress of science and regulate creativity,” he said. “If I had wanted chartered status, I would have become an accountant or surveyor.”

In order to become registered or chartered, scientists have to apply in writing outlining their skills and experience and provide evidence to the Science Council or a licensed professional body, such as the Institute of Physics or the Royal Society of Chemistry, for example. Some bodies require a face-to-face assessment as well.

Professor Barrett said that no scientists at any stage of their career would benefit from “yet another layer of pointless paperwork”. But he added that the idea could appeal to “third rate scientists” who may see it as a way “to overcome feelings of inadequacy or ineffectiveness”.

Philip Moriarty, professor of physics at the University of Nottingham, said that the key advantage of the scheme was that it would provide “much needed recognition” for technicians. “They’re the lifeblood of universities but too often this is forgotten,” he said.

The move could also “potentially lead to more rigorous standards with regard to publication ethics”, he said. But he added that it was not clear how this could be enforced.

Professor Moriarty added that the process could potentially involve “a massive bureaucratic workload”.

“I’d be concerned that some generic, jargon-ridden commitment to the principles of the scheme will need to be written up and evidence provided to demonstrate that those principles have been followed,” he said.

David Fernig, professor of biological chemistry at the University of Liverpool, said that he could not see how it would be useful for academia and doubted that registering all scientists would be achievable.

He added that the idea of chartered scientists “goes against the growing trend of ‘citizen scientists’ in their many different guises”.

“The status would most likely be used as a lever to protect the few against criticism. Critical thinking is in short enough supply already,” Professor Fernig said.

holly.else@timeshighereducation.com

Reader’s comments (1)

#1 Submitted by Jaswinder007 on April 20, 2017 – 10:15am

Great, who will pay for the registration? As in case I am SMIEEE (paying £150 per year), FHEA (luckily free). I have not applied for CENg because I cannot afford another £100 per year, not that I won’t be granted CEng if i apply because I do qualify. Where do we stop and what do we want to prove…?

(emphasis added)

 

Posted in Introduction | Leave a comment

EU 2017/625 recognizes expense of ISO accreditation. But still requires it.

Regulation (EU) 2017/625 of the European Parliament and of the Council of 15 March 2017 on official controls and other official activities performed to ensure the application of food and feed law, rules on animal health and welfare, plant health and plant protection products, amending Regulations (EC) No 999/2001, (EC) No 396/2005, (EC) No 1069/2009, (EC) No 1107/2009, (EU) No 1151/2012, (EU) No 652/2014, (EU) 2016/429 and (EU) 2016/2031 of the European Parliament and of the Council, Council Regulations (EC) No 1/2005 and (EC) No 1099/2009 and Council Directives 98/58/EC, 1999/74/EC, 2007/43/EC, 2008/119/EC and 2008/120/EC, and repealing Regulations (EC) No 854/2004 and (EC) No 882/2004 of the European Parliament and of the Council, Council Directives 89/608/EEC, 89/662/EEC, 90/425/EEC, 91/496/EEC, 96/23/EC, 96/93/EC and 97/78/EC and Council Decision 92/438/EEC (Official Controls Regulation)Text with EEA relevance.

recognizes that accrediting labs to ISO 17025 is costly,

(50)

Laboratories designated by the competent authorities to carry out analyses, tests and diagnoses on samples taken in the context of official controls and other official activities should possess the expertise, equipment, infrastructure and staff to carry out such tasks to the highest standards. To ensure sound and reliable results, those laboratories should be accredited for the use of these methods according to standard EN ISO/IEC 17025 on ‘General requirements for the competence of testing and calibration laboratories’. The accreditation should be delivered by a national accreditation body operating in accordance with Regulation (EC) No 765/2008 of the European Parliament and of the Council (16).

(51)

While accreditation is the instrument of choice to ensure high performance by official laboratories, it is also a complex and costly process, which would result in a disproportionate burden for the laboratory in cases where the method of laboratory analysis, test or diagnosis is particularly simple to perform and does not require specialised procedures or equipment, as is the case for the detection of Trichinella in the context of the inspection and, under certain conditions, in cases where the laboratory only carries out analyses, tests or diagnoses in the context of other official activities and not of official controls.

(52)

In order to ensure the flexibility and proportionality of the approach, in particular for animal health or plant health laboratories, provision should be made for the adoption of derogations aimed at allowing certain laboratories not to be accredited for all the methods they use. That happens in particular where validated methods for detecting particular pests of plants are not available. Moreover, accreditation of a laboratory for all the methods that it should use as an official laboratory might not be immediately available in cases where new or recently modified methods are to be used, in cases of emerging risks or in emergency situations. Under certain conditions, official laboratories should therefore be allowed to carry out analyses, tests and diagnoses for the competent authorities before they obtain the relevant accreditation.

(emphasis added)

It’s good to see the flow of revenue to waste and the inspection industry is now recognized.  Maybe in future years it will be understood that inspection does not guarantee quality and that no study has been done to check.  Oops!

Or maybe not, because revealing that EU officials don’t know the effectiveness of accreditation, or whether results from “official” labs are always perfect, would undermine several bureaucratic regimes.

Paragraph 52 also recognizes the slowness of the ISO and accreditation bodies to respond to new scientific developments.  It allows labs to investigate new hazards without accreditation for a year or so.  So will accreditation later make a difference to the quality of results?

Posted in Bureaucracy, Laboratory medicine, Science | Tagged , | Leave a comment

ISO/IEC quality coercion doesn’t work for software either

ISO 9000 doesn’t create quality for general management and ISO 17025 and 15189 don’t work for labs.  The authors below “conclude that ISO/IEC 9126 is not suitable for measuring design quality of software products.”

The use and usefulness of the ISO/IEC 9126 quality standard

Hiyam Al-Kilidar, Karl Cox, Barbara A. Kitchenham

This paper reports an evaluation the utility of ISO/IEC 9126. ISO/IEC 9126 is an international standard intended to ensure the quality of all software-intensive products including safety-critical systems where lives are at risk if software components fail. Our evaluation exercise arose from an experiment that required a quality assessment of outputs of the design process. Although ISO/IEC 9126 is intended to support evaluation of intermediate software products, both the experimental subjects (158 final year computer science and engineering student) and experimenters found the standard was ambiguous in meaning, incomplete with respect to quality characteristics and overlapping with respect to measured properties. We conclude that ISO/IEC 9126 is not suitable for measuring design quality of software products. This casts serious doubts as to the validity of the standard as a whole.

DOI: 10.1109/ISESE.2005.1541821
Posted in Computing | Tagged | Leave a comment

Exciting developments in healthcare

UKAS says Brexit will require more accreditation.  HM Government seems inclined to believe this. 

The Royal College of Physicians (RCP), NHS England, the Institute of Physics and Engineering in Medicine (IPEM), the Royal College of Radiologists (RCR) and the Society and College of Radiographers (SCoR) are the latest organisations of healthcare professionals to give themselves over to supporting accreditation by UKAS.

You can read these stories from UKAS here.

Are the profit-hungry US companies that want to use the NHS brand going to continue to pay for inspections by the quality cartel?

 

Posted in Cartel, Laboratory medicine, Medicine, NHS, Politics | Tagged , , , | Leave a comment

The Great NHS Heist

The NHS is entering the final phase of it’s destruction. In it’s place will be an American style private insurance based system. Successive Governments over a number of decades have laid down the key steps for the deformation disguised as ‘reorganisation’.

 

In our film “Sell Off “ we explained how the NHS had been undermined but things have moved on rapidly and there is an urgent need to decode and expose where we are headed.

The biggest obstacle to wider public awareness to the scale of the privatisation of the NHS has been a docile or even complicit mainstream media, choosing to echo Government propaganda over scrutiny and keeping the record straight. They simply have not held the our politicians to account.

It is vital that the public are given the opportunity to know the facts of what is happening with our NHS.

We are producing “The Great NHS Heist” documentary in order to provide the public with information that has been obscured and played down.

 

 

Posted in Economics, Medicine, NHS | Tagged , , , | Leave a comment

Evaluations, validations and verifications of diagnostic tests – except ISO accreditation

 

“Where two kits have equivalent performance characteristics, the one which is easier to use, cheaper, faster or requires a more easily obtainable sample might be preferred.”   UK Standards for Microbiology Investigations Q 1: Evaluations, validations and verifications of diagnostic tests, p9.

This guidance document describes the work necessary for the evaluation, validation and verification of laboratory tests.  It’s all good stuff, heavy and demanding.  Such thinking pre-existed the ISO management standards and UKAS.  But before accreditation, all of it wasn’t applied in every lab, all the time.

Maybe all previous testing must be suspect.  Maybe it was quite sufficient for specialist labs to do this work and for other labs to rely on their assurances.  Maybe great science doesn’t need it.  Who knows?

Now UKAS has to inspect every lab’s validation and verification of every variation in every test method.

It’s not putting labs out of service, so we may assume this extra work is making little difference to test results.  Internal quality control and external quality assurance would give labs an idea about that.

You’ll have to search elsewhere if you want to know how much difference this approach makes to the usefulness of laboratory testing.

Searching, you will find almost none of this laborious and necessary work applied to the inspection of ISO laboratory standards.

Here is a useful document.  Figure out how to apply it to UKAS inspections themselves.  Publish the results.

For all we know, millons are being wasted on screening labs for “quality” that actually makes no difference.  You’ll find a little evidence for that, but nobody cares.

It’s an equality issue – laboratory tests must always be validated.  Inspections of the ISO standards themselves must never be validated.

“Where two kits have equivalent performance characteristics, the one which is easier to use, cheaper, faster or requires a more easily obtainable sample might be preferred.”p9.

 

Posted in Forensic science, Laboratory medicine, Science | Tagged , , , , | Leave a comment

Spinal Manipulation for Back and Neck Pain: Does It Work?

Two American traditions of spinal manipulation.  Osteopathy, developed by a physician and surgeon.  Chiropractic, developed by a magnet healing quack who may have modified osteopathy or got ideas from his spirit guides.  Laird Harrison explains in the Medscape article, Spinal Manipulation for Back and Neck Pain: Does It Work?

In the comments that follow, Dr Mark Crislip has critically analysed claims in the article that such practices work.

Pubmed shows a gradually increasing number of low-quality studies supposedly validating the effects of ISO accreditation.  Just like spinal manipulation and the shift in wealth derived from it, the evidence for accreditation “keeps growing and growing”.  As UKAS spreads the ISO disease through laboratories and hospitals, this is the sort of analysis we need doctors to be doing for the claims of accreditation:

Dr. Mark Crislip,  Infectious Diseases
Feb 17, 2017

An excerpt from  Spinal Manipulation for Back and Neck Pain: Does It Work? Annotated.  https://sciencebasedmedicine.org/spinal-manipulation-for-back-and-neck-pain-does-it-work-annotated/

The article starts with:

In her office at McMaster University in Toronto, Anita Gross, MSc, has logged paper after paper showing that spinal manipulation can help control neck pain. “The evidence keeps growing and growing,” she says.

I have a eponymous law on RationalWiki. Another law of pseudo-medicine, eponymous 2, states that when a pseudo-medical intervention says there is growing evidence, there isn’t. It is a corollary to Betteridge’s law of headlines.

Well, perhaps the growing evidence is more of a Kellyanne Conway statement, an alternative fact, because while it is true that there is growing evidence, it is usually terrible evidence.

The question is whether an intervention is better than a sham or placebo intervention. To really know if an intervention is effective, especially an intervention that is not based in the known reality of anatomy, physiology, and biochemistry, you want to do placebo/sham controlled studies, preferably blind.

Let’s check out the ‘growing’ literature of chiropractic manipulation compared with sham chiropractic for neck pain.

And if you think there is no valid sham chiropractic manipulation, think again.

There is “Short-term usual chiropractic care for spinal pain: a randomized controlled trial” from 2013:

Short-term chiropractic treatment was superior to sham; however, treatment effects were not clinically important.

And it was those who knew they were getting treatment who had the most improvement. A classic placebo effect.

That’s it for manipulation vs. sham manipulation for neck pain. But that is growth. From zero to one.

Manipulation vs. sham manipulation for back pain? There is “Spinal Manipulation A Systematic Review of Sham-Controlled, Double-Blind, Randomized Clinical Trials” which included two studies of back pain. They found:

The most rigorous of these studies suggest that spinal manipulation is not associated with clinically-relevant specific therapeutic effects.

Or the recent review “The effectiveness of complementary manual therapies for pregnancy-related back and pelvic pain: A systematic review with meta-analysis” where:

Meta-analyses found positive effects for manual therapy on pain intensity when compared to usual care and relaxation but not when compared to sham interventions.

Here is the classic interpretation: When an intervention is equal to sham/placebo, it is not effective; the intervention does not work. It is why, for example, we no longer do mammary artery ligation for angina.

Or “Spinal Manipulative Therapy for Chronic Lower Back Pain in Older Veterans: A Prospective, Randomized, Placebo-Controlled Trial.”

The SMT did not result in greater improvement in pain when compared to our sham intervention; however, SMT did demonstrate a slightly greater improvement in disability at 12 weeks. The fact that patients in both groups showed improvements suggests the presence of a nonspecific therapeutic effect.

Again. Not better than sham equals no efficacy. And “nonspecific therapeutic effect” is code for placebo effect.

That’s it for the growing literature of clinical trials with meaningful methodologies, sham controlled studies, that I

found on Pubmed. There are numerous ‘A + B versus B’ studies, but they tell you nothing.

So the literature that is growing and growing is less than impressive in numbers or outcomes.

On to the second paragraph:

Gross, a physiotherapist and associate professor of rehabilitation science, helped write a 2015 Cochrane review of the literature and is already at work on updating that paper.

It implies, and maybe this is just me, that the Cochrane review is favorable for manipulation, since it follows the growing and growing quote. Just what does that review say?

No high-quality evidence was found, so uncertainty about the effectiveness of mobilisation or manipulation for neck pain remains.

Color me unimpressed.

The third paragraph?

Mounting evidence also supports spinal manipulation for low back pain, says Roger Chou, MD, professor of medicine at Oregon Health & Science University in Portland, Oregon, who led a similar review for the Agency for Healthcare Research and Quality last year.

And that similar review?

These include exercise, yoga, and tai chi; various psychological therapies; multidisciplinary rehabilitation; acupuncture; spinal manipulation (vs. an inert treatment); and low-level laser therapy. Effects were small to moderate in magnitude.

And they point to a table.

In that table there is spinal manipulation vs. inert treatment (7 trials) which had “small” effects BUT spinal manipulation vs. sham manipulation (1 SR (3 RCTs) 1 RCT), the gold standard for showing efficacy, had…

*…pause for dramatic effect while we decide which baker has to leave the tent…*

No effect.

Let me repeat: manipulation and sham manipulation are equal. That means it doesn’t work.

And that begs the question. You have back pain. I am going to send you for manipulation, but how do I order the manipulation that is vs an inert treatment and avoid the manipulation that is vs sham because it is the former works and the latter that does not. I can do that how? Puh-lease.

Kellyanne, Kellyanne, did it rain at the inauguration? Were the crowds yuuuuge? Was it the greatest Electoral College victory ever? Does chiropractic work better than sham? Is supportive literature growing and growing? Do these:

findings counter decades of accusations of quackery mounted against healers who massage or manipulate patients’ muscles or joints[?]

No. It is, at least in its chiropractic form, still quackery. But more on that later. But it does support the accusations that manipulation does nothing.

 

 

Posted in Medicine, Science | Tagged , , , , , , | Leave a comment

Disgruntled rogue scientists, but accredited

Uncertainty of measurement or just uncertainty?

“Rogue scientists” disgruntled with management at Randox forensic services allegedly doctored samples.  Unsurprisingly this led to hundreds of false convictions and other major injustices.  Here is the Daily Mail’s account.

Maybe privitising the forensic service wasn’t so smart.

Naturally, services so important to justice are accredited by UKAS to ISO 17025.  Here is Randox’s UKAS schedule of accreditation to prove it.

We pointed out previously problems with UKAS accrediting labs that issued false results.

And don’t forget that, since it can render up inspection fees, gold counterfeiting can be accredited.

Did Randox management look to UKAS to sort out their quality problem?  No, more sensibly they called the police to investigate this criminal activity.  Arrests have been made and the full story remains to be told after the investigation.

Randox management had more confidence in the police than the quality inspectors.

What is unlikely to be made clear is how the accreditation of falsified tests will be dealt with.  Will UKAS withdraw accreditation from Randox and sacrifice their inspection fees for the sake of probity?

Here is where you can check on the latest sanctions levied by UKAS.  Do you think Randox will appear?

Will the courts now wake up to the scam that is accreditation?

UPDATE 21 February 2017: Randox have issued a statement, Gross inaccuracies in media coverage of RTS, on the criminal act that is being investigated in their business.

Posted in Forensic science, Laboratory medicine, Law, Management | Tagged , , , , , | Leave a comment

Failing to repay

As university educations become impossibly expensive for ordinary people, Gary North advises,

“Here is a general rule. Any barrier to entry into any field that cannot be legally penetrated by means of a competitive examination is there for only one reason: to line the pockets of the people who have created the barrier to entry. Legal barriers to entry are marks of an unfree society. Politicians have abolished individual liberty for the sake of lining the pockets of members of cartels. They are using coercion to steal from excluded victims.”

As universities, so ISO-accredited organisations.

External quality assurance and user recommendations are the best indicators of the technical competence that ISO 9000 is inadequate to assure.  These are the equivalent of the difficult entrance exam that served some organisations well for centuries.

Accreditation is open to all who pay the inspection cartel and put up with its interminable, petty bureaucracy.

Coil of rope lies in ever-decreasing circles - Stock Image

 

Posted in Bureaucracy, Cartel, Economics, Education, History, Laboratory medicine | Tagged , , , | Leave a comment