British Ad Watchdog Agency Contradicts Own Rulings to Defend BSI

In 1997 the Advertising Standards Authority Ltd. (ASA) ruled against the BSI for making advertising claims it couldn’t support, chiefly about the cost-effectiveness of the ISO 9000 standard.

BSI have continued to print porkies about it ever since.

nbn-porkies-450x320

The Oxebridge blog has reported that in 2018 the ASA ev…ent…u…al…ly managed to reach a judgement about the latest complaint that BSI never stopped making false claims.

This time the ASA found the BSI was doing nothing wrong.  It’s now OK to make the claims they need to to keep the inspection cartel’s business on the rip.

Amazing!

The ASA’s inconsistent and plainly wrong decision about the BSI’s unsupported claims illustrates the decline in British bureaucracy in recent decades.  The ASA also didn’t uphold complaints against any of the ten most complained-about adverts.   Maybe they’re out of touch with the population they serve.

https://www.bbc.co.uk/programmes/b00wck32/contact

Advertisements
Posted in Bureaucracy, Law | Tagged , , , , , , | Leave a comment

The self-made women who created the Myers-Briggs

Nature carries an interesting review of the Myers-Briggs nonsense.  Popularised by the OSS and carried forward by corporate America.

Maybe one day the ISO accreditation racket will also be debunked.

Click through…

The self-made women who created the Myers–Briggs

S. Alexander Haslam enjoys the tale of how a questionable personality quiz went global.
Posted in Management, Psychology | Tagged , , , | Leave a comment

The most common nonconformities encountered during the assessments of medical laboratories in Hong Kong using ISO 15189 as accreditation criteria

This paper by Ho and Ho preceded that of Wilson et al. which examined ISO 17025 and was missed by these authors.  It used a similar approach to investigating the nature and number of ISO 15189 non-conformances over time.

There were some measurements in common between the two studies but Ho and Ho drew contrary conclusions.

Compare data from the two papers.

Wilson et al:

Ho and Ho:
Table 2. Most common nonconformities reported against management and technical requirements of ISO 15189 in two study periods (from 2004 to 2006 and in 2009).

 

Direct comparison is not straightforward because the two groups classified non-conformances into different categories, Ho: management/technical; Wilson: ISO/lab.

By assessing accreditation according to its own claims, Ho and Ho believed it to have brought improvements.  By assessing it to a standard outside itself, Wilson et al. demonstrated the opposite.

 

Internal assessment by Ho and Ho: “Similar nonconformities were reported in the two study periods though the frequency encountered decreased. The significant decrease in number of significant nonconformities encountered in the same group of laboratories in the two periods substantiated that ISO15189 contributed to quality improvement of accredited laboratories.”

External assessment by Wilson et al: “Most noncompliances arose solely from clauses in the standard and would not affect users. No effect was likely from 91% of these. Fewer than 1% of noncompliances were likely to have consequences for the validity of results or quality of service. The ISO system of compliance auditing has the performance characteristics of a poor screening test.”

Which shows that science is not necessarily objective; it may just be a display of presuppositions.  The story that controls the mind controls the data.

 

Original article:

Bella Ho*, Eric Ho. The most common nonconformities encountered during the assessments of medical laboratories in Hong Kong using ISO 15189 as accreditation criteria. Biochemia Medica 2012;22(2):247-57. http://dx.doi.org/10.11613/BM.2012.027
Hong Kong Accreditation Service, Innovation and Technology Commission, HKSAR Government, Hong Kong
*Corresponding author: bellaho@itc.gov.hk
 
Abstract
 
Introduction: ISO 15189 was a new standard published in 2003 for accrediting medical laboratories. We believe that some requirements of the ISO 15189 standard are especially difficult to meet for majority of laboratories. The aim of this article was to present the frequency of nonconformities to requirements of the ISO 15189 accreditation standard, encountered during the assessments of medical laboratories in Hong Kong, during 2004 to 2009.
Materials and methods: Nonconformities reported in assessments based on ISO 15189 were analyzed in two periods – from 2004 to 2006 and in 2009. They are categorized according to the ISO 15189 clause numbers. The performance of 27 laboratories initially assessed between 2004 and 2006
was compared to their performance in the second reassessment in 2009.
Results: For management requirements, nonconformities were most frequently reported against quality management system, quality and technical records and document control; whereas for technical requirements, they were reported against examination procedures, equipment, and assuring quality of examination procedures. There was no major difference in types of common nonconformities reported in the two study periods. The total number of nonconformities reported in the second reassessment of 27 laboratories in 2009 was almost halved compared to their initial assessments. The number of significant nonconformities per laboratory significantly decreased (P = 0.023).
Conclusions: Similar nonconformities were reported in the two study periods though the frequency encountered decreased. The significant decrease in number of significant nonconformities encountered in the same group of laboratories in the two periods substantiated that ISO15189 contributed to quality improvement of accredited laboratories.
Key words: accreditation; quality improvement; healthcare quality assurance; clinical pathology
Received: December 15, 2011                                                                                                                                            Accepted: April 19, 2012
 
 
Introduction
Before the publication of the international standard ISO 15189 for accreditation of medical laboratories, there was no official accreditation system for accrediting medical laboratories in Hong Kong. There are over 100 medical laboratories in Hong Kong including a large number of private laboratories as well as many in the public hospitals administered under the Hospital Authority, nevertheless, medical laboratories in Hong Kong are not required to be registered with the health authority or accredited for their operation. Standards of many medical laboratories in Hong Kong are of high quality and are well recognized by the international medical community through the regular participation of their staff in international conferences and publications in peer reviewed journals by the medical professionals of Hong Kong.
Hong Kong Accreditation Service (HKAS) is the only government run accreditation body in the economy but it did not offer accreditation for medical testing laboratories before 2004. HKAS has over 20 years of experience in providing accreditation to the non-medical testing laboratories and calibration laboratories and is a signatory to the mutual recognition arrangement of International Laboratory Accreditation Cooperation (ILAC) and Asia Pacific Laboratory Accreditation Cooperation (APLAC). Before the introduction of the accreditation programme for medical laboratories in 2004, laboratories were accredited using ISO/IEC 17025 and these did not include the medical testing laboratories. Some large hospital laboratories that enjoy international reputation sought professional recognition of their competence through accreditation by overseas accreditation bodies such as College of American Pathologists (CAP) or National Australian Testing Authorities (NATA) that have a long history of accrediting medical laboratories. Internationally, ISO/IEC 17025 has not been well accepted by the medical community as suitable for accrediting medical laboratories and so the ISO technical committee TC212 was given the task to draft a standard particularly for medical laboratories.Since ISO15189 first published in 2003, it has been widely used as the accreditation standard for medical laboratories internationally by various accreditation bodies (1-5). In 2004, the laboratory accreditation programme of HKAS was extended to cover medical testing laboratories using the new accreditation standard soon after its publication.
At that time accreditation is a new concept to most medical testing laboratories in Hong Kong. Most laboratories did not operate a management system before 2004. Medical laboratory practitioners were not accustomed to writing down their operations and the concept of record traceability was not fully understood. Internal auditing was unheard of. Though internal quality controls were run in routine testing and external quality assurance programmes were participated by most laboratories, the practice and frequency of participation varied greatly from laboratory to laboratory. Evaluation of new methods and new autoanalysers were carried out to various extents by large laboratories, while for small laboratories, new equipment at times were used immediately after installation without any method validation. Since the launching of the accreditation programme, laboratory personnel were trained in ISO 15189, and throughout the last few years, there had been obvious improvements in meeting the ISO 15189 requirements.
ISO 15189 is an accreditation standard prepared particularly for the medical laboratories. This standard is expected to help medical laboratories in raising the quality of services provided.
The aim of this study was to assess whether ISO 15189 contributed to the quality improvement of medical laboratories in Hong Kong and the laboratories had found particular difficulties to comply with which requirements of ISO 15189. The frequency of nonconformities (NCs) to requirements of the ISO 15189 accreditation standard, encountered during the assessments of medical laboratories in Hong Kong, during 2004 to 2009, was analyzed. This would show which are the more difficult requirements found by most laboratories and shorten the learning path of the medical laboratories. By looking at the change in number of NCs encountered in subsequent assessments of the same group of laboratories, the effectiveness of this new standard in improving the quality of accredited laboratory service is reviewed.
The types and number of NCs reported by the assessment teams in assessments of medical laboratories accrediting to ISO 15189 were reviewed in two periods and their performances were compared. The medical accreditation programme of HKAS started in 2004, the number of application for accreditation went slow in the first two years as the laboratories got themselves prepared but sharply increased in the third and fourth year. Data from 73 assessments in 2009 were compared to data obtained in an earlier period (2004-2006) when the accreditation programme was at its early stage. Hong Kong is an ideal place to demonstrate the usefulness of accreditation to ISO 15189 in raising the quality of services provided as accreditation is not mandatory in our community and it is not related to insurance claims. Improvement and changes are all self-initiated. Because accreditation is a new concept to most laboratories, they are excited to set up a management system that could meet international standards and they eagerly improve themselves in order to be accredited and recognized.
Materials and methods
Study design and data collection
The data for this retrospective observational study were retrieved from 2004 to 2009. From 2004 to 2006, HKAS had conducted assessments of 40 laboratories of different pathology disciplines using ISO 15189:2003 (6). Number of NCs reported were counted and categorized into “significant” or “minor” grading or reported as “recommendations” in each assessment. The number of NCs in each category reported during the assessments was grouped according to the clause number of ISO 15189 and their frequency of occurrence was presented as percentage of total no. of NCs reported for the laboratories assessed in the study period. Performance of the laboratories was presented in terms of average number of NCs per laboratory. Data collected from 2004 to 2006 represented performance of laboratories undergoing the first few assessments and are at learning phase of the ISO 15189 requirements.
In 2009, the number of laboratories assessed against requirements of ISO 15189:2007 (7) increased to 73. Similar types of data (total no. of NCs reported; the frequency of occurrence of NCs to requirements of ISO 15189 in terms of counts and percentages; and average number of NCs per laboratory for each category) were collected from these 73 laboratories assessed in 2009. Among these 73 laboratories, 61 of them had been accredited before 2008 (included those laboratories being initially assessed from 2004 to 2006), and were either having their surveillance visits or reassessments in 2009, whereas 12 laboratories had their initial assessments in the year. Among the 61 accredited laboratories, 27 were having their second reassessments in 2009, hence data collected from these 27 laboratories represented a more mature phase of implementing the management system established in accordance with the ISO 15189 requirements after being accredited for a few years. The performance of these 27 laboratories, in terms of number of NCs per laboratory, in their initial assessment was compared with their performance in the second reassessment in 2009. These figures would indicate whether accredited laboratories had any improvement in the quality of services provided after accreditation.
Nonconformities included in this study were either graded as significant or minor
A significant nonconformity is one which has serious adverse effect on the validity of an activity, its results or the competence of the organization or a deliberate violation of HKAS regulations for accreditation. For instance, when it is a widespread phenomenon that operating procedures are not documented and/or staff do not perform their tests in accordance with the documented procedures, a significant NC would be reported against clause 4.2 (quality management system). When a method was not evaluated to the extent as necessary to demonstrate its suitability for the intended use, a significant NC would be reported against clause 5.5. Significant NCs have to be corrected with documented evidence before accreditation is granted or reaffirmed.
NCs with no serious adverse effect on the validity of the activity, its results or the competence of the organization can be classified as minor. For instance, equipment was calibrated but the calibration label had not been updated with the calibration status or the laboratory failed to monitor the quality indicator established.Minor NCs also have to be corrected; and action plans have to be provided within one month after assessment, but the effectiveness of actions taken would be checked in the next assessment visit. Recommendations are those that are mainly good laboratory practices and laboratories may select to accept or reject the recommendations. For instance, phone reporting of critical results was found recorded on the back of the test request form, a more systematic way to record details of the reported results and the parties involved in the communication would be recommended for better record traceability.
Data collected from the periods of time mentioned above were analyzed according to the number and nature of NCs reported during the assessment visits, and the frequency of occurrence of NCs against each ISO 15189 requirement. In addition, the number of NCs reported per laboratory for the 27 laboratories undergoing their second reassessments was compared with the number reported in their initial assessments.
Statistical analysis
The Kolmogorov-Smirnov test was used to assess the normality of distribution of investigated parameters (i.e. number of NCs per laboratory in each grading reported in the initial and second reassessment of the 27 laboratories). Apart from the number of significant NC reported from the initial assessments of the 27 laboratories assessed in between 2004 and 2006 was not distributed normally, the other parameters were distributed normally. Selection of appropriate statistical methods was based on a recent publication by Simundic AM (8). Wilcoxon matched-pairs signed rank test was used to analyze if there was any significant differences between the performance of the 27 laboratories in terms of no. of NCs per laboratory encountered in their initial assessments and their second reassessments (8). P values < 0.05 were considered statistically significant. Statistical analysis was done using GraphPad Prism version 5.0 (GraphPad Software Inc., California, USA).
Results
Common nonconformities found against management requirements
The overall distribution of NCs per laboratory against different requirements of ISO 15189 found in the assessments from 2004 to 2006 was shown in Figure 1, whereas those found in the assessments in 2009 were shown in Figure 2. Data shown in Figure 1 were collected from all 40 laboratories assessed from 2004 till 2006, including data of the 27 laboratories undergoing initial assessments from 2004 to 2006 while those in Figure 2 were collected from 61 laboratories undergoing reassessments or surveillance visits in 2009. Since a comparison of the total number and distribution of nonconformities against each clause in Figure 1 and 2 would provide a general picture on whether laboratories had improvement after accreditation, data from the 12 laboratories that had initial assessments in 2009 were not included in Figure 2.
Figure 1. Distribution of nonconformities. S – signifi cant nonconformities; M – minor nonconformities reported against management and technical requirements of ISO 15189:2003 in assessment of 40 laboratories conducted from 2004 to 2006.
 
Figure 2. Distribution of nonconformities. S – signifi cant nonconformities; M – minor nonconformities reported against management and technical requirements of ISO 15189:2007 in reassessment and surveillance visits of 61 laboratories conducted in 2009.
Regardless of the accreditation history of the laboratories, the average number of NCs identified per laboratory reduced from 15.4 to 11.1 in the two periods (Table 1). For management requirements, the most common NCs were found against clauses 4.2 – Quality management system, 4.3 – Document control, 4.6 – External services and supplies and 4.13 – Quality and technical records in the assessments conducted in 2009, all were more or less related to documentation. Apart from an increase in number of NCs reported against clause 4.6 in 2009, similar NCs had also been reported against these management requirements in the earlier assessments from 2004 to 2006 (Table 2). Table 2 compared the performance of laboratories against the same requirement clause in the two periods. Those laboratories undergoing their initial assessments in 2009 were separately listed in Table 2. A decrease in number of NCs per laboratory was reported against clause 4.2 and clause 4.13 while a rise was reported against clause 4.6 in 2009 assessments of laboratories accredited before 2008. For the 12 laboratories undergoing their initial assessments in 2009, high numbers of NCs were also reported against these management requirements.
Table 1. Number of nonconformities (NCs) reported in assessments conducted from 2004 to 2006 and in 2009.
Nonconformities against Clause 4.2 – Quality Management System
Clause 4.2.1 stated that “Policies, processes, programmes, procedures and instructions shall be documented and communicated to all relevant personnel. The management shall ensure that the documents are understood and implemented. Laboratories were often found to carry out tests deviated from the documented procedures; and there were often procedures, steps or interpretation not described in the documented procedures. If documented procedures are not followed, this would imply that documents are not well understood and not the same procedures are implemented by all staff. There were also situations when procedures were thought to be well understood by all staff and hence they were not documented, nevertheless, different staff was found to carry out the same procedure with slight variation. Under these circumstances, when procedures are not documented and resulted in inconsistencies, this indicated a lack of communication among staff.
Nonconformities against Clause 4.3 – Document control
Before accreditation, laboratories are not used to having document control. There were posted instructions or notes as reminders to staff for interpretation criteria or for certain essential steps, nevertheless, these are instructions that could affect the quality of test results, they also have to be controlled but were not realized by laboratory staff in the early phase of management system implementation. Obsolete versions of controlled documents were found being used by staff, illustrating that they were still not used to the concept of document control. There were also forms and worksheets that had been used for years that were found to be not controlled. Instructions to patients for sample collection were distributed at the sample collection centers and they were slipped out of the document control system. Occasionally, instructions contained therein were found different from the controlled sample collection manual. Test kit inserts also formed part of the quality documentation system and should be controlled for use. Most laboratories retained test kit inserts for a defined period of time for reference. Nevertheless, there was no record on when that particular version of kit insert was used in the laboratory. This lack of period of use would cause problem in record traceability in case investigation was required for NCs identified. Another common observation was on the requirement to regularly review all documents that formed the management system. Quality documents were recorded as reviewed, but there were obvious errors or obsolete information not updated. Records were sometimes seen to be amended by staff without initial and date. These observations were all related to document control and were usually reported as minor NCs to the laboratories. The improvement in document control was not obvious when comparing laboratory performance in the two periods (0.45 NC per laboratory c.f. 0.48 in Table 2). As the management system becomes mature, it is envisaged that the occurrence of such NCs would decrease, but obviously it takes time for staff to get used to the system and change their quality mindset.
Table 2. Most common nonconformities reported against management and technical requirements of ISO 15189 in two study periods (from 2004 to 2006 and in 2009).
Nonconformities against clause 4.6 – External services and supplies
The number of NCs per laboratory reported against Clause 4.6 increased in 2009 (Table 2) because laboratories were assessed against requirements of ISO 15189:2007 in which the clause 4.6.3 required an inventory record system to include details of lot number, expiry date, date in use, etc. In the 2003 version, this is not a mandatory requirement. Inventory records were found to be incomplete with no recording of the date that a particular lot was put into service. It was a common practice that laboratories recorded date of opening on the box of the test kit, but when the box was discarded after being used up, the record was lost and this could not fulfill the requirement of keeping the in-use date in the inventory record. There were times when evaluation of new lot of reagents were done by parallel testing with the old lot, there was no defined acceptance criteria or the criteria established were inappropriate. The verification protocol was not documented, this resulted in different number of samples used in each evaluation and the results of evaluation were interpreted with slightly different criteria by different staff.
Nonconformities against clause 4.13 – Quality and technical records
The laboratory is required to retain records related to the management system and examination results. These include all the laboratory workbooks or sheets, raw observations, calculations, etc. A common observation is that interpreted results were recorded instead of the raw observations such as scoring, color reactions, fluorescence, actual reactions observed in tubes of different dilutions, etc. Operators could not be identified from the retained records and the supervisor who counterchecked the results was not recorded.
Common nonconformities against the technical requirements
A comparison on the number of NCs reported against the 8 technical requirements for the assessments conducted from 2004 to 2006 and in 2009 showed that NCs were reported against similar areas. Clause 5.3 – Laboratory equipment, clause 5.5 – Examination procedures and clause 5.6 – Assuring quality of examination procedures were consistently the three most common technical areas where NCs were found (Table 2). In general, more number of nonconformities was reported against the technical requirements than the management requirements (Figure 1 and 2).
Nonconformities against clause 5.3 – Laboratory Equipment
NCs were commonly reported against laboratory equipment, nevertheless, these are NCs easily corrected. Equipment used in the laboratory was required to be calibrated against a well calibrated reference equipment to attain metrological traceability. Before the era of accreditation, general equipment used in medical laboratories e.g. thermometers, balances, etc. were often not calibrated. They might have maintenance but were often not properly calibrated. In the early phase of accreditation, laboratories might employ a non-accredited laboratory to calibrate its equipment; or wrongly employ an accredited calibration laboratory to calibrate its equipment without noticing that the required calibration test was not on the laboratory’s accredited scope; or purchase equipment that came with a “calibration” certificate from the manufacturer, believing that it is an acceptable calibration certificate. There were situations where maintenance/calibration was carried out by a contractor, the records of which did not include any information on what parameters had been checked and calibrated. Equipment labels were not updated after calibration or maintenance being conducted. Usually these observations were graded as minor NCs and once being pointed out, laboratories quickly learned.
Nonconformities against clause 5.5 – Examination procedures
The top number of NCs was reported against clause 5.5.2 where laboratories have to validate procedures for confirming that the examination procedures are suitable for the intended use and that the validations shall be as extensive as are necessary to meet the needs in the given application or field of application. New autoanalysers were found putting into service before they were adequately evaluated. The number of patient samples used for evaluation was too few in some studies and the statistical method used for analysis was inappropriate (9) Evaluation reports did not consist of sufficient detailed information on the procedures and types of samples used, and acceptance criteria were not clearly defined. Some manufacturer’s claims were adopted without verification. Not only clause 5.5.2 is the most common requirement that non-conformity is found, NCs reported are usually of significant nature as an inadequately evaluated method could affect the quality of test results.
Another common observation is on the validation of reference intervals. Reference intervals are required to be reviewed regularly. There were cases where manufacturer’s reference intervals were adopted without validation; or previously established biological reference intervals were adapted to the new equipment without validation. Some biological reference intervals had been used for a long time, while the source was unknown; they were applied to the new equipment without any validation.
Nonconformities against clause 5.6 – Assuring quality of examination procedures
Clause 5.6.1 required the laboratory to establish an internal quality control system that verifies the attainment of the intended quality of results, so that mistakes are eliminated. Very often, in chemical pathology and haematology laboratories, Westgard QC rules are used for daily quality control monitoring. However, the documented QC rules were not followed. The most common observation being that when QC rules failed, actions taken by the laboratory to address these quality control failure incidents were not recorded and there was no record whether patient results had been reviewed. Another common observation was that the manufacturer’s given mean and standard deviation of a quality control material was used as the laboratory’s daily quality control range, this range was often too wide and inadequate to monitor the laboratory’s own performance. The control levels used sometimes did not cover the clinical decision level and the frequency of running quality controls was insufficient to monitor the autoanalyser performance. The use of third party quality control materials was always recommended when they were not being used by the laboratories. These observations were more commonly associated with chemical pathology and haematology laboratories.
Laboratories are required to participate in external quality assessment programmes and to handle EQAP samples in the same way as patient samples (10). EQAP samples sometimes were found handled with special treatment and sometimes consensus results were reported, particularly for anatomical pathology. When unsatisfactory results were returned, review was superficial and did not include any actions taken to address the root cause of failure; or observed trends were ignored.
Comparison on performance of laboratories at their initial assessments and their reassessments
In order to have objective evidence on the improvement brought by implementing a management system meeting the ISO 15189 requirements, we compared the number of NCs identified in 27 laboratories during their initial assessments with their second reassessments conducted around 3.5 to 4 years after establishing their management system. The comparison results on the average number of NCs reported per laboratory are presented in Table 3.
Table 3. Comparison of performance of 27 laboratories in initial assessments and in their second reassessments.
Discussion
When comparing assessments conducted in the early days when the accreditation programme was first implemented, i.e. those conducted from 2004 to 2006; with assessments conducted in 2009, it was found that most common NCs were reported against similar management and technical requirements (Figure 1 and Figure 2). For management requirements, the most common areas where NCs were reported were clause 4.2 – quality management system, clause 4.3 – document control, clause 4.6 – external services and supplies, and clause 4.13 – quality and technical records (Table 2). For technical requirements, they were clause 5.3 – equipment, clause 5.5 – examination procedures and clause 5.6 – assuring quality of examination results (Table 2). We believe that the common nonconformities encountered in assessments in Hong Kong are not unique to Hong Kong. Medical laboratories in other countries face similar difficulties in meeting the ISO 15189 requirements (2-4,11). The clause numbers that nonconformities were reported against could vary among different accreditation bodies as very often, more than one ISO 15189 requirements could be referred. For instance, lack of documented procedure could be reported against clause 4.2.1 where documentation of all procedures and instructions are required. Similarly clause 5.5.3 also required all procedures to be documented and be available at the workstation for relevant staff. Where document control problems (clause 4.3) were identified, this could also be related to insufficient training for staff (clause 5.1) or poor implementation of the quality management system leading to misunderstanding among staff (clause 4.2). Hence for similar observations, nonconformities could be categorized into different areas in different countries; this could be related to the training that the assessors received. A direct comparison of figures on nonconformities reported by other accreditation bodies without knowing the details of the findings may not be appropriate. Requirements of ISO 15189 give direction to the components of a good management system that contributes to patient care. The more important is the understanding of the quality concepts behind the requirements and implement continuous improvement.
When laboratories first established a management system according to requirements of ISO 15189, time is required for staff to buy in the idea of accreditation and to let themselves be familiar with the requirements. NCs were reported against issues of document control, use of obsolete documents, or carrying out procedures not as documented. This is understood as time is required for some laboratory staff to change their usual practices; some still do not realize the usefulness of these requirements and consider them as only unnecessary extra work required by the accreditation body. Compliance with the technical requirements also improved as the knowledge of the laboratory personnel increased through exchanges with the assessors, through seminars and contacts with other accredited laboratories. This was indicated from the decrease in the number of NCs reported per laboratory assessed in 2009 when comparing with those assessed from 2004 to 2006 (6.6 vs. 13.5), as well as significant reduction in the number of significant NCs reported in the same group of laboratories undergoing their second reassessments (Table 1 and 3). This is strong evidence that laboratories are improving as they try to comply with the requirements of ISO 15189. High number of NCs reported for the 12 laboratories undergoing initial assessment in 2009 reiterated the fact that accreditation helps to point out the deficiencies in laboratories and show their way for improvement.
Conclusion
ISO 15189 is an accreditation standard developed particularly for medical laboratories. A management system in compliance with requirements of ISO 15189 helps laboratory to improve quality of its service. As evident from the reduction in number of NCs reported in reassessments, improvement in quality of services resulting from accreditation becomes more apparent with time. Though the common types of NCs found in the two study periods are very similar, this did not indicate that laboratory personnel had not learnt from experience as detailed analysis of data collected in 2009 revealed that the number of NCs reported for each laboratory greatly decreased from initial assessments to reassessments/surveillance visits, thus showing that accredited laboratories are improving. Deficiencies of laboratories existed before accreditation, were pointed out in assessments and corrected, this led to general improvement in the quality of services provided. The ISO 15189 standard is contributing to quality improvement.
Acknowledgement
This article only represents personal opinions of the authors and did not represent the opinion of any organization and authority. The authors would like to thank Mr. SS Terence Chan of Hong Kong Accreditation Service (HKAS) to review this manuscript and all HKAS assessors in the field of Medical Testing for their contribution on providing professional judgment and comments for the improvement of our accredited laboratories.
Potential conflict of interest
None declared.
References
 1. Zima T. Accreditation in clinical laboratories. Biochem Med 2010;20:215-20.
 2. Aoyagi T, Kawai T. Validation of the ISO 15189 trial assessment results of clinical laboratories-effects of accreditation and interpretation of ISO 15189. Rinsho Byori (The Japanese Journal of Clinical Pathology) 2006;54:486-93. [Article in Japanese].
 3. Guzel O, Guner EI. ISO 15189 Accreditation: Requirements for quality and competence of medical laboratories, experience of a laboratory I. Clinical Biochemistry 2009;42:274–8.
 4. Gulderen Yanikkaya-Demirel. ISO 15189 accreditation: Requirements for quality and competence of medical laboratories, experience of a laboratory II.Clinical Biochemistry 2009;42:279–83.
 5. Kusum M, Silva P. Quality Standards in Health Laboratories – Implementation in Thailand: A Novel Approach. World Health Organisation Regional Office for South-East Asia New Delhi, 2005.
 6. EN ISO 15189:2003 – Medical laboratories – Particular requirements for quality and competence.
 7. EN ISO 15189:2007 – Medical laboratories – Particular requirements for quality and competence.
 8. Simundic AM. Practical recommendations for statistical analysis and data presentation in Biochemia Medica journal. Biochem Med 2012;22:15-23.
 9. Sonntag O. Quality in the analytical phase. Biochem Med 2010; 20:147-53.
10. Sciacovelli L, Secchiero S, Zardo, L, Plebani M. The role of the External Quality Assessment. Biochem Med 2010;20:160-4.
11. Harper JC, Sengupta S, Vesela K, Thornhill A, Dequeker E, Coonen E, Morris MA. Accreditation of the PGD laboratory.Human Reproduction 2010;25:1051-65.
Posted in Laboratory medicine, Science | Tagged , , , | Leave a comment

Margaret McCartney: A summary of four and a half years of columns in one column

Nothing to do with accreditation, just the poor thinking that lies behind it.  Dr McCartney is critical of inappropriate screening.  ISO accreditation is certainly that.  Some of her points most relevant to accreditation are below.  Click through to the BMJ to read them all.

Margaret McCartney: A summary of four and a half years of columns in one column

BMJ 2018; 362 doi: https://doi.org/10.1136/bmj.k3745 (Published 04 September 2018) Cite this as: BMJ 2018;362:k3745

Posted in Laboratory medicine, Management, Medicine | Tagged , | Leave a comment

When ISO is not ISO

We know how badly the ISO screws up lives and the workplace when it imposes its daft management standards. That doesn’t mean that its product standards are brilliant either.  They’re nothing more than another option in the marketplace and should have to compete as such without marketplace coercion.

DPReview explains how misleading ISO values may lead to bad photographic exposures –  You probably don’t know what ISO means – and that’s a problem.

Posted in Science | Tagged , , | Leave a comment

Nature’s Challenges in irreproducible research

Nature published a special on Challenges in irreproducible research.

Monya Baker wrote on the crisis of reproducibility in research – 1,500 scientists lift the lid on reproducibility.

The figures are grim for the discipline of science in which so many place so much trust.

Here are the reasons given for lack of reproducibility in research:

reproducibility-graphic-online4

Consensus on the importance of causes is limited and opinion probably lacks evidence in many cases.  Many of these issues are beyond the power of ISO 17025 to dictate.  The issues researchers consider the most frequent sources of problems are those accreditation is least able to deal with.

UKAS-sufferers would be astonished that Nature can publish a special on the topic and not deal with accreditation to ISO 17025/15189 as the supposed solution.  Perhaps it isn’t.  The journal does discuss quality assurance which, of course, can be done well without paying the inspection tax.

Researchers in the articles who wish to remain anonymous suggest that adequate quality assurance will add costs in the region of 30-50%, which is similar to accreditation.

Research is certainly disconnected from the world of scientific service delivery and pathology.

The unmeasured claims of the ISO inspection gang to deliver assurance have still not reached academia.  They will slow research progress greatly if they do.

But maybe academics would be tempted to measure the efficacy of accreditation.  The results might turn out to be worse than the confidence in research reproducibility.

The Oxebridge blog will keep you up to date with corruption within the quality cartel.  It’s time a journal did a special on this.

Posted in Practical problems, Science | Tagged , , , , , | Leave a comment

Three rules of groupthink: as global warming, so ISO accreditationism

Christopher Booker discovered Irving Janis’s Victims of Groupthink.  He used it as a framework for his report Global Warming: A Case Study in Groupthink.  We reported James Delingpole’s summary earlier.

https://www.thegwpf.org/content/uploads/2018/02/Groupthink.pdf

Here are Irving’s three rule of groupthink, useful because they also are relevant to ISO accreditationism and those who are deceived by it.

3 The three rules of groupthink

Rule one is that a group of people come to share a common view or belief that in some way is not properly based on reality. They may believe they have all sorts of evidence that confirms that their opinion is right, but their belief cannot ultimately be tested in a way that confirms this beyond doubt. In essence, therefore, it is no more than a shared belief.

Rule two is that, precisely because their shared view cannot be subjected to external proof, they then feel the need to reinforce its authority by elevating it into a ‘consensus’, a word Janis himself emphasised. To those who subscribe to the ‘consensus’, the common belief seems intellectually and morally so self-evident that all right-thinking people must agree with it. The one thing they cannot afford to allow is that anyone, either within their group or outside it, should question or challenge it. Once established, the essence of the belief system must be defended at all costs.

Rule three, in some ways the most revealing of all, is a consequence of that insistence that everyone must support the ‘consensus’. The views of anyone who fails to share it become wholly unacceptable. There cannot be any possibility of dialogue with them. They must be excluded from any further discussion. At best they may just be marginalised and ignored, at worst they must be openly attacked and discredited. Dissent cannot be tolerated.

Janis showed how consistently and fatally these rules operated in each of his examples. Those caught up in the groupthink rigorously excluded anyone putting forward evidence that raised doubts about their ‘consensus’ view. So convinced were they of the rightness of their cause that anyone failing to agree with it was aggressively shut out from the discussion. And in each case, because they refused to consider any evidence that suggested that their two-dimensional ‘consensus’ was not based on a proper appraisal of reality, it eventually led to disaster.

The insights should be also applied to the inspection cartel.  It may be a slightly smaller and more obscure lie than global warming but that is part of what may make it more durable.

Posted in Cartel, Laboratory medicine, Philosophy, Politics, Psychology | Tagged , , , , | Leave a comment

Scott’s parabola gives hope

Scott described the growth and death of fads in surgery:

It’s not so different from the curve based on real data published by Walsh which observed the fate of other management fads:

DOI: http://dx.doi.org/10.1093/intqhc/mzp012
Walshe, K. Pseudoinnovation: the development and spread of healthcare quality improvement methodologies. International Journal for Quality in Health Care 2009; Volume 21, Number 3: pp. 153–159.  DOI: http://dx.doi.org/10.1093/intqhc/mzp012
https://sci-hub.tw/10.1093/intqhc/mzp012

Walsh does not specify whether the slow growth of the accreditation line includes ISO accreditation.  It probably refers to non-ISO schemes since it doesn’t reflect the faltering curve for ISO 9001:

The significant life cycle of other methodologies is usually less than a decade.  Peaking for only a few years has been enough to ensure their decline because of undramatic effects.

The ISO inspection cartel has captured the minds of professional leaders.  Its compulsion to purchase a rehashed document every few years has kept it going longer than other pseudoinnovations.  Managers always love a new fad to waste money on and keep staff stressed.  It saves them from the dreariness of continual improvement or the risks of radical improvement.

Unlike surgery where the effects of innovations become clear and are audited, nobody bothers trying to measure outcomes meaningfully in many other fields.

UK governments boast of how much taxpayers’ money they have spent on the NHS.  They don’t say how much is wasted and few ask.  Money spent does not necessarily relate to benefits.  In an industry with price discovery through profit and loss accounts, managers would wise up or close down.

That’s why we need more academic investigation of why the ISO continues to not die.

 

Posted in History, Management, Medicine, NHS | Tagged , , , , , | Leave a comment

Weak evidence for ISO 9001 not being much good in hospitals

Uptake of ISO 9001 in European hospitals is lower than specialised healthcare accreditation schemes (15%) of 73 hospitals in this study.  So evidence on its effectiveness is weak.

“There is little hard evidence of the impact of these systems on hospitals to justify the amount of time and money spent on organizational assessment, or to choose between available programmes.”

Results “implied that hospitals that were accredited scored higher on composite measures of quality and safety than hospitals that were ISO certified.”

The effect of certification and accreditation on quality management in 4 clinical services in 73 European hospitals.  Charles D. Shaw et al. 2014, International Journal for Quality in Health Care, 26(suppl_1), 100-107, 2014.  https://doi.org/10.1093/intqhc/mzu023

Shaw et al had earlier published evidence in this chart:

Black box – non-ISO accreditation.  Shaded box – ISO 9001 certification. White box – neither.  So these interventions make little difference overall.  The authors say, “This analysis of data from the MARQuIs study suggests that in the samples of hospitals the impact of ISO certification on quality and safety may be less than with hospital accreditation, but it appears that either system is better than no system.”

 

Despite this, for airy-fairy justification of using ISO 9001 in hospitals try,

Why adopt ISO 9001 certification in hospitals? A case study of external triggers and sensemaking in an emergency department in Norway. Dag Tomas Sagen Johannesen and Siri Wiig. Safety in Health (2017) 3:7  DOI 10.1186/s40886-017-0058-5.

Posted in Medicine | Tagged , | Leave a comment

Change Board Recertification

US doctors have got themselves organized to oppose being exploited to prove their ability.

http://changeboardrecert.com/ is the place to read what’s happening in US medicine.

We need to see similar organization against labs being exploited for accreditation by the ISO publishing and inspection gang.

Dear Fellow Physicians,

Opposition to Maintenance of Certification (MOC) continues to gain momentum as a top national medical news story…

We take issue with the bulk of the MOC process: The Practice Improvement Modules and Patient and Peer Reviews amount to little more than busy work, and the costly, time-consuming Secure Examination—which requires an enormous amount of preparation and time off from work —is clinically irrelevant and has no place in the life of a practicing physician.

Of course, these criticisms are sticking points for our Boards, which derive income directly from this part of MOC. In fact, the ABIM claims it makes no money from MOC and that its members receive a minimal stipend to attend meetings and write exam questions—yet each such question costs $4,000 (through “stringent psychometric testing”), the ABIM President makes a salary of about $2,000 a day, and the costs associated with the MOC program since 2006 have inflated the ABIM’s revenue to $149,800,000—a number which grows yearly and with each modification to MOC.

Here’s how the figures look:

 

 

 

 

 

 

 

 

You can read more about the background of this commercial exploitation of doctors in the NEJM here:

Boarded to Death — Why Maintenance of Certification Is Bad for Doctors and Patients Paul S. Teirstein, M.D.

Check out the supportive comments beneath.

Similarly to the US medical certification Boards, the ISO derives income from selling ISO standards which are often not optional.  ISO standards get revised without the validation that would be required of laboratories because the ISO needs to sell everyone a new batch to keep up its income.

BSI and UKAS’s entitlements to regular income also goes unchallenged despite them offering no proof of their claims to assure quality and avoiding legal responsibility for the disasters that have occurred despite their accreditation.

And it has been alleged that the IAF is the personal piggy bank of a lady who keeps a low profile.  Some of the financial aspects are very much worth investigating:

7.) IAF is Elva Nilsen’s Personal Piggybank. The trick used by not-for-profit (NFP) organizations is to claim no salary on their tax return, but then pay exorbitant amounts third-party consulting firm, which then pays off the officers in some way. That way, the officers can claim no compensation on the NFP’s tax return, which faces the public, while actually cashing a check behind the scenes. We can’t know for sure if that is what IAF is doing, but it’s only major expense was an annual payment of $278,000 to Elva Nilsen’s consulting company, EJN Consulting. That payment represented a whopping 48% of IAF’s total annual expenses in 2015. Nilsen is the Secretary of IAF at the same time. So while the officers show no compensation by IAF, IAF can make a payment to Nilsen’s private company, whose tax returns are not available for public scrutiny, and Nilsen can then write checks to those officers without anyone knowing.

So let’s see similar organised resistance to the ISO accreditation scam.

 

Posted in Cartel, Medicine | Tagged , , , | Leave a comment