Flodgren et al. enquired, Can third party inspections of whether or not healthcare organisations are fulfilling mandatory standards improve care processes, professional practice and patient recovery?
Cochrane Database Syst Rev. 2011 Nov 9;(11):CD008992. doi: 10.1002/14651858.CD008992.pub2.
Effectiveness of external inspection of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour or patient outcomes.
They searched for high-quality studies including randomised controlled trials and Cochrane reviews.
We only identified two studies for inclusion in this review, which highlights the paucity of high-quality controlled evaluations of the effectiveness of external inspection systems. No firm conclusions could therefore be drawn about the effectiveness of external inspection on compliance with standards.
Want to know more? Here is their
Can third party inspections of whether or not healthcare organisations are fulfilling mandatory standards improve care processes, professional practice and patient recovery?
Third party (external) review systems are used within health care settings as a way to increase the compliance with evidence-based standards, but very little is known of their benefits in terms of organisational, provider and patient level outcomes, or their cost-effectiveness.
We searched the literature for evaluations of the effectiveness of external inspection of compliance with standards including; randomised controlled trials (RCTs), controlled clinical trials (CCTs), interrupted time-series (ITSs) and controlled before and after studies (CBAs). We found only two relevant studies: one RCT involving 20 hospitals in the Republic of South Africa and one uncontrolled before and after study (that required re-analysis as an ITS) involving all acute trusts in England.
The RCT evaluating the effectiveness of a hospital accreditation system showed improved compliance with COHSASA (the Council for Health Services Accreditation for South Africa) accreditation standards (involving 28 service elements), but little effect on eight indicators of hospital quality. Significantly improved compliance score with COHSASA accreditation standards was reported for 21/28 service elements: mean intervention effect (95% confidence interval (CI)) 30% (23% to 57%, P < 0.001). The score increased from 48% to 78% in intervention hospitals, while remaining the same in control hospitals (43%). A sub-analysis of 424 critical criteria, showed significantly improved mean compliance with the critical standards (P < 0.001). The score increased from 41% (21% to 46%) to 75% (55% to 96%) in hospitals with the accreditation programme, but was unchanged in control hospitals. However, only one of the nine intervention hospitals gained full accreditation status at the end of the study period, and with two others reached pre-accreditation status. Only one of eight indicators of hospital quality: ‘nurses perception of clinical quality, participation and teamwork’ was significantly improved (mean intervention effect 5.7%, P = 0.03).
Too few studies were identified for inclusion in this review to draw any firm conclusions about the effectiveness of external review of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour or patient outcomes. Instead, this review highlights the lack of high-quality studies evaluating the effectiveness of external inspections on compliance with standards in healthcare organisations.
The best they found was a minor perceived improvement that had low statistical significance and was only a measure of nurses’ perception. This is more likely to be an indirect measure of the power of suggestion and desire to please management than a quantification of genuine improvement that might be measured in more meaningful ways.
As we saw in Wilson et al.‘s paper regarding laboratory accreditation, Flodgren et al. reported a similarly clear absence of evidence for inspections in hospitals bringing improvement, even outside the laboratories.
We must ask again and again, why is so much money and effort wasted where there is no evidence of effectiveness?