Listening to the evidence of Roger Taylor from Dr Foster Intelligence on day 99 of the Mid Staffs Public Inquiry the last piece of the jigsaw that I needed to see how we got into a hopeless mess about Mortality information at Stafford fell into place. I had had it all the time, but just had not seen that this is the piece that counted!
There was a happy time back in early 2009 when I had never heard of HSMR or the Hospital standardised Mortality Rates. But since 17th March 2009 the matter has never been far from my thoughts.
The reason I know this date, when I have difficulty remembering family birthdays is that this is the day when the press and TV brought a media hurricane to Stafford.
We had known for 18 months before that there was a band of relatives who had suffered real tragedies at Stafford. There were demonstrations in the town square and outside the hospital, they were working closely with the press to tell their stories and had already forced the resignation of the Chief Executive officer and Chair of the trust. What no one had reason to suspect that we would see press headlines that 400-1200 people had died as a result of poor care in Stafford.
The problem with the mortality rate figures in Stafford were known to a relatively small number of people – I knew of them, but did not even know the name “Dr Foster Intelligence”at that time. What I did know is that a considerable amount of research had been carried out which showed that there were serious problems with the coding of the data which was producing the apparently high mortality figure.
I was not directly involved in the chaos that followed the arrival of the world’s media in Stafford so what I did was I sat down and read the Healthcare commission report from cover to cover 4 times. It took that many times to convince myself that the “excess death figures” which appeared in every press item and every TV interview were not there, and that there was no other material contained within the body of the report that could imply these figures. This was clearly a hospital with problems, a hospital where lots of people would have had an uncomfortable time, I could imagine that there will have been some people who will have died, I found an interview with Sir Ian Kennedy who said it is likely that “some people may have died”. This seemed to me to be a completely fair balanced and reasonable statement. The rest looked like media induced hysteria.
There are clear caveats in the HCC report about the limitations of what the data can tell us.
For the data to accurately reflect hospital
activity, there must be clear, accurate and timely
information recorded in the patient’s notes;
accurate and consistent clinical coding; and
clear procedures for collecting and processing
the data. There also needs to be appropriate
training and accreditation of staff.
This is the passage which came to dominate my reading of the report.
The highlighting is mine. This seems to me to be the very simple human story at the heart of this chaotic chain of events.
Background
We were told by clinical staff and managers
that the trust had a long history of poor quality
information about its services. ..
A report by CHKS was commissioned by the
trust in early 2007, due to concerns about the
coding of clinical data. The report identified
deficiencies in the clinical coding entered in
the Patient Information Management System
(PIMS). This was manifested by inaccuracies
in coding and under-reporting of co-morbidities
(that is, patients’ other health
problems).
The coding manager at the time
had been on long-term sick leave and the rest
of the team were working part-time. Contact
with clinicians was also poor, with coders
being reluctant to approach them about
unclear notes. Clinicians had little
understanding of the need to make notes
clear for the coders.
The trust recruited a new coding manager in
July 2007, when the previous post holder
retired. More investment was put into the
department and new members recruited to
the team. Staff told us that the new coding
manager had had a positive impact on the
quality of coding. The new manager built
better relationships with clinicians and
motivated her staff to attend training courses
and gain accreditation. Examples of positive
developments included having consultants on
the clinical coding and data quality group, and
systems that the coders could use to crosscheck
information, such as radiology and
pharmacy reports.
The coding manager told us that she still had
concerns about some of the clinical input to
coding. It was reported that junior doctors
could present a problem because of their
frequent job rotations. They were often
imprecise about diagnoses, whereas the main
problem encountered with senior consultants
was the tendency to under-report comorbidities.
The trust had a history of poor performance on
mortality. The data from Dr Foster showed that
the three-year HSMR for 2003-2006 was 125.
This was the fourth highest ratio in England.
The trust had only begun to monitor clinical
outcomes after the publication of Dr Foster’s
Hospital Guide in 2007, and had relied on the
use of the Dr Foster ‘real-time monitoring
tool’ to identify areas of concern (‘red bells’).
This tool was used by the trust’s lead clinician
for clinical governance.
In response to its apparently high mortality
rate identified by Dr Foster, the trust initially
focused on the poor quality of the clinical
coding of the cases involved. It also
established a group to consider mortality
outcomes. The group’s follow-up of high
mortality rates had focused on reviews of
individual case notes of patients who had died.
This was conducted by clinicians at the trust
over a period of time. The general conclusions
of the follow-up were that the deaths were
predictable and that no problems with care
were identified.
There is further on in the report the description of the work of the mortality report – which carried out a limited case review of deaths, and found that around 80% of these had been miscoded. ***** I have still to find this reference.
As the story has moved on and the press headlines have kept on coming I have watched for every bit of information which might help to explain what happened here. How is it that the world in general has accepted this mysterious assertion that 400-1200 people died unnecessarily in Stafford when this information does not appear in any official document and there is clear evidence that the coding on which the information was based was of exceptionally poor quality.
Here are some of the pieces of the jigsaw.
My Attitude to HSMR
I am not a statistician though I have had some experience of using statistical systems. I have only learnt about any of this because of the truly dreadful experience that the people of Stafford have suffered. I am a fan of statistical information and believe that where people understand how to use them and use them well that they can often give a really good way of seeing what is happening. I accept that HSMR and some of the newer systems that Dr Foster are developing in partnership with other organisations are of considerable value, but they are, as Roger Taylor himself has clearly stated very vulnerable to misuse. Rubbish in will result in rubbish out.
No one left to fight the story.
When the story broke the only people with enough knowledge to contest the story had already gone. Everyone else was in a state of severe shock.
25 out of 75
It would be easy to assume from all that we have heard about HSMR that this is a universally used and accepted system, tried and tested, the single measure for comparing Quality of care and performance on Mortality in the country. This is not the case. We now have a definitive statement signed by many prominent names within the NHS including DFI which clarifies the limits of what HSMR can do. It is a system used by some hospitals. It would like to be the dominant system accepted by all hospitals. Maybe one day it will be, but it is not there yet. It has not won universal acceptance, and the huge amount of controversy surrounding the Stafford Hospital story has not helped it to win this. I would suggest there is a real opportunity now for DFI to mend some fences and put the matter right!
All hospitals have to try and set systems in place to monitor quality. Stafford was using a whole raft of systems. For whatever reason – and this is lost in the mists of time, they did not regard HSMR as a high priority. They had bought into it but were not enthusiastic supporters. On their prioritised list of indicators I believe (this information was given at one of the hospital board meetings) that the HSMR was ranked 25 out of the 75 indicators that they used.
I suspect that this means that staff were going through the motions of making returns rather than giving it any kind of priority. A hospital which was seriously understaffed, and where admin staff had been cut, and the coding manager was off on long term sick leave will not have had the time, training or motivation to do this tricky job well.
Lack of understanding.
Professor Jarman was shown a report by Helen Moss which showed her understanding of the function on Primary coding. She was struggling with the reality that they often did not know what a patient was admitted for, they were there for investigation, the diagnosis would come later, and she would then wish to code it with the dominant condition. He was clearly very puzzled by this. Robert Francis asked him to clarify – if you get a case with admission for a broken hip and the person later develops CDiff, then what is the right code. Professor Jarman would not give an answer to this. It was clear that Helen Mosses understanding did not match with his, so there was confusion right at the centre.
Counter intuitive coding.
What a hospital would want this system to do is to give a clear indication of the overall health of the patient, so that it is possible to spot if a death occurs unexpectedly.
Roger Taylors very clearly stated example, of a diabetic being admitted for an in-growing toe nail and being coded by the in-growing toe nail is very helpful, but I can see that it would really be of serious concern to a clinician.
Reflecting the state of health.
DFI in their analysis of the coding for Stafford say that the Primary coding seemed ok and that therefore the results are valid. I would suggest that for a hospital with the kind of case load Stafford actually has, with many elderly patients suffering from 5-10 chronic conditions, that if the system does not clearly reflect this and weight it adequately then HSMR based largely on primary coding will give highly misleading results.
Poor recording of Co-morbities.
The evidence in the HCC report shows us that recording of co-morbidities was completely inadequate, it had been for years, and it continued to be so for some time after 2007.
(follow up – I would say that Dr Foster has work to do to satisfy hospitals that it can pick up on cases of people dying when they come in with a broken toe nail – but also accurately represent people whose general health makes death a likely outcome. The “information revolution” probably gives the way forward on this. I am certain that Roger Taylor left the Inquiry with the clear understanding that there is work to be done to ensure people understand the system better, and that it is responsive to their needs.)
The PCT investigation
The PCT carried out its own check – comparing the coded cases to the GPs notes. They thought it was the coding.
CHKS
Carried out an investigation and thought it was the coding.
The SHA investigation
through Birmingham university thought it was the coding.
Who thought what?
In the run up to the HCC report all the health experts close to Stafford were confident that though there were clearly some problems at the hospital that so far as the mortality problems were concerned they were looking at a coding problem. Doctor Foster, Some elements of the HCC, Bill Cash, The pressure group, elements of the local press and possibly the Conservative central Communications team though that there had been huge numbers of deaths.
The piece of paper with the figures.
We learn from the evidence of Bill Moyes what happened at the meeting held in Alan Johnsons office immediately before the release of the HCC report. The HCC appeared at the last minute with a piece of paper with the Excess deaths. None of this had been discussed fully with other parts of the health service before. There was a huge row and a consensus view was formed that the material could not be released because the foundation of the figures was insufficiently sound and because it would cause widespread misunderstanding and concern.
The leak to the Daily Mail
We still do not know who leaked this figure to the Daily Mail. Bill Cash was asked directly about this by the BBC and said that he would not talk about it. The figures appeared with quotes from Bill Cash and from Julie Bailey.
Promotion of the figures
Whoever chose to put these figures into the public arena, they have been heavily used by the press, the Conservative party (including David Cameron, and Andrew Lansley) and the Pressure group.
Why the numbers changed
One of the mysterious facts about these figures only became clear to me through the evidence of Professor Jarman and Roger Taylor. The Hospital had been expecting their HSMR figure to come out as 114, when in fact the 2007 figures it came out as 127. This was a huge shock to them. There are two explanations for this, as Brian Jarman made clear.
The base line for HSMR shifts. The general trend is downwards, and so hospitals are “re-benchmarked” each year. To do this they have to get returns in and Stafford failed to do this. The other big element was that a new code for Palliative Care had been brought in, and that this had entirely escaped the notice of the people struggling with the coding in Stafford. They had not used the palliative care core, but lots of other hospitals had, so there was a major shift occurring at this point. The 2007 League table quickly highlighted this problem and brought about action to recode cases.
The Palliative care recoding first pass
What really exited the press from the Professor Jarman evidence is that he clearly took the view that a group of hospitals in West Mids, under the guidance of the SHA had set out to game his system by recoding loads of people as palliative care. So we have again had lurid headlines involving many thousands of deaths!
Roger Taylors evidence under careful questioning has I think helped us to get to the bottom of this. I am sure that there was a concerted action to get the coding sorted out. I think that the hospitals will have taken the immediate action of switching primary codes. This was sometimes the wrong thing to do. It certainly made an impressive graph!
Getting the coding right
Once the understanding of the system improved then I think what happened is that the hospitals recoded again, doing it the hard but right way in entering all the co-morbidities to give a balanced view of the health of the patients.
David Stone
David Stone was acting Chair after March 2009. He gave evidence to the Health scrutiny committee indicating that the coding at that time was robust – with an HSMR of 88 at that time. with the implication that it may not have been previously.
The 80% miscoding and Roger Taylors surprise.
What made a real impression on me was that Roger Taylor gave the bulk of his evidence with the conviction that though there were some real areas where the HSMR figure could be a little misleading, that there was no way that it could be very far out. He is used to thinking that everyone always says “it’s the coding”.
He was visibly shaken by the idea that the coding could be 80% wrong and clearly thought that if this was the case then the admin must be pretty chaotic. I could see him thinking (of course this is just my impression) if the coding is really out by that much then the implications would be major!
Today I spotted a tweet about the Titanic being designed by professionals. I think that the HSMR system is designed by very clever professionals and is an elegant system capable of being really helpful to the NHS. I think it is also vulnerable to being holed under the water line as comprehensively as it was by the unique series of events in Stafford.
I hope that if anyone from DFI gets to read this that they will think about this.
After looking very closely at this issue for three years, I think it is the coding!
