The evidence from Dr Foster intelligence is another of the days that it would have been impossible for me to miss at the inquiry. This blog is again me working under time pressure, and I cannot do Roger Taylor justice this way. I will come back to it in a lot more detail later. This evidence turned out to be one of the most important days so far. It has certainly provided me with the piece of the jigsaw that I had missing.
Dr Foster Intelligence is central to the Stafford Story. This is the company that published the Dr Foster league table of mortality rates in 2007. This is what set the hare running that would eventually lead to the Healthcare Commission investigation and all the subsequent inquiries.
Dr Foster intelligence is the commercial company set up to sell the information systems designed by Professor Jarman to Hospitals and a range of other users. There is a price tag of £35,000 per hospital per year for that service. If you want to see what commercialisation within the Health service looks like then Dr Fosters is a good place to start. If you want to see how we face the challenge of incorporating commercial organisations into the health service, and the incomprehension this can bring with it then Dr Fosters and the Stafford experience is a very good place to start.
The company has always seemed shadowy to me. People talk about their large plush offices, sales techniques that seem a little alien to many of the people who work within the NHS and their reluctance to share commercially sensitive material.
It was important to me to be able to put a face to this company. What I saw was the face of an energetic company that is capable of doing some really innovative things, some imperfectly, a company that is subject to the pressures which come with the need to generate profit. This is “a different country. They do things differently here! “
Robert Taylor is an imposing, or even handsome man, perhaps one of the most articulate speakers we have heard, his previous job was as a journalist with the Financial times. He came across as very professional in a business world kind of way. I came prepared to be hostile but I liked him.
The Lawyer wasted no time – we were straight to the point.
This is the bold claim that Dr Fosters made in their publication of 2007 "
Hospital mortality ratios are an effective way to measure and compare clinical performance, safety and quality. Deaths in hospital are important and unequivocal health outcomes. They are also relatively straightforward to measure, being clearly defined events that must be reported in law."
The lawyer wanted to know if this was an over claiming of the position. Can the System actually act as measure of performance and quality? Or are they over claiming?
Roger Taylor came up with a different statement
in measuring clinical outcomes and clinical performance there are no perfect measures. What we try to do at Dr Foster is provide the best possible information that can be derived from the data that is available
I'm saying an effective measure of safety is one that helps you identify the risk of something being wrong.
This is of course in line with many of the statements made by people throughout the inquiry that the system can act as a warning system – a prompt to go and look. That is the limitations of what it can do.
The lawyer asked if it was right to put information into the public domain – and he felt that it is, with the right kind of caveats, which he believed were in place – in that the wording was in his publication.
Roger Taylors views on the public are interesting.
The public as well, I should stress, are all -- although they are not statistically very -- on average very knowledgeable, they are very cognisant of the fact that this type of performance information is – is imperfect and they are very aware of that fact.
In general I would completely accept his assertions that the Public recognise garbage when they see it, What of course has to be remembered is the way in which the garbage about Stafford has been packaged and repeated through channels that people want to trust, that they can no longer make these judgements. He talks about the scepticism of the public about statistical information which he sees as essentially a healthy thing, This exists, It will be increased if the Stafford story is ever properly seen or understood by the public, and it is a tragedy because good statistics, appropriately used can really help us pin point and solve problems.
He talks about the way that what is on the public’s mind affects the thinking of health service managers, for instance the public priorities in 2006/7 were waiting times and infection. He wants the data they use to be part of the process of informing the public effectively, and helping them to be a part of the process.
He also thinks that being transparent has the effect of helping to improve the quality of data and of data systems.
I hope he is right. A large part of the press coverage that has occurred in the Stafford Case has been about trying to
Dig for bits of information that were withheld. Assuming that these contained the real truth and making them the story. A large part of them has also been the focus on one single indicator – the HSMR, which in the case of Stafford was, for complex reasons (will blog later) fatally flawed. As we here in the evidence from the NHS information centre (blog to follow) A vast amount of data will become publicly available this autumn as part of the public information revolution and the NHS will be working on 300 different indicators rather than one eye catching but misleading figure.
The lawyer raised the matter of acceptance – and it was clear that within the NHS there was a very mixed response to the information produced in 2007 People did not trust either the methodology or the quality of the data.
The backing of the DoH did something to help to establish the company as legitimate, but there was still suspicion.
A lot of people felt uncomfortable with the idea of a private company making profit from public information, and especially uncomfortable with the idea that the methodology was not transparent. This suspicion still persists.
Roger Taylor was asked about the relationship with the government. It is clear that this is not a simple matter – a government that is at the start of its term may welcome transparency – but once in a difficult position, and subject for criticism then they may become less supportive.
Roger Taylor was asked about the tricky relationship with the SHA. He was very open about this – It was a difficult relationship, it is worth reading the transcript on this – P17>. It appears to me that these two men now understand each other pretty well. – in a good way!
This exchange with the Chairman (Robert Francis) is interesting.
3 that. You publish your annual guide, which to make no
4 bones about it, seems to result in the odd chief
5 executive resigning, but it seems to follow --
6 A. I would say that that isn't actually the case. I know
7 there has been speculation about that but --
8 THE CHAIRMAN: But that is a potential perception and it is
9 a perception that may matter here.
10 You also sell services to trusts which allows them,
11 in effect, to get some of the information that ends up
12 in the guide in advance and, therefore, an opportunity
13 to do something about it. Is there not a potential
14 conflict between your commercial activities in that
15 regard and the use of publicity in a way which may
16 promote those commercial activities?
Robert Taylor says that the hospitals who buy in do not at present get an advantage, but because this “enrages customers” they will be changing this policy, and in future customers will get a chance to challenge figures before publication.
There is P23 a discussion on methodology. DFI had the impression they were being transparent on this – Tim Straughan picks up on this in the afternoon. The openness on methodology was open in that all ingredients declared, but not how you mix or bake them.
There is a move towards greater openness all the time – with a lot of material being published on NHS choices.
There is a very interesting exchange with Steve Allen – begins p27 which shows the tension before the publication of a Dr Foster Guide. Roger Taylor made it clear that they are always worried about legal challenge.
Steve Allen is making the points that DFI should Consult, reach agreement on methods and then measure. Roger Taylor accepts that there is a need towards much better agreement.
My own observations as I listened to this ( and maybe this is unfair) was that what we have at the moment is a commercial organisation outside the NHS, which depends on publicity to sell its services and is essentially free to lob bombs into the NHS.
I think this is a perception many hospitals share, and the essential “commercial” challenge for DFI is to overcome this distrust.
We looked briefly at the relationship with Prof Jarman. See P33.
The issue of HSMRs was discussed P35 There has been an active debate about their use and limitations since 2000. The Stafford Case has brought that out into the open and has forced statisticians to work together, bringing together a consensus around the SHMIs These will not be a fixed entity The SHMIs will keep developing over time.
I am now going to have to curtail this piece – to get to today’s hearing on time, but there are a number of important themes that come through.
The issue of the palliative care coding which first arose in Professor Jarman’s evidence was explored in more detail. The Lawyer I think correctly raised the possibility that the correction of data which led Professor Jarman to suspect gaming – the main theme in the media coverage of his evidence – was in fact perfectly possibly the correction of very bad miscoding. This is certainly the way that I interpret it. (more to come)
We heard how the hospital at the first attempt of correcting their data first piled lots of people into Palliative care code, which was not right, but later moved to much better coding of the co-morbidities – which corrected the method and had the effect that the HSMR has stayed low.
Robert Francis raised the matter of the 400-1200 excess deaths and has had a clear statement from Roger Taylor that these figures should never have been used. They are an illegitimate interpretation of the data. Roger Taylor referred to the document he had co-signed on this issue which clarifies the limitations of HSMR. We saw this document in the afternoon. (more to come)
Robert Francis raised his real concern at the way in which the media continue to misuse the death rates and excess deaths. (more to come)
A pivotal moment for me came with the matter of the claim that 80% of the cases in Stafford had originally been coded wrongly. This clearly came as a bolt out of the blue to Roger. It was a major shock. It people could get the methods this wrong then that is a fundamental blow to the integrity of his system. He clearly did not want to believe that this was possible.
I would have to advise anyone with a doubt about quite how bad the data was to go back to the healthcare commission report which deals with this in some detail. The data was that bad, for a complex mix of reasons. (more later)
The other important moment came when Roger Taylor was asked if they make any attempt to control the quality of the data used by their system. He said that they have simple ways of checking if the material is complete – enough entries, and timely- they have no way of checking of their customers are operating the system in entirely the wrong way.
He was asked if there needs to be better ways of checking the quality of the data. His answer was Yes.
More Later!
