Hospital league tables produce highly questionable rankings for the purpose of creating an artificial market in the NHS, and must be opposed, argues Graham Kirkwood

In November last year, the UK Labour government announced there would be a new set of NHS hospital league tables compiled from a range of performance criteria. These tables will be produced every three months with the stated aim of supporting individual NHS trusts that need to improve. Hospitals demonstrating persistently low performance will receive prompt intervention, while those performing well will be rewarded with additional freedoms, including more power over how they spend their money.

The first set of tables were published in June this year. The scores used to rank hospitals are compiled from seven different areas including waiting times for operations and time spent in A&E. In addition, hospitals running a larger-than-expected financial deficit are given a low ranking without examining the causes of the deficit, e.g. exorbitant annual charges for PFI costs that a hospital can do nothing about, the need to recruit agency staff, etc, nor is there any consideration given to the impact in terms of resources available to pay for staff and services. Hospitals that are struggling financially cannot be ranked higher than the third of the four tiers created to categorise performance. Ambulance services and mental-health trusts are also being ranked.

What is measured

Performance monitoring of public services began in the 1980s, but it wasn’t until the 1990s that it became embedded in many areas, most of all education. School ranking has created a pseudo market in education where parents compete to get close to a ‘good’ school, renting and buying property nearby to ensure their first child can get into the school, and siblings then follow. Performance management has also been used across local government.

In a landmark paper published in 1996, Harvey Goldstein and David Spiegelhalter critiqued the use of league tables in education and health. They list three ways performance can be measured. First, input factors, such as nurse-to-patient ratios, give a measure of the resources available to a hospital. The Royal College of Nursing highlight how patient care is put at risk when there are too few nurses on the ward, with the risk of death increasing by 3% each day a patient experiences a registered nurse staffing shortage. The second measure is of processes and organisation, for example, the percentage of surgery provided as day cases, in and out the same day, rather than overnight inpatient care.

The third measure focuses on quality of care and is the most problematic and the most used by politicians. While the first two types of measure may give some useful insight into why a hospital may be having difficulties, measures of supposed quality, such as hospital-level mortality, are riddled with problems. Goldstein and Spiegelhalter list a number of issues with such measures of ‘quality’, including the social and political considerations in the choice of measures used to quantify quality. In addition, models used to compile these measures typically adjust by a range of factors to attempt to compare like with like, for example, if a hospital has a larger than average number of older patients, then you would expect mortality at that hospital to be higher. Adjusting for age should remove most of the differences between hospitals due to age alone. Adjustments can be made for a number of other factors, including gender and deprivation or poverty measured at a small residential area level.

However, not everything can be adjusted away. Case mix, the profile of patients in terms of their diagnosable health issues on coming into hospital, can be adjusted for, but this is known to be inadequate. There are no routinely collected data available which accurately measure the true health status of patients entering hospitals. Differences in case complexity not captured by diagnosis coding on entry to hospital are likely to result in what is termed residual confounding, in other words, differences which cannot be adjusted away. These differences might include a patient’s general health, severity of symptoms, support (or lack of it) at home on discharge, disease severity and medical complexity.

The politics of league tables

Finally, of utmost importance, the way these rankings are used is extremely problematic. Goldstein and Spiegelhalter stressed that uncertainty must be taken seriously. Measures are normally given as a mean (average) value with a range, normally 95%, where the true mean might lie; so for example the measure may be 3 but statistically there is a 95% chance that it actually lies somewhere between say 2 and 5 (the mean doesn’t have to sit in the middle and range values are dependent on what is being measured, how it is modelled and the hospital population or sample population size). Where these ranges overlap between two hospitals, no difference between them should be implied.

Of course, the rankings produced by the NHS for the government take no account of these nuances as Goldstein and Spiegelhalter predicted; once you produce such measures, ranking is inevitable. Rankings are simply given in order, so Moorfields Eye Hospital is the best acute hospital in England and the Queen Elizabeth Hospital in King’s Lynn, the worst, full stop.

Just as the Blair Labour government did in the 2000s, Starmer’s Labour government is undermining the very foundations of the NHS. Competition is being embedded in the NHS at every level and just as in education, a pseudo market in NHS hospitals created. This goes against everything the NHS is supposed to be. Unions representing doctors, nurses and other hospital staff need to come out against these unreliable and unscientific league tables. This fraud being inflicted on the NHS needs to be brought to a halt before it becomes embedded as it has in education.