First day of the PROMs

Tags: consent Darzi DH Quality

10 Feb 2009

The Department of Health has issued instructions to the NHS on the action that needs to be taken to deliver Patient Reported Outcome Measures from April.

The use of PROMs in the health service was first outlined by Lord Darzi in the NHS Next Stage Review. Now commissioners and service providers have been given detailed guidance on how the system should work.

PROMs will collect information on the clinical quality of care as reported by patients, who will be asked to answer the same set of questions before and after an operation. The comparable data will then be used to calculate a numerical value for the improvement to their health.

From April, all licensed providers of hip replacements, hernia surgery, and varicose vein surgery will be expected to invite patients to complete a pre-operative questionnaire. About 250,000 patients are expected to be asked to fill out a PROMs questionnaire in 2009-10.

A contractor will then be responsible for collecting the pre-operative data and administering post-operative questionnaires.

The DH has said it is proceeding with Northgate Information Solutions as its preferred bidder for administration and data aggregation and MORI as its preferred bidder for analysis. Contracts are due to be signed this month.

Health minister Lord Darzi said the beauty of PROMs was its ability to measure the success of operations as reported by patients themselves.

He added: “While a surgeon may judge a hip replacement successful because the procedure has been performed perfectly on the day, the patient will rightly disagree if they are still in pain and continue to have a poor quality of life six months down the line.”

Lord Darzi said the programme would be the first of its kind in the world and empower patients to choose a hospital that achieves the best results for the operation they need.

He added: “It will also strengthen commissioning across the NHS by offering primary care trusts the evidence they need to buy the best services based on patient experiences.”

The NHS Information Centre is to play a central role in delivering the PROMs programme, converting the PROMs questionnaires into health status measurements, and linking the identifiable record-level data to existing routinely collected data such as hospital episode statistics.

The questionnaires will be provided in a booklet and include a patient consent form and patient information sheet. Pre-operative questionnaires will include questions on general health status and condition-specific health status.

Post-operative questionnaires will include questions about complications, re-admissions to hospital and re-operations. The questionnaires are available on the DH N3 website.

The NSR indicated that there is an intention to link payments to PROMs data. The DH says research was underway to identify other areas where PROMs might be feasible.

Fiona Barr

Reader's Comments

Snake oil

10 Feb 09 09:02

At the same time as 'evidence based' medicine is being forced upon clinical workers (at least those parts which cut costs) - the notion of evidence based policy is willfully neglected.

Where is the evidence that this approach will improve patient care as opposed to becoming a vapid PR exercise?

The approach described is epidemiologically illiterate: see http://en.wikipedia.org/wiki/Randomized_controlled_trial for why this approach cannot produce valid data

Even with diligence and even handedness there can be little value in data collected in this fashion. However in practice such data will be enormously vulnerable to gaming - if there is a casemix component patients will be systematically massaged into the most advantageous categories. More probably casemix adjustment will be simply inadequate so centres operating on younger, healthier and richer patients will see their stars and incomes rise. Just like school league tables - unto those who have shall be given, from those which have not shall be taken away.

Meanwhile the policy will tie up hospital staff in yet more data collection for the sake of data collection. It will also distract attention from clinically useful information technology projects just as 18 week wait did before it.

NHS patients and staff deserve so much better.

PROMs at last

tim.benson@abies.co.uk

10 Feb 09 14:02

First, I must declare an interest. I am involved in setting up a business to collect PROMs as part of routine care. Lord Darzi is quite right in saying that we need to measure the effectiveness of what we do and that the patient's perception of this is paramount. Not only should we measure outcomes, but we need results transparency. The first big issue is to adjust for case mix and risk. Everyone in the field recognises that you have to we compare apples with apples and not with pears. This means that need to record all of the key determinants of outcome, including principal diagnosis, number of co-morbidities, treatment provided, patient's age/sex and locality (deprivation, stress etc). Th enext generation of outcome measures will be integrated with electronic health records, where most of this data is available.

The second big issue is to make the data collection process beneficial for patients and clinicians, as well as the bean-counters at the Department of Health. Outcome measurement can be part of routine care and need not take more than a few seconds. The patient's perception of how they feel is part of their presenting history, which ought to be formalised and inserted into the patient's record so that it can be charted to show progress and facilitate continuity of care. The scheme being introduced by the DH from April is based on trials carried out some years back and is likely to work for hip and knee replacements, but is unlikely to be scalable to the other 99% of clinical care. However it is a vital start, which should be welcomed all round.

As EHI says, it is the first day of the PROMs. To suggest it is snake oil and that RCTs can deliver solutions completely misses the point. The essence of the RCT is that it is blind (preferably double blind); the essence of outcome measurement is transparency.

Tim Benson

Input State

roy.dainty@ntlworld.com

10 Feb 09 16:02

What I have not seen nor heard in connection with this or other 'outcome based' documents and statements is anything about capturing the patient state from the time of referral and onwards through the patient journey. Outcome measurement without input measurement is meaningless. Tim, or someone from the centre, perhaps you can illuminate us on this? Many are expecting input states to become part of standard capture and thus form part of the national clinical data sets?

Poster 1 replies

10 Feb 09 16:02

Tim

>>need to record all of the key determinants of outcome, including principal diagnosis, number of co-morbidities, treatment provided, patient's age/sex and locality<<

These might one day fall out of the Detailed Care Record as you describe - but parallel dedicated data collection using paper or computer systems will meanwhile plug the gap.

"18 week wait" could perhaps have fallen out of routinely collected data but - owing to delay in CRS delivery - diverted resource from both CfH and non-CfH clinical projects. PROM will too - q.v. "show me the money".

>>first big issue is to adjust for case mix and risk<<

Let's hope they plan to do this before publishing league tables, villifying 'poor performers' and sending the resources and voting classes away from 'sink' hospitals.

Once one has faithfully stratified cases - numbers in comparable groups are likely insufficient for even egregious clinical outcome aberrations to reach statistical signficance. While confidence intervals will be so wide that seemingly large variations in rarer complications might be wholly attributable to chance.

Conversely unarguably statistically significant differences in patient's satisfaction ratings may be easy to achieve. Putting resources into (say) free parking (0-100% approval rating in 6 seconds) while cutting corners on prevention of rare serious clinical complications may emerge as the optimum financial strategy.

>>that RCTs can deliver solutions completely misses the point<<

I picked the randomised controlled trial Wiki article not to suggest they were the only possible tool - but because the article summarises the pitfalls of other approaches to comparing apples with apples. IMO PROM falls headlong into most of them. Perhaps a clinical epidemiologist not pursuing a seat in the Upper House might care to arbitrate?

>>[PROM is] unlikely to be scalable to the other 99% of clinical care. However it is a vital start, which should be welcomed all round.<<

I suggest the converse invoking the McNamara Fallacy. Specificially so much political attention and resource over the past 30 years has been devoted to NHS elective surgery - that opportunity costs must be falling elsewhere - maternity for example.

But let's say I'm wrong and PROM works brilliantly - there remains the avowed use of these data - financial penalties for low performers. Because that's worked so well in education?

I fear we shall have to agree to differ on this one!

Regards, Malcolm H Duncan

Inputs and Silos

tim.benson@abies.co.uk

10 Feb 09 17:02

The links between inputs, outcomes and IT are limited because these disciplines exist in separate silos. Outcome measurement ought to be of interest to anyone with a quality and effectiveness agenda. However most outcomes knowledge is with academic psychologists, economists and epidemiologists who live in ivory towers. Input measurement has been primarily of interest to cost accountants in the finance department. Health informatics is yet another silo.

Agree - snake oil

10 Feb 09 22:02

90% of users of "Darzi's Patent Hip-and-Knee-Salve" said their hips and knees felt better after using it.

Ah, "perceptions of how you feel", such a fickle, slippery thing.

Are operations the right use of PROMS?

maryhawking@tigers.demon.co.uk

12 Feb 09 12:02

I feel there are some fundamental problems with using joint replacements as a model for PROMs - and a good many unanswered questions on the thinking. My patients quite often have unrealistic expectations - and a poor memory for the pain and disability prior to surgery. Many expect to jump off the operating table, free from pain and fit to run the marathon the following day: their post-operative PROM is likely to reflect their unrealistic expectations. How long after surgery would you measure a post-operative PROM? probably about 12 months would be sensible - but, like the 4 week quitter measurement of smoking cessation, politicians will probably demand a max of 28 days!

Using PROMS as an ongoing monitoring tool in LTC management makes sense: the DH version doesn't.

DOI - GP

PROMs vs RTCs vs Feedback for Quality

13 Feb 09 10:02

Malcolm Duncan may be right to be sceptical about PROMs as they are planned to be implemented, but I really don't see what RTCs have got to do with this. RTCs are great for resolving scientific questions under near laboratory conditions; they tell us nothing much about what actually happens in the messy real world, where incentives and disincentives change how things are done *in practice*. Part of the problem is that there is simply far too little feedback in medicine to guide proper practice; there is oodles of evidence that what is known to be clinical good practice is simply not being delivered - and that it is not just cost that is the barrier. PROMs may actually help clinicians, teams, and healthcare organisations deliver better and more effective care – if done properly. Clinicians should be welcoming and helping direct this rather than doing the usually ‘not invented here’ response. After all if the patient does not actually get better (relative to the likely outcome without treatment, of course), then any surgical intervention is a waste of everyone’s time and effort, no matter how well performed. Having been at the receiving end of NHS care, it is only too obvious how little clinicians actually understand about what they do to people in all aspects, not just rearranging the plumbing or dealing out drugs. It is about time they woke up and ‘got real’.

Bonus culture - yet again!

13 Feb 09 14:02

Darzi is wrong to believe he has defined "quality", which like beauty is in the eye of the beholder, and he is wrong to suggest the use PROMs as one dimension of its measurement. Obviously it is the (ab)use of PROMs that will create problems. There is good evidence in the QA literature that attaching payments (bonuses or penalties) to a numerical measure is tantamount to paying people for chance effects (see poster 1). It is little better than betting on roulette - however clever the analyses. Sadly the current banking crisis has not taught Government that this method of incentivisation is a recipe for long-term disaster. No one in healthcare can control the results, apparently measured by PROMs, and if they delude themselves that they have, then this will be a recipe for chaos. As regards what should happen: the answer is not straightforward. There is no magic bullet. We need honesty, we need teamwork, we need co-operation rather than blame and we need much better information systems. Most of all we need to permit the highly trained staff of the NHS to have pride in what they do.

Just a few seconds...........

17 Feb 09 22:02

Tim Benson writes; "Outcome measurement can be part of routine care and need not take more than a few seconds" Presumably this will be "at the touch of a button", and doubtless the same button, much beloved of Health Ministers, that enables us to make Choose and Book referrals quicker than you can say "Spin Doctor".

Just a few seconds

tim.benson@abies.co.uk

18 Feb 09 14:02

Check out www.routinehealthoutcomes.com if you don't believe me!