Study questions value of large EPRs

Tags: Greenhalgh Out-of-hours SCR UCL

15 Dec 2009

Local electronic patient record systems are often more effective than larger scale projects, according to a new study by the academic leading the independent evaluation of the Summary Care Record.

Professor Trish Greenhalgh and colleagues from University College London’s Department of Open Learning analysed 24 previous systematic reviews and 94 primary studies on EPRs all over the world.

The study, published in Milbank Quarterly, concluded that larger scale EPR projects promise much but sometimes deliver little.

It also suggested that researchers and policymakers need to do much more work on how to get EPRs to work in the real world and called for an interdisciplinary debate on priorities for the EPR research and policy.

Professor Greenhalgh said that while EPRs are often depicted as the cornerstone of a modern health service, clinicians and managers all over the world struggle to implement them.

She added: “Depressingly, outside the world of the carefully-controlled trial, between 50 and 80% of EPR projects fail – and the larger the project, the more likely it is to fail.”

The researchers said they found no evidence that large-scale commercial IT systems in health care produce the benefits anticipated by their architects, and that a few high quality studies suggest that they did not.

However they added: “We also found recent evidence that if EPR systems are developed organically and in-house, scale per se may not be a bar to their success.”

The study says the National Programme for IT in the NHS in England is based on six assumptions: that the EPR is primarily a container for information about the patient; that it can be integrated seamlessly and unproblematically into clinical work; that it will increase the effectiveness and efficiency of clinical work; that it will drive changes in how staff interact with the patient and one another; that it should replace most if not all forms of paper record; and that the more comprehensive and widely distributed it is, the more value it will add.

It says much of the literature covered in the review ran contrary to these assumptions.

In particular, the study found that while secondary work like audit and billing could be made more efficient by EPRs, primary clinical work could be made less efficient.

It said paper could offer greater flexibility for many aspects of clinical work and concluded that seamless integration between systems was unlikely ever to happen “as human input will probably always be required to re-contextualise information for different uses.”

Professor Greenhalgh said the results provided no simple solutions and did not support an anti-technology policy.

Instead, she added: “They suggest it is time for researchers and policymakers to move beyond simplistic, technology-push models and consider how to capture the messiness and unpredictability of the real world.”

The UCL team led by Professor Greenhalgh that is evaluating the SCR is due to present its final conclusions in May next year.

A report presented to NHS Connecting for Health in September showed that the SCR sometimes adds value in out-of-hours consultations, but that it had so far had made a limited contribution in secondary care.

Fiona Barr

Reader's Comments

Story for main EHI site too

15 Dec 09 13:12

Professor Trish Greenhalgh and colleague's paper (and implications thereof) are in no way restricted to Primary Care.

It is secondary care where the hubristic mega-system failures are most prevalent.

This paper deserves the widest possible reading.

These issues have long been well understood in the Healthcare Informatics community yet they are systematically ignored. Sadly it is too often the case "you either try to do it the wrong way or you don't get work at all".

Not sure where this leaves us..?

15 Dec 09 15:12

It's a pretty exhaustive piece of research from someone who is held in high regard.

I'd agree with the previous comment that these findings must have a profound impact on the wider health service in the UK and must also pose questions about creating single shared records across disciplines which do affect Primary Care in some areas. "Is patient safety being compromised?" must be a concern based on views expressed in the report

Who will stand up and provide a view from the 'highly centralised' camp?

Huge amounts of resources (not just cash) are invested in 'big is best' and it appears that insufficient research was conducted prior to deciding on the direction of travel in 2002/3.

Where are we people??

There have been NHS successes in the past

john.aird@uhl-tr.nhs.uk

15 Dec 09 15:12

Before we write-off any large scale system design and implementation let us consider a couple of factors.

We have done this before:

Back in the old days of Regional IT depts, the South West Regional IT service was one of the best, not only did it develop an integrated suite of hospital IT systems, (way ahead in scope and functionality than most of their time) many are still running today. If the NHS IT leads of the day had sought to maintain development rather than selling of (family silver) to commercial organisation, we would probably already have the modern EPR system++ we wish for today. So the NHS has successfully specified, financed, built and run integrated (not just interfaced) IT systems on a big scale.

Single of multi instance:

This is a massive issue. If NHS IT had started off on the basis of single instance then all our codes, numbering, names, data quality etc, would be fully harmonised, but we didn’t. So moving from multi to single instance will quite possibly pose huge difficulties for the NHS. Is it necessary? Well that depends on what the perceived benefits are and if they are worth the trouble. Because otherwise the LSPs could make their and our lives much simpler by implementing individual instances of Lorenzo (or what ever) and using SUS to move shared data.

Radically Altering Medical Care

16 Dec 09 05:12

This measured scholarly report comes as no surprise.

The systems to which the authors refer radically change how medical care is provided and are not team players because they were not designed to be that way. The delays in care and random mistakes facilitated by such systems must be quantified. The CPOE ordering systems can be particularly risky and are, indeed, medical devices, as has been determined by Sweden and others.

With this in mind, it would be best for users to report their adverse events or near accidents to the MHRA. If it is a system made in the USA, the FDA of the US has become interested, so you should report adverse events there also, at FDA.gov.

Again, congratulations to the authors for a phenomenal work and congrats to e-health for the coverage.

Quel surprise!

16 Dec 09 10:12

Isn't it well known that the only successful large projects are those that start small and expand/join up?

In 1997, Information for Health prescribed just such a project - local health communities linked up to initiate local electronic health records.

Just as they were at the point of signing up their suppliers, a whim of government resulted in a complete change of tactics to create NPfIT with its centralist approach.

If I4H had been allowed to procede, we'd be sitting in our hospitals with a fairly mature EHR with several years of data by now!

Is this the counter spin now we need to save money?

16 Dec 09 13:12

All seems a bit fishy to say the least that now the Chancellor's given all our money to his banker mates we get the reports casting doubt on the benefits of large EPR's.

Unfit for purpose, shoddy American-model EPR systems that are implemented by unskilled people are not delivering the benefits.

Well implemented UK NHS compatible systems are delivering the benefits. Up until now they've tended to be home grown or from suppliers who really did make the effort to make it work.

Care needs to be taken here as the baby will be thrown out with the bathwater if we're not careful and we will be copying information by hand for the next decade or two.

And remember you can always find a paper or results to support your case, and if not, then massage it in a Climategate style. It's the nurses and doctors who will tell you what really makes a difference to patient care not in-vitro academic studies which at the best of times are once-removed from the front line reality.

What IS the EPRs work?

16 Dec 09 14:12

>how to get EPRs to work in the real world<

Each generation of healthcare management and politicans it seems falls for the promise that the next (and always unfinished let alone proven) terminology / messaging standard / data model / hardware technology will make the big all encompassing EPR 'work'.

The first step I suggest is rather to understand what the EPR's work is... the candidate jobs include

Medico-legal clinical record
Intra-professional communication (e.g. community nurse to community nurse)
Inter-professional communication (e.g. physiotherapist to doctor)
Communication with patient / carers
Test ordering and results platform: lab, x-rays etc.
Document management
Electronic prescribing platform
Active decision support (e.g. prescribing decision support, hot-links to knowledge bases)
Enterprise scheduling
Provision of "care pathways" e.g. pre-defined templates for cross disciplinary or cross-institution data exchange plus workflow management for patients with certain conditions
Clinical performance audit
Administrative target monitoring (e.g. 18 week wait)
Billing
Activity statistics

As EMR projects grow in size, scope and ambition so does expectation that the system will do all those jobs AND that seamless integration will spontaneously result. In particular that "[someone] enters data once and [everyone] (re)uses it many times". Thereby cash releasing benefits and improved patient safety materialise as if by magic.

I suggest healthcare is not and can never be delivered that way... even with more than the typical token or wholly absent process redesign.

The illusion starts IMHO with the layperson's observation "I keep getting asked the same question" while in many cases..

they are being asked a subtly different question
they are being asked the same question but in a different context which may radically alter the response
they are being asked a question where the answer might have changed (even if this is very unlikely)
a question is deliberately repeated where it is judged too dangerous to trust the previous response as being correctly recorded during a previous episode of care or by a different system user

The other illusion is that the same data are useable for multiple tasks when this is patently untrue!

The clinical information to relay in (say) a discharge summary to a GP may will in no way adequately document the billable (or commissioned) services delivered during the admission and vice versa.

The clinical information to relay to (say) the occupational therapist, may be uninterpretable by or irrelavant to the opththalmologist and vice versa.

Only examine each of the 'jobs' above rigourously applying these criteria

1. "Just the facts mam"
2. Just the data which need to be machine readable and can be machine read

and the puff and pixie dust falls away - leaving far more arbitrary opinion, single use / use case specific free or semi-structured text, manual processes and paper than those funding these projects would wish for.

"However with cloud technology {insert gimmick of choice} none of these constraints applies" - just add new government and round the futile loop we shall go again.

See you same place, same article in another £12 billion and ten years time. Sigh :-(

What we have known for a long time

neelam.dugar@gmail.com

16 Dec 09 17:12

We do need an Integrated Health Record in NHS. I agree that single vendor monolithic systems will not work. We do need good departmental systems that support clinical workflow inNHS. Many such systems already exist.

However, what is also required are robust interoperability standards to bring them together in an EPR. Such a registry repository model of EPR is XDS (global standard from IHE)

we discussed this at our last meeting 23/10/09- theme was vendor neutral open standards based EPR using multiple departmental systems.

All speaker presentations are available on our forum www.pacsgroup.org.uk

Does the study really say this?

maryhawking@tigers.demon.co.uk

20 Dec 09 11:12

This is a fascinating if difficult read - but it does seem to me to be as much about approaches to evaluating EPRs (still not clearly defined and unlikely to be the same in all studies) as about EPRs themselves.

There are some real problems highlighted:- " When used as a formal tool (e.g., with structured templates and a requirement for data to be coded), the EPR often slows down and frustrates the clinical encounter, but it greatly accelerates the secondary uses of clinical data. Rather than promising that the EPR will “save time” or “make clinical care more efficient,” a more honest message would be that creating accurate and complete clinical records requires the sacrifice of time and effort by frontline clinical and administrative staff but that this is (sometimes) justified by more benefits for efficient business processes (e.g., billing), governance, and research. Appropriate incentive structures are needed to ensure that those who do the work reap the appropriate rewards "

Very true - and one of the reasons GP EPRs have been accepted by GPs - even before Targets and QOF - but not by secondary care as a whole.

"The interpretivist, critical, and recursive traditions problematize the very notion of success in an EPR project or program These traditions also recognize that the most immediate and easily measurable impacts of a new EPR system (such as more time needed to enter data or frustrations stemming from the model-reality gap) may fail to capture more subtle or distant potential benefits (such as the easier and more reliable production of aggregated data or greater capacity for research). Accordingly, just as the “success” of a project may be talked up for political reasons, so “failed” projects should not be dismissed out of hand" " the success of an EPR project also has an ethical dimension, asking, for example, who has the power to define what counts as success, who sponsors the evaluation and what its hidden aims are, and whose interests are (and are not) represented in the evaluation "

A thought-provoking evaluation of the various approaches and conclusions in some of the literature - but I'm not clear about the conclusions being drawn from it - certainly as regards general practice and primary care EPRs in the UK!

A Free for All is no solution

21 Dec 09 12:12

Can we please not get too misty eyed about the pre NPfIT way of doing things. It took almost 5 years to implement a single message exchange for Pathology results. Interfacing a huge range of heterogeneous systems will be just as much of a challange as building a monolythic system.

The report makes a key point about the fact that information created in one context may not translate to another context just by moving the data around. This challange is even greater for a data messaging approach than for an integrated approach. This is because in the integrated approach the original context can be made accessible. In the messaging approach, unless extreame care is taken, the context can be lost.

The real challange the report throws down is about how we take an information approach to these systems rather than a data approach.