EHR safety, a false sense of security, and epistemological problems in the Medical Informatics community

Neil Versel is an author of many articles about healthcare information technology.  He recently published "Patient safety in the balance: Questions mount about EHRs and a wide range of patient safety concerns" (http://www.healthcareitnews.com/news/patient-safety-balance) in Healthcare IT News, August 27, 2014, also appearing in the Sept. 2014 print edition.

His article contained a sidebar on my views "A 'false sense of security'" (http://www.healthcareitnews.com/news/false-sense-security).

I thought my feedback back to him would be of interest, centering on a published comment of the typical hyper-enthusiast variety made by a prominent member of my Medical Informatics professional community.  The view exemplifies a widely-held but erroneous (and decidedly non-scientific) viewpoint in my professional community - and many others, including government -  regarding drawing conclusions in the face of significant evidentiary gaps and impediments regarding harms.

Here are my emails to Mr. Versel.  They speak for themselves, especially the short second supplementary email seen at the bottom of this blog post

-- SS

-------------------------------------------------------------

---

EMAIL #1:

From: Silverstein,Scot
Sent: Friday, September 05, 2014 10:05 AM
To: Neil Versel
Subject: Re:" A 'false sense of security'"

Neil,

Thanks for citing me in your article at http://www.healthcareitnews.com/news/patient-safety-balance, http://www.healthcareitnews.com/news/false-sense-security etc.

Want to call your attention to one of my largest disappointments and frustrations in health IT that evolved over the past decade or two, namely, cheerleading for the technology in the face of commonly understood Western science and epistemology ("the theory of knowledge, especially with regard to its methods, validity, and scope. Epistemology is the investigation of what distinguishes justified belief from opinion").

My concerns are best exemplified by the comment posted at the article site by a prominent member of my Medical Informatics professional community:

"The question is not whether EHRs are safe or not; sometimes they are, and sometimes they are not. The real question is whether they are safer than paper records, and the weight of scientific evidence is that they are."

The actual truth, considering the explicit statements with explicit reasons given by IOM, FDA, ECRI, and others as to why what we know about electronic harms is incomplete ("tip of the iceberg", see below), is that "while the potential for electronic systems to be safer than paper exists, the scientific evidence is inadequate to make any statement about comparative safety on the ground."

The data inadequacy is made far worse by the known and stated systematic impediments to discovery and diffusion of the harms, and a lack of robust studies on ACTUAL harms due to paper-based record keeping, especially and critically regarding optimal and well-staffed paper systems.  (Try a Medline search on that topic!)

The hyper-enthusiasts who favor the theoretical over reality on the ground and who push for speedy national rollout without proper safeguards, surveillance and regulation, are, in fact, killing people. [I.e, whose injuries and deaths at the hands of bad health IT were otherwise preventable had appropriate industry safeguards been in place - ed.]

There is actually nothing to argue, nothing to debate on this issue.

Some material backing up these assertions is below, mostly already covered in my blog-based 11 points.  [As at http://hcrenewal.blogspot.com/2014/04/fda-on-health-it-risk-reckless-or.html - ed.]

Cheers,

Scot

------------------------------

Examples: 

1)  IOM report on HIT safety, 2012 (as I report at http://hcrenewal.blogspot.com/2012/03/doctors-and-ehrs-reframing-modernists-v.html , midway down):

.. While some studies suggest improvements in patient safety can be made, others have found no effect. Instances of health IT–associated harm have been reported. However, little published evidence could be found quantifying the magnitude of the risk.
Several reasons health IT–related safety data are lacking include the absence of measures and a central repository (or linkages among decentralized repositories) to collect, analyze, and act on information related to safety of this technology. Another impediment to gathering safety data is contractual barriers (e.g., nondisclosure, confidentiality clauses) that can prevent users from sharing information about health IT–related adverse events. These barriers limit users’ abilities to share knowledge of risk-prone user interfaces, for instance through screenshots and descriptions of potentially unsafe processes. In addition, some vendors include language in their sales contracts and escape responsibility for errors or defects in their software (i.e., “hold harmless clauses”). The committee believes these types of contractual restrictions limit transparency, which significantly contributes to the gaps in knowledge of health IT–related patient safety risks. These barriers to generating evidence pose unacceptable risks to safety.[IOM (Institute of Medicine). 2012. Health IT and Patient Safety: Building Safer Systems for Better Care (PDF). Washington, DC: The National Academies Press, pg. S-2.]

Also in the IOM report:

… “For example, the number of patients who receive the correct medication in hospitals increases when these hospitals implement well-planned, robust computerized prescribing mechanisms and use barcoding systems. But even in these instances, the ability to generalize the results across the health care system may be limited. For other products— including electronic health records, which are being employed with more and more frequency— some studies find improvements in patient safety, while other studies find no effect.
More worrisome, some case reports suggest that poorly designed health IT can create new hazards in the already complex delivery of care. Although the magnitude of the risk associated with health IT is not known, some examples illustrate the concerns. Dosing errors, failure to detect life-threatening illnesses, and delaying treatment due to poor human–computer interactions or loss of data have led to serious injury and death.”

2)  FDA Internal memo on HIT risk, 2010 (as i report at http://hcrenewal.blogspot.com/2010/08/smoking-gun-internal-fda-memorandum-of.html):

... In summary, the results of this data review suggest significant clinical implications and public safety issues surrounding Health Information Technology. The most commonly reported H-IT safety issues included wrong patient/wrong data, medication administration issues, clinical data loss/miscalculation, and unforeseen software design issues; all of which have varying impact on the patient’s clinical care and outcome, which included 6 death and 43 injuries. The absence of mandatory reporting enforcement of H-IT safety issues limits the number of relevant MDRs and impedes a more comprehensive understanding of the actual problems and implications.

 This is especially true considering the FDA's own noted limitations of their information sources:

Limitations of the MAUDE search and final subset of MDRs include the following:

1. Not all H-IT safety issue MDRs can be captured due to limitations of reporting practices including

... (a) Vast number of H-IT systems that interface with multiple medical devices currently assigned to multiple procodes making it difficult to identify specific procodes for H-IT safety issues;
... (b) Procode assignments are also affected by the ability of the reporter/contractor to correctly identify the event as a H-IT safety issue;
... (c) Correct identification by the reporter of the suspect device brand name is challenged by difficulties discerning the actual H-IT system versus the device it supports.
2. Due to incomplete information in the MDRs, it is difficult to unduplicate similar reports, potentially resulting in a higher number of reports than actual events.
3. Reported death and injury events may only be associated with the reported device but not necessarily attributed to the device.
Memo: H-IT Safety Issues
4 Correct identification by the reporter of the manufacturer name is convoluted by the inability to discern the manufacturer of the actual H-IT system versus the device it supports.
5 The volume of MDR reporting to MAUDE may be impacted by a lack of understanding the reportability of H-IT safety issues and enforcement of such reporting.

3)  Jeff Shuren MD JD, head of FDA CDRH (as I report at http://hcrenewal.blogspot.com/2010/02/fda-on-health-it-adverse-consequences.html):

The Office of the National Coordinator for Health IT held a meeting of the HIT Policy Committee, Adoption/Certification Workgroup on February 25, 2010. The topic was "HIT safety." The agenda, presenters and presentations are available at this link.

At this meeting FDA testimony was given by Jeffrey Shuren, Director of FDA’s Center for Devices and Radiological Health. Dr. Shuren noted several categories of health IT-induced adverse consequences known by FDA. This information was striking:

He wrote:

... In the past two years, we have received 260 reports of HIT-related malfunctions with the potential for patient harm – including 44 reported injuries and 6 reported deaths. Because these reports are purely voluntary, they may represent only the tip of the iceberg in terms of the HIT-related problems that exist.

Even within this limited sample, several serious safety concerns have come to light. The reported adverse events have largely fallen into four major categories: (1) errors of commission, such as accessing the wrong patient’s record or overwriting one patient’s information with another’s; (2) errors of omission or transmission, such as the loss or corruption of vital patient data; (3) errors in data analysis, including medication dosing errors of several orders of magnitude; and (4) incompatibility between multi-vendor software applications and systems, which can lead to any of the above.

4)  ECRI Institute Deep Dive study of health IT safety (as I report at http://hcrenewal.blogspot.com/2013/02/peering-underneath-icebergs-water-level.html):

Participating facilities submitted health IT related events during the nine-week period starting April 16, 2012, and ending June 19, 2012. ECRI Institute PSO pulled additional health IT events that were submitted by facilities during the same nine-week period as part of their routine process of submitting event reports to ECRI Institute PSO’s reporting program. The PSO Deep Dive analysis consisted of 171 health IT-related events submitted by 36 healthcare facilities, primarily hospitals

(There were 8 injuries and 3 possible deaths associated with these 171 IT-related events over just 9 weeks, suggesting a damn serious problem way ahead of paper errors.)

5)  Statement by Medical Director of ECRI, same link as above:

Karen Zimmer, MD, medical director of the institute, says the reports of so many types of errors and harm got the staff's attention in part because the program captured so many serious errors within just a nine-week project last spring.  The volume of errors in the voluntary reports was she says, "an awareness raiser."

"If we're seeing this much under a voluntary reporting program, we know this is just the tip of the iceberg; we know these events are very much underreported."

6)  ECRI's continuing concerns based on continuous data submitted by their PSO member hospitals that health IT systems are "one of the top ten (and in fact in 2014, risk #1) technology risks in healthcare" (as I report at http://hcrenewal.blogspot.com/2014/04/in-ecri-institutes-new-2014-top-10.html)

This is a short list, but you get the idea about making any statements about comparative risks of paper v. electronic.

-----------------------------------------------------------------------------------------
 

---

EMAIL #2: 

 From: Silverstein,Scot 
Sent: Friday, September 05, 2014 10:08 AM
To: Neil Versel
Subject: RE: Re:" A 'false sense of security'"

Not to mention that the "real" question isNOT "whether they [electronic systems] are safer than paper records."

The real question is:  is it ethical, based on the current clinical reality and incomplete state of knowledge, to push this technology on patients?

I also could add to that question:  "...especially without informed consent?"

Scot

-----------------------------------------------------------------------------------------

-- SS  

7:33 AM

"Why Do We Always Need to Blame Somebody?" - An Investment Banker Pushes Back Against Health Care Leadership Accountability

One of the many dramatic stories generated by the destructive Hurricane Sandy illustrated, oddly enough, the influence of big finance on American academic medicine.

Vivid video showed patients being carried down darkened stairways after flooding and a power failure at Langone Medical Center in New York (for example, see this CNN story.)  Amazingly, all the patients survived, thanks to heroic work by health care professionals and first responders.  CNN noted, "Some 1,000 staff members -- doctors, nurses, residents and medical students -- along with firefighters and police officers evacuated the patients."

The medical center suffered significant damage beyond that caused by the blackout.  A New York Times story about the problems was entitled, "A Flooded Mess that was a Medical Gem."  It noted the hospital's basement flooding destroyed major equipment like MRI machines, a linear accelerator and a gamma knife.  An animal research facility was destroyed and most of the animals died.  A renovated lecture hall and the library were ruined.

What Went Wrong?

However, soon after the debate began about why the hospital flooded and power failed.  A Bloomberg story stated,  

New York University Langone Medical Center, the 705-bed hospital in lower Manhattan that assured city officials it was ready for Hurricane Sandy, stood dark and empty a day after the storm rolled through.

That story raised questions whether hospital leadership gave adequate priority to infrastructure like generators, or put too much emphasis on spending likely to produce more rapid rewards.

Blame is being placed on the building’s outdated backup power system, which has raised concern that aging infrastructure at U.S. hospitals has created a risk for similar outages that jeopardize patient care.


'Hospitals are careful to get the latest and greatest medical equipment, but then they don’t spend on the infrastructure,' Michael Orlowicz, a principal at consulting company Lawrence Associates LLC, said....

A story on ProPublica (available on Salon) noted,

Experts say such failures are troubling but not entirely surprising. Dr. Arthur Kellermann founded the emergency department at Emory University and headed it from 1999 to 2007. Now, he’s Paul O’Neill-Alcoa Chair in Policy Analysis at RAND Corporation think tank.


The other night, as the NYU evacuation was unfolding, he tweeted, 'Hospital preparedness and well-functioning backup systems are a costly distraction from daily business, until they are needed. Like now.'


In an email interview with ProPublica, Kellermann elaborated: 'I have no doubt when the hospital assured the Mayor that their backup systems were ready, they believed they were. They were wrong. What I find most remarkable about this story is that [more than seven] years after Hurricane Katrina, major hospitals still have critical backup systems like generators in basements that are prone to flooding.'

Similarly, a Reuters story included another quote from Dr Kellermann,

'I've been asking hospitals to look at their own survivability' after a natural or manmade disaster, 'and I just can't get it on their radar screens,' said Dr. Art Kellerman,...

 
It added,

For hospital administrators trying to keep their institutions in the black, disaster-resistant infrastructure is expensive and lacks the sex appeal of robotic surgery suites and proton-beam cancer therapy to attract patients.

'People don't pick hospitals based on which one has the best generator,' Kellerman said. 

The notion that hospital leaders may put short-term revenue ahead of long-term infrastructure development, even when such development might be critical for patient safety, should not surprise Health Care Renewal readers.  Hospitals are often lead or influenced by those who believe maximizing short-term revenue should be the main goal of all management, an over-generalization of the idea promoted in business schools for a generation that business leaders should maximize "shareholder value," which has come to be defined as short-term stock price (see this post).

  Who Defended the Disaster Planning

 In response to this or anticipated criticism, leaders of Langone Medical Center deployed.  Not unexpectedly, one was Richard Cohen, the vice president for facilities, as reported by ProPublica, via the Huffington Post,  

After Hurricane Irene, officials at NYU Langone Medical Center spent several million dollars protecting its backup power system from flooding, according to Richard Cohen, vice president of facilities operations.

The hospital removed a fuel tank and a set of emergency generators at street level and chose to depend on what Cohen termed an 'extremely modern, extremely reliable' system of rooftop generators.

The hospital also built a new, flood-resistant house for pumps that draw fuel from the hospital's sealed underground tank and feed it to the generators that make electricity when New York City's power fails.

One vulnerability remained, and it proved to be the system's Achilles Heel. A portion of the hospital's power distribution circuits, which direct the generated electricity out into various areas of the hospital, were located in the hospital's basement.

'It's like what happens when you have a flood in your basement and the electrical panel is in your basement,' Cohen said.

Oops.  Why a crucial component of the system meant to protect the back up power system from threats including flooding was placed in an area at risk from flooding was not clear.   Only one story I could find (in the NY Times) included a response by the Dean of the Medical School and CEO of the Medical Center Dr Robert I Grossman.

At this point, Dr. Grossman said, he could only theorize as to why the generators had shut down. All but one generator is on a high floor, but the fuel tanks are in the basement. The flood, he said, was registered by the liquid sensors on the tanks, which then did what they were supposed to do in the event, for instance, of an oil leak. They shut down the fuel to the generators.

Oops again.  Why an effort to flood proof the hospital included an undeground fuel tank which could not be operated if water got near it was also not clear. 

The most voluble defender of the hospital's management proved to be one Mr Kenneth Langone.  As noted in a blog post in the Wall Street Journal, Mr Langone is the medical center's "board chair and benefactor."  In fact, as the NY Times reported in 2008,

Kenneth G Langone, a billionaire financier and founder of Home Depot, is giving another $100 million donation to New York University Medical Center, matching the one he made anonymously in 1999. 

In return, the university plans to name the medical center the N.Y.U. Langone Medical Center,....

The WSJ blog post asserted,

Langone said the hospital 'frequently' tested its generators and they had passed the tests, and the hospital was prepared for a 12-foot storm surge. 'We anticipated 12-foot surges, which we knew we could handle. We got 14-foot surges,' he said.


Some of the hospital generators were in the basement, which flooded. Langone acknowledged that the generators were 'not in the right location,' but that was an artifact of aging facilities undergoing an extensive upgrade. 'They’ve been there for years,' he said of the generators in the basement. As part of a $3.2 billion modernization, NYU Langone was planning on buying new generators and locating them in better locations than the basement, Langone said.

Oops one more time.  Mr Langone seemed to only offer inertia as an excuse for why some generators remained in the basement after an effort to flood proof the back up electrical system.
Langone was quoted in the CNN story mentioned above,

Kenneth Langone, the chairman of the hospital's board of trustees who also happened to be a patient there until he was discharged Tuesday morning, said that regulations require the generators to be tested regularly and that they've worked every time.


Langone said the hospital is in the midst of an 'enormous' building campaign. The generators are going to be replaced in a renovation, he said.

In a Bloomberg story, Langone was quoted again,

'We believed the machines would work, and we believed everything we were told about the scope and size of the storm,' Langone said.

 In that story, he tried to deflect attention from tha apparent infrastructure failure, and presumably the responsibility of the organization's leadership for it, to the efforts of health professionals,

'The backup generators failed, it’s that simple, but the story here is the magnificence of the effort of all of our people and what they did,' Langone, 77, said yesterday....

He also defended the relatively silent Dean and medical center CEO,

'What this dean has done is nothing short of spectacular, in every respect,' Langone said of Grossman. 'So last night God decides to give us a test and our machines failed.'

The story ended with yet another of his attempts to deflect attention to management's responsibility,

'Machines fail, airplanes take off in great shape and they have malfunctions,' Langone said. 'Why do we always need to blame somebody for something that could just have happened? Why not write a story about what people did because things happened? Let’s be a little positive once in a while.'

And in the WNYC News Blog, Langone appeared yet again with this apologia, 

He said hospital pumps failed, because they were overwhelmed by an event that was 'unprecedented' and 'an act of god.'


'The generators are on the seventh floor, and the fuel supply is in cement vaults in the basement, where they're supposed to be according to code,' Langone said. 'Moisture sensors shut down the pumps, but they did what they're supposed to do.'

Summary

Certainly the survival of all the former patients at Langone Medical Center due to brave efforts by health care professionals and first responders ought to be celebrated.  From the discussion so far, it is not clear whether the infrastructure failures were unavoidable due to the scope of a huge natural disaster, or whether the failures were the results of poor planning and insufficient attention to and investment in infrastructure.  Celebration of personal and professional dedication, however, ought not to distract from determining what lessons could be learned about making health care infrastructure safer in cases of natural disaster. 

It also ought not to distract from concerns about management accountability.  In this day and age, it is not surprising that no executive at Langone Medical Center would accept any responsibility for an effort to protect its electrical back-up power from flooding that included an underground fuel tank which would be shut down if any water affected it.  However, these executives are rewarded handsomely supposedly for their "spectacular" leadership.  (Dean Grossman received $1,744,780 in the 2010-2011 period according to the NYU Hospitals Center 2010 form 990.  That document listed four other executives who made over $1 million.)  One would think they would at least try to substantively address how their patients got put into such a precarious situation.

It is surprising that the silence from management was supplanted by the opinions of a very wealthy board chairman who paid hundreds of millions for some of the improvements to the hospital that were destroyed by the storm, but improvements that may not have included fully flood proofing the hospital's back up electrical system.  Why he may well be disappointed about the loss of what he spent so much to build, it is not clear why his opinions about technical aspects of disaster preparation should replace responses from those who were responsible for disaster preparedness.  After all, Mr Langone, while very wealthy, has no evident expertise in engineering, science, or anything pertaining to protecting infrastructure from natural disasters.  (Mr Langone's biography showed his background seems to be only in investment banking and finance.)  One wonders whether Mr Langone's prominence in the discussion suggests how influential the views of investment bankers, versus those of health care professionals, engineers and scientists, have become in the operation of health care systems.

Again, it appears that the culture of finance has intruded progressively into the cultures of health care and academics during an era in which finance has been increasingly irresponsible, as shown by the global financial collapse and our current economic woes.  Instead, true health care reform would develop leadership and governance that upholds health care professionals' values rather than worshiping short term revenue.

2:03 PM

Let's Deregulate Pharmaceutical Information Technology

... after all, a full Electronic Medical Record/Order Entry/Decision Support system is far more complex, with potential for far more immediate patient impact, then pharma's research IT systems. Yet the former is unregulated.

PhRMA, you should start lobbying to create your own private Certification Commission For Pharmaceutical IT (CCPIT) now. You can start certifying all your IT by going through a checklist of features, and avoid those pesky FDA inspections just as hospital do!

Seriously, my views are identical to that of Hoffman and Podgurski.

They expressed their views in the article "Finding a Cure: The Case for Regulation And Oversight of Electronic Health Records Systems", Hoffman and Podgurski, Harvard Journal of Law & Technology 2008 vol. 22, No. 1, and now have summarized and amplified them in the short piece "Why Electronic Health Record Systems Require Safety Regulation", Bioethics Forum, Sharona Hoffman and Andy Podgurski, March 20, 2009.

Emphases mine:

...In light of these risks to patient care, federal regulations must establish rigorous quality control mechanisms. Currently, initial approval of EHR systems is conducted by a certification program operated by the Certification Commission for Health Information Technology, a private, industry-based organization. Our review of the certification criteria revealed that they are inadequate to ensure the safety and efficacy of EHR systems . For example, testing is conducted in just one eight-hour period , and the criteria do not explicitly address important issues such as safety and usability.

We believe that EHR systems should be scrutinized through a careful premarket approval process, including field testing at several facilities for at least six months. We also recommend local system oversight committees, based on the IRB model , that would oversee field testing and conduct postmarketing monitoring throughout the life of the product. Adverse event reporting would be mandated under this system to ensure that problems are detected and addressed swiftly and that the government intervenes when necessary to safeguard patient welfare.

Federal regulations should also require that all EHR systems meet specific quality standards. Audit trails and capture-replay capabilities should be required to facilitate discovery of both system and user errors, much as black boxes allow the reconstruction of conditions that led to aviation incidents .

... Because EHR systems will manage patient care to a significant degree, they must be subject to government oversight akin to the highest level of scrutiny required, in principle, by the Food and Drug Administration for complex medical devices ... Federal regulations should establish appropriate oversight and quality control through EHR system standards, approval processes, and ongoing monitoring requirements. It is only with careful oversight that providers can be assured of investing in high quality EHR products. And it is only with appropriate safeguards that the benefits of this very promising technology will be maximized–that health outcomes will improve and risks as well as costs will decline.

I do not think these suggestions are unrealistic.

After spending considerable time in pharma, to be consistent I say either we discontinue FDA regulation of pharma clinical IT, or we continue to regulate pharma's IT and add the even more complex provider healthcare IT to the pool of regulated medical devices.

-- SS

12:38 PM