A new "Better EHR" book and an observation re: health IT regulation, health IT amateurs, and user centered design (UCD) - "responding to user feature requests or complaints?"

A new book has appeared on improving usability of electronic health records.  The result of government-sponsored work, the book is available free for download.  It was announced via an AMIA (American Medical Informatics Association, http://www.amia.org/) listserv, among others:

From: Jiajie Zhang [support@lists.amia.org]
Sent: Tuesday, December 02, 2014 6:00 PM
To: implementation@lists.amia.org
Subject: [Implementation] - New Book on EHR Usability - "Better EHR: Usability, Workflow, and Cognitive Support in Electronic Health Records"

Dear Colleagues,

We are pleased to announce the availability of a free new book from the ONC supported SHARPC project: "Better EHR: Usability, Workflow, and Cognitive Support in Electronic Health Records".The electronic versions (both pdf and iBook) are freely available to the public at the following link:https://sbmi.uth.edu/nccd/better-ehr/

First, this book appears to be a very good resource at understanding issues related to EHR usability.  I particularly like the discussion of cognitive issues.

However, this book also holds messages about the state of the industry and the issue of regulation vs. no regulation, and impairment of innovation:

I think it axiomatic that user-centered design (UCD) is a key area for innovation, especially in life-critical software like clinical IT.  (I would opine that UCD is actually critical to safety and efficacy of these sophisticated information systems in a sociotechnically complex setting.)

I think it indisputable that the health IT industry has been largely unregulated for most of its existence, in the manner of other healthcare sectors such as pharma and traditional medical devices.

Yet, even in the absence of regulation, the book authors found this, per Section 5 - EHR Vendor Usability Practices:

a)  A research team of human factors, clinician/human factors, and clinician/informatics experts visited eleven EHR vendors and conducted semi-structured interviews about their UCD processes. "Process" was defined as any series of actions that iteratively incorporated user feedback throughout the design and development of an EHR system. Some vendors developed their own UCD processes while others followed published processes, such as ISO or NIST guidelines.

Vendor recruitment. Eleven vendors based on market position and type of knowledge that might be gained were recruited for a representative sample (Table 1). Vendors received no compensation and were ensured anonymity.

and

b)  RESULTS
Vendors generally fell into one of three UCD implementation categories:

Well-developed UCD: These vendors had a refined UCD process, including infrastructure and the expertise to study user requirements, an iterative design process, formative and summative testing. Importantly, these vendors developed efficient means of integrating design within the rigorous software development schedules common to the industry, such as maintaining a a network of test participants and remote testing capabilities. Vendors typically employed an extensive usability staff.

Basic UCD: These vendors understood the importance of UCD and were working toward developing and refining UCD processes to meet their needs. These vendors typically employed few usability experts and faced resource constraints making it difficult to develop a rigorous UCD process.

Misconceptions of UCD: These vendors did not have a UCD process in place and generally misunderstood the concept, in many cases believing that responding to user feature requests or complaints constituted UCD. These vendors generally did not have human factors/usability experts on staff. Leadership often held little appreciation for usability.

About a third of our vendor sample fell equally into each category.

In other words, a third of health IT sellers lacked the resources to do an adequate job of UCD and testing; and a third did not even understand the concept.

Let me reiterate:

In an unregulated life-critical industry, a third of these sampled sellers thought 'responding to user feature requests or complaints constituted UCD'.  And another third neglected UCD due to a 'lack of resources'.

I find that nothing short of remarkable.

I opine that this is only possible in healthcare in an unregulated healthcare sector.

Regulation, for example, that enforced good design practices and good manufacturing practices (GMP's) could, it follows, actually improve clinical IT innovation considering the observations found by these authors, through ensuring those without the resources either found them or removed themselves from the marketplace, and by making sure those sellers that did not understand such a fundamental concept either became experts it UCD, or also left the marketplace.

I can only wonder in what other fundamental(s) other sellers are lacking, hampering innovation, that could be improved through regulation.

As a final point, arguments that regulation hampers innovation seems to assume a fundamental level of competency and good practices to start with among those to be freed from regulation. In this case, that turns our to be an incorrect assumption. 

As a radio amateur, I often use the term "health IT amateurs" to describe persons and organizations who should not be in leadership roles in health IT, just as I, as a radio amateur, should not be (and would not want to be) in a leadership role in a mission-critical telecommunications project.

I think that, inadvertently, the writers of this book section gave real meaning to my term "health IT amateurs."  User centered design is not a post-accident or post-mortem activity.

-- SS

12/4/2014 Addendum:

I should add that in the terminology of IT, "we don't have enough resources" - a line I've heard numerous times in my CMIO and other IT-related leadership roles - often meant: we don't want to do extra work, to reduce our profits (or miss our budget targets), or hire someone who actually knows what they're doing because we don't really think that the expertise/tasks in question are really that important.

In other cases, the expertise is present. but when those experts opine an EHR product will kill people if released, they find the expert 'redundant', e.g., http://cci.drexel.edu/faculty/ssilverstein/cases/?loc=cases&sloc=lawsuit.

Put in more colloquial terms, this is a slovenly industry that has always made me uncomfortable, perhaps in part due to my experience having been a medical safety manager in public transit (SEPTA in Philadelphia), where lapses in basic safety processes could, and did, result in bloody train wrecks.

Perhaps some whose sole experience with indolence and incompetence-driven catastrophe has been in discussions over coffee in faculty lounges cannot appreciate that viewpoint.

Academic organizations like AMIA could do, and could have done, a whole lot more to help reform this industry, years ago.

-- SS

2:39 PM

Health IT Amateurs, Logical Fallacy and Hospital Leadership: An Inhibitor to Good Health IT

An excellent three-part article on local providers' efforts to "join the electronic medical record/clinical IT movement", including a middle section "One doctor has reservations about EMR's design, usability", was published yesterday in the Dubuque (IA) Telegraph Herald.

I was cited as that doctor (subscription required).

The article began:

Dr. Scot Silverstein travels the country attending conferences [and other countries as well - ed.], speaking on panels and voicing concerns about health care's headlong rush into a reliance on electronic medical record systems.

"Headlong rush" is an accurate description of my beliefs, as in my "cart before the horse" posts here.

The newspaper reporter continues:

"Older and younger physicians alike are increasingly concerned about the poor design and poor usability of clinical IT," Silverstein said.  

Not only that, but so is ONC, the Institute of Medicine and the National Institute for Standards and Technology, among others, who I indicated were the source of my quotes.

The Drexel University College of Information Science and Technology faculty member calls "EMR" an anachronistic term from a time when the systems were merely storage tools for records.  "What is meant in 2012 is not just an innocuous 'filing cabinet,' but an enterprise clinical resource management and workflow control system - not just storing records, but regulating and governing all clinical behavior and action," Silverstein said.

That is, indeed, my own observation.

Silverstein contends such systems are inappropriate for some health-care environments, such as emergency rooms and intensive-care units.  "They slow down and distract clinicians due to their generally poor user interfaces, in the worst possible setting, and disrupt clinician cognition," he said.

Again, not only me.  As I had written to the reporter:

... These devices are not appropriate for high-risk, high-intensity environments such as ED's, ICU's etc.  They slow down and distract clinicians due to their generally poor user interfaces, in the worst possible setting, and disrupt clinician cognition.  But don't take my word for it.  See the 2012 report from the Institute of Medicine on healthcare IT safety (I wrote about it at http://hcrenewal.blogspot.com/2011/11/iom-report-on-health-it-safety-nix-fda.html), the National Institute of Standards and Technology report on clinical IT usability (http://hcrenewal.blogspot.com/2011/10/nist-on-ehr-mission-hostile-user.html) and the literature I collated at http://www.ischool.drexel.edu/faculty/ssilverstein/cases/?loc=cases&sloc=readinglist.

In fact, document image management systems and human data abstractors are a good tradeoff to meet the needs of the most time-pressed clinicians whose time is a hospital's most valuable asset, and to make patient charts available when and where needed.  This is as opposed to "digital data field and form-based" (i.e., conventional) EMR's that force the clinicians to waste their time in clerical functions and distract them from pressing clinical matters and informational accuracy. 

He then reports:

Silverstein defines such cognition as the decision making and problem solving necessary to provide quality health care.

Precisely.

... Silverstein thinks current clinical IT programs focus too much on raw data and not enough on supporting a physician's decision-making abilities. "As a result, valuable time and energy is spent managing data as opposed to understanding the patient," he said. "Ideally, IT systems would place raw data into context with current medical knowledge to provide clinicians with computer models that depict the health status of the patient, including information on how different organ systems are interacting, epidemiological insight into the local prevalence of disease and potential patient-specific treatment regimens."

 
Actually, it's not just that I think and said these things.  The quote came verbatim, as I had indicated, from the prestigious National Research Council of the United States and their 2009 study on health IT, led by healthcare IT pioneers Drs. Octo Barnett and William Stead.

Any time logically consistent, ethics-based, common sense observations and opinions are expressed about health IT, however, one can always rely on a pundit or hospital executive for a misdirecting, illogical, and/or impertinent comment.

The expected came from Kay Takes, vice president of patient care services at Mercy Medical Center-Dubuque:

 ... Kay Takes, vice president of patient care services at Mercy Medical Center-Dubuque, said the hospital finds electronic medical records a help in the critical environment of the ICU.

"Specifically in the ICU, in the last 13 months we've gotten enhancements that allow us to download values from the medical equipment - it's automatically pulled into the medical record," she said "It's been fantastic. The availability of the information is enormously valuable. It's been a lot more of a benefit than a hindrance."

I speak from experience, in having been involved in developing those exact same capabilities in the mid 1990's (or, most accurately, protecting patients from the dangers created by the IT department in the project), that they are a convenience to those who formerly had to collect the data manually and write it on the ICU flowsheet.

The capabilities are also a mild convenience to clinicians who view the data, although the surface area of a paper flowsheet is a great advantage in seeing more data in one's field of view at one time than the usual small computer screen (to illustrate see my Feb. 2012 post EHR Workstation Designed by Amateurs at this link and my Jan. 2012 post An 'Anecdotal Complaint' About An ICU EHR at this link).

From the latter post:

... And we do still talk to each other – but even that doesn’t always “work out”, because we’ve lost our operational minds (collectively) – the shared-by-all compact, visually all data in one place, and temporally arranged – i.e., the shared nurse/doc/resp flowsheet [traditionally in an ICU, a long tabular scroll of paper for "at a glance" patient status overview - ed.] – where everybody was looking at the same page, which we no longer are – as the team is slowly discovering.

And which required no logon for sign-over bedside rounding (~40 minutes for 20-30 babies was the allotted time). The flowsheet needed only a 10-15 second glance to spot developing problems; “the computer” is effectively inaccessible in the time allotted for the twice daily sign-n-out “rounds”.

Ultimately, though, Ms. Takes, if quoted accurately, commits the logical fallacy of ignoratio elenchi ("ignorance of refutation", missing the point) – an argument that may in itself be valid, but does not address the issue in question.

For this convenience does not at all justify the downsides of EHRs, especially in an ICU:  increased time for task completion, increased risk of errors of commission or omission, and the other risks as outlined in sources such as FDA's 2010 Internal Memo on H-IT Risks, and recently by AHRQ in their IT Hazards Manager project (Appendix B).

Let's review those risks and failure modes from the AHRQ report, all observed empirically in the real world.  From the report:

A health IT hazard is a characteristic of any health IT application or its interactions with any other health care system (e.g., the people, equipment and work spaces of an ICU) that increases the risk that care processes will be compromised and patients harmed.

The potential outcomes of these hazards include medical privacy breach/identity theft, medical misadventures such as errors of commission or omission resulting in "close calls" (errors barely averted), or patient injury, or death, stress on clinicians reducing their performance, and documentation errors or data corruption increasing the risk of errors in the short, medium or long term.  In short, nothing you'd likely desire to occur while you or a family member was a patient:

Usability
• Information hard to find
• Difficult data entry
• Excessive demands on human memory
• Sub-optimal support of teamwork (situation awareness)
• Confusing information display
• Inadequate feedback to the user
• Mismatch between real workflows and HIT
• Mismatch between user mental models/expectations and HIT

Data Quality
• IT contributed to entry of data in the wrong patient’s record
• Organizational policy contributed to entry of data in the wrong patient’s record
• Patient information/results routed to the wrong recipient
• Discrepancy between database and displayed, printed or exported data
• Faulty reference information
• Unpredictable elements of the patient’s record available only on paper/scanned documents
• Lost data
• Inaccurate natural language processing
• Virus or other malware

Decision Support
• Excessive non-specific recommendations/alerts
• Faulty recommendation
• Missing recommendation or safeguard
• Inadequate clinical content
• Inappropriate level of automation

Vendor Factors
• Sub-optimal interfaces between applications and devices
• Faulty vendor configuration recommendation
• Unusable software implementation tools
• Non-configurable software
• Inadequate vendor Testing
• Inadequate vendor software change control
• Inadequate control of user access
• Faulty software design (specification)

Local Implementation
• Faulty local configuration or programming
• Inadequate local testing
• Inadequate project management
• Inadequate software change control
• Inadequate control of user access
• Suboptimal interface management

Other factors
• Inadequate training
• Excessive workload (including cognitive)
• Inadequate organizational change management
• Inadequate management of system downtime or slowdown
• Unclear policies
• Compromised communication among clinicians (i.e., during handoffs)
• Interactions with other (non-HIT) care systems
• Physical environment (e.g., hardware location, lighting, engineering)
• Inadequately secured data
• Hardware Failure
• Use error in the absence of other factors

The convenience of automated data collection through an EMR system comes at, one might say, a slight cost that may not be realized by health IT amateurs.** 

Unfortunately, their lack of knowledge of these issues reduces the caution and "pushback" required for good health IT to become the norm, and permits bad HIT to be sold.


-- SS

** Amateurs in the sense that I am a radio amateur, not a professional, formally trained telecommunications engineer, and would never take a major role in an enterprise telecommunications project, especially a mission-critical one.

8/27/12  Addendum:

A typical, anonymous, irrational response of the type we've seen on this blog before (as for example here and here) has been posted as a comment to the newspaper story:

"nemesis" posted at 1:19 pm on Mon, Aug 27, 2012. 

He travels the country complaining about existing systems? Perhaps the good prof could spend a little time designing the ideal interface/system and see if he can get anyone interested. Surely his design would win them all over.

The problem with this type of comment, of course, is that it's logically fallacious (a form of ad hominem and an odd "appeal to perfection") and irrelevant.  It is no way responds to or refutes the issues I raise in the article.   

It likely comes from a health IT pundit of some sort, but hopefully not someone in a position of authority.  In medicine, the rule of thumb is simple:  "Critical thinking always, or your patient's dead."

-- SS

9:42 AM

Why non-medical amateurs need to be kept away from authority roles in health IT ... lest their ignorance kill people

This example of a disaster waiting to happen, in the form of an error-promoting CPOE, is a poster example of why the net of litigation needs to be cast far wider than just clinicians when EHR-related errors result in injury or death:

CPOE selection screen for crucial blood thinner, coumadin (Warfarin).  Click to enlarge.

This order entry screen, from a production system (of a vendor whose stock price has recently taken a dive) shows the following.  In all fairness, I do note it's unclear if the vendor or the customer's configuration "experts" were responsible for this:

COUMADIN (warfarin) tablet 2 mg Oral daily once.
CAUTION: Potential look-Alike or Sound-Alike medication - this product is COUMADIN

with similar entries for other doses.

Below and not indented as is the selection, where the clinician is liable not to look very carefully, is the helpful interpretation:  "warfarin (COUMADIN) Tablet 2 milligram Oral daily for 1 Times."

"Oral daily for 1 Times?"

This drug needs to be given daily, generally for a very long term.  Its effect on blood clotting varies for numerous reasons in an individual over time, and needs to be checked frequently via a blood test (International Normalized Ratio or INR) to ensure the level of effect is neither too little (which could result in clots) or too much (which could result in serious or fatal bleeding).

In this case, the clinician wanted Coumadin to be administered "daily", as in "each and every day", but this was the default - daily, but only once.  "Oral daily for 1 Times."

Brilliant!  

Daily Coumadin (i.e., daily EVERY DAY), the clinician related, could be ordered only with "painstaking difficulty."

"X mg Oral daily once" is an unimaginably absurd and bizarre dosing selection to have on a CPOE system for such a critical drug - or any drug.  "Daily - once?"  

It should not, and does not, take a rocket scientist to realize this selection could quite easily lull the busy clinician into believing they have selected a dose to be continued every day - i.e., "once daily" - as per the standard usage of this drug.

To order this drug for (true) daily administration, a user must find a "repeat" icon and click the number of days the drug is to be administered.  The "repeat" icon is not readily apparent amidst screen clutter.

For other drugs, the order choices are "## mg oral daily" or similar. 

This semantic and human-computer interaction ineptitude is truly a disaster waiting to happen, especially with the medical/nursing/trainee staff turnaround that goes on in hospitals, and with the reality that clinicians are working at various hospitals with different CPOE/EHR systems.

Is this some sort scheme to prevent endless-administration Coumadin errors when the drug is actually deliberately discontinued, I ask?  If so, it's ill-conceived and dangerous at best.

By way of further information, this drug is a common anticoagulant whose use is often protective of injurious or fatal blood clots that can cause strokes or death in people with common conditions such as atrial fibrillation or prosthetic heart valves:

Warfarin is used to decrease the tendency for thrombosis or as secondary prophylaxis (prevention of further episodes) in those individuals that have already formed a blood clot (thrombus). Warfarin treatment can help prevent formation of future blood clots and help reduce the risk of embolism (migration of a thrombus to a spot where it blocks blood supply to a vital organ).

The type of anticoagulation (clot formation inhibition) for which warfarin is best suited, is that in areas of slowly-running blood, such as in veins and the pooled blood behind artificial and natural valves, and pooled in dysfunctional cardiac atria. Thus, common clinical indications for warfarin use are atrial fibrillation, the presence of artificial heart valves, deep venous thrombosis, and pulmonary embolism (where the embolized clots first form in veins).

This is an example of the kinds of mission hostility (other equally bizarre examples presented here) that results when amateurs attempt to play doctor.

I add that this type of "errorgenicity" is inexcusable.  If patients suffer harm from this type of "feature", the net of liability needs to go further than just the clinician who was caught in a web of cybernetic clinical toxicity.

-- SS

5/1/2012 Addendum:

More EHR madness and another physician, a cardiologist and electrophysiologist, who also believes these should be considered medical devices.

From DrWes blog (excerpts, and emphases mine; see entire post at link below):

The Electronic Medical Record Should be Viewed as a Medical Device 
Apr. 30, 2012

This week I received a medical record from a large academic medical center somewhere in the United States (the details were are unimportant) that has one of these new pioneering EMR systems manufactured by $13 billion-dollar company, Cerner Corporation ... what I saw was one of the better examples of how EMRs are contributing to misinformation and confusion when health care is delivered.

I received a copy of an internal medicine consult that was performed on a patient at this outside hospital. I have extracted the "medications" portion of the internist's note exactly as it was displayed in the note below ... Needless to say, I was terrified at what the system had listed as the patient's medications:

In this example, we see multitudes of medications listed more than once. We see drugs of similar classes (antihistamines, beta blockers) on the same list. We see warfarin, one of our most dangerous drugs dispensed, without a dose included. We see what seems to be outpatient meds listed with inpatient meds, I'm not sure. Honestly, we really have no idea what medications are actually being taken from this list. And yet this list of medications is listed by the EMR as the patient's "Active Medications."

Med list (page 1).  Click to enlarge; see original post for part 2.


... What the heck have we created? 

Certainly, any capable physician who cares for patients would describe this medication list as worthless.

This "med list" is similar to the list I showed at part 4 of my multi-part series on the mission hostile user experience of most commercial EHR's, from yet another system, redrawn by me in redacting the vendor ID.  These lists reflect a mercantile computing person's view of a med list as an inventory of pills:

 Another "what the heck have we created?" EHR med list, on screen. Click to enlarge.

 Dr. Wes also asks:

... So how will we measure problems with EMRs? It seems industry representatives would rather not address these concerns. We should ask ourselves, is anyone thinking about this?

Yes, they are.  And we are spreading our thinking to one place where action might actually occur sooner rather than later:  to the Plaintiff's Bar.

-- SS

7:42 AM