MBA-holding Informatics Fellow's Portfolio: Revolutionizing Healthcare Through Plagiarism

I highlighted the MBA culture at least once before on this site, on April 16, 2010 at "Healthcare IT Corporate Ethics 101: 'A Strategy for Cerner Corporation to Address the HIT Stimulus Plan'", http://hcrenewal.blogspot.com/2010/04/healthcare-it-corporate-ethics-101.html.

In that post, I noted MBA candidates/Cerner employees happily conspiring in a paper at Duke's Fuqua School of Business towards combination in restraint of trade through "recommending that Cerner collaborate with other incumbent vendors to establish high regulatory standards, effectively creating a barrier to new firm entry. "

Combination in restraint of trade: An illegal compact between two or more persons to unjustly restrict competition and monopolize commerce in goods or services by controlling their production, distribution, and price or through other unlawful means. Such combinations are prohibited by the provisions of the Sherman Anti-Trust Act and other antitrust acts.

The paper was highlighted at  professor David Ridley's page "Duke University Fuqua School of Business: Past Papers" - that is, until a few days after my blog post went up and he was informed of it.   You can see cached copies of the paper and page at the post at link above.

Today, I've had another experience with an MBA holder who has decided to enter the field of Medical Informatics.

I received an unsolicited Cc: of an email, sent by a professional in my field I do not know at a university in Australia.  The email was directed at a postdoctoral fellow at a U.S. medical informatics program in the Midwest, advising the fellow that his 'Portfolio' brag page page was plagiarized directly almost verbatim from a personal essay I'd written ca. 1999 and now archived at my current Drexel site at http://cci.drexel.edu/faculty/ssilverstein/informaticsmd/infordef1.htm, and that plagiarism was bad for informatics careers:

Date: Tue, May 5, 2015 10:28 pm
To: [Name of recipient MBA-holding informatics fellow redacted - ed.]

I was disappointed to find the following three paragraphs on the homepage of your site ([URL redacted] - ed.)

"It became apparent to me and many informatics professionals that significant confusion and misconceptions exist in hospitals, industry, and the world at large about what medical informatics is, and what experts in medical informatics do (and are able to do if given the opportunity). Also, there is confusion as to what medical informatics is not.

"The available quantity of information in most subject areas ("domains") has grown rapidly in recent decades. Issues about information and its use have become quite complex, and the issues themselves have undergone scientific study. Informatics is information science. In other words, informatics is a scientific discipline that studies information and its use.

"Both theoretical and practical issues are studied. Examples of theoretical issues include terminology, semantics (term meaning), term relationships, and information mapping (translation). Practical issues include information capture, indexing, retrieval, interpretation, and dissemination. Medical informatics, an informatics subspecialty, is the scholarly study of these information issues in the domain of biomedicine."

This text is an almost perfect copy of the introduction to Scott Silverstein’s page (http://cci.drexel.edu/faculty/ssilverstein/informaticsmd/infordef1.htm).

Plagiarism has no place in Medical Informatics, and could harm your career. I would appreciate it if you could rewrite or remove this content on your site

Best Regards 

[Professor name redacted - ed.]

There was other copied material after these paragraphs as well; almost the entire page was my words and ideas.  The page shamelessly concluded with this:

Shamelessly copied from http://cci.drexel.edu/faculty/ssilverstein/informaticsmd/infordef1.htm#importance

I do not know how the Australian professor detected the plagiarism, if he had involvement with the fellow, or the context of the interaction.

This fellow had an MBA and the title of his "portfolio" page was about his passion for 'revolutionizing healthcare.'

It's clear he thought his stealing my words and ideas would never be noticed. In other words, exploiting my creativity for his own gain and image-enhancement was fine.

Obviously in our connected world, plagiarism is not a good idea. Perhaps not so obvious are the predatory values of the MBA degree and the damaging effects on all our healthcare when such individuals 'revolutionize' it.

I sent a demand for the material's immediate removal along with a polite suggestion of unpleasantness if he does not comply.

I am not naming the postdoc due to having bigger fish to fry.

-- SS

Update 5/6/2015: 

The fellow has removed about 3/4 of my material from the webpage in question, but a passage remains verbatim.

I've sent another request backed by a screenshot and link to my material, and a rather more direct consequence of failure of complete removal.

Between the IT invasion of health IT and the MBA invasion, perhaps patients need to hire fulltime medical advocates for everything more serious then getting a boil lanced.

-- SS

Additional thought 5/7/2015:

I should add the misleading credentials exaggeration of minimal exposure to informatics (a seminar or AMIA short course at best) leading to a claim of a non-existent "American Medical Informatics Certification for Health Information Technology" by an erstwhile NextGen VP who also apparently holds a MBA with a concentration in Health Administration, see http://hcrenewal.blogspot.com/2009/02/nextgen-and-vendordoctor-dialog-yet.html.

8:11 AM

AMIA Board: specification of core competencies in Biomedical Informatics

In 1998 I launched a website called "Medical Informatics and Leadership of Clinical Computing" (now entitled "Contemporary Issues in Medical Informatics- Common Examples of Healthcare Information Technology Difficulties" at this link).

Its theme was that leadership of IT in healthcare was severely lacking in the formal competencies needed to reach any measure of success, and in fact the lack of informatics competencies in the usual IT actors was causing wasted resources and patient harm.

I had also commented that the term "Medical Informatics" itself was being misappropriated by anyone claiming to do anything with computers in medicine, even the creation of trivial and/or low-value programs.

Sadly, little has changed in that regard since 1998; in fact things are much worse.  The meaning of the term "Medical Informatics" itself has become severely blurred, and job listings that use the term are largely misguided.  They often seek a nurse (most common) or doctor (less common) without formal education in the domain, who's dabbled with hospital IT systems, to lead clinical IT projects.  This is a totally inappropriate and even dangerous approach (example here).

The American Medical Informatics Association has released a paper "AMIA Board white paper: definition of biomedical informatics and specification of core competencies for graduate education in the discipline" that is long, long overdue.  As of this writing, full text is available a this link:  http://jamia.bmj.com/content/early/2012/06/07/amiajnl-2012-001053.full.

This paper certainly provides a robust affirmation of ONC's recommendations on healthcare IT leadership roles that I wrote of in my Oct. 2009 post "ONC Defines a Taxonomy of Robust Healthcare IT Leadership."

Some highlights of the new AMIA paper:

Abstract

The AMIA biomedical informatics (BMI) core competencies have been designed to support and guide graduate education in BMI, the core scientific discipline underlying the breadth of the field's research, practice, and education. The core definition of BMI adopted by AMIA specifies that BMI is ‘the interdisciplinary field that studies and pursues the effective uses of biomedical data, information, and knowledge for scientific inquiry, problem solving and decision making, motivated by efforts to improve human health.’ Application areas range from bioinformatics to clinical and public health informatics and span the spectrum from the molecular to population levels of health and biomedicine. The shared core informatics competencies of BMI draw on the practical experience of many specific informatics sub-disciplines. The AMIA BMI analysis highlights the central shared set of competencies that should guide curriculum design and that graduate students should be expected to master.

Note that Biomedical Informatics, which the Board feels is a broader term encompassing all of the information-science disciplines in healthcare and biomedical research, is defined as "a core scientific discipline underlying the breadth of the field's research, practice, and education."  One does not acquire expertise in a scientific discipline without first rigorously studying that discipline, e.g., as is done in medical school to gain optimal understanding of clinical medicine.

... The present articulation of BMI core competencies is intended to support AMIA and its members in promoting the discipline as a career choice, and to provide guidance to students and curriculum developers when choosing, designing (and implementing), or re-designing graduate-level academic BMI programs.

(Who needs graduate education in Biomedical Informatics when all that seems to be needed is a little on-the-job dabbling?)

... Defining BMI as the scientific core of a discipline that has broad applications across health and biomedicine highlights its foundational role and refutes the kind of reductionism that superficially explains BMI simply as the application of information technology (IT) to biomedical and health problems.

I termed that phenomenon "Medical Instamatics" on that late 1990's site.  Unfortunately, the "reductionism" is all too prevalent today.  People whose BMI education and skill levels (which I define as the ability to apply deep knowledge and experience to successfully manage the unexpected, not just manage traditional activities via a book of "process"), are often at the amateur level -- in the same sense that I am a radio amateur, not a telecommunications/engineering professional -- or worse.  This wreaks havoc (as here) in health IT, especially when led by senior management also incognizant of the issues.

Definition: Biomedical informatics (BMI) is the interdisciplinary field that studies and pursues the effective uses of biomedical data, information, and knowledge for scientific inquiry, problem solving, and decision making, driven by efforts to improve human health.

Scope and breadth of discipline: BMI investigates and supports reasoning, modeling, simulation, experimentation, and translation across the spectrum from molecules to individuals and to populations, from biological to social systems, bridging basic and clinical research and practice and the healthcare enterprise.

Theory and methodology: BMI develops, studies, and applies theories, methods, and processes for the generation, storage, retrieval, use, management, and sharing of biomedical data, information, and knowledge.

Technological approach: BMI builds on and contributes to computer, telecommunication, and information sciences and technologies, emphasizing their application in biomedicine.

Human and social context: BMI, recognizing that people are the ultimate users of biomedical information, draws upon the social and behavioral sciences to inform the design and evaluation of technical solutions, policies, and the evolution of economic, ethical, social, educational, and organizational systems.

There is also a call for experts to:

• Acquire professional perspective: Understand and analyze the history and values of the discipline and its relationship to other fields while demonstrating an ability to read, interpret, and critique the core literature.

In effect, health IT amateurs, including those in traditional business computing, have little to no formal education or experience in reasoning, modeling, simulation, experimentation, and translation; developing, studying, and applying theories; building on and contributing to computer, telecommunication, and information sciences and technologies; and drawing upon the social and behavioral sciences to inform design of these complex systems.

BMI is the core scientific discipline that supports applied research and practice in several biomedical disciplines, including health informatics, which is composed of clinical informatics (including subfields such as medical, nursing, and dental informatics) and public health informatics (sometimes referred to more broadly as population informatics to capture its inclusion of global health informatics). There are related notions, such as consumer health informatics, which involves elements of both clinical and public health informatics. BMI in turn draws on the practical experience of the applied subspecialties, and works in the context of clinical and public health systems and organizations to develop experiments, interventions, and approaches that will have scalable impact in solving health informatics problems. However, it is the depth of informatics methods, shared across the spectrum from the molecular to the population levels that defines the core discipline of BMI and provides its coherence and its professional foundation for defining a common set of core competencies.

Here is the diagrammatic represention of the above in the full article:

Biomedical informatics and its areas of application and practice, spanning the range from molecules to populations and society

Finally, excerpts from the meat of the article on Prerequisite knowledge and skills.  This depth and breadth of knowledge does not come from studying business computing, dabbling with systems by nurses or physicians lacking formal domain education at the graduate level or beyond, or by guessing by the seat of one's pants:

• • Fundamental knowledge: Understand the fundamentals of the field in the context of the effective use of biomedical data, information, and knowledge. For example:
    • ... Healthcare: screening, diagnosis (diagnoses, test results), prognosis, treatment (medications, procedures), prevention, billing, healthcare teams, quality assurance, safety, error reduction, comparative effectiveness, medical records, personalized medicine, health economics, information security and privacy.
  • Procedural knowledge and skills: For substantive problems related to scientific inquiry, problem solving, and decision making, apply, analyze, evaluate, and create solutions based on biomedical informatics approaches.
    • Understand and analyze complex biomedical informatics problems in terms of data, information, and knowledge.
    • Apply, analyze, evaluate, and create biomedical informatics methods that solve substantive problems within and across biomedical domains.
    • Relate such knowledge and methods to other problems within and across levels of the biomedical spectrum.
• Theory and methodology: BMI develops, studies, and applies theories, methods, and processes for the generation, storage, retrieval, use, management, and sharing of biomedical data, information, and knowledge. All involve the ability to reason and relate to biomedical information, concepts, and models spanning molecules to individuals to populations:
  • Theories: Understand and apply syntactic, semantic, cognitive, social, and pragmatic theories as they are used in biomedical informatics.
  • Typology: Understand, and analyze the types and nature of biomedical data, information, and knowledge.
  • Frameworks: Understand, and apply the common conceptual frameworks that are used in biomedical informatics.
    • A framework is a modeling approach (eg, belief networks), programming approach (eg, object-oriented programming), representational scheme (eg, problem space models), or an architectural design (eg, web services).
  • Knowledge representation: Understand and apply representations and models that are applicable to biomedical data, information, and knowledge.
    • A knowledge representation is a method of encoding concepts and relationships in a domain using definitions that are computable (eg, first order logics).
  • Methods and processes: Understand and apply existing methods (eg, simulated annealing) and processes (eg, goal-oriented reasoning) used in different contexts of biomedical informatics.
• Technological approach: BMI builds on and contributes to computer, telecommunication, and information sciences and technologies, emphasizing their application in biomedicine.
  • Prerequisite knowledge and skills: Assumes familiarity with data structures, algorithms, programming, mathematics, statistics.
  • Fundamental knowledge: Understand and apply technological approaches in the context of biomedical problems. For example:
    • Imaging and signal analysis.
    • Information documentation, storage, and retrieval.
    • Machine learning, including data mining.
    • Networking, security, databases.
    • Natural language processing, semantic technologies.
    • Representation of logical and probabilistic knowledge and reasoning.
    • Simulation and modeling.
    • Software engineering.
  • Procedural knowledge and skills: For substantive problems, understand and apply methods of inquiry and criteria for selecting and utilizing algorithms, techniques, and methods.
• Human and social context: BMI, recognizing that people are the ultimate users of biomedical information, draws upon the social and behavioral sciences to inform the design and evaluation of technical solutions, policies, and the evolution of economic, ethical, social, educational, and organizational systems.
  • Prerequisite knowledge and skills: Familiarity with fundamentals of social, organizational, cognitive, and decision sciences.
  • Fundamental knowledge: Understand and apply knowledge in the following areas:
    • Design: for example, human-centered design, usability, human factors, cognitive and ergonomic sciences and engineering.
    • Evaluation: for example, study design, controlled trials, observational studies, hypothesis testing, ethnographic methods, field observational methods, qualitative methods, mixed methods.
    • Social, behavioral, communication, and organizational sciences: for example, computer supported cooperative work, social networks, change management, human factors engineering, cognitive task analysis, project management.
    • Ethical, legal, social issues: for example, human subjects, HIPAA, informed consent, secondary use of data, confidentiality, privacy.
    • Economic, social and organizational context of biomedical research, pharmaceutical and biotechnology industries, medical instrumentation, healthcare, and public health.
      
      
      
      

While nobody is an expert in all of these areas, skills in many of them are essential for successful and safety-promoting leadership in the health IT domain.

I repeat, this depth and breadth of knowledge does not come from studying business computing, dabbling with health IT, or by guessing by the seat of one's pants.  It comes about from rigorous education and experience in the appropriate domains at the graduate and (especially) post-doctoral levels.

Amateurs mistakenly put in leadership positions, and their organizations, are going to increasingly find themselves in legal hot water over mistakes in design and implementation that result in patient harm, security breaches, overbilling and other issues.

That is probably what it will take to have hospitals manage health IT talent more appropriately.

Finally, I plead guilty to tooting my own profession's horn.

Somebody needs to when the stakes are so high for patients.

-- SS

5:49 AM

Have we suffered a complete breakdown in the scientific method with regard to EHR and clinical IT?

Have we suffered a complete breakdown in the scientific method with regard to EHR and other clinical IT?

I read announcements like this with trepidation:

http://govhealthit.com/articles/2009/03/31/sebelius-confirmation.aspx
“The goal,” Sebelius said, “is to provide every American with a safe, secure electronic health record by 2014." The nominee also endorsed efforts to use data gleaned from electronic medical records to conduct “comparative effectiveness research" (CER) to provide information on the relative strengths and weaknesses of alternative medical interventions to health providers and consumers.”

Recovery Act funds have been allocated to NIH specifically for comparative effectiveness research. NIH has further specified the definition of CER as:

"[A] rigorous evaluation of the impact of different options that are available for treating a given medical condition for a particular set of patients. Such a study may compare similar treatments, such as competing drugs, or it may analyze very different approaches, such as surgery and drug therapy."

NIH states that such research may include "the development and use of clinical registries, clinical data networks, and other forms of electronic health data that can be used to generate or obtain outcomes data as they apply to CER."

The problems I foresee concern the word "rigorous" as in the above definition.

The use of EHR data to reliably detect uncommon (but strong, discrete) early warning signals from a single drug or treatment -- to then be subject to more rigorous study with reasonable controls -- is itself a Medical Informatics "Grand Challenge." An example would be finding VIOXX's association with myocardial infarction earlier than we did, via an EHR-based automated postmarket surveillance process.

Doing this is a "grand challenge" due to the nature of EHR data, which is as far from "clinical trials clean" as possible. It is what might be called highly uncontrolled. The statistical methods needed to reliably pull signals out of the muck for even a single drug are still exploratory, the problems formidable if one wants to stay scientifically sound. I wrote about the experimental nature of such efforts a few years ago here, and believe an effort got underway at U. Indiana/Regenstrief to test such methodologies for postmarket surveillance about the same time.

Now we have had what appears to be a leap of faith and logic of irrationally exuberant proportions, and probably a deviation from sound science as well. The government has announced enthusiasm for EHR data-based comparative effectiveness research (CER) not to aid science, but to cut costs (implying skipping the rigorous confirmatory phases) through elimination of more costly drugs and treatments deemed less effective or at effectiveness parity compared to less expensive choices. Following this thinking, perhaps in the future a metric will be developed for an "acceptable" improved benefit/cost ratio for expensive drugs that are better than cheaper alternatives?

This overconfidence in EHR data is of concern. To detect relatively less concrete (i.e., than major ADE) "outcomes differences" between two or more drugs or treatments via EHR data - did treatment A lower blood pressure more than drug B, did drug C lessen depression more than drug D - rises to the level of "grand overconfidence in computing." To accomplish this task with reasonable scientific certainty from reams of EHR data, originating from different vendor systems, input by myriad people of different backgrounds with differing interpretations of terminologies (students/MD's/RN's etc) under different pressures (time, reimbursement maximization), and so forth, seems a stretch. What will the p values and predictive values be for such studies? Yet our incoming HHS secretary touts such methods?

Ironically, the gold standard in medical science is the controlled clinical trial, yet EHR-based comparative effectiveness research itself as a research methodology, now touted by our government, seems to have gotten a pass.

Even what I would consider minimum requirements for scientific treatment comparisons, such as well designed and reasonably controlled registries as developed here for interventional cardiology, with hundreds of granular, finely defined and "tuned" data elements, appear to be bypassed in EHR "miracle claims." Such precise registries take months or years to develop, implement, and train users to interact with properly. Further, such registries are not portable and must be created for individual medical domains and subdomains. Uncontrolled EHR data is no substitute for such efforts.

The following question arises:

Where are the comparative effectiveness studies that compare 1) EHR-based comparative effectiveness studies of drugs and treatments to 2) controlled clinical trials-based comparative effectiveness studies?

In other words, where are the meta-clinical trials that compare EHR data mining-based comparative effectiveness research as a methodology, vs. the "traditional" gold standard methodology of controlled clinical trials to compare drugs or treatments? How do we know EHR-based CER studies will not produce GIGO that will cause harm through ham-fisted elimination or defunding of useful treatment options?

While there are initial efforts underway to increase understanding of CER, e.g., "Broad Challenge Area 5" (PDF) of the NIH RC1 Challenge Grants in Health and Science Research, ominously, there is a lot of potential advantage to be had with terabytes of uncontrolled data and a political agenda.

I fear that what will come from "comparative effectiveness research" that draws upon uncontrolled EHR data will be politics masquerading as comparative effectiveness research. Good luck to private practitioners and medical innovators. Good luck, pharma. Good luck, patients.

This movement towards EHR uncontrolled data alchemy represents a further deviation from medical science towards the Syndrome of Inappropriate Over-Confidence in Computing (a.k.a. SICC Syndrome) writ large.

It seems the IT industry has now rendered a scientific approach to HIT and its use obsolete. We see this "post scientific era" phenomenon in the takeover of clinical IT by vendors who contractually demand suppression of sharing of problems, we see it in a remarkably uncritical push for EMR's by 2014 now involving force of government (only financial at present, but will punitive licensure issues and other measures be off the table?) despite a growing body of literature advising caution, we see a consortium of big business/payers/vendors/myriad secondary feeder organizations gunning full blast for this technology without consideration of the possible downsides.

Biomedical informatics, a scientific discipline (at least those parts of it not yet compromised by conflicts of interest), as a relevant field is very much a minority player in today's health IT.

Even the contributions from experts and pioneers in the field of Biomedical Informatics, in the form of the Jan. 2009 National Research Council's report that "Current Approaches to U.S. Health Care Information Technology are Insufficient" (here) has not had much impact.

I see Biomedical Informatics' death as a relevant discipline that anyone of importance pays attention to, not too far down the road as well.

-- SS

Addendum April 20:

We've seen this phenomenon in our economy. WSJ "Information Age" writer L. Gordon Crovitz notes:

... In a paper for the scientific journal of the Royal Society back in 1994, Harvard economist Robert Merton wrote that "any virtue can become a vice if taken to extreme, and just so with the applications of mathematical models in finance practice." We know even better now that some risks can be calculated and thus reduced, while some unknowns cannot be turned into probabilities. "The mathematics of the models are precise, but the models are not, being only approximations to the complex, real world."

I believe EMR data at best is a very loose approximation to the real world. It contains many "unknowns" regarding quality and reliability that cannot be turned into probabilities no matter how fancy the math. Asking too much of EHR data becomes a vice, not a virtue.

-- SS

9:25 AM

Health IT Project Success and Failure: Recommendations from Literature and an AMIA Workshop

A preprint article "Health IT Project Success and Failure: Recommendations from Literature and an AMIA Workshop" by Bonnie Kaplan and Kimberly D. Harris-Salamone (doi:10.1197/jamia.M2997) has just been published and made available in the Journal of the American Medical Informatics Association (JAMIA). It will appear in the May/June 2009 issue of JAMIA.

[Addendum 1/2010: a link to a free PDF of the published article is here.]

There is a history to this publication, that summarizes the findings and recommendations of a surprisingly well attended workshop ironically named "Avoiding The F-Word: IT Project Morbidity, Mortality, and Immortality." The workshop was held at the 2006 national meeting of AMIA.

Ten years ago, in early 1999 I started a website hosted on AOL, and called at that time "Medical Informatics and Leadership of Clinical Computing: Common Examples of Healthcare IT Failure." A historical version of the site (with a "politically corrected" title) is here; the modern site is now here. The website was unique at the time and today, remarkably, remains nearly so (see this pdf poster from AMIA's 2006 annual meeting).

Yet I was to soon learn through correspondence about the website that these problems were common as well as international in scope.

Healthcare IT failure was (and is) a somewhat "taboo" subject, in no small part because of contractual gag clauses on users and vendor lack of legal accountability for defects, as well as a striking overconfidence in computing and timidity about discussing organizational failure and IT mismanagement. Its open discussion has therefore been severely impaired and minimized.

I pushed this topic hard among the Medical Informatics community, yet for years there was resistance to bringing this issue to the fore. I took significant "heat" for my views as well, making me somewhat of an outcast in my own professional community and even to one of my former mentees. This phenomenon may have been a combination of academic concerns about a topic that might dilute the field, preferential and understandable focus on successes, conflicts of interest, and other issues.

[May 2009 addendum: the "taboo" nature of the subject of HIT difficulty and failure might in fact have been engineered by the HIT industry; see the article "The Machinery Behind Healthcare Reform: How an Industry Lobby Scored a Swift, Unexpected Victory by Channeling Billions to Electronic Records" by Robert O'Harrow, Jr., May 16, 2009, Washington Post -- ed.]

I, on the other hand, having been (pre-informatics fellowship) a Medical Review Officer in the medical department of a regional mass transit authority, and observing a fatal subway elevated accident related to drug abuse, was not going to back down.

The accident was the worst in perhaps sixty years in Philadelphia, with hundreds of injuries and several deaths in part due to overriding of medical judgment by non-medical personnel on drug testing at time of union-demanded reinstatement of persons known to have problems. This was as a quid pro quo for labor contract approval.

Worse, there were IT errors as well -- again with overriding of medical personnel, namely, me. I'd written a program on our departmental PC for random drug test date selection, but by MIS edict, use of that program was overridden in favor of some slop they authored for their machines - which "forgot" to enroll the operator behind the accident in random testing altogether. Months went by with no tests, and then at the time of the accident the operator's cocaine metabolite levels "blew the lid off the GCMS machine", according to the toxicologist performing the test.

Having been informed by my earlier transit authority experience (and gaining some lessons in fortitude from the militant mass transit labor unions I had dealings with), I started the HIT difficulties website in late 1998-early 1999 after watching ICU patients needlessly being put at risk and the high risk Invasive Cardiology cath lab being disrupted as CMIO at Christiana Care in Mr. Biden's home state. These problems were once again due to the ill-informed, capricious edicts of overempowered MIS leaders, sanctioned by equally ill-informed executive leadership.

In fact, I quit that CMIO role, no longer being able to tolerate being a marginalized "Director of Workarounds to Dangerous Health IT Mismanagement" (as are many of today's CMIOs), unable by management caprice to make even the most basic of corrections to obvious and risky - and grossly negligent, perhaps even criminally negligent - design and implementation errors.

With the exception of a period of time in pharmaceutical research IT from 2000-2003, I periodically sought additional case information on healthcare IT difficulties from colleagues via email and AMIA message board postings, but the results were lukewarm at best.

I'd hoped upon returning to a focus on the healthcare provider IT sector in late 2003 that conditions in that sector would have improved. I was wrong. I continued to comment on these issues but with results similar to the past.

However, as more informaticists became familiar with the organizational and clinician chaos that ill-conceived clinical IT and/or suboptimally managed electronic medical records projects were causing, a "critical mass" of interest and determination appeared to have been reached by early 2006.

At my request my assigned research assistant at Drexel sent out the following message to a number of AMIA message boards, the clinical information systems (cis), ethical, legal and social (els), evaluation special interests (eval-sig) and the people and organizational issues (poi) workgroups.

From: poi-wg-bounces@mailman.amia.org [mailto:poi-wg-bounces@mailman.amia.org] On Behalf Of Yunan Chen
Sent: February 27, 2006 8:46 AM
To: cis-wg@mailman.amia.org; els-wg@mailman.amia.org; eval-sig@mailman.amia.org; poi-wg@mailman.amia.org
Subject: [poi-wg] Healthcare IT failure cases

Hello:

My name is Yunan Chen and I am a research assistant for Dr. Scot Sliverstein in the Institute of Healthcare Informatics , College of Information Science & Technology at Drexel University . Currently, we are working on a project regarding Healthcare IT infrastructure failures.

The purpose of our project is to investigate why Healthcare IT infrastructure may fail from a social- technologic perspective. We will collect cases published in scientific papers and newspapers, as well as unpublished stories by the Healthcare IT practitioners. Then, we will categorize these cases and build a casebase for future implementation guideline. We have already collected some cases which are listed in: http://home.aol.com/medinformaticsmd/failurecases.htm . The cases listed here were collected from 1998-2001. Now we hope to gather newly happened cases to enrich our collection. If you experienced or heard of any interesting story, please do not hesitate to write it down and send it to me. We hope to hear more from healthcare IT practitioners' hand-on experience. Any story is valuable to us. Any posting of the cases will be keep anonymous and we won't reveal identities of people or organizations .

We will move the Healthcare IT failure cases webpage to Drexel University server later, so all your stories will be listed on that site. We will inform you once the new website is ready.

If interested, please write back to me at: yunan.chen@ischool.drexel.edu . Comments and suggestions are welcome. I am looking forward to hearing back from you. Thanks very much.

Yunan Chen
Doctoral student & Research assistant
Institute of Healthcare Informatics
College of Information Science & Technology
Drexel University

It soon became apparent that a critical mass had been reached. A torrent of shared experiences of HIT difficulty in the various message boards appeared from all over the country and from other countries as well.

Calls for a formal AMIA panel or workshop on this issue arose. This call was energetically supported by a heterogeneous and widespread group of informatics professionals.

This momentum prompted the "Avoiding The F-Word" workshop to be collaboratively formulated in a wonderful volunteer effort led by former colleague Bonnie Kaplan at the Yale Center for Medical Informatics (where I'd completed my postdoc in medical informatics), and Kim Harris-Salamone, along with the AMIA workgroup members.

Ten AMIA workgroups and over fifty HIT experts ultimately participated in the workshop.

In Sept. 2006, before the workshop occurred, I wrote at "The holes in the quest to enable the electronic clinical trial ...":

... I also think some answers to the question "Where are the holes in the quest to enable the electronic clinical trial?" will be found in an upcoming Nov. 2006 American Medical Informatics Association Annual Conference workshop on healthcare IT failure entitled "Avoiding The F-Word: IT Project Morbidity, Mortality, and Immortality", the first of its kind:

SESSION DESCRIPTION

Recent studies of health care computer applications and the reported failures of well-known systems surprised the medical informatics community, leading to questions of how to increase the chances of IT systems success and the reduction of errors.

Similar problems plague a variety of different systems, whether for institutions as a whole, for ancillary services, or for consumer health, and have done so for many years. Despite an accumulation of best practices research that has identified a series of success factors, some 40% of information technology developments in a variety of sectors are either abandoned or fail, while fewer than 40% of large systems purchased from vendors meet their goals. According to the recent CHAOS Report by The Standish Group, which surveyed failures of IT in general (not just in health care), only 34% of IT projects were considered truly successful. Similar numbers have been estimated for health care, and the number has unfortunately remained approximately the same for at least the last 25 years. While there have been some published reports of failures, removals, sabotage of systems, or how failures became successes or were otherwise redefined, there has been too little opportunity to learn from studies in which technology interventions resulted in null, negative, or disappointing results.

The purpose of this session is to examine why this happens and what might be done to improve the situation, and to collaboratively develop a series of frameworks for various types of systems and healthcare settings to aid in implementation and evaluation. The session builds on a lively exchange by numerous members of a number of AMIA Working Groups concerning success and failure in medical informatics.

The session will be devoted to better defining or characterizing "success" and "failure." From there, participants will break out into smaller groups to continue the discussion, develop a set of important issues, action items, and recommendations.

The report, now published, is filled with pearls such as these:

• With the US joining other countries in national efforts towards the many benefits health information technology use can bring for health care quality and savings, recent sobering reports recall the complexity and difficulties of implementing even smaller-scale systems. Despite best practice research identifying success factors for health information technology projects, a majority, in some sense, still fail.

• With the United States Congress appropriating more than $20 billion for health information technology as part of the February, 2009 economic stimulus package, the US joined other countries in national efforts towards the many proven benefits such technology use can bring for health care quality and savings. Moreover, Medicare, along with private and commercial health plans, is implementing a new paradigm for paying for health care services in the US, known as Value-Based Purchasing (VBP), or pay for performance initiatives (P4P). These initiatives rely heavily on using electronic health records to provide clinical documentation that proves the value of their services. Tempering the fervor, though, are recent sobering reports that raise concerns about how the technology is designed and deployed.

• Similar failure rates [to other types of IT] have been reported for health IT.(14, 15) Hospitals are among those organizations where delays and cancellations of software projects are endemic.(16) For years, problems have plagued the implementation of health IT applications, whether for ancillary services, for whole institutions, for regional or national systems, or for consumers. Today's problems are reminiscent of those analyzed since at least the 1970s in classic studies of hospital information and patient record systems.(17-19) In 1990, Dowling estimated that staff interfere or sabotage with "nearly half" of projects(20), while Heeks noted in 2006 that it is his "best estimate...that most HIS [health information systems] fail in some way."(15) Recent studies and newspaper accounts cite difficulties in a variety of health information technology applications. Over the years, in many countries, patterns of severe problems repeatedly have beset a variety of efforts: hospital information systems and electronic records(21-26); ambulance services(27, 28); community, regional, and national health information networks (28-33); public health systems (34, 35); patient education (36); and physician order entry.(18, 19, 37-41)

• Yet, while there have been some published research reports of health care IT failures, unfortunately, there has been an absence of systematic and thoughtful publication of lessons learned from IT interventions where there have been null, negative, or disappointing outcomes.(27, 53) Despite calls for increased research, there are insufficient numbers of published research reports of health care IT failures, removals, sabotage of systems, or how failures became successes or were otherwise redefined. As in other sectors (69), IT-related failures in health care often are covered up, ignored, or rationalized, so mistakes are repeated.

• Participants emphasized that communication and workflow issues add to project complexity. Health care requires collaboration, as does system implementation, yet there is difficulty in translating among specialties, stakeholders, clinicians, and implementers, sometimes to the point of a seeming "culture clash."

• The workshop concluded with reports from break-out groups charged with discussing ideas for how AMIA could address health informatics failure. Break-out groups made suggestions concerning: research and publication, best practices, advocacy, education, certification, and databases and knowledge integration.

The workshop findings concluded with these words of wisdom:

Much has been learned about success and failure in IT implementation, but we need to understand more. There are legal issues when a system "fails," including just what constitutes "failure." There are social issues, ranging from how such failures affect various groups and health informatics as a whole (including possible policy and regulatory reactions), to the social aspects of what makes for a "successful" implementation. Finally there are ethical issues involved in evaluating system "success" or not sufficiently attending to previously-identified success factors and best practices.(24)

Most "failures" are failures to properly apply managerial wisdom that has been substantiated by research and experience. Perhaps the worst aspect of failure is failure to learn from past experiences so that the same issues and problems are not perpetuated.

While the workshop was a culmination of the kindling on this crucial topic I've been involved in since being overruled on patient-endangering healthcare IT mismanagement, I believe we as a society, and the HIT sector as a for-profit, entirely unregulated industry lacking accountability (the accountability rests on clinical end users, as in, in the courtroom in malpractice suits), are still very far from solutions to that final observation on failure to learn from history.

The observations of this workshop are only made more acutely important by what I believe are ill advised, rushed plans to coerce US healthcare organizations and practitioners to adopt these evolving, arguably still-experimental virtual medical devices by 2014, or face penalties.

-- SS

10:43 AM