Lest We Forget

Print Friendly, PDF & Email

Issues, Impacts and Insights Column


Judith A. Effken, PhD, RN, FACMI, FAAN


Effken, J. (2014). Lest We Forget. Online Journal of Nursing Informatics (OJNI), 18(1), Available at  http://ojni.org/issues/?p=3083


Issues, Impacts and Insights ColumnA major part of the United States’ Affordable Care Act (casually known as “Obamacare”) went into effect January 1, 2014.  This part of the act provides healthcare insurance for Americans who were previously unable to obtain  health insurance due to cost or preexisting conditions by allowing them to select an insurance option through an online marketplace ( HealthCare.gov) with the help of government subsidies for those in need.  Because he is by nature very proactive and tends to be an early adopter, my husband was among the first to test the new website in an attempt to help enroll a friend.  True to form, my husband persisted in these attempts for the better part of 3 months.  However, in the end his considerable effort was worth it, He was able to enroll our friend in an excellent, subsidized health plan at a price he can afford. My goal in this column is to critically reflect from the perspective of an  informatics professional on the design and implementation of the HealthCare.gov website, as I understand the issues from media reports and our own experience.  I do so, not to denigrate the government’s efforts, but to use this real-world case study as a way to reconsider how better to design and implement systems in health care generally.  What I suggest based on 20-20 hindsight may not be new to OJNI readers, but hopefully will reinforce what we have learned about best practices in design and implementation of health information systems in general. It is common knowledge that HealthCare.gov was less than a rousing success the day it went live—in fact, it was more dead than alive for a couple weeks.  What happened?  What can we learn from this?

  • The massive scope of the project.  The fact that Obamacare would require coordination of efforts by the Department of Health and Human Services (its director, Kathleen Sibelius, had primary responsibility), the Social Security Administration, the Internal Revenue Service , a variety of insurance companies and Experian should raise red flags because these entities are independent and each has its own unique software applications.  (Note that these are just the entities I am aware of; the list may be much longer.)  In addition, this was not to be a straightforward, information-providing website, but a site where applicants filed an application form, then once their application was validated, would be able to use the site to search the online “marketplace” for plans available in their states, compare the plans, and enroll as desired.  The application itself would then be transmitted to the selected insurance company along with the first payment.  With 20-20 hindsight, it is clear that this is a monster of a website application—perhaps as challenging than an integrated health information system.  The entities being connected have very different goals and procedures.  We in informatics are well aware of the challenges entailed by different languages, multiple coding systems, and security and privacy requirements. With 20:20 hindsight, it is not clear that this complexity was fully understood at the outset, particularly by leadership.  From an informatics perspective, this is why informatics professionals learn to do extensive planning, anticipate barriers and solutions, and conduct extensive, detailed analysis.  Further, the results of these preliminary efforts must be shared with leadership and subsequent planning modified as needed to reduce or eliminate identified barriers and ensure a positive, timely, within budget outcome.
  • Scope creep. Informatics professionals are familiar with scope creep; indeed many of us have encountered it when the specifications for an application change and the designer has to go back to square one (or at least modify code substantially).  In the case of HealthCare.gov, the requested changes in design specifications were substantial, requiring a great many program changes.  For example, originally the specifications allowed the public to research the various healthcare plan options without registering. That was changed later (probably to simplify the search so that people only viewed the insurance policies that were applicable to their state and circumstances).  The fixes to accommodate the specification changes  were rather messy, resulting in the need to reenter the same data on multiple pages in some cases.  As I write this, the “back end” of the program is still being corrected so that the data entered are accurately transmitted to insurers and insurance payments are appropriately credited.  With 20-20 hindsight, the core solution is better planning and analysis—and the approval, in writing, of all the players.
  • Project Management.  The New York Times reported that the initial project manager had little prior experience with informatics and could not make the day-to-day decisions that the designers needed without going to the Administration.  For example, it took weeks for an administrative decision about whether social security numbers should be used as part of the database.  My 20-20 hindsight solution?  Someone with an excellent understanding of the project and the various groups that would be involved—both within and outside the government—and not only with the responsibility and authority necessary to make day to day decisions, but also a direct line to the President to report progress and problems on a regular basis, should have been managing this project since it was so important to the President’s agenda and the nation’s healthcare future.  It is worth noting that a technology executive from Silicon Valley was recently hired to manage the project.
  • Design Process.  Based on the testimony given to Congress by programming leaders, teams from different agencies were responsible for designing different aspects of the website. The team leaders were reluctant, or unable, to name a single person in charge of the project when queried by the Congressional committee.  The obvious 20-20 hindsight solution is better coordination under the auspices of a single project manager and regular, honest communication among all involved parties.
  • An Unrealistic Timetable.  The timetable set initially left little room for testing and fixing problems before going live.  With 20-20 hindsight, the timetable clearly needed to be extended significantly.  It almost always takes more time than expected to develop an application—but sufficient time must be allowed for thorough testing.
  • Politics.  HealthCare.gov was a particularly risky project because it was the public face of the President’s signature policy achievement.  Reports suggest that the initial programming of the website may have been delayed till after the midterm election, which put additional pressure on the programmers and left little to no time for testing and making the necessary changes because the Administration was unwilling to move back the launch date.  Whether or not this is true, politics clearly plays a role in many, if not most, healthcare informatics implementations.  The rewards of the extensive financial investment need to be made visible quickly and the original timetable adhered to if people are to have faith in the process and its outcome.  In the case of HealthCare.gov, the failure to have a website that worked efficiently when it went live inflicted considerable damage on the public’s view of the Affordable Care Act (Obamacare) and on President Obama himself, while giving opponents fuel for their continuing efforts to kill the program.   My 20-20 hindsight solution:  Planning needs to be very realistic and allow, not only adequate time for analysis and design, but also adequate time for testing and modifications so that the system is as stable and reliable as possible.  Political needs should be acknowledged and accounted for in the initial planning process, but must not compromise the design and testing process.
  • Ease of Use.  The President wanted the website to be as easy to use as an ATM or at least as easy as shopping online.  That certainly wasn’t true when HealthCare.gov went live.  I received updates from my husband at least daily on how far he had gotten with the application before the screen froze.  Having to reenter the same data multiple times and encountering broken or relabeled links, as well as cryptic error messages, didn’t enhance his experience.  Although there were options to call a “human” for help, initially the designated persons were unable to provide the needed answers (probably because the system was being changed on the fly).  Problems reported would be “referred up the ladder” for someone else to resolve, but no response was ever received.  It was a red letter day indeed when one of the people my husband called to assist him in getting past the validation section said, instead of referring him up the chain of command, “Would you like me to try to help you?”  After asking him a few additional questions not previously encountered on the site and entering his responses, our friend was finally registered!  My husband still raves about this miracle worker!  With 20-20 hindsight:  Any application needs extensive testing—with the anticipated volume and type of users—before going live.  Because the HealthCare.gov site was an amalgam of different applications built for different purposes by different designers, not only the individual sections needed to be tested, but also the overall application.  In addition, the help team needed to be well prepared to assist users when they had problems because there will always be issues—even in the best planned applications–and responses should be provided in the time-frame specified.
  • Theory, Model or Framework.   I have a personal passion for the use of conceptual models or frameworks as a basis for any major effort in design, implementation, or evaluation.  We are fortunate to have numerous models available to guide technology development and implementation (e.g., Rogers’ technology adoption model, Lorenzi and Riley’s seminal book on managing technological change, or Vicente’s text on cognitive work analysis).  Nursing informatics too has developed a plethora of models (e.g., Effken, 2003; Garcia-Smith & Effken, 2013; Staggers & Parks,1993). I have no way of knowing if any such guidance was used by the group who developed HealthCare.gov, but I suspect not. A comprehensive model or framework might have helped the group identify the multiple levels, from individual to political, that needed to be recognized from the outset; and come to a common conception of the problem and potential solutions.  A comprehensive model could have helped them emphasize to leadership the need for ongoing evaluation at each stage of the project (although this would certainly not be the first health information project to cut evaluation from the project budget and rue that decision later on). Still, theories and models are not necessarily to be shared aloud with those less theoretically inclined lest they run kicking and screaming from the room!  Depending on the group, it can be more effective to apply the guidance the abstract models provide behind the scenes while keeping the external focus on the concrete.

Today our friend is enrolled in an excellent healthcare plan, HealthCare.gov is operating efficiently, and many more Americans are acquiring much needed health insurance each day.  Still those of us in informatics have to be saddened by the angst the website created initially—for leaders, designers and users alike—because many of the problems could have been avoided.  On the other hand, the trials of the HealthCare.gov team can remind us of what happens when we ignore or forget the lessons learned from theory, research and prior experience.  It is my hope that this reflection on the saga of HealthCare.gov will be instructive for that purpose—lest we forget.


Effken, J. A. (2003).  An organizing framework for nursing informatics research.  CIN: Computers, Informatics, Nurses, 21(6), 316-323.

Garcia-Smith, D., & Effken, J. A. (2013). Development and initial evaluation of the Clinical Information Systems Success Model (CISSM), International Journal of Medical Informatics, 82, 539-552.

Lorenzi, N, M., & Riley, R. T. (1995).  Managing technological change: Organizational aspects of health informatics. New York, NY: Springer.

Staggers, N., & Parks, P. L. (1993). Description and initial applications of the Staggers & Parks nurse-computer interaction framework.  Computers in Nursing, 11, 282-290.

Vicente, K. J. (1999).  Cognitive work analysis: Toward safe, productive, and healthy computer-based work.  Mahwah, NJ: Lawrence Erlbaum Associates.  

Back to Issue Index

Back to Issue Index

Be Sociable, Share!

Comments are closed.