OJNI

Information

This article was written on 23 Mar 2012, and is filled under Volume 16 Number 1.

Current post is tagged

, , , , , , , , , , , , ,

Stakeholder Dissonance as a Critical Determinant of an E-health Initiative: A Case Study

by

Elizabeth A. Samaras RN, MSN, NP, AOCN and

George M. Samaras, PhD, DSc, PE, CPE, CQE

This edition of OJNI was made possible by our Sponsor
Chamberlain College of Nursing

CITATION

Samaras, E. and Samaras, G. (February 2012). Stakeholder Dissonance as a Critical Determinant of an E-health Initiative: A Case Study. Online Journal of Nursing Informatics (OJNI), 16 (1), Available at http://ojni.org/issues/?p=XXX

Abstract

Health Information Technology (HIT) systems are developed and deployed solely because of real or perceived value by their stakeholders.  Proper identification of all stakeholders and assessment of their Needs, Wants and Desires (NWDs) is a fundamental principle of human-centered systems engineering and should be undertaken before launching new e-health or HIT projects.  The State of Colorado, in the United States, commissioned a feasibility study of a proposed internet portal for information on statewide cancer services.  This study uncovered profound disagreement regarding the domain and range of information for inclusion in the proposed portal, as well as information quality control (e.g. what constituted eligible, safe, credible, and timely cancer services information).   The authors termed this conflict between stakeholder NWDs Stakeholder Dissonance (SD), which they recognized threatened the project’s viability if left unresolved.  The strategic importance of recognizing, measuring, and controlling SD before, during, and after launching any HIT initiative and the implications of SD management for future research and practice are discussed.

Keywords

e-health; stakeholder dissonance; health information technology; outcomes

We have seen the Internet evolve as a tool in health decision-making (Daniels et al., 2007), a factor in patient-centered care and outcomes (Eysenbach, 2003; Ziebland et al., 2004), and a facilitator of evidence-based practice (Tan et al., 2006).  We are engaged in rapid escalation in healthcare system information access and delivery with the introduction of compelling incentives under the HITECH provisions of the 2009 ARRA Bill for the implementation and “meaningful use” of electronic health records, e-prescribing, and computerized provider order entry, as well as the Internet’s instrumental role in the expansion and development of Health Information Exchanges.  This revolutionary pace threatens to outpace our ability to 1) plan what is important to do, 2) validate whether we did what we planned, and 3) determine if what we did met the Needs, Wants, and Desires (NWDs) of healthcare delivery system stakeholders.  Further complicating matters is the tendency of stakeholders to have different, and sometimes opposing, priorities and ideas about how to best utilize HIT initiatives. A study involving the feasibility of a proposed unified cancer services information internet portal provides examples of the concept of stakeholder dissonance (SD), which we defined as the conflict between the NWDs of the various stakeholder groups.  In this case, it is those groups who are invested in the successful implementation of the portal, i.e., consumers, providers, regulators, and disseminators. The existence of errors, workarounds, threats to patient safety, and outright rejection of HIT are necessary and sufficient for a diagnosis of SD (Samaras & Samaras, 2010).  Anticipating, identifying, prioritizing, and mitigating SD before launching any HIT initiative is of strategic concern, if desirable patient outcomes and stakeholder satisfaction are to be achieved.  Managing SD has implications that are important for future research and health informatics practice.

 

Background

Cancer-specific health information is prevalent on the Internet, and those with cancer are particularly high users of this resource (Ziebland et al., 2004).  There are many reputable academic, governmental, and non-governmental sources for cancer information on the Web; however, despite this wealth of resources, there is no single Internet site dedicated to information on Colorado-based cancer services.  For this reason, the authors were tasked under a state-sponsored research contract with analyzing the feasibility and perceived merit of a proposed unified internet portal of Colorado cancer services information designed for cancer information stakeholders.  The proposed initiative of the Colorado Department of Public Health and Environment’s Comprehensive Cancer Prevention and Control Program was envisioned as a way to provide residents with an integrated means of accessing, understanding, and using information relevant to cancer care services throughout the state.  Successful implementation of HIT presumes meeting the NWDs of all stakeholders.

There is an extensive body of work in social sciences regarding the definition, roles, responsibilities, and rights of stakeholders (Child and Marcoux, 1999; Freeman,1994; Mitchell, Argyle &Wood, 1997); the importance of the stakeholder to public and health policy and systems development (Brugha & Varvasovszky, 2000; Curran & Totten, 2010; Keown, Van Eerd, & Irvin, 2008; Littau, Jujagiri, & Adlbrecht, 2010; Rudin, Simon, Volk, Tripathi & Bates, 2009; Stafinski, McCabe, & Menon, 2010), as well as to the organization’s mission, values, and vision (Curran & Totten, 2010; Gilmartin, 2001) . The coupling of stakeholder and dissonance is not widely found in the literature.  A keyword search of selected health, social science, and engineering databases, as well as in a popular search engine search, found only two published papers within systems engineering, human factors, and project management literature (Taylor & Tofts, 2008; Verner, Beecham, & Cerpa, 2010).  Those authors conceptualize SD as disagreement regarding project outcomes; this is not quite the same and is more limited than our definition of conflicts between the NWDs of the various stakeholders.   There is also the current authors’ previous use of nurse stakeholder dissonance as an important consideration in the design and deployment of new healthcare technologies (Samaras & Samaras, 2010) and other articles that allude to the concept without using this specific terminology (e.g., While, Forbes, Ullman, & Mathes, 2008).     Cognitive dissonance is a concept that may seem to be related and is linked by language (dissonance) and structure (dyad) to SD, but is only tangentially related to SD conceptually.  In addition to the presence of dissonance, something is also “at stake” in both concepts.  Just as the stakeholders’ interests in a given project are “at stake” in SD, an individual’s strongly held beliefs – as well as the actions consistent with those beliefs – are “at stake” in cognitive dissonance.  But then the similarities between the two concepts strongly diverge.  Cognitive dissonance involves the internal process of rationalization and adjustment within a single individual; it occurs when that person is confronted by incontrovertible evidence that “disconfirms” the individual’s strongly held beliefs and actions related to that belief (Festinger, Riecken, & Schachter, 1954).  Cognitive dissonance is a “cognitive” covert response that occurs solely within a single individual.

In contrast, SD always requires two or more stakeholders having dissonance pairwise between them, often manifested through some outward or external display of verbal or behavioral conflict.  In SD, usually one or more stakeholders perceive an imbalance in power or legitimacy, whereby the NWDs of another stakeholder are thought to receive excessive consideration, invariably at the expense of one or more other stakeholder(s). It is worth noting that stakeholders as individuals may indeed experience cognitive dissonance.  This may result in the individual reacting with even more vehemence than other stakeholders and taking a more non-negotiable stand, thereby further contributing to SD.

Methods

Methods used in the original feasibility study

A thorough discussion of the human-centered systems engineering approach for HIT may be found in Samaras and Horst (2005) and in Samaras (2011).  The approach taken in the original feasibility study is beyond the scope of this paper; it is described in detail in a companion methodological paper published in the proceedings of the International Ergonomics Association Triennial World Congress held in Beijing, China (Samaras & Samaras, 2009). The following provides a brief description of methods used in this study, which began with an analytical and heuristic approach to pinpoint ergonomic factors (Samaras & Samaras, 2009; Samaras & Samaras, 2010; Samaras, 2011) relevant to the portal’s development and deployment.

Figure 1: Cycle of Care: Prevention, Screening/Surveillance (S/S), Diagnosis (Dx), Treatment (Rx), Post-Treatment Care (post-Rx)Figure 1: Cycle of Care: Prevention, Screening/Surveillance (S/S),

Diagnosis (Dx), Treatment (Rx), Post-Treatment Care (post-Rx)

The assessment of stakeholders’ NWDs consisted of pre- and post- analytical components and an empirical field study component.  The initial needs analysis began by identifying the universe of stakeholders, the cycle of care (see Figure 1), and Colorado-based cancer-related services.  A number of functional decompositions (work domain analysis, cross-functional flow analysis, and function & task analysis) and an initial risk analysis (see Table 1) were used to develop the field study materials (moderator’s guide, informational handouts, etc.)

RISK CATEGORY

STAKEHOLDER(S)

RISK

RELIABILITY

 C

No Access Requisite Information

P

Poor Presentation of Services

R

Poor Auditing of Compliance

D

Lack of Relevance

AVAILABILITY

C, P

Not Affordable

C, P, R, D

Not Accessible

MAINTAINABILITY

C, P, R, D

Not Complete

C, P, R, D

Not Correct

C, P, R, D

Not Current

SAFETY

C

Physical Threat

C, P

Psychological Threat

C, P

Financial Threat

P, D

Legal Threat

P, D

Ethical Threat

SECURITY

C, P

Privacy Failure

C, P

Confidentiality Breach

C, P, R, D

Integrity Failure

C, P, R, D

Authenticity Failure

USABILITY

C, P

Information Access Frustrated

D

High Design & Maintenance Costs

Table 1: Summary of a priori risk analysis. Adapted from Samaras & Samaras, 2009.

The Safety, Efficiency, Effectiveness, and Satisfaction (SEES) worksheets (see Figure 3) were populated prior to the field study and then revised after the field study.  The SEES results were converted to NWDs using Kano’s (1984) method (see Figure 2).  These NWDs were elicited from stakeholder focus group attendees to inform the portal’s engineering design requirements, which would be the basis for the portal’s implementation validation.  Four focus groups were conducted at key geographic locations around the state with a varying number of participants (Denver-9; Grand Junction-8; Pueblo-7; and with a Cancer Center Committee-11).  To the extent it was believed there was underrepresentation of categories of stakeholders in the structured focus groups (such as among rural, American Indian, and African American participants), 1:1 interviews with six key informants were conducted.

 Figure 2: Stakeholder Response Matrix. Adapted from Kano, 1984Figure 2: Stakeholder Response Matrix. Adapted from Kano, 1984

Figure 3: The SEES Worksheet: Organizing Commensurable Stakeholder NWDs – each box is populated as part of the initial analytic phase and then revised based upon the field study results.

Figure 3: The SEES Worksheet: Organizing Commensurable Stakeholder NWDs –

each box is populated as part of the initial analytic phase and then revised based upon the field study results.

The study consisted of theoretical analyses combined with empirical data collection.  Random assignment could not be used (so it was not a “true” experiment), but multiple measures at different geographic sites were used (consistent with a “quasi” experimental design).  Complete validity can never be assured in any research design; however, threats to each of the four major types of experiment validity for this study were considered.  The design was, in part, a “quasi” design because the independent variable was not being manipulated; we did not know what types of individuals would yield specific NWDs.  Relevant threats to internal validity for the study included selection bias, history bias, and instrumentation/experimenter bias.  Every effort was made to include “samples” of all cancer stakeholders, but selection bias cannot be excluded – especially given the obvious gender skewing of our study sample (N=41; M/F=0.22). However, there is no reason a priori to believe selection bias had a profound effect on the results.  Since one of us was the moderator (EAS) and recruited each participant prior to participation, history bias cannot be excluded – but there is no evidence that the participants would have in any way modified their comments, opinions, or fundamental positions based upon the preliminary description of the study mandate and/or the moderator instructions at the beginning of each focus group.

Instrumentation/experimenter biases, which are essentially the same in this case, since the recruiter, the moderator, and the first transcript analyst were the same individual (EAS), cannot be excluded.  However, every effort was made to limit this threat by using predetermined and consistent scripts, an experienced focus group moderator (EAS), and a qualified second opinion by the second author (GMS) on the transcript analyses.  Demand characteristics – participants trying to be good subjects and following along with other participant’s positions or the position they believe is the moderator’s – are always an issue with this type of group design, but did not seem to be an obvious problem in this study.

There appeared to be considerable face validity, content validity, and concurrent validity, but at this early stage, we were unable to ascertain predictive validity.  Conclusion validity is a crucial issue in qualitative research (Shadish, Cook, & Campbell, 2002).  We were attempting to identify and discriminate between the NWDs of a set of identified stakeholder categories, which were deemed to be cancer information services’ consumers, providers, disseminators and regulators.  There were two threats to the study. The first threat was the risk of concluding there was a set of NWDs, when, in fact, there was not. The second threat was the opposite – concluding there was not a set of NWDs, when there really was.  We identified a large number of putative NWDs and categorized them by stakeholder, so the latter threat may not be particularly relevant.  We were unable to exclude the threat that the identified NWDs were not the real NWDs (NWDs devolve, decaying over time, D?W?N?{null set}).  NWDs are a moving target; over time what we once “Desired” devolves into a “Want” as technology changes, which may devolve into a “Need” because it becomes an expectation.  Ultimately, a “Need” may become obsolete and disappear, just like buggy whips.  This devolution can occur extremely rapidly as the pace of technological change escalates (e.g. see how each new generation of cell phone features renders its predecessor obsolete within a shorter period of time.) Threats to external validity are “sampling” threats and “proximal similarity” threats.  Sampling threats would have been minimized by the use of a random representative sample of the Colorado population; this was not possible in our study.  We used purposive sampling (accessing specific, predefined groups) followed by convenience sampling (potential participants self-selected based upon their schedule and other matters of convenience to themselves).  Mitigating proximal similarity threats include such maneuvers as balancing demographics.  In this study, participants were drawn from three distinct geographical areas: Denver, Grand Junction, and Pueblo.  There was not great disparity between the study population and Colorado’s population with respect to race/ethnicity.  However, the overwhelming majority of study participants were female and were somehow involved, directly or indirectly, in Colorado-based cancer services as consumers, providers, regulators, or disseminators, i.e., our stakeholder groups.

All focus groups and interviews were audiotaped with the written consent of participants and transcribed by a professional transcriptionist.  Transcripts were analyzed for common themes that could help shape the portal’s engineering design requirements.  However, after scrutinizing the focus group and interview transcripts further, we began to recognize a pattern of competing NWDs among the identified stakeholders.  While aligning the system with the NWDs of all stakeholders is crucial, we soon realized that identifying and resolving dissimilar or competing  NWDs was equally or more important to the portal’s long-term success.  This was particularly true because some of these competing NWDs were significant enough to threaten the projects’ viability, both during development and then again during subsequent deployment and maintenance.  Recognizing this became the impetus for the authors’ to coin the term Stakeholder Dissonance (SD) to articulate the importance of their evolving concept of conflicting NWDs among stakeholders (Samaras & Samaras, 2010) and its emergence as a critical determinant for the conceptualization, development, deployment, and maintenance of the proposed e-health initiative under consideration.

 e-Health Initiative Analysis

Methodological difficulties developing and fielding stakeholder-appropriate information and technology systems have been well recognized (Aarts & Gorman, 2007; De Rouck, Jacobs, & Leys, 2008; Johnson, Johnson, & Zhang, 2005; Rudin et al., 2009; Scandurra, Hagglund, & Koch, 2008). Whitten, Kreps, and Eastin (2005) summarized the research conducted on this topic by focusing on “how online information is used, what users want, outcomes or impacts, and quality and credibility” (p.e34). The general idea behind undertaking the feasibility study for the proposed portal was that it would facilitate getting the right information to the right people at the right time – a fundamental premise of nursing informatics.

To delineate the domain of information we were investigating, one of the first tasks was to define the range of Colorado-based cancer information services covered by the portal.  Since cancer care is cyclic in nature (regardless of whether before or after a cancer diagnosis), activities include prevention, screening/surveillance (S/S), diagnosis (Dx), treatment (Rx), and post-treatment (post-Rx) care (see Figure 1).  Consider the example of breast health.  Prevention includes prophylactic mastectomy for extremely high-risk individuals; screening/surveillance includes clinical breast examination and mammography; diagnosis includes biopsy and histopathology; and treatment may include surgery, radiation, and/or chemotherapy.  Once treatment ends, post-treatment care for latent adverse sequelae of the disease itself or its treatment(s) commences, and the cycle continues with new efforts for prevention and long-term surveillance.  The domain of information for the proposed Colorado-based portal included this range of cancer services associated with the cycle of care.

While recognizing the relevant universe of stakeholders is a generally acknowledged challenge, our initial analytic approach helped us identify appropriate stakeholder categories and putative NWDs by category.  We identified four major categories of service information stakeholders (see Figure 3): Consumers (patients, family members and other caregivers), Providers (clinicians and therapists, hospital and clinics), Regulators (federal, state, local governments and accrediting bodies), and Disseminators (information technology contractors, non-governmental organizations, and various media).  In the field studies, we also recognized that many individuals often belonged to more than a single stakeholder group.  Members of the focus groups agreed that our four broad categories subsumed the “universe” of possible stakeholders. The Venn diagram in Figure 4 depicts these various stakeholder groups and examples of their interfaces.

 Figure 4: Stakeholders and their Various Interfaces

Figure 4: Stakeholders and their Various Interfaces

One of the principle study objectives was to construct and document an initial needs analysis because our own interest was to recognize common, disparate, and incompatible needs among stakeholders.  The analysis was predicated upon a well-known consumer response matrix based upon Kano’s (1984) approach (see Figure 2); this was used in the field study to discriminate between NWDs of the respective stakeholders. Needs were defined as basic needs that the stakeholder must have; Wants were considered performance needs that stakeholders would like to have; and Desires were latent needs that stakeholders would know when they saw it.  Since NWDs are not stable and devolve over time (D?W?N?{null set}), we reasoned they must all be commensurable.  We postulated that ALL stakeholders have the same top-level NWDs, as identified by the SEES model: Safety, Effectiveness, Efficiency, and Satisfaction, although each stakeholder may define these differently.  The last need, stakeholder Satisfaction, comprises five subjective measures (perceived effectiveness, perceived efficiency, engaging, error tolerant, and easy to learn) as presented in the worksheet shown in Figure 3 (Samaras & Samaras, 2009)Stakeholders typically have significant difficulty defining their NWDs, but seem to have no problem discussing their opinions of SEES; this may be cultural, it may be related to language comprehension, or it may be something entirely different.  If you consider how nurses versus biomedical engineers view their SEES differently, you realize that it may very well be related to differences in the two subcultures or nursing vs. engineering language or jargon.  To access the underlying NWDs, these top-level NWDs were used to develop questions for the focus groups and structured interviews regarding the portal, e.g. “What would make the portal safe?”, “What would make it effective?” and so forth. We also discussed the a priori risk analysis for the portal, designed to identify the range of risk categories and specific threats to the respective stakeholders; this further informed the process and is summarized in Table 1.

Results

Both consonance and dissonance emerged at the interfaces of different categories of stakeholders, as well as (less frequently) among those in the same category, thereby implying the existence of structured subgroups.  Consonance is relatively easy to deal with as there is agreement as to how to proceed; dissonance is where the potential problems lie, and as such, is the focus of this paper.  The following provides a discussion of some of the examples of SD that were uncovered.

Consumers <<  >> Providers

Both consumers and providers expressed a high “Need” for uniform rating and presentation criteria on the portal, so that fairness was assured (for example, see discussion below regarding physician-providers’ particular concern with high-cost websites or large advertising budgets that might eclipse service providers with more limited resources).  Survivors, other consumers, and non-physician providers in the groups, e.g. nurses, social workers, and patient navigators, generally expressed excitement over the potential benefits of the proposed portal (rapid access to  Colorado-based cancer-related services information organized by stakeholder, location in the cycle of care, geographic location, etc.). In direct contrast, many physician-providers observed that the cost to implement and maintain the portal would be prohibitive and that limited state resources would be better utilized in direct patient care.

Consumers <<  >> Consumers, Regulators

One consumer stakeholder, a cancer survivor, expressed the “Need” for information about lingering side effects from pelvic radiation treatment and the value that diet and nutritional supplements had played in her ability to cope with these latent effects. She advocated for the inclusion of information regarding these types of supplementary and nutritional services in the portal.  This was in direct conflict with another consumer (a family caregiver) concerned about safety (both physical and emotional) related to nutritional supplements.  His family had been targeted by unscrupulous businesses trying to take advantage of his wife’s dire situation with advanced cancer by barraging their family with unfounded promises of cure through dietary supplementation.  In addition to these differing concerns among consumers, inclusion of this type of resource may also generate dissonance between consumers’ and regulators’ concerns about vetting the credibility of nutrition-related information on the web.

Consumers  <<  >>  Disseminators

Consumers and providers expressed a high “Need” for information currency and accuracy for technical and health related matters, as well as for ancillary services and resource availability.  These are elements of information quality control and were deemed important for the safety, credibility, and usefulness of the proposed portal.  For example, consumers wanted access to current eligibility criteria for clinical services available through the portal, not only for un- and under-insured, but even for the insured.  Disseminators contrasted this with the inevitable economic resource allocation decisions required to support the proposed portal to that desired degree of currency and accuracy.  Failing to do so, however, would result in a major unmet consumer-identified “Need” for the portal.

Consumers expressed a “Want” for communication among portal users (e.g., instant messaging, chat rooms, etc.) with the expectation this would improve their ability and reduce the time required to identify problems and solutions of relevance to them.  This community “self-help” approach appeared to be an alternative to the formal “consumer help lines” that were not rated highly by participants, apparently in large part due to automated responses and delays in speaking to a “real person”.  This is a potential source of dissonance between consumers and disseminators with existing “help lines”; it also raises some potential conflicts with regard to regulators’ security concerns.

Many consumers and their advocates recognized the probable lack of information technology infrastructure and user expertise among rural and/or economically disadvantaged consumers.  Using locally-trained resource librarians would only partially mitigate this concern.  Even with access to the portal, stakeholders still had reservations regarding front-end usability.  This extended to issues of risk, language, literacy, cultural beliefs, special physical needs, geographic isolation, and other factors – all of which are inherent challenges for the disseminators.

Providers  <<  >> Regulators

Providers “Need” to preserve information security for both legal (e.g., privacy, confidentiality, integrity, and authenticity) and commercial reasons. Regulators involved with the proposed portal “Want” access to providers’ information to confirm that it is complete and correct before making it available to consumers.  This posed a possible source of conflict between these stakeholder groups.  Issues related to credentialing and eligibility also arose between these stakeholders, and those issues are addressed further in the next section.

Providers <<  >> Disseminators

The “Need” for information security and prevention of illegitimate alteration of services information places a burden on disseminators and their infrastructure.  Selection criteria and credentialing for inclusion of services information was an issue for providers and would be expected to be of concern to disseminators.  Beyond the previously identified domain of information, we have to address issues related to the range of information. For example, the whole issue of nutritional supplementation provoked cultural and value issues among and between stakeholder groups.  Stakeholders often expressed conflicting viewpoints about what is “legitimate” or “credible” cancer service information for inclusion in the portal, how this should be determined, and by whom.

Regulators <<  >>  Disseminators

The analytical and heuristic effort to identify stakeholder NWDs uncovered potential conflicts related to the timing of information availability, especially in the case of periodic updates.  The following types of questions arose: (1) Would disseminator’s information updates have to be reviewed prior to release or at some specified interval immediately after release?  (2)Would regulators be limited to periodic audits and disseminators tasked with internal audits as part of their quality management system?  (3) How would mechanisms of information quality control impact the cost of regulation and the operational costs to disseminators?

Consumers, Providers <<  >> Disseminators

Providers in urban-based cancer centers and rural areas, as well as consumers, “Want” bi-directionality of information about care services.  Bi-directionality refers to ways that Colorado’s unified portal could provide information to rural consumers and providers about urban-based cancer treatment centers; conversely, cancer center providers sought information about health services in remote regions of the state where they could refer their rural patients for post-treatment care (e.g., home healthcare, physical therapy).  This posed a central challenge for the information disseminators charged with expending the requisite resources to identify, maintain, and periodically update information about remote services.  There also seemed to be general consensus that uniform rating criteria must be established and consistently applied.  “Fairness must be assured” so that those  providers with high cost websites or large advertising budgets neither eclipse providers with more limited resources or mislead consumers with lavish presentations.  While evenhandedness was sought in principle, implementation of uniform rating criteria was regarded as non-trivial in practice.

Consumers, Providers, Disseminators  <<  >> Regulators

Consumers, select providers, and disseminators strongly agreed that information on non-clinical services, such as housing, transportation, and financial assistance programs were a very important “Need” and must be included in the development of a unified portal.  This posed a likely conflict with regulators’ legitimate concerns about liability and all stakeholders’ NWDs for currency of information. The burden and expense of keeping up with all these services, as well as the frequent address and eligibility criteria changes would be a major challenge.

 

Discussion

While we purposefully chose representatives from all identified stakeholder categories, targeted three regions throughout Colorado, and included subjects generally reflective of ethnic breakdowns within the state, our ability to generalize our empirical findings are nevertheless limited by our small sample size and restricted geographic region.  This is not true for our theoretical concept of stakeholder dissonance derived from this case study.  All systems are developed and deployed solely because of real or perceived value by their stakeholders.  Therefore, SD management arguably has enormous external validity.  It should be considered a critical strategic concern in the development, deployment, and maintenance of HIT systems intended to improve patient outcomes, increase provider satisfaction, and improve organizational effectiveness and efficiency.

It is beyond the scope of this paper to give an exhaustive account of all SDs found among and between these four major stakeholders groups in this case study; rather the goal here was to articulate some of the major examples of SD that arguably posed a threat to this specific system’s long-term success, as it was initially conceived.  Our mandate was also a very specific one, having to do with information regarding cancer-related services within Colorado and meeting relevant stakeholder NWDs regarding the proposed portal.  For this reason, the specific SD findings cannot necessarily be generalized beyond this proposed initiative and its specific set of stakeholders.

The critical importance of threats to safety in all its forms was demonstrated both in the a priori risk analysis (Table 1) and as a pervasive theme expressed by all stakeholder groups. Careful attention to all facets of safety – including providing current evidence-based information, information quality, security, confidentiality, and accuracy, as well as emotional and financial safety – were all considered “Need” requirements for the portal.  As a top level NWD for all stakeholders, emphasis on safety in all its various forms was not unexpected.  However, as we have seen in this analysis, efforts at meeting this top level NWD can be laden with stakeholder dissonance, precisely because “safety” for one group may not always be perceived as “safety” for another, as illustrated in Table 1.  As such, this poses a central challenge for managing any system’s viability both initially and over time; it is critically important to recognize that the “maintenance tail”, and not just the initial development and deployment effort, is critically important to the success of any HIT endeavor.

One of the chief tasks before moving forward with the proposed portal, had it been deemed viable, would have been to identify, prioritize, and resolve conflicting NWDs, develop a mechanism to compensate for evolving NWDs over the portal’s lifespan, and formulate a set of design inputs that satisfice the NDWs of all the stakeholders.  Satisfice is a term coined by Simon (1957) and presumed to be a contraction of satisfy and suffice. It is an optimality criterion that means to obtain a good result that is good enough, though not necessarily the best, for each stakeholder.  The rationale is that no system can fully satisfy all the NWDs of ALL the stakeholders.  Furthermore, different stakeholder groups have evolving and conflicting NWDs, presenting a moving target that must be continually reconciled.  In the final analysis, the quality of any system will be the degree to which the NWDs of all the stakeholders are satisficed (Samaras & Samaras, 2010).

The existence (or absence) of a political and public policy commitment to developing and, more importantly, sustaining (Wetter, 2007) a high quality information portal over the planned life of the system was a central consideration.  This was particularly true as we estimated that the cost of the “maintenance tail” over the portal’s lifespan would be orders of magnitude greater than the cost of its initial development and deployment.  This added another important dimension of difficulty to system sustainability that is by no means unique to the system of this case study.  System structure and function must evolve as the stakeholders’ NWDs evolve; if not, the system will devolve rapidly to obsolescence.

Effectively managing SD in all phases of the HIT system’s lifecycle, from “lust to dust”, is critical to obtaining successful outcomes from any HIT system.  While it is true that sometimes SD may be accidently or inadvertently minimized during development and/or deployment, this is not generally the case.  Figure 5 illustrates the linkages between the development, deployment, and post-deployment surveillance lifecycles of a HIT system and the loci of SD creation, avoidance, and reduction.  Avoiding development-induced SD requires the rather uncommon activity of managing SD in the development organization; inevitably, this will be imperfect.  The deploying organization then has the opportunity to reduce overall SD or else it can compound development-induced SD as well as introduce new SD associated with intentional or inadvertent changes in workflow, workload, and worker satisfaction.  As with the development process, SD management inevitably will be imperfect in deployment design.  The final safety net is post-deployment surveillance.  Keen attention to SD management (recognizing, measuring, and controlling SD) offers continuous learning and improvement opportunities for the deploying (and developing) organization.

If we wish to achieve specific beneficial outcomes for deployed HIT that are stable and enduring – improving patient care, increasing provider satisfaction, improving healthcare profitability, reducing healthcare cost – we need to manage SD to prevent stakeholders from working against each other, so they may fully cooperate in meaningful use of the HIT.

  Figure 5: Linkages among HIT Development, Deployment, and post-Deployment Surveillance Lifecycles. Adapted from Samaras (2011).

Figure 5: Linkages among HIT Development, Deployment, and post-Deployment Surveillance Lifecycles. Adapted from Samaras (2011).

Conclusion

By employing the human-center systems engineering approach rigorously in advance of system development, complex challenges such as those posed by SD were identified, prioritized, and could have been mitigated before finite human and capital resources were ill-spent.  There are increasing numbers of both unpublished anecdotal and, more recently, published reports of HIT implementation failures (e.g., Leviss et al., 2010; Peute, Aarts, Bakker, & Jaspers, 2010).  A recent retrospective analysis of three acute care case studies (Samaras, Real, Curtis, & Meunier, in press) further demonstrates that nurse stakeholder dissonance was an element in poor patient outcomes and reduced nurse satisfaction. A priori use of this model for development and deployment of complex health information systems is one means of husbanding scarce health resources.  A prospective study involving a new Computerized Provider Order Entry (CPOE) system implementation is ongoing.  One objective is to reduce SD with the expectation that it will improve SEES for the identified CPOE stakeholders: patients, prescribers (MD, NP, PA), nurses, pharmacists, unit clerks, pharmacy technicians, administrators, and regulators.

Managing SD arguably forwards the goal of fitting new HIT systems to the system stakeholders, much as the evolution in industrial ergonomics shifted the paradigm away from fitting the worker to the workplace to fitting the workplace and tools to the worker.  Managing SD is worthy of further study in other healthcare settings, as it arguably has profound cost, safety, and use (effectiveness, efficiency, and satisfaction) implications in our rapidly changing, increasingly HIT-reliant health delivery system.  Making a tough decision not to go forward with the development or deployment of a new HIT because it fails to adequately satisfice the conflicting NWDs of its professed stakeholders may also be a legitimate option, as we strive to carefully steward limited health resources.

Acknowledgement

This work was funded in part through a contract (CCCP # PPG07000175) awarded to Samaras & Associates, Inc. by the Colorado Comprehensive Cancer Program. The authors wish to thank the many stakeholders who participated in the empirical portion of this study in 2007.  We also want to express our appreciation to Catherine Dingley, PhD, RN, FNP, Lynn Phillips, Michele Ellis, and the OJNI reviewers for their constructive comments.   The analysis and conclusions expressed are solely those of the authors.


 

References

Aarts, J. & Gorman, P. (2007).  To err is system.  International Journal of Medical  Informatics, 76 S1-3. doi:10.1016/S1386-5056(07)00078-0

Brugha R, Varvasovszky, Z. (2000). Stakeholder analysis: a review. Health Policy Plan, Sep; 15(3), 239-46. Retrieved from PubMed.

Child, J.W., & Marcoux, A. M. (1999). Freeman and Evan: Stakeholder theory in the original position. Business Ethics Quarterly, 9(2), 207-223. Retrieved from EBSCOhost.

Curran, C., & Totten, M. (2010). Mission, strategy, and stakeholders. Nursing Economic$, 28(2),116-118. Retrieved from EBSCOhost

Daniels, H., James, N., Rahman, R., Young, A., Derry, J. & McConkey,C. (2007). Learning about cancer. Mind, Culture and Activity, 14(1-2), 28-141. Retrieved from EBSCOhost.

De Rouck S., Jacobs A., & Leys M. (2008). A methodology for shifting focus of e-health support design onto user needs: A case in the homecare field. International Journal of Medical Informatics, 77(9), 589-601. doi:10.1016/j.ijmedinf.2007.11.004

 Eysenbach, G. (2003). The impact of the Internet on cancer outcomes. CA: Cancer Journal for Clinicians, 53(2003), 356–371. Retrieved from EBSCOhost.

Festinger, L., Riecken, H.W., &  Schachter, S. (1954). When prophecy fails. Minneapolis: University of Minnesota Press.

Freeman, R. E. (1994). The politics of stakeholder theory: Some future directions. Business Ethics Quarterly, 4: 409-421.

Gilmartin, M.J. (2001). Humanism in health care service: the role of stakeholder management. Nursing Administration Quarterly, 25(3), 24-36.

Johnson, C.M., Johnson, T.R., & Zhang, J. (2005). A user-centered framework for redesigning healthcare interfaces. Journal of Biomedical Informatics, 38(1), 75-87.

Kano N. (1984) Attractive quality and must-be quality. Journal of the Japanese Society for Quality Control, April, 39-48.

Keown, K., Van Eerd, D., & Irvin, E. (2008). Stakeholder engagement opportunities in systematic reviews: Knowledge transfer for policy and practice. Journal of Continuing Education in the Health Professions, 28(2), 67-72. Retrieved from EBSCOhost.

Leviss, J. (Ed.), Gugerty, B., Kaplan, B., Keenan, G., Ozeran, L, Rose, E. & Silverstein, S. (associate eds.). (2010).  H.I.T. or Miss: Lessons Learned from Health Information Technology Implementations (abstract). A publication of AMIA, AHIMA. Retrieved from: http://marketplace.himss.org/acct618b/Default.aspx?tabid=44&action=INVProductDetails&args=1638&BookTitle=HIMSS%20Store%20-%20H.I.T.%20or%20Miss

Littau, P., Jujagiri, N., & Adlbrecht, G. (2010). 25 years of stakeholder theory in project management literature (1984–2009). Project Management Journal, 41(4), 17-29. doi:10.1002/pmj.20195

Mitchell, R. K., Agle, B. R., & Wood, D. J. (1997). Toward a theory of stakeholder identification and salience: Defining the principle of who and what really counts. Academy of Management Review, 22(4), 853-886.

Peute, L.W., Aarts, J., Bakker, P., Jaspers, M. (2010). Anatomy of a failure: A sociotechnical evaluation of a laboratory physician order entry system implementation. International Journal of Medical Informatics, 79(2010) e58-e70. Doi:10.016/j.ijmedinf.2009.06.008

Rudin, R., Simon, S., Volk, L., Tripathi, M., & Bates, D. (2009). Understanding the decisions and values of stakeholders in health information exchanges: Experiences from Massachusetts. American Journal of Public Health,             99(5), 950-955.

Samaras E.A., Real S.D., Curtis A.M., & Meunier ,T.S. (in press). Recognizing nurse stakeholder dissonance as a critical determinant of patient safety in new healthcare information technologies. Accepted for publication in Special Issue: WORK: A Journal of Assessment, Prevention, and Rehabilitation, Proceedings IEA 2012, Recife, Brazil, 2/12-16/2012.

Samaras, E.A. & Samaras, G.M. (2010). Using human-centered systems engineering to reduce nurse stakeholder dissonance. Biomedical Instrumentation & Technology, 44(1), 25-32. doi: 10.2345/0899-8205-44.s1.25.

Samaras, G.M. (2011). Human-centered systems engineering: Managing stakeholder dissonance in healthcare delivery. In A. Kolker & P. Story P (Eds.), Management engineering for effective healthcare delivery: Principles and practices (pp.148-171). Hersey PA: IGI Global.

Samaras G.M., &  Horst, R.L. (2005). A systems engineering perspective on the human-centered design of health information systems. Journal of  Biomedical Informatics, 38(1), 61-74.

Samaras, G.M., & Samaras, E. A. (2009). Feasibility of an e-health initiative: Information NWDs of cancer stakeholders. Proceedings International Ergonomics Association. Beijing, China. August 9-14, 2009.

Scandurra, I., Hagglund, M., & Koch, S. (2008). From user needs to system specifications: Multidisciplinary thematic seminars as a collaborative design method for development of health information systems. Journal of Biomedical Informatics, 41, 557-569.

Simon, H.A. (1957). Models of man: social and rational. New York: Wiley.

Shadish, W.R., Cook, T.D., & Campbell, D.T. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Belmont, CA: Wadsworth.

Tan, E.L., Stark, H., Lowinger, J.S., Ringland, C., Ward, R., & Pearson, S.A. (2006). Information sources used by New South Wales cancer clinicians: A qualitative study. International Medical Journal, 36, 711-717. Retrieved from EBSCOhost.

Taylor, R. & Tofts, C. (2008). Taking a RaSP to enterprise stakeholder dissonance (abstract). HPL Laboratories, HPL-2008-82. Retrieved from http://www.hpl.hp.com/techreports/2008/HPL-2008-82.html .

Verner J., Beecham S. & Cerpa, N. (2010). Stakeholder dissonance: disagreements on project outcome and its impact on team motivation across three countries (abstract).  SIGMIS-CPR’10, Proceedings of the 2010 Special Interest Group on Management Information System’s 48th Annual Conference on Computer Personnel Research; pp. 25-33. Retrieved from http://dl.acm.org/citation.cfm?id=1796918.

Wetter T. (2007). To decay is system: The challenges of keeping a health information system alive. International Journal of Medical Informatics. doi:10.1016/j.ijmedinf.2006.05.011

While, A., Forbes, A., Ullman, R., & Mathes, L. (2008). The role of specialist and general nurses working with people with multiple sclerosis. Journal of Clinical Nursing, 18, 2635-2648. doi: 10.1111/j.1365-2702.2008.02459.x

Whitten P., Kreps G., & Eastin M.S., (2005). Creating a framework for online cancer services research to facilitate timely and interdisciplinary applications. Journal Medical Internet Research, 7(3) e34. doi:10.2196/jmir.7.3.e34

 Ziebland S., Chapple A., Dumelow C., Evans J., Prinjha S., & Rozmovits, L. (2004). How the internet affects patients’ experience of cancer: A qualitative study. British Medical Journal, 6328(7439):564.

Author Bios:

Elizabeth A.Samaras, RN, MSN, NP, AOCN

Elizabeth A. Samaras is an Assistant Professor of Nursing at Colorado State University-Pueblo, where she teaches undergraduate and graduate nursing courses including health informatics, evidence-based practice, and clinical medical surgical nursing.  She is a Family Nurse Practitioner and Advanced Oncology Certified Nurse having received her BFA from University of North Caroline-Chapel Hill (1976) and her MSN from Yale University (1984); she is currently a DNP candidate at the University of Northern Colorado (expected 2012).  Ms. Samaras spent the early part of her career in occupational/environmental health, where she initiated numerous health informatics programs including a personal health record for occupationally-exposed workers. Before joining academia, she served as the clinical trials coordinator and breast health specialist for a Community-based Cancer Center.  Her research interests include evidence-based health informatics, nurse stakeholder dissonance, and promoters of minority recruitment to breast cancer treatment and prevention clinical trials. She served as a peer-reviewer for the upcoming 11th International Congress on Nursing Informatics (Montreal 2012). Contact: elizabeth.samaras@colostate-pueblo.edu

 

George M.Samaras, PhD, DSc, PE, CPE, CQE

George M. Samaras is a professional engineer (PE, 1980) in private practice.  His firm provides technical, regulatory, and management consulting services – primarily to medical device and pharmaceutical manufacturers.  He has a number of biomedical engineering patents and numerous peer-reviewed publications in physiology and engineering (hardware, software, human factors, and quality engineering).  He attended the University of Maryland, where his basic training was in electrical engineering (BSEE, 1972); he also received a MS (1974) and PhD (1976) in physiology.  He received a DSc (1992) from The George Washington University, Department of Engineering Management and Systems Engineering.  He is a board-certified human factors engineer (CPE, 1998) and an ASQ-certified quality engineer (CQE, 2005).  His past employment includes serving as a professor at both the University of Maryland School of Medicine and the George Washington University Graduate School of Engineering, founding and managing a contract biomedical engineering firm, and working for the USFDA/CDRH as a reviewer and manager.  He is a senior member of numerous professional organizations.

 

Back to Issue Index

Back to Issue Index

Be Sociable, Share!

Comments are closed.