This article was written on 23 Mar 2012, and is filled under Volume 16 Number 1.

Current post is tagged

, , , , , , , , , , , ,

Electronic Health Records and Quality of Care: Mixed Results and Emerging Debates

Print Friendly, PDF & Email

Achieving Meaningful Use in Research with Information Technology Column

by Maxim Topaz, RN, MA Doctoral Student

and Kathryn H. Bowles, PhD, RN, FAAN Associate Professor,

University of Pennsylvania School of Nursing

This column was made possible by an educational grant from
Chamberlain College of Nursing


Topaz, M. & Bowles, K. H. (February, 2012). Electronic Health Records and Quality of Care: Mixed Results and Emerging Debates. Achieving Meaningful Use in Research with Information Technology Column. Online Journal of Nursing Informatics (OJNI), 16 (1). Available at http://ojni.org/issues/?p=1262


Achieving Meaningful Use in Research with Information TechnologyIn the United States (US) and many other countries, Electronic Health Records (EHRs) have been proposed as a sustainable solution for improving the quality of medical care. Recently, the implementation of EHRs was made compulsory by the American Recovery and Reinvestment Act (2009) that allocated 27 billion dollars to the implementation of EHRs in clinical settings and introduced the concept of Meaningful Use (Blumenthal & Tavenner, 2010; Hillestad et al., 2005). By the end of 2015, healthcare providers across the US are expected to prove they are meaningful users of “certified EHR technology in ways that can be significantly measured in quality and in quantity” (US Department of Health and Human services, 2011), otherwise, be financially penalized when providing services for Medicare and Medicaid clients. On the other hand, providers who do meaningfully implement EHRs in their practices are eligible for financial incentives of as much as $44,000 (through Medicare) and $63,750 (through Medicaid) per clinician (Blumenthal & Tavenner, 2010).

Based on the compelling assumption “EHRs will improve caregivers’ decisions and patients’ outcomes” (Blumenthal & Tavenner, 2010. p 501), these legislative acts and financial incentives create a strong urge to implement EHRs in clinical settings across the US. Surprisingly, mixed evidence exists about the effect of EHRs on the actual quality of care. This editorial briefly reviews some of the shortcomings related to the emerging body of research evaluating the impact of EHRs on the quality of ambulatory care. Suggestions for future research are also provide

Based on the findings of several recent studies examining the impact of EHRs on quality of care, several authors have raised concerns about the ability of health information technology to improve the quality of outpatient care (Linder, Ma, Bates, Middleton, & Stafford, 2007; Romano & Stafford, 2011). Although the results of these studies have influenced the policy and scientific discourses (Classen & Bates, 2011; Radecki & Sittig, 2011; Shih, McCullough, Wang, Singer, & Parsons, 2011), they also provoked almost immediate controversy among other researchers who attempted to understand the validity and generalizability of these mixed findings (McDonald & Abhyankar, 2011; Mohan & Hersh, 2011; Oetgen, Mullen, & Mirro, 2011). The debate centers on study design flaws that weaken the credibility of the study finding

One example is a critique of a recent study examining the impact of EHRs and Clinical Decision Support (CDS) on quality of care (Romano & Stafford, 2011). The investigators analyzed a nationally representative sample of ambulatory patients’ visits during 2005-2007 (N=50,554). The data was taken from the National Ambulatory Medical Care Survey conducted by the National Center for Health Statistics (CDC, 2011). The investigators found only two of the 20 assessed Quality of Care Indicators (CIs) were greater in EHR-assisted visits or EHR- and CDS-assisted visits (“diet counseling in high-risk adults” and “lack of routine electrocardiographic ordering in low-risk patients”) than in non-EHR-assisted visits. The authors concluded “these results raise concerns about the ability of health information technology to fundamentally alter outpatient care quality” (p. 897). However, others questioned some of the study methods.

McDonald and Abhyankar (2011) and Mohan and Hersh (2011) expressed concern about how the investigators defined the presence of an EHR. The study respondents were simply asked, if they have an EHR, and does it include CDS. This approach weakens the study because a positive response to this question does not provide any information on the characteristics and application of the EHRs or CDS. Because of the lack of standards in construction and functionality of EHRs and CDS, these tools may differ significantly from one eligible healthcare provider to another and across different clinical settings. For example, some EHRs include advanced CDS tools allowing clinicians to track detailed patient information in real time while others are used only for billing purposes. The lack of detailed information on EHRs and CDS used by the reporting eligible health care providers limits the ability of the study to make generalizable conclusions (McDonald & Abhyankar, 2011; Mohan & Hersh, 2011).

The second significant study limitation lies in its measurement of the quality for care (McDonald & Abhyankar, 2011; Mohan & Hersh, 2011). Romano and Stafford (2011) examined process quality indicators (QIs) (for example: “ACE inhibitors use for CHF”) that were computed as the percentage of applicable visits receiving appropriate care. However, the National Ambulatory Medical Care Survey included only three diagnosis and three reasons to visit; therefore, other comorbidities cannot be accounted for, neither in severity adjustment nor in contraindications for the suggested treatment. For example, according to one of the QIs, all the patients with CHF, excluding the ones with hyperkalemia and angioedema, were supposed to receive ACE inhibitors, otherwise the quality of care for this indicator was not considered to be fully achieved. It is not clear how researchers learned of these comorbid conditions if only three diagnosis and three reasons for visits were captured by the survey. Moreover, the survey included information for only eight drugs and ACE inhibitors might be number nine on the list, thus it would not have been included. Also, several other medical conditions traditionally considered as contraindications to ACE inhibitors were not listed as the exclusion criteria for this QI (for example renal artery stenosis or hypersensitivity to the drug) (Bicket, 2002). Further, it is not clear how EHRs without CDS tools are supposed to encourage clinicians to increase the quality of their clinical processes. By definition, EHRs serve as a repository database for storage and representation of clinical data, therefore the lack of the correlation between these QIs and EHRs is not surprising but rather expected. On the other hand, outcomes other than the assessed QIs (for example readmission rates) may have improved in EHR visits but they were not evaluated in these studies. Finally, the National Ambulatory Medical Care Survey is a cross-sectional survey not designed to assess EHRs and therefore the ability to claim EHRs, or CDSs, are not producing better quality care (causal relationships) is very limited (McDonald & Abhyankar, 2011; Mohan & Hersh, 2011).

In contrast, a smaller study that allowed for more flexible and comprehensive QIs when examining the impact of CDS tools on the quality of care found a significant improvement in similar process measures (Persell et al., 2011). In this study, the EHR allowed providers to enter patient and medical reasons for not following CDS guidelines and recommendations presented by the QIs (i.e., exceptions) as part of routine workflow. These exceptions were further excluded from the final quality estimation. Therefore, a significant number of patients, sometimes up to 7.4% of the total number, who were not treated according to the guideline suggested for them were excluded from the final quality calculation. Adding this flexibility to exclude patients is actually a desirable option because clinicians and patients have a legitimate right to agree or refuse certain treatment for a good reason and it should not affect the QIs. In this study, clinicians who identified a significant reason (other than stated in direct contradictions to the drug or procedure included in the guideline) not to prescribe were able to do so without being penalized for the diminished adherence to the QIs. For example, prescribing cancer screening to a frail elderly person might cause more suffering and harm than benefits. It is important to mention all exceptions in this study were assessed by a group of peer reviewers and most of them were found legitimate (Persell et al., 2010). Additionally, clinicians involved in this project were explicitly informed about the quality goals and they were receiving monthly notices with their score on the suggested QIs.

To conclude, the effect of EHRs (and CDSs) on the quality of care is yet to be comprehensively explored. To date, most of the research examining national samples of ambulatory practices is limited by how QIs are defined, measured and the quality of the data sources. Moreover, most of the nationwide studies examine outcomes related to processes only. We suggest further research in this field should be based on sound conceptual or theoretical frameworks that explicate the logical relationships among the database elements, the EHR or CDS intervention components, and the outcomes of interest. Hopefully, this will be possible in the near future as the implementation of EHRs across most healthcare settings in the US is supposed to happen by 2015 (US Department of Health and Human services, 2011). Additionally, QIs analyzed in future studies should be more comprehensive and flexible, reflecting the complex picture of healthcare services. Given this complexity, mixed method studies are also suggested to fully capture how the EHR or CDS is used to influence quality of care. Finally, a mix of structure, process, and outcome components should be included in examination of the effects of EHRs on quality of care to fully understand the interplay of these variables.


Bicket, D. (2002). Using ACE inhibitors appropriately. American Family Physician, 66(3), 461-468.

Blumenthal, D., & Tavenner, M. (2010). “TheMeaningful Use” regulation for electronic health records. New England Journal of Medicine, 363(6), 501-504. doi:10.1056/NEJMp1006114

CDC. (2011). The National Ambulatory Medical Care Survey (NAMCS). Retrieved from http://www.cdc.gov/nchs/ahcd.htm

Classen, D. C., & Bates, D. W. (2011). Finding the meaning in “Meaningful Use.” New England Journal of Medicine, 365(9), 855-858.

Hillestad, R., Bigelow, J., Bower, A., Girosi, F., Meili, R., Scoville, R., & Taylor, R. (2005). Can electronic medical record systems transform health care? Potential health benefits, savings, and costs. Health Affairs, 24(5), 1103-1117. doi:10.1377/hlthaff.24.5.1103

Linder, J. A., Ma, J., Bates, D. W., Middleton, B., & Stafford, R. S. (2007). Electronic health record use and the quality of ambulatory care in the United States. Archives of Internal Medicine, 167(13), 1400-1405. doi:10.1001/archinte.167.13.1400

McDonald, C., & Abhyankar, S. (2011). Clinical decision support and rich clinical repositories: A symbiotic relationship. Comment on “electronic health records and clinical decision support systems.”Archives of Internal Medicine, 171(10), 903-905. doi:10.1001/archinternmed.2010.518

Mohan, V., & Hersh, W. R. (2011). EHRs and health care quality: Correlation with out-of-date, differently purposed data does not equate with causality. Archives of Internal Medicine, 171(10), 952-953. doi:10.1001/archinternmed.2011.188

Oetgen, W. J., Mullen, J. B., & Mirro, M. J. (2011). Electronic health records, the PINNACLE registry, and quality care. Archives of Internal Medicine, 171(10), 953-4; author reply 954. doi:10.1001/archinternmed.2011.189

Persell, S. D., Dolan, N. C., Friesema, E. M., Thompson, J. A., Kaiser, D., & Baker, D. W. (2010). Frequency of inappropriate medical exceptions to quality measures. Annals of Internal Medicine, 152(4), 225-U49.

Persell, S. D., Kaiser, D., Dolan, N. C., Andrews, B., Levi, S., Khandekar, J., . . . Baker, D. W. (2011). Changes in performance after implementation of a multifaceted electronic-health-record-based quality improvement system. Medical Care, 49(2), 117-125. doi:10.1097/MLR.0b013e318202913d

Radecki, R. P., & Sittig, D. F. (2011). Application of electronic health records to The Joint Commission’s 2011 National Patient Safety Goals. Journal of the American Medical Association, 306(1), 92-93.

Romano, M. J., & Stafford, R. S. (2011). Electronic health records and clinical decision support systems impact on national ambulatory care quality. Archives of Internal Medicine, 171(10), 897-903. doi:10.1001/archinternmed.2010.527

Shih, S. C., McCullough, C. M., Wang, J. J., Singer, J., & Parsons, A. S. (2011). Health information systems in small practices improving the delivery of clinical preventive services. American Journal of Preventive Medicine, 41(6), 603-609. doi:10.1016/j.amepre.2011.07.024

US Department of Health and Human services. (2011). CMS EHR meaningful use overview. Retrieved from https://www.cms.gov/ehrincentiveprograms/30_Meaningful_Use.asp

Proofed by Monica Key



Back to Issue Index

Back to Issue Index

Be Sociable, Share!

Comments are closed.