Investigative First Steps: Appropriate Identification and Ethical Review of Research and Quality Improvement
Theodore D. Cosco, Alana Knopp, BA and Doris L. Milke, MSc, PhD
Cosco, T., Knopp, A. & Milke, D. (October, 2007). Investigative first steps: Appropriate identification and ethical review of research and quality improvement. Online Journal of Nursing Informatics (OJNI), 11, (3) [Online]. Available at http://ojni.org/11_3/cosco.htm
Acquiring data ethically is a cornerstone of scientific investigation. This is especially important in the field of nursing informatics because the vast majority of data deals directly with people and their personal health information. Before investigators collect or use data from surveys, charts, or samples, the appropriate ethics review and approval must occur. Determining the appropriate ethics review depends on the investigative method employed; therefore, correct method identification is integral. From an ethics review standpoint, two of the most common methods, research and quality improvement (QI), are fundamentally dissimilar yet difficult to differentiate. Inaccurately identifying a study as research or QI could mean that it is inappropriately reviewed or conducted without appropriate ethics approval. Nursing informatics requires the appropriate identification of research and QI methods to prevent investigators from unnecessarily putting themselves, their studies, their study participants and their data integrity at risk.
Key words: Research, Quality Improvement, Ethics Review, Method Identification, Nursing Informatics
Nursing informatics (NI) is inherently an information intensive field, generating knowledge that informs decisions about patient care while increasing the quality of nursing practice (Goosen, 1996; Hannah, 2007). NI practices become more efficient, effective and productive through scientific investigations (Simpson, 2007). However, before investigators can collect or use data from surveys, charts or samples, the appropriate ethics review and approval must occur. This protects investigators, their studies, study participants and data integrity from unnecessary risks associated with the misapplication of research and quality improvement (QI).
Ethical advancements in knowledge using research and QI methods combine information, technology and nursing practice to increase the quality of care. NI synthesizes these components using primary data, collected by nursing informaticians, and/or secondary data, collected by investigators in other fields. An example of a nursing informatician using primary data would be if s/he developed an on-line survey regarding perceived benefits of a nursing-specific search engine and personally retrieved the survey data. Conversely, a nursing informatician would be using secondary data if s/he were to analyze the perceived benefits of a nursing-specific search engine using data collected by an occupational therapist. Primary and secondary data analyses require the same method identification and ethics review process (Huston & Naylor, 1996). Dealing with personal health data requires nursing informaticians to treat any study as if they conducted the original data collection and go through an appropriate ethics review process.
Determining the appropriate ethics review process depends upon correctly identifying the investigative method. Unfortunately, distinguishing between two of the most common methods, research and QI, is not always a straightforward task. A pivotal case at the University of Pittsburgh illustrates this difficulty (Lynn, 2004). A group of investigators published a paper on a project described as QI, but the university’s Institutional Review Board (IRB) retrospectively identified it as meeting the criteria for research. The issue was appealed to the Federal Office for Human Research Protection, which eventually ruled that the project was research and should have had an IRB review (Lynn, 2004). Consequently, the investigators were criminally and civilly liable for any negative repercussions experienced by the study participants.
Provincial laws in Canada, such as Alberta’s Health Information Act, explicitly require that research – but not QI – projects receive ethics approval from a Research Ethics Board (REB). This REB approval dichotomy has generated much academic controversy and debate. Because of the difficulty differentiating research and QI, investigators may identify their projects incorrectly. Projects inaccurately dubbed QI that more closely resemble research may bypass REB review and approval, but not the ethical and legal implications of conducting a study without it. Because REB review is “quite formal, rather ponderous and costly” (p. 68, Lynn, 2004) and “overburdened, understaffed and under funded,” (p. 320, Davidoff & Batalden, 2005) projects classified as QI can generally begin at a quicker pace than research. As more of these inappropriately (un)reviewed projects surface, the long-avoided issue facing health care providers of correctly distinguishing research and QI is becoming one of greater significance and concern (Lynn, 2004); it is especially true for those, like nursing informaticians, who use personal health information.
Differentiating Research and QI
Distinguishing between research and QI is not always a straightforward task. Many of the best evaluation methods overlap with methods used in research (Jarvis, 2000); thus, many QI projects exhibit research-like qualities and research projects exhibit QI-like qualities. However, using the characteristics of a study helps to differentiate research and QI, thereby aiding investigators in ensuring their projects are forwarded to the appropriate ethics review. Three criteria help differentiate research and QI: a study’s intents and purposes, degree of generalizability and associated risks.
The literature shows that a useful tool in differentiating a study as research or QI are the intents and purposes of a study (Beyea & Nicoll, 1998; Koschnitzke, McCracken & Pranulus, 1992; Lee, Anderson, Pringer Boone & Henderson, 1992; Martin, 1995; Mcnelly, 1997; Newhouse & Pettit, 2006; Patton, 1993; Pritham, 1996; Saunders, 1993; Solberg, Mosser & McDonald, 1997; Thurston, Watson & Reimer, 1993). QI intends to increase the efficacy of processes at a local or internal level as part of ongoing care, tailoring studies to the needs of the study participants themselves (e.g., implementing a computerized model of care tailored specifically to a geriatric dementia unit) (Committee on Assessing the System for Protecting Human Research Participants (CASPHRP), 2002). Conversely, research intends to generate knowledge and establish fact, independent of routine care (e.g., testing and comparing the efficacy of a computerized model of care designed for use in all health care settings).
The audience the project is intended for is another fundamental difference between research and QI. Research is associated with breadth of application and QI is associated with depth (Diamond, 2004, Reinhardt & Ray, 2003). Research projects intend to be generalizable, with acquired knowledge aimed at the entire scientific community in order to benefit a large population (Department of Health and Human Services, 2005; Diamond, 2004; Perneger, 2004, Reinhardt & Ray, 2003). QI intends to satisfy a local or internal population, with a smaller, more distinct, target audience, specifically members of the local or internal population (i.e. staff, patients, caregivers and the study participants themselves) (CASPHRP, 2002; Perneger, 2004, Reinhardt & Ray, 2003). Research results are not assumed to be immediately incorporated into local care delivery, whereas QI studies are conducted almost solely on that principle (CASPHRP, 2002; Department of Health and Human Services, 2005). An example would be the development of a standard set of behavior terminologies. Approaching this project from a QI perspective, investigators would tailor their set of terminologies to their ward, systematically going through cases looking for specific identifiers. Conversely, a similar research project would be concerned with the applicability of the set of terminologies to more than just one ward. Therefore, the behavior terminology set would need general categories that permit its application to a larger population. Its reliability and validity would need pilot testing on external populations before any sort of practical application would be possible.
In the interest of generalizing to a broad population, research, in direct contrast to QI, imposes risks beyond those of everyday care (Bellin & Dubler, 2001; Casarett, Karlawish & Sugarman, 2000, Reinhardt & Ray, 2003). To establish a causal relation between two variables that can generalize to a larger population research must eliminate or, at the very least, control any potentially confounding variables. This often involves establishing a baseline or null variable or the use of controls, strengthening causation, but adding risks and ethical issues associated with multiple treatment assignment (Bellin & Dubler, 2001, Jarvis 2000). Processes that go above and beyond routine care for the purpose of generalization have many associated risks, necessitating REB approval (Casarett, Karlawish & Sugarman, 2000). These projects are considered research. QI, on the other hand, tends to use standards or benchmarks from previously established best practices so that objective referencing can occur without deviating from routine care and creating additional risk (Diamond, 2004). Consequently, REB approval is not necessary.
Research is associated with having greater than minimal risk, whereas QI projects have minimal risk (Casarett, Karlawish & Sugarman, 2000; Harris, 1997; Lo & Groman, 2003). According to the Canadian Tri-Council Policy Statement on Research Involving Humans, minimal risk is risk “no greater than [the risks] encountered by the subject in those aspects of his or her everyday life” (p. 1.5). To illustrate, an example would be testing a software program designed to increase the accuracy of medication. A research trial would use differing experimental groups; one ward could use the new software and one (control) no software at all. In a QI trial, there would be no need to have a null comparison group because of the use of previously established standards or benchmarks as a reference, consequently all wards would receive the same treatment (Diamond, 2004). Because of the differential use of the software programs in the research trial, scenarios could arise in which study participants are put at risk greater than minimal risk (e.g. if the new software proved to be less effective than having no software, the study participants in the “new software” group would be at a higher risk of receiving incorrect medication). Research projects must be reviewed and approved by REBs to ensure that the risk is acceptable given the potential benefits of the study.
Using publication as a means to differentiate between research and QI is the subject of ongoing debate in the literature. Generally, QI dissemination is at the local level (i.e. staff, caregivers, patients, etc.) whereas research dissemination is on a broad scale (i.e. to journals, conferences, etc.) According to Lynn, et al. (2007), some believe that if a project is published it is intended for a generalized audience and therefore, regardless of the characteristics of the study itself, is a research study. Consequently, QI manuscripts may be rejected because they have not received REB approval, regardless of the fact that review was neither ethically or legally required (Lynn, et al., 2007). To categorize all manuscripts as research would prevent the knowledge acquired from QI studies from dissemination and application elsewhere (Davidoff & Batalden, 2005). Byers and Aragon (2003) maintain that QI studies can be applied beyond the original intended audience. Disseminating QI studies’ valuable knowledge acquired from local improvement studies, i.e., QI studies, may serve to benefit beyond the boundaries of its original audience (Lynn, et al., 2007), as has been demonstrated by the emergence and prosperity of QI journals (i.e., Quality and Safety in Healthcare, International Journal of Productivity and Quality Management, etc.). Therefore, as the literature suggests, publication should not be used as a definitive distinguishing criterion between research and QI.
Nursing informaticians are in a unique position to use research and QI to connect nursing practice, information and technology to create evidence-based knowledge and improvements in care. The ability to distinguish between research and QI, and subsequently ensure studies receive appropriate ethics approval, is an essential component of furthering NI. Three characteristics aid in the identification of any study as research or QI: a study’s purpose, degree of generalizability, and associated risks. The use of properly identified, reviewed and applied research and QI benefits NI as a field, health care as a whole, and most importantly, patients.
Bellin, E., & Dubler, N. (2001). The quality improvement – research divide and the need for external oversight. American Journal of Public Health, 91(9), 1512-1517.
Beyea, S. C., & Nicoll, L. H. (1998). Is it research or quality improvement? Association of Perioperative Registered Nurses Journal, 68, 117-119.
Byers, J. F., & Aragon, E. D. (2003). What quality improvement professionals need to know about IRBs. Journal for Healthcare Quality, 25(4), 4-9.
Casarett, D., Karlawish, J. H. T., & Sugarman, J. (2000). Determining when quality improvement initiatives should be considered research; Proposed criteria and potential implications. Journal of the American Medical Association, 283, 2275-2280.
Committee on Assessing the System for Protecting Human Research Participants. [CASPHRP] (2002). Responsible research: A systems approach to protecting research participants. Washington, DC: The National Academies Press.
Davidoff, F., & Batalden, P. (2005). Toward stronger evidence on quality improvement. Draft publication guidelines: the beginning of a consensus project. Quality and Safety in Health Care, 14, 319-325.
Department of Health and Human Services (2005) Code of Federal Regulations Title 45 Public Welfare Part 46: Protection of Human Subjects. Retrieved on June 22, 2007 from http://www.hhs.gov/ohrp/documents/OHRPRegulations.pdf
improvement projects: How do we protect patients’ rights? American Journal of Medical Quality, 19(1), 25-27.
Goossen, W. T. F. (1996). Nursing information management and processing: A framework and definition for systems analysis, design, and evaluation. International Journal of Biomedical Computing, 40, 187-195.
Hannah, K., & Kennedy, M. A. (2007). Representing nursing practice: Evaluating the effectiveness of a nursing classification system. Canadian Journal of Nursing Research, 39(1), 58-79.
Harrington, L. (2007, May/June). Quality improvement, research, and the institutional review board. Journal for Healthcare Quality Online, 195. Retrieved June 22, 2007 from http://www.nahq.org/journal/ce/article.html?article_id=277
Harris, M. R. (1997). Code of ethics and scientific integrity: What relevance to outcome activities? Advanced Practice Nursing Quarterly, 3, 36-43.
Huston, P., & Naylor, D. (1996). Health services research: Reporting on studies using secondary data sources. Canadian Medical Association Journal, 155(12), 1697-1702.
Jarvis, H. (2000). Differentiating among research, evaluation and measures to assure quality. Healthcare Management Forum, 13(4), 34-36.
Koschnitzke, L., McCracken, S. C., & Pranulis, M. F. (1992). Issues in clinical nursing research: Ethical considerations for quality assurance versus scientific research. Western Journal of Nursing Research, 14, 392-396.
Lee, L., Anderson, V., Piringer, P., Boone, J., & Henderson, D. K. (1992). An epidemiologic approach to quality improvement, quality assurance, and clinical research. Infection Control and Hospital Epidemiology, 13, 545-552.
Lo, B., & Groman, M. (2003). Oversight of quality improvement. Archives of Internal Medicine, 163, 1481-1486.
Lynn, J., Baily, M.A., Bottrell, M., Jennings, B, Levine, R.J., Davidoff, F., et. al. (2007). The ethics of using quality improvement methods in health care. Annals of Internal Medicine, 146, 666-673.
Lynn, J. (2004). When does quality improvement count as research? Human subject protection and theories of knowledge. Quality and Safety in Health Care, 13, 67-70.
Martin, P. A. (1995). Is it research? Applied Nursing Research, 8, 199-201.
Nerenz, D. R., Stoltz, P. K., & Jordan, J. (2003). Quality improvement and the need for IRB review. Quality Management in Health Care, 12(3), 159-170.
Newhouse, R., & Pettit, J. (2006). The slippery slope differentiating between quality improvement and research. Journal of Nursing Administration, 36(4), 211-219.
Patton, M. D. (1993). Action research and the process of continual quality improvement in a cancer center. Oncology Nursing Forum, 20, 751-755.
Perneger, T.V. (2004). Why we need ethical oversight of quality improvement projects. International Journal for Quality in Health Care, 16(5), 343-344.
Pritham, C. H. (1996). Quality improvement process/outcome research. Journal of Rehabilitation Research and Development, 3, vii-viii.
Reinhardt, A., & Ray, L. (2003). Differentiating quality improvement from research. Applied Nursing Research, 16(1), 2-8.
Saunders, M. J. (1993). Director of quality improvement research. Journal of Nursing Care Quality, 7, 39-43.
Simpson, M. R., Stevens, P., & Kovach, C. R. (2007). Nurses’ experience with the clinical application of a research-based nursing protocol in a long-term care setting. Journal of Clinical Nursing, 16(6), 1021-1028.
Solberg, L. I., Mosser, G., & McDonald, S. (1997). The three faces of performance measurement: Improvement, accountability, and research. Joint Commission Journal of Quality Improvement, 23, 135-147.
Thurston, N. E., Watson, L. A., & Reimer, M. A. (1993). Research or quality improvement. Journal of Nursing Administration, 23, 46-49.
Working Group of the Three Federal Funding councils: Medical Research Council of Canada, Natural Sciences and Engineering Research Council of Canada, and Social Sciences and Humanities Research Council of Canada. (1998). Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans. Retrieved on June 16, 2007 from http://www.cs.ualberta.ca/~wfb/ethics/ethics-e.pdf
Theodore D. CoscoMr. Cosco is a third year BSc Specialization Psychology Industrial Internship Program student at the University of Alberta. Currently he holds a Research Intern position at CapitalCare where he has been involved with developing and disseminating research and quality improvement projects as well as creating and implementing a behavior observation tool used to decrease psychotropic drug use. His research interests include research ethics and the interface between technology and practice.
Alana Knopp, BAMs. Knopp earned a BA in Psychology from the University of Alberta in 2007. She holds the position of Research Assistant at CapitalCare and manages the organization's longitudinal database that contains information about Alzheimer residents and has been involved in several evaluation and research projects regarding the relocation of long-term care residents. She is enjoying her experiences with applied research projects before choosing a graduate degree program. Her main research interest is the effect of health care policy and service delivery on the quality of health care as well as the individual psychosocial experience.
Doris L. Milke, MSc, PhDDr. Milke is the Senior Researcher for CapitalCare, where she heads a research unit that supports corporate research and engages in original research funded through external agencies. She chairs a Research Facilitation Committee for her organization. She is an Adjunct Associate Professor in the Faculty of Rehabilitation Medicine, the Faculty of Nursing, and the Department of Psychology at the University of Alberta. Currently she is an investigator on two research teams funded by the Canadian Health Services Research Fund that are studying the delivery of facility-based continuing care.