DIFFERENCE IN NURSING DOCUMENTATION
BEFORE AND AFTER COMPUTERIZATION:
A PILOT STUDY (Part 2)

by

Sandra Boldreghini, MSN, RN, CS
Family Nurse Practitioner
Desoto Family Medical Practice, P.C
Olive Branch, MS
sboldre@memphisonline.com

June H. Larrabee, PhD, RN
Research Scientist
Center for Nursing Research
Camcare Health Education & Research Institute
june.larrabee@camcare.com



Measurement

Nurse-Perceived Quality
Quality was operationalized as nurse-perceived quality (NQUAL), which was the average score on process criteria for a maximum of three nursing diagnoses and was measured using the Nursing Care Plan Research Data Collection Instrument (NCP DCI) (Larrabee et al., 1995), a criterion-referenced instrument. This instrument uses nursing interventions transcribed from an individualized nursing care plan as process indicators of nursing care quality. These indicators or criteria were different for each patient's care, since the care plans were individualized to guide nurses in meeting specific patient needs. Transcription of three interventions for the nursing diagnosis "impaired gas exchange", for example, would become the following process criteria: "During the patient's stay on the unit, there is documentation that: 1. The patient was positioned every 8 hours to improve gas exchange (head of bed in high-Fowler's, support arms with pillows, or place pillow on overbed table for patient to lean on.) 2. Oxygen was dministered as ordered q hours. 3. Dyspnea is assessed at least q hours. (Larrabee, 1992)." After transcribing the process criteria onto the NCP DCI, the data collector reviewed the nursing care from the time the care plan was begun to the time the care plan was discontinued or the time of discharge. Each process criterion had two questions: "How many times should it have been done?", the denominator, and "How many times was it done?", the numerator. If an action should have been done once every eight hours and the patient was in the hospital for 72 hours after the nursing intervention was selected, then the action should have been done nine times. Nine became the denominator for that criterion. The number of times the action was actually done became the numerator. If the nurse neglected to chart an action, that decreased the numerator; therefore, the percentage score accounted for missing data. The percentages for all process criteria were averaged to arrive at the NQUAL score. In the original study measuring NQUAL (Larrabee et al., 1995), interrater reliability for NQUAL was demonstrated to be acceptable using both interrater agreement (84.2%) and Wilcoxin sign rank t-test (p = .82).

Because of the study site's size, this operationalization of quality was not limited to a few nursing diagnoses or a few interventions, because excessive time would elapse before acquiring a sufficient sample of charts that shared nursing diagnoses. Also, there was no mechanism at the study site for identifying which nursing diagnoses were included in which charts prior to pulling charts in the Medical Records Department. Care pertaining to a maximum of three nursing diagnoses, goals, and interventions was reviewed, because that is the maximum number that gave some variability of the data generating the scores and was still feasible to collect. This approach to operationalizing quality enabled sampling documentation comprehensiveness rather than doing a census review.

Nurse Goal Achievement
Beneficence was operationalized as nurse goal achievement (NGOAL), which was the average score of the outcome criteria for a maximum of three nursing diagnoses and was measured using the NCP DCI. This instrument uses expected outcomes on an individualized nursing care plan as indicators of nurse goal achievement. These outcome indicators or criteria were different for each patient, since specific patient needs were different. As with the interventions, as many as three expected outcomes for each of the three randomly selected nursing diagnoses were transcribed onto the NCP DCI. Then, the data collector reviewed the nursing care from the time the care plan was begun to the time the care plan was discontinued or the time of discharge. Each outcome criterion had two questions: "How many times was it assessed?", the denominator, and "How many times was it met?", the numerator. If an expected outcome was not met, that decreased the numerator; therefore, the percentage score accounted for missing data. The percentages for all outcome criteria were averaged to arrive at the NGOAL score. In the original study measuring NGOAL (Larrabee et al., 1995), interrater reliability for NGOAL was demonstrated to be acceptable using both interrater agreement (89.3%) and Wilcoxin sign rank t-test (p =.70).

Procedure
This study was conducted using retrospective chart review. Interrater reliability was established by the first author and the instrument originator separately reviewing three randomly selected charts from August 1995 and two charts from April 1996. Interrater agreement for the August 1995 NQUAL (100%) and NGOAL (86.9%) and for the April 1996 NQUAL (100%) and NGOAL (89.2%) scores was acceptable.

Twenty charts with a nursing care plan for August 1995 and 20 charts with a nursing care plan for April 1996 were randomly chosen by medical records personnel from a master patient list and data collected by the first author. The first author prepared the NCP DCI for data collection by transcribing onto the form as many as three expected outcomes and three nursing interventions for each of the three randomly selected nursing diagnoses. When there were more than three nursing interventions in a nursing diagnosis, three were randomly selected for review by a roll of a die. Some care plans contained as few as one nursing diagnosis and one nursing intervention. Then, each chart was reviewed for documentation of the selected expected outcomes and interventions. The data were entered into an EXCEL spreadsheet and uploaded for analysis using the Statistical Analysis System (SAS) software, version 6 (1989a; 1989b) on the VAX cluster located in the Computing and Telecommunications Center at the University of Tennessee, Memphis.

Analysis
Mean, standard deviation, and range were used to describe NQUAL and NGOAL scores. Two-tailed t-tests were used to identify differences in NQUAL and NGOAL scores before and after implementing the NIS. Because only the postimplementation NGOAL score approximated normal distribution, Spearman rank correlation (Burns & Grove, 1993, p. 505) was used to identify relationships between NQUAL and NGOAL scores before and after implementing the NIS. An alpha level of significance of < .05 was used.

Results
No statistically significant difference (p = 0.52) was demonstrated between the preimplementation NQUAL score (mean = 71.9, SD = 29.5) and the postimplementation NQUAL score (mean = 77.4, SD = 23.8), even though the postimplementation score was higher, as anticipated. An unanticipated finding was that the preimplementation NGOAL score (mean = 90.2, SD 25.7) was significantly higher than the postimplementation NGOAL score (mean = 56.9, SD = 29.5). Finally, there was no relationship demonstrated between quality (NQUAL) and beneficence (NGOAL) before (r = -0.04 , p = 0.87) or after (r = -0.27 , p = 0.26) implementing the computerized NIS.

Discussion
Difference in Preimplementation and Postimplementation NQUAL Scores
Finding that there was not a statistically significant difference between nurse-perceived quality (NQUAL) before and after implementing the NIS was unanticipated. This finding differs from other studies that demonstrated improved documentation of nursing data, including nursing interventions (Pryor, 1989), after computerization (Barnoud, 1994; Churgin, 1994; Kahl et al., 1991; Pabst et al., 1996).

In a quality study, the anticipated result when collecting data on desired interventions is that the scores will be negatively skewed, approaching 100%, as both the pre- and postimplementation NQUAL scores were in this study. These scores compare favorably as quality indicators, therefore, in comparison to the postcomputerization NQUAL score (mean = 51, SD = 16) in an earlier study (Larrabee, 1992). This finding suggests that nurses participating in selecting an NIS should consider how well the prospective NIS facilitates intervention documentation.

A probable explanation for the high preimplementation NQUAL score in this study is the nurses' awareness that, legally, "If it is not documented, it was not done" and the emphasis at the study site on the necessity for comprehensive documentation. A likely explanation for the higher postimplementation NQUAL score is that the "Document Interventions" screen actually prompts nurses to document the nursing interventions selected in the nursing care plan. This feature contrasts with the system at the site of an earlier study (Larrabee, 1992) and may account for the large difference in mean postimplementation NQUAL scores at the two sites. This represents an advantage of the NIS at the current study site and supports Pryor's (1989) conclusion that charting programs influence the quality and quantity of data documented.

Failure to demonstrate significant improvement in nursing intervention documentation (NQUAL) postimplementation may result from two factors. First, the preimplementation NQUAL score was high, indicating good nurse awareness of the necessity to document interventions. The preimplementation care plans were parsimonious with most having only one or two nursing diagnoses and only one or two nursing interventions. Perhaps nurses handwriting care plans were highly selective, choosing only very pertinent nursing interventions and avoiding others that they would be less likely to document. Second, ease of care plan generation using the NIS enabled nurses to include more nursing problems or diagnoses and their interventions postimplementation, with most care plans addressing three or more nursing diagnoses and, in some care plans, dozens of nursing interventions. During data collection, it appeared that some nurses had selected standardized care plans without eliminating less pertinent or even inappropriate interventions. For instance, the standardized care plan for pain management includes interventions pertaining to use of a patient-controlled analgesia pump. If these interventions were inappropriately left on the care plan of a patient who was not on a patient-controlled analgesia pump, nurses would appropriately not chart such interventions as having been done. Yet, the NQUAL score for the patient-controlled analgesia pump interventions would be low because NQUAL was based on the assumption that the nursing interventions selected for the care plan were appropriate. Even when the selected interventions were appropriate and even though the system would prompt nurses to chart all planned interventions, the nurses may not have had enough time to do so and may have only charted those thought to be most important. This explanation challenges an underlying assumption of the study--that nurses generate individualized, appropriate care plans. The implication for practice is to determine the appropriateness of care plans generated and, when necessary, reeducate nurses regarding priority selection when generating individualized nursing care plans and to stress the need to include only the most important nursing interventions. In summary, the findings suggest that nurses using an NIS should assess how NIS characteristics or nurse behaviors influence intervention documentation and implement corrective actions, as appropriate. The findings and their probable explanations indicate that the validity of NQUAL as a measure of quality is suspect. Improving the completeness of intervention documentation is necessary to demonstrate data validity, an important prerequisite for using chart data to investigate the influence of interventions on goal achievement.

Difference in Preimplementation and Postimplementation NGOAL Scores
Finding that the preimplementation NGOAL score was significantly higher than the postimplementation score was unanticipated. Both NGOAL scores were inconsistent with a previous finding that one-third of charts were missing documentation of outcomes (Ehnfors & Smedby, 1993). The postimplementation NGOAL score in the current study (mean = 56.9, SD = 29.5) was quite different from the postimplementation NGOAL score (mean = 89, SD = 14) in an earlier study using a different NIS (Larrabee, 1992). There are two plausible explanations for this unanticipated finding. First, as with nursing diagnoses and interventions postimplementation, the ease of generating nursing care plans resulted in many more nursing goals being selected. Some of these goals may not have been very important for that patient or were even inappropriate. For instance, the nursing goal "acknowledge impending death" was probably inappropriate for a young patient in stable condition who was hospitalized for a liver transplant evaluation due to hepatic cancer. "Impending death" was probably not going to be acknowledged due to that patient's hope of a longer life with a transplant. This observation suggests lack of goal congruence that occurs when nurses establish goals that are inconsistent with patient goals and is thought to be a reason that NGOAL was not a predictor of patient-perceived quality (Larrabee et al., 1995). The implication for practice is that nurses should evaluate the appropriateness of goals selected during care planning, and, if appropriate, be reeducated regarding generating nursing care plans and emphasizing selection of goals that are appropriate and, when possible, congruent with patient goals.
The second explanation for failure to demonstrate improved nurse goal achievement (NGOAL) postimplementation involves three NIS characteristics. First, unlike documenting interventions, this NIS did not provide prompts to document goals. The system contrasts with the NIS in an earlier study (Larrabee et al., 1995) that facilitated the documentation of goal achievement by allowing the nurse to click on one field for attainment of a goal, thereby contributing to a high postimplementation NGOAL score (mean = 89, SD = 14). Because the NIS in the present study did not have this documentation feature, the only way for many of the goals to be documented was by typing narrative notes. Because of the extra time this required, goal achievement was not documented in many cases. For instance, anxiety as a nursing diagnosis has the goal "the patient will report feeling less anxious." Assessment of the level of anxiety was documented as a nursing intervention, but there was no documentation describing change in level of anxiety. This resulted in a low NGOAL score postimplementation. The implication for practice is that nurses at hospitals using this NIS should request programming changes to facilitate documenting goal achievement.

The second NIS characteristic that may have prevented improved NGOAL documentation pertained to the intervention field "check incision for redness, swelling, or drainage." There was no companion field for documenting the actual status of the incision and, because typing narrative notes takes more time than pointing and clicking, these observations were often not documented. Such fields should be added to the NIS to facilitate documenting goal achievement.

The third NIS characteristic that may have prevented improved NGOAL documentation pertained to teaching the patient. Although there were fields to document teaching interventions, there were no fields to document patient learning, such as the patient demonstrating or verbalizing an activity. Again, such fields should be added to the NIS to facilitate documenting goal achievement.

Identifying these three NIS characteristics that may have impeded nurses documenting goal achievement has implications for practice. First, nurses using an NIS that likewise does not facilitate such documentation need to assess how well goal achievement is being documented in their setting. Second, these nurses should consider requesting the NIS vender to modify the system. And, third, nurses participating in selecting an NIS should consider how well the prospective NIS facilitates documenting goal achievement. The findings and their probable explanations indicate that the validity of NGOAL as a measure of beneficence is suspect. Improving the completeness of goal achievement documentation is necessary to demonstrate data validity, an important prerequisite for using chart data to investigate factors that influence goal achievement.

Go to Part 3 of the article.