and Penelope Glynn, PhD, RN
Frontiero, L. and Glynn, P. (October 2012). Evaluation of Senior Nursing Students’ Performance With High Fidelity Simulation. Online Journal of Nursing Informatics (OJNI), 16 (3), Available at http://ojni.org/issues/?p=2037
Studies have shown that nursing students and new graduate nurses are recognized as lacking critical thinking skills. Often, nursing students are only assigned a one or two patients which limits their ability to develop and enhance clinical judgment in preparation for the real nursing-practice environment. This descriptive correlational study evaluated student performance using high-fidelity simulation in a convenience sample of senior nursing students in an undergraduate nursing program. Student performance was evaluated with a two- and a four- patient assignment in the nursing simulation laboratory, situations that mirrored a realistic patient-care assignment. A previously validated Simulation Evaluation Instrument, developed by Todd, Manz, Hawkins, Parsons, and Hercinger (2008), was utilized in this pilot study. Key findings indicated that student performance improved between the sequence of a two- and four- patient assignment.
simulation, evaluation, competency, critical thinking, clinical judgment.
Nurse educators are challenged to prepare nursing students to function safely and effectively while providing quality nursing care in a complex healthcare environment. Nurse managers, educators, and administrators in various healthcare settings often identify the inability of graduate nurses to “put it all together” while managing a multiple-patient care assignment. Some graduate nurses may even be apprehensive about taking a patient-care assignment in the work setting after only having a one or two-patient assignment while attending nursing school. In fact, the inability of graduate nurses to function safely and prioritize care while managing a real patient assignment has been reported in the nursing literature, and this inability has been identified as resulting from a lack of critical-thinking skills and poorly developed clinical judgment (Del Bueno, 2005).
Childs and Sepples (2006), Horan (2009), Jeffries (2005), and Sportsman, Bolton, Bradshaw, Close, Lee, Townley, and Watson (2009) acknowledged that nursing is a practice profession where active learning can be utilized to achieve competencies. Childs and Sepples affirmed that nursing students attained knowledge, acquired critical thinking and psychomotor skills, and developed confidence in their abilities through various active learning strategies and transferred these skills into the clinical setting in caring for patients. One method being used to teach these skills, enhance student competencies, and develop students’ critical thinking is simulation.
Simulation, for the purpose of this study, is an event or situation made to resemble clinical practice as closely as possible (Jeffries, 2005). The effectiveness of simulation as a teaching methodology is well documented in the literature, and its use has increased dramatically in the last few years. Most recently, simulation has been used for purposes of evaluation. Radhakrishnan, Roche, and Cunningham (2007), Sportman, et al. (2009), and Todd, Manz, Hawkins, Parsons, and Hercinger (2008) utilized simulation to evaluate student nurses’ critical-thinking skills and competencies. The study by Radhakrishnan et al. was the first to measure students’ performance with a complex two-patient simulator assignment and the study by Todd et al. evaluated student performance with a complex patient assignment using a researcher-developed simulation evaluation instrument.
Given the documentation of employers’ concern regarding nurses’ ability to transfer their acquired skills and clinical judgment into the real-world setting and the increasing use of simulation for both teaching and evaluation, it would seem that the use of simulation to measure a student’s clinical judgment and competencies in handling a complex patient assignment of greater than two patients might be of value with these students prior to their entry into practice.
Thus, this study measured student performance utilizing simulation by evaluating student performance in a complex two-patient simulator assignment and then, a four-patient assignment in the nursing simulation laboratory. A debriefing after each scenario focusing on individual student results, allowed the researcher to recommend strategies that would improve practice upon transition to the actual work setting.
Like their employers, student nurses are apprehensive about their transition to practice and their ability to meet the expectations of the workplace (Gerrish, 2000). Gerrish reported that student nurses need to be supported with active-learning strategies that foster their clinical, organizational, and managerial skills prior to their transition to their future nursing role. Utilizing critical thinking skills to prioritize care, effectively manage systems based practice, and foster team work and communication is essential in the workplace. To address students’ apprehension, nursing faculty need to plan and implement strategies that will enhance students’ critical thinking and competencies.
The fact that many new graduates are not prepared for entry-level practice is well documented. Del Bueno (2005) has identified that only thirty-five percent of new registered nurse (RN) graduates meet entry-level expectations for clinical judgment regardless of educational preparation and credentials. Therefore, this pilot study utilized multiple-patient simulation assignments that fostered critical thinking skills and evaluated student performance prior to transition to nursing practice. The research question that guided this study was: is there a difference in student performance when senior nursing students are exposed to a two and then a four-patient simulation assignment? The hypothesis was that last-semester senior nursing students will perform better when caring for a two-patient simulation assignment as compared to a four patient simulation assignment as measured by the Simulation Evaluation Instrument.
At this time, there are a limited number of quantitative studies that measure critical thinking and competency development or the transferability of these experiences to the practice environment. According to Jeffries, the transferability of skills developed through simulation to the workplace environment is affected by the realism of the scenario development in the simulation laboratory. The literature identifies that simulation can be utilized in the development of critical thinking, clinical judgment, and competencies (Lasater, 2007). It is also clear that after initially utilizing simulation as an effective teaching modality, the simulation community has moved toward the use of simulation as an effective evaluation strategy for measuring students’ competencies and critical thinking abilities prior to entry into practice. The necessity for a standardized method to quantitatively measure students’ performance outcomes during simulation scenarios is beginning to be addressed by nurse educators.
The importance of evaluation is evidenced in a quantitative pilot study by Radhakrishnan, Roche, and Cunningham (2007), the results of which indicated that simulation can influence nursing clinical practice parameters and student performance and that its effects can be measured. The researchers assessed the performance of twelve senior second-degree baccalaureate nursing students in five categories: basic assessment skills, safety, prioritization, problem-focused assessment, interventions, delegation, and communication skills while caring for a complex two patient assignment.
Study results indicated that the six students who cared for a two patient simulation assignment in addition to a normal clinical rotation with a preceptor performed better in the simulation laboratory at the end of the semester as compared to the control group that had a clinical rotation without any designated simulation assignment. The results of this study emphasized the need for educators to evaluate student performance in a controlled simulation environment with predetermined learning objectives. The strong correlation between improved student competencies with the use of innovative technology supports the use of simulation as an active learning strategy to develop critical thinking skills and improve student performance as well as evaluation measures.
A quantitative study conducted by Todd, Manz, Hawkings, Parsons, and Hercinger that focused on measuring student performance, (similar to the study of Radhakrishnan et al.), resulted in the creation of an effective simulation evaluation instrument (SEI). Todd et al. created and evaluated a quantitative evaluation tool used to assess student performance with simulated clinical experiences. The tool identifies twenty-two behaviors that are evaluated in the categories of assessment, communication, critical thinking, and technical skills. Results indicated that students scored higher in the categories of assessment and communication and lowest in demonstrating technical skills.
This pilot study utilized a descriptive correlational design to evaluate student performance with high-fidelity simulation while measuring critical thinking and designated student competencies in last-semester senior nursing students. The objective was to measure students’ use of critical thinking and demonstrated competencies in assessment, communication, and technical skills when managing a two patient assignment and then a complex four patient assignment. The study took place in a large nursing program at Salem State University in Salem, Massachusetts.
The target population was 132 last-semester senior nursing students in a traditional daytime undergraduate nursing program. The sample was a convenience sample of ten students functioning as the primary nurse in each of two scheduled sessions in the simulation laboratory. Students in the study selected participation in the scenarios from a list of available simulation sessions. A sample size of ten primary-nurse students was chosen because of the time-intensive nature of the simulation session; making the sample size manageable.
Following institutional review board approval, the researcher requested student volunteers to participate by visiting senior students in designated classrooms. The purpose of the research study and time commitment for each participant was explained to all senior nursing students. Senior students who wanted to volunteer were asked to sign up for two scheduled simulation sessions during the first month of the semester.
Each student functioned as the primary nurse in the scenarios and completed a demographic questionnaire prior to participation in the scenarios. Information was kept confidential. The researcher assigned student numbers from one through ten to each testing session and coded the student numbers and information. The only person able to distinguish individual student identities, demographic information, and scored student performance in this study was the nurse researcher.
The researcher facilitated each simulation session by providing instructions and patient reports to the students as well as debriefing each student after every simulation session. Two scenarios were utilized for the first simulation session, and four scenarios were utilized when students returned for a second session. The time lapse between sessions was between two to three weeks based upon student availability. The scenarios were chosen from the set of National League of Nursing (NLN) simulations developed with the Laerdal Medical Corporation in 2010 (Table 1). The researcher explained the expected process and the environment, and provided a patient report including staffing levels for the unit. Students were scheduled for 30 to 45 minutes to provide nursing care for each simulation assignment and thirty minutes for each debriefing session. A student-informed- consent document identified the importance of confidentiality regarding student participation.
Table 1: National League for Nursing (NLN) Scenarios
Two faculty members measured the students’ performance using the SEI instrument after receiving two training sessions on the use of the SEI (Table 2). The training sessions facilitated consistency in inter-rater reliability, the administration of the testing sessions, and discussion of expectations for student performance. Two staff members from the simulation laboratory were trained to manage the computerized scenarios and to verbally respond to students’ interactions with the patient-care simulators. A predetermined script of responses was available to staff in the control booth. Potential researcher bias was controlled during the data collection and simulation sessions as the researcher had no input in the scoring of the SEI for each student or the assignment of students to a simulation session.
Table 2: Simulation Evaluation Instrument (SEI)
Name of Student: ____________________
Student Number _____________________
Faculty Member: _________________________________________________________
Grade: _____________________________ Date________________________________
Adopted tool and permission granted by Hawkins, K., Hercinger, M., Manz, K., Parsons, M. and Todd, M., Creighton University School of Nursing
The researcher conducted a debriefing for each student after the two- patient and the four-patient simulation assignment. Dreifuerst (2009) and the pilot studies by Radhakrishnan et al. and Todd et al. validated the concept of debriefing after each simulation where significant learning can occur through reflection during debriefing. The Massachusetts Nurse of the Future competencies including system-based practice, leadership, teamwork, and communication were integrated into the scenarios, and these concepts, along with the student performance as measured by the SEI, were discussed in the debriefing (Massachusetts Department of Higher Education, 2010). The Nurse of the Future Core Competencies were collaboratively developed by nurse educators in academia and nurse administrators and educators in the practice setting who identified the essential knowledge, skills, and attitudes necessary for the Nurse of the Future to demonstrate while caring for increasingly complex patients (Table 3).
Table 3: MA Department of Higher Education Nurse of the Future Core Competencies
Upon completion of the study, the data collected from the demographic questionnaire and the SEI instruments were analyzed. Descriptive statistics were utilized when analyzing the demographic data with SPSS V18 software program. The statistics utilized for the SEI include the percentile scored for each subcategory and overall performance including percentages, the mean, frequency, and standard deviation. Paired t-tests and Pearson correlation coefficients were used to answer the research question. Each participant achieved a performance-based score, utilizing the SEI with an established minimum passing score of 75 % and a maximum achievable score of 100 %.
The prospect of caring for a simulated patient assignment provided senior nursing students with the opportunity to care for multiple patients while prioritizing nursing care, delegating tasks to team members, and integrating leadership skills into the scenario while utilizing critical thinking skills and demonstrating designated competencies in the performance of caring for a realistic patient assignment. The simulation technology allowed the researcher to utilize patient scenarios that project patient baseline information on a bedside monitor, program vital signs in response to student actions in caring for the simulated patients, and integrate other technology into the care of the simulated patient including accessing computerized medical records, IV pumps, and medication administration. The following sections describe the sample population and study results utilizing the SEI tool.
The sample was comprised of nine last-semester senior nursing students. One student did not perform well in the first simulation session and chose not to continue participation in the study. The sample demographics exclude this participant. The participants’ age ranged from 21 years to a maximum of 34 years, which resulted in a mean age of 23.67 with a standard deviation (SD) of 4.03. Eighty-nine percent of the participants were female (N=9), 100% of students were white, 89% of the participants’ primary language was English, one student had a child, and one student(11% ) reported a previous bachelor’s degree in another field. Eighty-nine percent of the participants were employed, had healthcare experience primarily as a certified nursing assistant, and worked a mean of 17.77 hours per week with a SD of 8.57 and a range of 12 to 30 hours worked each week.
All the students had previous simulation laboratory experience with 50% reporting attending the simulation laboratory three to four times. The mean GPA of the students was 3.32 with a range of 2.54 to 3.78 and a SD of .46. Eight of the participants (89%) did not have any science failures while 33% (N = 9) of the study participants had two nursing-course failures (see Figures 1-3).
Figure 1. Demographic Information
Figure 2. Mean age, hours worked each week, and the grade point average of the sample.
Faculty observed the students’ care of the simulated patients and scored twenty-two behaviors as either 0 (does not demonstrate competency) or 1 (demonstrates competency). If not applicable, no score was given. The passing score equaled 0.75 multiplied by the number of items used. The SEI measured 22 behaviors in the categories of assessment, communication, critical thinking, and technical skills. In exploring the results for the research question, the percentages were analyzed for each category of the SEI as well as the total mean scores for the two and four patient assignment. Statistical analysis of paired t-tests revealed no significant difference in student performance between the two and four-patient assignments. The hypothesis that students would perform better in the two-patient simulation assignment as compared to the four-patient simulation assignment was not supported. The null hypothesis was supported in that there was no significant statistical difference in student performance between the two and four-patient assignment.
Figure 3. Failure rates in one science course and in two nursing courses
In order to better understand the results, the researcher conducted additional data analysis on variables measured in the SEI that may have affected the results. Paired t-tests were conducted based on the results of the scores from the SEI tool used by two faculty members. The components of the SEI were analyzed and are reported in Table 1. In both the two and four- patient assignments, students’ total mean scores were higher in the categories assessment, critical thinking, and communication. The mean score for the category technical skills was lower in both the two and four-patient assignment than the other categories (see Table 4).
Table 4: Mean Scores for Two and Four-Patient Assignments
A Pearson correlation coefficient was calculated for the average of the total mean score for the two and four-patient assignments. A strong positive correlation was found with a value of (r(7) =.718, p < .05), indicating a significant relationship between the performance on two patient assignment, and performance on the four-patient assignment. Study results also indicated a strong positive relationship between the category of technical skills and the total mean score.
The study results indicated a non-statistically significant difference in the students’ mean total performance scores between the two and four-patient assignment. Statistical analysis indicated significant differences in students’ mean performance scores for technical skills, which were lower when compared to mean scores in the categories of assessment, communication, and critical thinking skills. The study findings of lower technical scores were consistent with the previous research results cited by Todd et al. (2008).
One might expect that students would perform better with the two-patient assignment and demonstrate difficulty in caring for a four-patient assignment, since students typically have experience in caring for only one or two patients as a nursing student. The results did indicate that students’ mean total scores increased from the two to the four-patient assignment, but the results lacked statistical significance due to the small sample size. Another factor that may have positively affected student performance was that the students’ four patient assignment included the two patients from their two-patient assignment and two additional patients, instead of four new patients.
Study results provide support for nurse educators utilizing a simulation evaluation instrument to measure students’ performance in a summative evaluation. Benner (2010) has acknowledged simulation as a method to improve student salience with use of realistic patient-care scenarios. During the debriefing after each simulation session, the researcher integrated practice standards and knowledge that would improve student salience. Students reported that practice with the first two patient assignment and the subsequent debriefing assisted them with managing the four-patient assignment.
In caring for more than one patient, patient identification is an essential component in providing safety in patient care especially medication administration (Radhakrishnan et al. 2007, Todd et al., 2008) In this study, students were not consistent in checking the patient wristband, with two of the students (N=9) giving the wrong pain medication to the simulated patient. A focused debriefing after evaluation of the simulation integrated safety and systems concepts, which would foster future student performance and emphasized appropriate patient-safety measures. It may be that students’ demonstrated difficulty with technical skills while managing a multiple-patient assignment because of as their experience as a student is with only a one or two-patient assignment. Students reported feeling stressed in organizing care between the patients, and this may have affected students’ performance under the category of technical skills.
The study results suggest that students need to practice patient identification with multiple- patient assignments, safely administer medications, and utilize standard precautions including handwashing on a consistent basis while caring for multiple patients. Nurses practice in an increasingly complex environment while caring for multiple patients and are exposed to resistant communicable diseases, therefore, adherence to hand-washing protocols is essential. Practicing accurate identification of patients will result in improved patient safety. The students demonstrated consistent skills in prioritizing care and delegating skills to other team members in both assignments. Communication was a strength that was evident in the students’ interactions with their team members. Students were aware of the tasks that could be safely delegated to the nursing assistant, while having difficulty with delegation to licensed practical nurse (LPN) as a team member, this could be attributed to their own experience as a certified nursing assistant. It should be noted that delegation is a skill that improves with a nurse’s experience, and students acknowledged recommendations to change future practice regarding delegation.
Students’ strengths in critical thinking, assessment, and communication were evident in both simulation sessions. The debriefing integrated the Massachusetts Nurse of the Future Core Competencies in the care of multiple patients and was conducted in a positive manner by the researcher. Students reported that the feedback was essential in identifying the strengths of their performance, while identifying information that would improve future performance. Students cared for the two-patient assignment first and then returned to care for the four-patient assignment, all the participants reported that this progression eased the transition to the more complex four-patient assignment.
The limitations in this study were the small sample size, the use of one state university and geographic location in the northeast, and one type of nursing student. Due to students’ volunteering for the study, student diversity was not representative of the student population at the university. Self-selection by the students limited the researcher’s understanding of students’ motivations for participating in the pilot study.
The research results suggest that a research study with a larger study population and a diverse group of undergraduate students would be beneficial as would be utilizing a larger group of faculty evaluators. In utilizing a larger sample size and offering an incentive, such as reducing clinical hours based on study participation, it is possible to increase diversity by capturing participants who may have worked more hours or had children that limited their availability to participate in the study. Due to the time intensive nature of the simulation sessions, utilization of a team of nursing students with assigned roles might be beneficial. Recommendations for future studies include measuring student performance with four completely new patients for the four-patient assignment to better assess possible differences in student performance.
This study contributed to identifying the importance of simulation in the evaluation of critical thinking skills and designated competencies that measure student performance. The use of the SEI assisted faculty in identifying performance issues in students including patient safety and delegation of care that could be addressed during debriefing. The faculty’s summative evaluation as evidenced in the debriefing focused on improving future students’ performance. The integration of technology in the programming of the scenarios, accessing information on the computer, and in the use of technology in medical equipment was essential to faculty and students in developing competency in technology and in creating a realistic patient care environment.
The study results support the belief that student performance can be enhanced by practicing a multiple patient assignment. By focusing on a realistic nursing patient care assignment and incorporating best practices while caring for the patients, the results should be improved patient outcomes and fostering ease of transition into practice. In an increasingly complex health care environment with limited resources, it is essential for nurse educators to utilize technology and active learning strategies to increase students’ critical thinking skills and competencies in preparation for the transition to nursing practice.
Benner, P., Sutphen, M., Leonard, M., & Day, L. (2010). Educating nurses. A call for radical transformation. San Fransciso, CA: Josey-Bass.
Childs, J. & Sepples, S. (2006). Clinical teaching by simulation. Lessons learned from a complex patient care scenario. Nursing Education Perspectives, 27(3), 154-158.
Del Bueno, D. (2005). A crisis in critical thinking. Nursing Education Perspectives, 26(5), 278-282.
Gerrish, K. (2000). Still fumbling along? A comparative study of the newly qualified nurse’s perception of the transition from student to qualified nurse. Journal of Advance Nursing, 32(2), 473-480.
Horan, K. (2009). Using the human patient simulator to foster critical thinking in critical situations. Nursing Education Perspectives, 30 (1), 28-30.
Jeffries, P. (2005). A framework for designing, implementing, and evaluating simulations used as teaching strategies in nursing. Nursing Education Perspectives, 26 (2), 96-103.
Laerdal Medical Corporation. (2010). NLN Simulation in Nursing Education Sim Man Scenarios. Volume 1. Wappinger Falls, NY.
Lasater, Kathie. (2007). High-Fidelity simulation and the development of clinical judgment: Students’ experiences. Journal of Nursing Education, 46 (6), 269-276.
Massachusetts Department of Higher Education. (2010). Creativity and connections. Building the framework for the future of nursing education and practice. Nurse of the future nursing core competencies, Boston, MA. , Author.
Radhakrishnan, K., & Roche, J., & Cunningham, H. (2007). Measuring clinical practice parameters with human patient simulation: A pilot study. International Journal of Nursing Education Scholarship. 4(1), 1-11.
Sportsman, S., Bolton, C., Bradshaw, P., Close, D., Lee, M., Townley, N., & Watson, M. (2009). A regional simulation center partnership: Collaboration to improve staff and student competency. The Journal of Continuing Education in Nursing, 40 (2), 67-73.
Todd, M., Manz, J., Hawkins, K., Parsons, M. & Hercinger, M. (2008). The development of a quantitative evaluation tool for simulations in nursing education. International Journal of Nursing Education Scholarship, 5(1), 1-16.
Linda A. Frontiero is Director of the Nursing Simulation and the Nursing Resource Center and Assistant Professor of Nursing at Salem State University, Salem, MA. Linda has 14 years of teaching experience in higher education nursing education programs and as director of the simulation laboratories. She earned her DNP from the Regis College, MSN and BSN from Salem State University. Dr. Frontiero has received grants funds from the MA Department of Higher Education for simulation and curriculum revision utilizing the Nurse of the Future competencies as well as numerous grants for Scholarships for Disadvantaged Students from Health Resources and Services Administration (HRSA).
Penelope Glynn is Dean of School of Nursing and Associate Professor of Nursing at Regis College, Weston, MA. Glynn has 20+ years of teaching experience in several higher education programs. She earned her Ph.D from Boston University and is known for making presentations on evidence-based practice.