Volume 12 Number 1 - February 2008 Issue Submit Your Manuscript for Publication!

Online Journal of Nursing Informatics (OJNI) Spring 2008 Volume 12, Number 1
ISSN # 1089-9758 Indexed in CINAHL © 1996 - 2014

Abdallah, L. (February, 2008). Reflective Teaching with Technology: Use of a Personal Response System and Publisher's web site to Enhance Students' Performance in a Nursing Assessment and Skills Course. Online Journal of Nursing Informatics (OJNI), 12 (1) [Online]. http://ojni.org/12_1/abdallah.html


Reflective Teaching with Technology: Use of a Personal Response System and Publisher's web site to Enhance Students' Performance in a Nursing Assessment and Skills Course

by Lisa Abdallah PhD, RN


Nursing Informatics This article describes a Carnegie Scholarship of Teaching Research Project on Reflective Teaching with Technology conducted in the fall 2006 at the University of Massachusetts Lowell under the guidance of the Faculty Institute. The purpose of this project was to enhance student interaction in a lecture based nursing course by introducing a multimedia approach with technology. Specifically, a Personal Response System for students to use remotely during class and student access to the textbook's web site to complete online case studies and NCLEX based examinations. Reflection and evaluation of these technologies and their relationship to students' understanding of course objectives were examined. In addition, students were asked to evaluate the above technology at midterm. The summary of data analysis of student performance on quizzes and midterm examinations and student evaluation of the technology both supported its implementation as being an effective teaching activity that enhanced students' learning. Future implementation of technology into other nursing courses and continued research on the use of technology in the nursing classroom and its relationship to quality education is recommended.

Key Words: Teaching with Technology, Nursing Education, Reflective Teaching with Technology, Personal Response System in the Classroom


Nursing faculty strive to provide high quality nursing education for their students. In doing so they must reflect on their teaching methods and continue to improve the educational environment in order to achieve the best teaching outcome. One approach is to incorporate innovative teaching strategies such as technology in teaching and evaluate the effectiveness of newly instituted teaching methods in order to contribute to the future of nursing education.

Black and Watties-Daniels (2006) reviewed the literature relating to technology and enhanced learning and found a large amount of educational literature supporting technology as an enhancement to the learning environment. However, research conducted on the use of technology specific to nursing classroom instruction between 1988 and 2006 was limited. Of the nursing literature reviewed technology was stated to have improved clinical reasoning and decision making skills (p.104). Simpson (2003) discussed how technology has transformed nursing education and stressed that nursing instructors should consider incorporating technology into traditional nursing classrooms as a mechanism to enhance instruction. Many computer programs can be added to course instruction as a way of simulating the clinical setting or a clinical scenario prior to the student actually entering into the actual clinical situation. The nursing literature also discusses the integration of multimedia as a mechanism to stimulate critical thinking, facilitate student interaction (Kennerly, 2001), and foster student participation in the classroom(Billings & Kowalski, 2006) allowing them to be active contributors to class dialogue and content. In addition, Walsh & Seldomridge (2006) recommend that nursing faculty rethink how classroom content should be delivered. They suggest that nursing educators consider moving away from the model of delivering all the details to re-structuring content based on principles. By doing so faculty will allow students more opportunities for active dialogue and discussion and promote critical thinking in the classroom.

This article describes a Carnegie Scholarship of Teaching Research Project on Reflective Teaching with Technology conducted in the fall 2006 at the University of Massachusetts Lowell under the guidance of the Faculty Institute. The purpose of this project was to enhance student interaction in a lecture based course by introducing a multimedia approach with technology. Specifically, a Personal Response System for students to use remotely during class and student access to the textbook's web site to complete online case studies and NCLEX based examinations. Reflection and evaluation of the use of these technologies and their relationship to students' performance in this course will be discussed This project was instituted in the Nursing Assessment and Skills course (33:313) for junior nursing students (N=71). This course introduces nursing students to the foundations of health assessment and psychomotor skills and emphasis is placed on the integration and application of these skills through the use of critical thinking. Prior to beginning this project the course information was delivered to students using a typical lecture format with PowerPoint technology and students demonstrated application of critical thinking with quizzes, exams and in the accompanying laboratory sessions. The students taking this course were all in the nursing major and had all completed the same courses necessary to enter this course. Although students were taking the same course, they were quite diverse in their learning needs and styles. These needs were considered when this technology enhanced active learning strategy was implemented to this course.

The plan was to enhance the students' learning experience for the fall 2006 semester by adding two additional technological enhancements methods to this traditionally lecture-based course. The specific goal of this project was to focus on one specific learning objective where students could benefit from using the technology to enhance their critical thinking and to have a better understanding of the course content. This specific learning objective was: upon the completion of this course the student would be able to formulate health assessment data and nursing diagnoses based on data collected. The project introduced the following teaching and learning activities into the course in order to accomplish this goal.

The first technology-enhanced teaching activity incorporated was a Personal Response System (PRS). The PRS is an electronic student response system that uses individual battery powered transmitters or clickers. Each student participating in the course was asked to purchase through the university bookstore their own clicker that had an individual ID number. Students were responsible for bringing their clickers to each class. Students were presented with questions throughout the lecture that assessed their understanding of content presented. Students could choose their appropriate number or letter responses to the questions presented in the classroom via the PowerPoint presentation and aim their clicker at the PRS receiver which was installed in the front of the classroom. The class was held in a lecture hall already equipped with the PRS receiver and the software was loaded on the computer in the front of the room. The PRS software can take all of the students' responses and display the group responses on a bar graph to show the overall results of students' responses. This technology provided immediate feedback to the instructor and students about understanding of the material being presented and allowed further clarification or dialogue between the students and the instructor. This enhancement to the learning environment employed the principles for quality undergraduate education of active student involvement and prompt feedback (Chickering & Erhmann, 1996).

Comeaux (2005-2006) discusses the constructivist learning model as active engagement of students in the learning process by making them be reflective and analytical learners. By encouraging this in students they can best obtain the information being taught. In addition Comeaux states that the expectations of the learner must be clearly stated and the bar should be raised high, but not too high that the learner can not achieve the expected outcome. Assessment should be used in order to determine just what the students do know or do not know about the material presented. Students in this course were also assigned to access the publisher's web site that accompanied their textbook and complete case studies and the NCLEX based examinations that accompanied two specific chapters relating to respiratory and cardiac assessment. These exercises required the students to utilize critical and analytical thinking in order to complete questions? By participating in such exercises the belief is that students would further synthesize the content they are learning in the course.


During the first few class sessions students were introduced to the PRS system by providing them with general multiple choice questions to answer posted via the PowerPoint presentation. Students were to use their clickers and choose the best response as a mechanism to allow students feel comfortable with the technology. Then in each class content relating to the specific course objective (formulate health assessment data and nursing diagnoses based on data collected) was introduced to the students and case based questions with multiple choice responses were given via the PowerPoint presentation for students to answer. Students were also assigned to go to the textbooks free website and complete the assigned case studies and NCLEX based exams each week. Students were introduced to the publisher's website in the first class and shown how to maneuver through it.

The case studies and NCLEX based exams which accompanied the Respiratory and Cardiac systems were assigned as graded activities. Students were required to complete the case studies, print out their results and submit to the instructor at each class. Students also completed the NCLEX based exams for the respiratory and cardiac assessment chapters and were expected to email their results to the instructor. It was a course requirement for students to participate with the PRS, complete the website case studies and NCLEX based exams. Because the textbook website provided students the ability to use an expert response as a resource when completing the case studies, specific exam questions relating to collection of health assessment data and formulation of appropriate nursing diagnoses were incorporated into the first two quizzes and to the midterm examination as an objective measurement of the extent to which students comprehended the information related to the course objective chosen for this project. This project was granted exempt status by the Institutional Review Board of the University of Massachusetts Lowell .


The hypothesis for incorporating these technologies into this lecture based course was that student's performance on the first two quizzes and on the midterm examination would improve and student feedback on the midterm evaluation of the use of these technologies would indicate a positive response. The rationale to support this hypothesis is that incorporating the PRS system into the lecture component of the nursing assessment and skills course and adding the publisher website case studies and NCLEX based exams would enhance student involvement and motivation to be active learners of the complex clinical information being presented. It also would provide for immediate feedback of students' understanding of the information being provided and allow for further discussion and clarification of material. Most importantly, the use of case studies in class and as additional assignments would stimulate critical thinking in students and assist them in meeting the course objective of being able to collect health assessment data and in turn use critical thinking to formulate nursing diagnoses accurately.

Resources required for accomplishing this project:

All students had access to computers in the nursing labs so that they could access the publisher's website if they did not own a computer. All students were sent a notification via email two weeks before classes began that they were required to purchase PRS clickers prior to the first day of class. In addition, the PRS equipment in the lecture hall had been previously installed. As a mechanism to prevent a student forgetting to bring his/her clicker to class, students received an incentive for attending every class with their clicker. They received participation credits on the syllabus and in order to receive full credit they must have attended each class with their clicker. The data print out from the generated PRS class roster collected data on each student who used a clicker each class in order to keep track of this. Therefore by default it also became a great attendance tracker as well.

Data Collection and Analysis

A class grading roster with student ID numbers and PRS clicker ID numbers was developed and loaded into the PRS software program on the computer in the lecture hall. This allowed for data to be generated by the PRS system for each class session and be saved to a disk for future analysis to evaluate students understanding of the course material. To evaluate the effect of the newly incorporated technology based activities on student learning, additional exam questions that measured the student's ability to collect health assessment data and formulate nursing diagnoses based on data presented was added to the first two quizzes and the Midterm Examination. Data generated from the first two quizzes and midterm exam analysis was transferred onto an excel spreadsheet for analysis. To gather student feedback on the use of the technologies, a midterm assessment of student feedback on the PRS system and publisher web site was given to all students to complete anonymously after the midterm examination. The computer generated grade book data of student PRS participation for each class and data generated from the quizzes and midterm examination were formed into a data set and using SPSS 10.0 statistical analysis software package cross tabulations and Chi square analyses (Agresti, 2002) were conducted to examine whether the use of the Personal Response System enhanced student performance on the first two quizzes and midterm examination. This was accomplished by analyzing the overall class performance on the PRS questions related to health assessment and nursing diagnoses content answered in class and comparing this performance to the overall class performance on the same questions included in the first two quizzes and midterm examination (Table 1). To prevent inflated p values from multiple comparison testing, significance of p values was examined using the Bonferroni correction and with the use of the Simes test (Simes, 1986).

A comparison of class average performance on the Respiratory and Cardiac NCLEX based examinations assigned to students from the publisher's website and class average performance on Quiz 2 and the Midterm examination is presented in Table 2. In order to examine whether the use of the PRS questions in class, and the Respiratory and Cardiac NCLEX based examinations enhanced students' performance on Quiz 2, the Midterm examination, the final exam, and final course grade, four separate Multiple Linear Regression analyses were conducted (Tables 4,5,6,7). The first regression analysis used performance on the Midterm examination as the dependent variable and the independent variables were categorical variables for class average performance on the two NCLEX based examinations. The categorical variables were for High average (90-100), Medium average (80-89) and Low average (below 80) on the combined NCLEX scores. In addition another set of categorical independent variables were included in the regression analysis for the number of PRS questions correct, the number of PRS question incorrect, and the number of PRS questions not answered. Three other regression analyses were conducted using Quiz 2, the final exam, and the final course grade as dependent variables. In order to collect information about the students who participated in this class, students were presented with PRS questions in class on the first day to demonstrate the ease of the system and to receive preliminary feedback relating to the technology that was going to be instituted into this class (Table 3). Finally students were asked to anonymously complete an evaluation of this technology after the midterm examination (Appendix 1).

Findings and Reflections

Findings from the cross tabulation and chi square testing on student performance on PRS questions in class compared to student performance on same or similar content questions during examinations (Table 1) demonstrated that student exposure to the PRS questions in class was associated with a high overall class percentage getting the same or similar content questions correct on the first two quizzes or the midterm examination. Whether a student answered the PRS question correctly, incorrectly, or did not answer the PRS question in class, was related to a 70% or greater number of students in the class answering correctly (range 70% to 92.5% of the class) the same or similar questions when incorporated into the first two quizzes and/or the midterm examination. Three of the PRS questions that were tested on subsequent quizzes or the midterm (Best indicator of the general health of a child, Nursing Diagnosis of Decreased Cardiac Output, and Nursing Diagnosis of Immobility) were found to be statistically significant. However, the remaining eight content areas were not. The specific content on these questions were related to the purpose of the general survey, and nursing diagnoses of ineffective airway clearance, ineffective breathing pattern, and alteration in comfort and activity intolerance. This content is challenging to learn and students may need more in depth information in order for them to synthesize this information well. Interesting to note is that students performed at a high level ranging from 80%-92.5% of the class answering questions correct on the midterm that were previously presented to students via PRS questions in class. Reasons for this may be attributed to the fact that after students recognized that material presented in class via the PRS system began showing up on examinations, they were more attentive to studying this and demonstrated so on their improved performance by the midterm.

There were no significant relationships found in review of the regression analysis on the NCLEX average scores and PRS questions and their relationship to class performance on Quiz 2, Midterm, Final and final course grade,. The overall class average scores on both NCLEX examinations were higher (89% and 92%) than the overall class average scores on course examinations that covered this content which were 79% and 87% respectively. This may be attributed to the fact that the NCLEX examination exercises allowed students to review each question for the correct answer and rationale using an expert response icon on the website and this in turn could assist them in performing at a higher level than when given the actual test. In retrospect however, the class average score on the midterm examination was quite high (87%) demonstrating that the students were well prepared. It is important to note that the students' evaluation of the NCLEX based examinations supported its contribution to their performance.

The initial survey of students conducted with the PRS system on the first day of class demonstrated that 100% of students had access to a computer with the internet, 85% of the students felt that the use of NCLEX question and case study assignments would enhance their ability to meet course objectives, and 15% were not sure. Only 56% of the class felt that the use of the PRS questions in lecture would enhance their ability to learn the material presented, 6% did not and 38% were not sure. It is important to note that only 50 out of the 71 students enrolled in this class answered this survey using the PRS clickers on the first day of class. The reason for this was that although an email was sent to all registered students in August informing them how to purchase this clicker at the cost of $29.99 at the bookstore and to have this for the first day of class some students did not check their email and this led to only 69% of the class being prepared to use the PRS on the first day of class.

Findings of the student evaluation at midterm demonstrated the following characteristics of the class: 73% had greater than five years of experience with technology, 23% had up to five years experience and 3% had less than 1 year, 81% were first degree students and 17% were second degree students, 87% were female and 11% male, 56% were 18-22 years of age, 21% were between the ages of 23-27, 6% were 28 to 32 and 10% were older. Interestingly, 17% reported to be students with English as a second language and 74% were not. Results of the student evaluation at midterm (n=70) demonstrated an increase in the percentage of students (67%) who either agreed or strongly agreed that the PRS enhanced their ability to understand the information presented to them in comparison to those asked on the first day of school (56%). In addition to this, 52% of the class either agreed or strongly agreed that the case study assignments assisted them to utilize critical thinking and enhanced their learning of class content, 23% were neutral and 25% either disagree or strongly disagreed. Of interest, however, is that 80% of the class strongly agreed or agreed that the use of the NCLEX class assignments assisted them to utilize critical thinking and therefore enhanced their learning of the content, only 7% were neutral and 13% disagreed, supporting continuation of this technology in future courses. When students were asked if all of the technology incorporated into this course assisted them to perform better on required quizzes and the midterm 64% either strongly agreed or agreed, 17% were neutral and 18% disagreed. When students were asked to indicate how each of the incorporated technologies assisted them to meet the course objective of: Demonstration of the ability to collect health related data systematically through history taking and selected psychomotor skills, 76% of the class indicated the PRS as being either good to excellent to enhance their ability; 77% indicated the case studies; and 82% indicated the NCLEX assignments as being in this range as well. For the course objective: Formulation of health assessments and nursing diagnoses based on data collected, 87% indicated that the PRS was good to excellent; 85% indicated the case studies, and 82% indicated the NCLEX questions to be in this range. These data demonstrate that the students felt strongly that the use of all of the technologies incorporated into this course enhanced their overall performance.


Implementation of this project faced several challenges. Some of the students were coded in the data as no response to PRS questions. The reasons for this were students forgetting to bring their PRS clicker to class and the occasional ineffective clicker reception from equipment installed in the lecture halls. This poor reception was due to the excessive number of students (71) trying to use their clickers at the same time. When contacting the personnel in media technology to ask about this after the first few classes, it was determined that the large class size overwhelmed the current PRS system because it could not receive multiple students aiming at the receivers at the same time. The ideal size for the current clicker system was 30-40 students. The University would need to invest in a better system that works with radio waves instead of infrared. The cost of this system was estimated to be approximately $5,000 to install but it would allow all students to aim simultaneously and be recorded. This would be necessary if one wanted to continue the use of a PRS system in large lecture halls with a large number of students. To accommodate for the current systems limitations students were instructed to aim one row at a time in order to be received and this unfortunately took the spontaneity out of the intent of this system making it harder to use the system and requiring a limit on the number of questions to be used per class. Another problem noted when using the PRS system with PowerPoint presentations was that the use of any slide transition or builds in the program would interfere with the PRS system. Therefore one has to add the new technology of the PRS but at the cost of making the presented information free from animation.

The use of the publisher's web site activities were well received by the students but due to the large class size (71) the emailed NCLEX based examinations that students were instructed to complete and send to the instructor each week were perceived as overwhelming. In discussion with media support about the overload to the email system, it was suggested that a filter be added so that all of these messages would go into a folder. Unfortunately, these messages do not all come from the same site so this filter idea did not work. The intent for future classes will be to have a Web CT course management site and all of the NCLEX responses would be filtered to be received at this site instead of through email. The case studies were printed by each student and given to the instructor in the beginning of class; this also was found to be very overwhelming for one instructor without any assistant to review and grade each week's assignment. It was found to be a worth while exercise for students and certainly required critical and analytical thinking on the part of the student. It is recommended to continue with this assignment in future classes. It is important to note that faculty when managing a class of 71 students could be overwhelmed with the addition of grading of case study assignments on a weekly basis. This may deter faculty who teach large classes away from incorporating such time consuming assignments unless they have some assistance with this.

Solutions to Challenges

Plans for developing additional technology-enhanced teaching activities for this course would primarily be focused on development of a Web CT course management site. This would allow students to receive the syllabus and PowerPoint handouts ahead of time, communicate with the instructor and each other in threaded conversations and allow all required assignments to be received at this site as not to overwhelm the instructor's personal email system. In addition, future consultation with the university technology department to consider purchase and installation of appropriate PRS system to accommodate large classes.

Implications for Future Research

Future research that examines the use of technological enhancements in other nursing courses should be considered as a mechanism to add to the body of knowledge relating to quality nursing education. Studies which examine the use of the PRS system and its effect on student performance in other nursing content areas should be considered. In addition, the use of critical thinking exercises and web based technologies should be examined and evaluated for their contribution to the quality of undergraduate nursing education.


The incorporation of this technology-enhanced teaching activity has been a rewarding experience for the faculty and student participants. This experience supports this instructor's personal philosophy of teaching which is based on the premise that in order to enlighten students one should employ a variety of teaching methods, including, but not limited to, technology, case studies, lectures, focus groups, hands on demonstration, and critical thinking exercises. The summary of data analysis of student performance on quizzes and midterm examination and student evaluation of the technology both support its implementation as being an effective teaching activity that enhanced students overall performance in class and in turn led to a quality learning environment. This study finding lends further support to incorporating technology based teaching activities into nursing educational process. Nursing faculty should consider such technologies for future application into their course design and delivery.



Agresti, A. (2002). Categorical Data Analysis, 2nd Edition. John Wiley & Sons, Hoboken, N.J. Chapter 3.

Billings, D.M. & Kowalski, K. (2006). Keeping participants enrolled: Participation. The Journal of Continuing Education in Nursing, 37 (4), p.152-153.

Black, C.D. & Watties-Daniels, A. D. (Summer 2006). Cutting edge technology to enhance nursing classroom instruction at Coppin State University. ABNF Journal, p.103-106.

Chickering, A.W. & Ehrmann, S.C. (1996). Implementing the seven principles: Technology as lever. AAHE Bulletin, (3), p.3-6.

Comeaux, P. (2005-2006). Assessing students' online learning: Strategies and resources. Teaching Excellence, 17(3).

Kennerly, S. (2001). Fostering interaction through multimedia. Nurse Educator, 26 (2), p.90-94.

Simes, R. J. (1986). An improved Bonferroni procedure for multiple tests of significance. Biometrika, 73, 751-754.

Simpson, R.L. (2003). Welcome to the virtual classroom: How technology is transforming nursing education in the 21st century. Nursing Administration Quarterly, 27(1), p.83-86.

Stemwedel, J.D. (2005, November). Rubrics, roles and successful online discussions. Online Classroom, p.3, 8.

Walsh, C. M. & Seldomridge, L. A. (2006). Critical thinking: Back to square Two. Journal of Nursing Education, 45(6), p.212-219.

Author's Bio

Lisa Abdallah PhD, RN

Dr. Abdallah is an Assistant Professor in the Nursing Department of the School for Health and Environment at the University of Massachusetts Lowell and teaches in both the undergraduate and graduate nursing programs. She is a John A. Hartford Institute Geriatric Nursing Scholar. Dr. Abdallah strives to incorporate new technologies into the classroom and clinical setting to enhance her current teaching strategies.


I would like to thank E. Andres Houseman Sc.D. Assistant Professor, Department of Work Environment, University of Massachusetts Lowell for his statistical expertise and guidance during the analysis of the data and Lin Zhan PhD, RN, FAAN, Professor, Department of Nursing, University of Massachusetts Lowell for her editorial review of this manuscript.