J. Seymour (Dr. Roz), EdD, RN
East Tennessee State University
1998 the National Education Association (NEA) and Blackboard Inc., jointly
commissioned The Institute for Higher Education Policy to examine the
multitude of quality assurances (QA) benchmarks which have been produced since
1994. These benchmarks or QA strategies for distance education have been in
existence for many years. The question the commissioning organizations wanted
to answer was are there specific benchmarks applicable to Internet-based
distance education. The answer was found in the literature, interviews with
faculty, administration, and students with substantial experience in distance
education (as in offering more than one degree program via online distance
learning). Of the 45 identified benchmarks found using these methods, 24 were
determined to be essential to ensure quality in Internet-based distance
education. They are recommended for use by all levels of administration,
accrediting bodies, state legislatures as well as faculty and students to make
reasonable and informed judgments with regard to the quality of
Internet-based distance education both before and after it is offered.
an effort to assess the degree to which the benchmarks for Internet-based
distance education where met I adapted the program benchmarks to be able to
use them to evaluate one course offered at ETSU. The course will be presented
here as a case study using all 24 benchmarks, resulting from the commissioned
study, as evaluation questions.
course is open to all graduate or undergraduate students interested in health
care informatics. The course consists of one in-classroom face-to-face meeting
on the first night of class, four scheduled field trips (one is optional the
other three are required), and a final face-to-face meeting on the last day of class, at a local restaurant,
for exit interviews.
course incorporated WebBoard for asynchronous conferences and Chat for
synchronous conferences. Three small group, peer moderated, asynchronous
conferences consisting of no fewer than three students were established by
week three of the course. Peer moderators evolved as leaders do in any group.
Specific rules were set up for moderator duties. A plenary, faculty moderated,
asynchronous conference was established in week one. This conference was the
constant for all members, a place where all small conference teams reported in
summary fashion, on their discussions, every three weeks.
There were five synchronous chats. Three chats with experts in a
variety of informatics areas (library database and search engine use by two
librarians, clinical information systems by a nurse computer expert, OASIS by
the state manager and the state nurse consultant to all home care sites
implementing it). The other two chats where with the faculty and topics for
these where established in the course calendar. One of
these chats was a discussion of a case study, posted two weeks before the
chat, regarding the implementation of a health information system in a clinic
assignments consisted of reading chapters in the required text. Chapters were not assigned but selected by student as
appropriate to the content listed on the calendar for that week. Answering
questions, in the class conference, on the first three chapters of the text
reading, was assigned during the first three weeks to get students used to the
chat space as where introductions of each other to the class. Students were
required to take part in the class and small conferences at least once a week,
posting meaningful content.
of a Web Site Evaluation for any site used in any part of the course
(referring class to useful site, used as reference in a project or paper)
using a Web Site Evaluation Tool posted on the class web page.
evaluations of health information systems, visited on field trips to local
facilities, were completed, by students in pairs, using a tool posted on the
class web site. The first was a report of the first visit and the second
incorporated the first visits evaluation with the second including a
discussion comparing the system capabilities at the two sites. The sites where
a large veterans administration hospital and a private, acute care facility.
The other two field trips, one to visit a classroom using A.D.A.M. and other
computer based software for the teaching and learning of anatomy and
physiology and the second to the Advanced Visualization Laboratory (AVL). The
AVL which is part of a degree program on the ETSU campus is one of only nine
in the world doing virtual three dimensional work. The student could chose not
to attend one of these last two field trips. No other assignment were
associated with these last two field trips. Student only had to post follow up
discussion on WebBoard regarding their value to the achievement of course
final assignment was a five page, health
informatics ethical/social issue paper, submitted
using creative electronic means (most used Microsoft Power Point), to every
student in the course. Each student was required to read, comment on, post
relevant questions and responses in discussion on the class conference, and
lastly post a grade on each paper to the faculty. The ethical/social issues
paper must have been publication ready using
guidelines for publication from the On-line Journal of Nursing Informatics All
papers/products could be submitted to faculty via attachment in email, in
person, or Faxed and required use of the appropriate publications format for
the students discipline (APA format for nursing).
were provided access to a help system via the university distance education
department and in addition to open access to faculty via e-mail, phone, fax
and/or office hours for which a 48 hour turn around time was established. Each
signed a university wide permission form to participate in chat.
large reference list was placed on the course web site including
web addresses (URLs), print journal articles, books, personal email addresses
for outside resources, and other documents. Both hypertext and embedded links
were placed throughout the reference list for ease of use.
was conducted at each face to face field trip and using an evaluation tool
from the course web space during week six of the course. Evaluation was also
conducted on the last week of the course via the same tool and via exit
interviews and students’ writing letters to students needing to take this
course next semester telling them how it went and how to get ready. Paper and projects where graded using the preset guidelines and
class participation was evaluated by monitoring the value of comments placed
in class conference, chats with experts, small group conferences and the
statistics on the number of times a student was on the chat space.
Web Site consisted of a welcome page with the faculty email
address, dates of class, number and time of face-to-face meetings and sites
for field trips, links to a pre-course assessment of readiness to take an
online course and a link to an article about computer rage. Class conference
was explained as was required participation in WebBoard.
welcome page was a modified listed version of the course described above and
provided a hyperlink to a course registration site, a link to acquire a
password to enter the course web site itself, and an access point to WebBoard.
WebBoard was set up by faculty, after students registered for the course. Web
Board sends out an automatic Welcome message with the individual, faculty
assigned password for access to chat and conferences. WebBoard has a large
help component. During the first three weeks the assignments on WebBoard where
designed to increase comfort with use and to begin to build a community of
online learners for this course, such as introductions and required reading of
the text and responding to specific questions on the reading. Other course requirements such as email addresses, types of
computer hardware and software needs were spelled out.
I will proceed to describe in seven categories: a) institutional, b) course development, c) teaching/learning, d) course structure, e) student support, f) faculty support, and g) evaluation and assessment if and how each of the aforementioned 24 benchmarks under those categories were achieved during the offering of this course.
Institutional Support Benchmarks
to the benchmarks study there are three components to examine under
institutional support when evaluation is implemented. The questions that must
be answered yes t establish high quality in this area are: 1) Is a documented
technology plan including electronic security measures such as
password protection, encryption, and back-up system in place and
operational and does it insure quality, integrity, and validity of
information, 2) Is the reliability of the technology delivery system as
failsafe as possible, and 3) is there a centralized system which provided
support for building and maintaining the distance education infrastructure?
technology plan. As can be
seen by the course description both the faculty and students had log in and
passwords assigned which were private to them. The password assigned to
students for use in WebBoard could be changed by the student if desired.
Participation on the chat space required signatures with an explanation of the
public nature of any chat space so students would be aware others would be
seeing all posts made to the site.
one point during the course a student from another technology intensive course
was visiting and participating in this class (because of many factors she was
unable to participate in the course chat for her course enough times). The
students in this course agreed she could participate in our discussions. They
gave permission for this stranger to take part. Faculty entered this student
and gave her a password and log in to chat space only.
measures documented and in place.
After about the third week of the course the course administrator in the
distance education department notified the faculty that several students had
dropped the course and that the password for entrance into the course web site
should be changed. The course faculty chose a new password and sent an email
to all students, those who dropped and those remaining, to announce that the
new password would go into effect in one week and why. This email also
explained that students no longer in the course would also be dropped from
WebBoard and it requested that they complete one of the posted evaluation
tools and send me an email explaining how I could have helped them to stay in
the course. A second email was sent to only those remaining in the course,
with the new password and log in and the exact date for the change. At this
time the dropped students where also deleted from the WebBoard conferences and
students in small conferences were notified of missing members names.
reliability. Whenever the
entire system was to be down for any reason the faculty was notified by the
distance education administrator. The faculty passed this information on to
students via email. This administrator was available on what seemed like a 24
hour a day basis for answering any faculty questions and had a posted help
site for students to which the course had an embedded link. On one occasion,
one student had used the administrator help site over five times in two days
and the administrator made the faculty aware of this. This allowed the faculty
to intervene and counsel the student along with the help person.
At one time during the course the administrator thought students who were not officially in the course had entered and completed mid-course evaluations. Again this allowed the faculty to investigate. It was found that these were the students who had dropped the course complying with a request to complete this tool, made in the final email to them.
one time during the course a student not in this course but in the
Introduction to Professional Nursing course, and completing an informatics
assignment on the topic of computer use in education, chose to request to
enter the health informatics web site. He was given the password to the site
and all students in the course where informed that this was happening. He was
not given access to WebBoard so no breach of confidentiality occurred here.
Course Development Benchmarks
benchmarks in this area require the evaluator to ask four questions: 1) Are
there guidelines for minimum standards for course development, design, and
delivery, 2) Do learning outcomes determine the technology to be used to
deliver content rather than the availability of existing technology, 3) Are
instructional materials reviewed, 4) Is the course designed to require
students to engage themselves in analysis, synthesis, and evaluation?
development, design, and delivery.
The guidelines that exist for the development of any course in the College of
Nursing were followed in the development, design, and delivery of this course.
It has a structured syllabus with all outcomes, teaching/learning methods and
evaluation methods spelled out. It includes guidelines for projects with
outcomes determining technology use.
There was much more technology available to be used in this course, as in
other software such as Real Audio, Real Video, and Acrobat Reader, which could
have been used. Students were provided with links to these packages which
could be downloaded free from online sites but they were not required because
they were not needed to achieve learning outcomes. A minimum of computer
technology was used: email, Internet, synchronous and asynchronous chat, fax,
and phone. Students needed to know how to send attachments via email, how to
search the web using various browsers and search engines, how to post to a
conference or chat space, and how to access on line or in-library data bases
to accomplish outcomes.
Instructional materials were submitted to peers on other campuses, who are
teaching a similar course. Feedback from them was excellent. Peers teaching
other online courses at ETSU reviewed the course web pages and the strategies
for learning. The distance education administrator reviewed the public page
and made suggestions for improvement which were followed. The department chair
of the PMNU department, the Associate Dean for Undergraduate and Graduate
Programs, the Dean of the College of Nursing, and several faculty where sent
the course web site and a password to review the pages prior to the course
offering. The PMNU Chair gave feed back which was used. The National League
for Nursing was making an accreditation visit to the college during the time
this course was being offered. It in no way hindered that review.
engagement in analysis, synthesis, and evaluation.
Students, as can be seen in the previously described assignments, were
consistently engaged in analysis of text material and systems and readings;
they conducted multiple syntheses of data collected from the field, and
conducted evaluations of web sites, papers submitted by other students, self
evaluation to assess outcome achievement, and course and faculty evaluation.
Early in the course they complained that one other assignment (the preparation
of several annotated bibliographies) would make the course a burden and
demonstrated how other assignments incorporated the same tasks and outcomes.
This assignment was dropped from the course and students determined how they
wanted the grade points reassigned.
Teaching and Learning Benchmarks
questions asked of a course, faculty, and students here are: 1) is student
interaction with faculty and other students seen as an essential
characteristic and is it facilitated in various ways, 2) Is feedback on
assignments and questions constructive and provided in a timely manner, and 3)
are students instructed in the proper methods of effective research, including
assessment of the validity of resources.
Student to student, faculty to student, and student to outside expert
interactions were essential to meeting course outcomes. Interaction was
facilitated by face-to-face, email, fax, office hours, online scheduled
synchronous chat and unscheduled online conferences.
and timely feedback.
Feedback on assignments was extensive, given by faculty and peers, revisions
were allowed and encouraged. Feedback via email was guaranteed within 48 hours
but was usually within minutes. The faculty member was on vacation during
spring break but continued to respond to students who were participating in
chat or with her via email.
Effective research methods and assessment of resources. Students were instructed in ways to collect, sort, and cluster data from a variety of sources to construct various types of reports and projects. They were instructed in synchronous chat with two librarians (one education librarian and one medical librarian) on how to use online data bases and search engines to conduct effective searches. They were instructed on how to use tools to assess data acquired from Web sites and documents using a tool designed for the course. They were instructed how to collect verbal data from health systems administrators, who had previously provided them with hands on experience with health information systems, to use in various reports. They were encouraged to check the validity of resources by calling or e-mailing experts such as members of the Nursing Informatics News group, the persons who gave them health systems information during field trips, and to use on line dictionaries and their text book to define computer terminology they were not familiar with. Every project required use of the skills of data collection, data analysis, synthesis, evaluation and revision.
Course Structure Benchmarks
questions an evaluator must ask in this area are: 1) Do students pre-assess
to see if they are ready and possess
the self motivation and commitment to learn at a distance, 2) do they have
access to the technology required by the course design, 3) Are they provided
with supplemental information and learning outcomes for the entire course and
each course assignment, 4) Is there access to sufficient library resources
including virtual library resources, 5) have faculty and students agreed to time lines for assignment due
dates and faculty responses?.
for distance learning. A
readiness scale which helps students assess their readiness to take part in an
online course appears on the welcome page for the course. It is a tool used by
many colleges for the same purpose. Students are asked to respond to the tool,
on-line and email the results to the faculty member before registering for the
course. Students who achieve less than a score of 20 on the instrument are
reminded that they may want to reconsider signing up for an online course at
this time. All of the students did not take the tool so much of this data is
missing. It is clear from the attrition rate and the comments from those who
dropped the course that many where not ready (this information will be
elaborated on under the last benchmark section, evaluation). Students were
also encouraged too hyperlink to and read an article on Computer Rage from the
USA Today web site. No feedback is available to assess whether any read the
A meeting with the advisor with many of the RN/BSN students who enrolled in
but did not complete the course, later validated that they had not taken even
the basic computer course required of all ETSU graduates.
A meeting with the advisor with many of the RN/BSN students who enrolled in but did not complete the course, later validated that they had not taken even the basic computer course required of all ETSU graduates.
The course design required minimal technology. It was assumed that
students wishing to take the course would read the welcome page which had
specific technology requirements listed. Any student registered at ETSU has an
e-mail address and access to computers on any of its campuses. However, access was not an issue but where the computer being accessed
for this course was located was an issue. For example, it was discovered in
week three that several students did not have computers at home. These
students seemed to think they could take this entire course from work
computers. They soon discovered that firewalls on work computers would not let
them use synchronous chat or email for course work. Many also discovered they
could not get into the course from work computers because it required a
password. Many realized they would not have the time to participate via work
computers only, at the level required. Several students had never owned nor
used a computer. They purchased them during the first week of the course and
so, even though they had access, had severely miscalculated the steep learning
curve. The students with all of
the required hardware and skill using word processing, presentation software
as well as e-mail and Internet access had little difficulty (these issues will
be addressed in more detail under assessment benchmarks).
course information. All
course requirements were spelled out in detail in an online syllabus,
calendar, assignment guidelines, and reference list. WebBoard was explained
and practice sessions were set up in the first three weeks. On the first class
night the course met face-to-face and at that time some articles were distributed, several
free journals were recommended as were ways to acquire them made available.
Library resource sufficiency. Students had email access to the Sherrod Library Education Librarian and the online library services with access to multiple data bases. They were instructed as to how to use data bases and search engines by the education librarian and a medical librarian in a ‘chat with the experts’ session. Pubmed was available to them in the virtual world as well. Many resources were emailed as links by faculty. All students had easy access to interlibrary loan from the medical library as well as Sherrod and many local hospital and town libraries. One student commented on the overwhelming amount of available resources.
on completion of assignment times and faculty responses.
It was agreed early in the course that due dates were to be hard and fast but
that extensions would be given where needed. The field trips were not
scheduled until the first night of class in order to get agreement from all
students as to the best time. These were scheduled during the pre-set class
time of Tuesdays from 6-9 so that a regular schedule could be adhered to. All
students were working full time. One field trip was optional and the student
could choose, from two, which one they could not attend. It was agreed that
faculty would respond to email no later than 48 hours. Faculty office hours
appeared on the class calendar along with email, phone, and voice mail
information. It was agreed that a first draft of any project would gain high
critique with in one week and there would be one opportunity for revising.
Revisions would be resubmitted within one week and receive a final grade
within one week of re-submission.
course motto which was stated early on the calendar was, “Blessed are the
flexible for they shall not be bent out of shape.” This motto was used often
as in when asking for extensions, when asking for one project to be taken out
of the course, when asking for more time for anything.
Student Support Benchmarks
support is a broad area and requires the evaluator to ask: 1) do students
receive information including admission, tuition and fees, books and supplies,
technical and proctoring advice, and student support services, 2) do students
get hands-on training and information to aid them in securing material through
electronic databases, interlibrary loans, government archives, news services,
and other sources, 3) do students have access to technical assistance,
detailed instruction about the electronic media used, practice sessions prior
to beginning of the course and convenient access to technical support staff,
4) are questions directed to students service personnel answered accurately
and quickly and are student complaints handled in a structured way?
Students receive information. There is a catalogue published, there is a home page for the distance education learner and there is a course welcome web site. Each of these sources describes and provides access to information or direct links to access for admissions, the book store or online text book sellers. Student support services are in place as for any campus student.
training and information.
Librarians conducted a two hour ‘chat with the experts’ session to assist
students with all issues regarding access and use of library and on line
services, data bases and other sources. The faculty provided access to free
related journals. Students were asked to join the nursing informatics listserv
and were given directions as to how to accomplish this. Students at ETSU are
all required to take a computer hands on course during the lower division
component of the curriculum. Practice sessions were held in chat spaces during
the first three weeks of class by having students introduce themselves and
respond to questions, posted by the faculty, from the first three chapters of
the text book. No other practice sessions were conducted. This will be
commented on again in evaluating segment of paper.
to technical assistance.
Technical assistance was available from the distance education department via
a hot link for questions. These were responded to with in a few hours. Faculty
provided technical assistance as well which speaks to the need for faculty to
be knowledgeable not only about content of the course but also about the
courseware they are using.
services. In the ETSU
College of Nursing there is a student services department. The staff in this
department recruited the majority of the students who signed up for this
course. They provide ongoing academic counseling for all students in the
nursing program regardless of what type of course they are taking. Students
from other majors also have university student services departments to get
assistance from. Each of the student services groups and the college of
nursing has formalized ways to address student complaints no matter what
courses they are in or what method those courses are offered in.
Faculty Support Benchmarks
It is in this area that one must be pro-active as well as with students. If faculty do not have the support needed they cannot do justice to any curriculum in any format. In this area it is suggested that the following questions be asked in evaluation: 1) do faculty have technical assistance in course development and are they encouraged to use it, 2) are faculty assisted in making a transition from classroom teaching to online instruction and are they assessed during the process, 3) is there training and assistance including peer mentoring continuous and ongoing throughout the course, 4) are faculty provided with written resources to deal with issues arising from student use of electronically-accessed data?
assistance in course development.
The only technical assistance this faculty member required was how to use
Front Page to design the course Welcome and Web site, how to set up, manage,
and use WebBoard for synchronous and asynchronous chat. This was provided by
request of the faculty member to the distance education department via three
private tutorial sessions during the summer, on the faculty’s own time. The
department provided support by upgrading the faculty’s office computer and
providing Front Page 2000 for installation at home and on campus for ease of
managing the course. Access to the web on campus was by high speed ISDN line
and at home the faculty changed from a slow 56K modem, which tied up the phone
lines to a more expensive, faster, cable modem with no financial support, even
after a formal request. There was little encouragement to use technical
support. The faculty member often became the technical support for others
attempting to use one or two online techniques in their courses.
from classroom to online.
Little help was needed in this area. This course had been taught in
traditional classrooms three times previous to the online version. The course
remained virtually the same only the space and distance changed.
Training, peer mentoring continuous. There would have been plenty of help had I needed it. As a matter of fact a year prior to offering the class online I attend a formal session at ETSU on another online course system which was helpful. I also read volumes about online teaching and learning in various print and online journals and consulted with peers on other campuses who are teaching similar courses. I also feel this will be one of the hardest areas for many faculty. It bears mentioning that this course faculty member attended an on campus faculty development session provided by two peers who had just completed an online course. This session provided some clues to the need for clearly stated project guidelines and descriptions of how to conduct the course. Technical support also came from a faculty list set up by the Dean of Distance Learning and a list set up by the nine peer faculty from other colleges, currently teaching online courses at ETSU.
resources regarding student use of electronically-accessed data issues. I was never provided with anything in this category. I did however use
the web to find out this information on my own and so perhaps did not seek it
on campus. That does not mean it does not exist. A policy was developed for
the university, during this semester and that is now in writing and will be
very helpful in the future regarding what constitutes a totally online course
versus a partially on line course and other policy areas. However I still see
nothing on the horizon on this area regarding issues related to student use of
electronically-accessed data. At
this time I cannot say I am certain what this benchmark means. I will need to
conduct a literature search.
Evaluation and Assessment Benchmarks
is here that one sees that the benchmarks are for programs and not one course
but I feel I can still demonstrate my success or lack of success in one course
using the following questions from this area: 1) is the educational
effectiveness and teaching/learning process assessed through an evaluation
process that uses several methods and applies specific standards, 2) are data
on enrollment, costs, and successful/innovative uses of technology used to
evaluate program effectiveness, and 3) are intended learning outcomes reviewed
to ensure clarity, utility, and appropriateness?
An evaluation process using several methods and applying specific standards. In this course several methods were used to evaluate the teaching/learning process and its educational effectiveness. There were two evaluation tools made available on line. Students were asked to complete the tool at mid-course and changes were made based on this evaluation. They were asked to evaluate if they dropped the course. These results will be used to revise the course for future offerings, where appropriate. Students were asked to write a letter to a friend they new would be taking the course next semester to tell them how to get ready for it. Standards established as guidelines for projects were matched with student accomplishment of the projects and students in exit interviews were asked if all projects helped them to achieve course outcomes.
interviews also were conducted on the last day of class to ask for feedback on
ways to change the course for the next offering. Suggestions were do not put
the Annotated Bibliography assignment back in, continue to require the final
paper in publishable form but do not require submission of it as part of the
course, during the first three weeks make it clear that the questions will be
put in chat to assist us to learn how to use the system, cut down the
reference list, especially web sites they are overwhelming, plan synchronous
chats more thoroughly with experts, keep all field trips, keep the text book
and continue to not have assigned readings just keep using the discussion
questions, discontinue small conferences if less than two people remain in
them, create a small conference or space for current links, keep posting
calendar changes in a different color and keep sending email when changes or
additions are posted. Students felt the chats went well and that the class
conference was a valuable and interesting space that took much more time than
they had allotted for in their schedules. This time factor was true in
general. Most commented the course required more than they expected. Some were
taking another online course in the science department and felt the two
together were not possible to take because of the time. Not that the time was
greater than other class time but that it seemed to be when spread out rather
than being in two or three hour time blocks in the week. They felt like the
work was never done because they had to check the site and the class
conference on WebBoard everyday in order not to miss anything. They felt they
established a unique class, made friends, had good faculty support. They were
concerned about all of the drop outs but seemed to understand why they
occurred. Many admitted they were taking this course as an elective and so
dropped it when they found it requiring more time than they expected it would.
Exit interviews were also conducted with the five online experts who took part in synchronous chats. Their feedback was that they felt they went well, they wished they had done as I encouraged them which was to post an outline to me ahead of time along with a web site or two for students to visit prior to the chat. The students felt these went well but would have liked faster bandwidth to increase the speed so the time lag between responses did not get them off track. They also wanted a bit more direction from the experts. They felt the best asynchronous chats were with the course faculty but even then they go kicked offline by their service provider several times during one hour chats.
on enrollment, costs and innovation.
I really have no data on cost but have been asked to give report to the Dean of Distance Education for his use to evaluate these aspects. We
actually enrolled 12 paid students who started the course in the first week,
six (6) more late enrolled and started the course in the second week for a
total of 18. Of those 18, 9 were undergraduate RN/BSN students taking the
course as an elective, two (2) were non-health related majors and seven (7)
were nursing masters students taking this as a required course. Six students
completed the course: two undergraduate RN/BSN, three graduate students and
one undeclared non-health related student. There are many recommendations in
the literature about the need for small size classes in online environments.
Students commented at the end of the course that they do not see how they
would have kept up with the discussion on the WebBoard in class conference if
everyone who signed up had remained in the course. Students dropped for
illness in the family, conflicts with other courses, because it was a
non-required elective and as such they thought it was too time consuming. Two
dropped because of lack of knowledge about use or lack of ownership of a
computer. The students who enrolled late all dropped. Two stated they could
never get caught up. I feel the course was innovative but that no high level
or high speed computer was needed to complete it successfully.
review of learning outcomes.
Like most courses there were times when students did not understand guidelines
for projects and so became frustrated trying to successfully achieve outcomes.
It is worth noting here that to nursing students only a grade of A+++ seems to
mean successful achievement. The most common example was in the health
information system evaluation comparison project. Most felt that it could be
done in pairs even after the first assessment was conducted. This was never
said and was not true but I had to allow two pairs to complete the entire
project this way because of the lack of clarity or understanding.
did not know the difference between a web page, web document, or web site.
This made it impossible for them to accurately cite sources, in text and on
reference lists, using APA or any other paper format. There was much
frustration expressed about APA.
Students did not want to conduct complete web site assessments until later in the course when they began to see the value as they found some very unreliable patient education sites. Learning outcomes will be reviewed.
course met most if not all 24 benchmarks successfully. All students who stuck
it out stated we need to offer more online courses. This appears to be a
testament to establishing a cadre of students familiar with this learning
environment who are or can becomes self motivated learners and can manage time
in fundamental ways. The faculty was intimately knowledgeable about the
software and hardware and its use in the course which probably was responsible
for keeping down student complaints about the web site, software, and hardware
requirements. Faculty was available in multiple ways as in office hours,
email, fax, synchronous chat, asynchronous class conference, and telephone.
Students took advantage of each of these as they felt they needed them.
were available to faculty and each other in most of the same ways. They used
each other frequently for various consultations about this and other courses
they were taking at the same time. They gave support to each other and to the
faculty via many virtual mechanisms. The innovative method of ‘chat with the
expert’ was only partially successful. In the future, use of the same
experts will help. The experts now know how important it is to post an outline
ahead of the chat and can anticipate some of the problems inherent in this
format. Most common problem was response time delays making the chat a bit
disjointed at times.
The other innovative method of field trips, with some being not required, was very successful. The field trips took us to exciting implementation sites where students could see informatics in action. The pharmacy robot, the VA medication administration using scanners, ADAM as a learning tool, and the Advanced Visualization lab have been much discussed. A video tape about the future impact of technology on health care and one on a telehealth session in progress were circulated and viewed be each student and discussed on WebBoard. This learning tool gained high praise.
in a list. Problems are
broken down into I. technological, II. computer literacy, and III. pedagogical
teacher and student. Section IV is a list of problem prevention strategies
used by faculty but not by students. Section V lists strategies used during
the course to try to reduce student time. Section VI is a list of the results
from the exit interviews with students on the last night of the course.
Section VII are the changes planned for the next course offering.
The most common technological problems were:
getting kicked off chat by service provider during one hour chat sessions.
changes of email addresses for both students and faculty.
3. computer down time (although this was minimal and on weekends).
II. The most common computer literacy problems were:
students never participated in chat space before so many students did not
enter the class conference space until week three finding themselves very far
behind. Even when given time to catch up, they did or could not.
2. reading online is hard to do if large amount of material. Students did not begin to print until late in course
no computer at home
4. new computer at home with no knowledge of how to use it
did not know how to send attachments via email
6. no knowledge of how to cut and paste into email or WebBoard
from another software program.
no knowledge of how to create hot links in email or WebBoard
no knowledge of how to format printer to print in landscape or to create
charts in word processing package.
9. no knowledge of how to download and install free software from the web (although no downloads were required to succeed in the course options to do this were offered).
most common pedagogical issues for faculty and students
too many assignments
first two weeks in class conference seemed to have no purpose.
scheduling of a class day and time caused conflict. Faculty expected students
to set aside that time for face-to-face options and students expected to have
that time free if they wanted to.
you never know if the excuses are real such as my computer was down, my email
crashed, I have to work so I cannot do that live chat or field trip. Students
did not seem to see this as cutting class as they would have in a regular
students lacked awareness of time and organization skills needed to take
course on-line and to do off-line reading
lack of knowledge of use of APA or any other standard for formatting papers.
7. students expressed frustration with lack of ability and skill to learn on their own.
IV Prevention strategies that were not used by students:
pre-course survey regarding readiness to take distance course via online.
article about computer rage.
web site hyperlink to a free, short course on how to use the Web including
help mechanism in WebBoard
5. Link to APA Web site to download APA Helper 2.0 free.
Strategies used during the course to reduce time use, based on student
Cut one assignment,.the annotated bibliography
allowed students to work in pairs on Health Information Site Visit report and
3. accepted issues paper ready to be submitted for publication with minor revisions
Exit interview with students: results
break up course calendar into more pages.
tell students early to print and read
keep conferences and chats
give a bit more instruction on chat space in a synchronous environment
keep chat with experts
reduce the number of required website evaluations or shorten the format
keep required sign on to informatics listserv very helpful for external
all nursing students need this course and need to take it on online for
need to learn content and method
offer more on-line courses
in first face-to- face class put all students on web or computer and go over
course online together as in logging on, passwords etc.
embedded links at top of calendar to each week were great
reference list was great, especially links to web sites
field trips excellent
allowing pairs for one project great
17. requirement for graduate students to submit issues paper for publication excellent but cannot be accomplished until course is over.
Changes projected for next course offering. Based on student, peer,
expert, and self feedback I plan to make the following changes in this course
for the next offering.
1. take a careful look at all project guidelines
keep the annotated bibliography requirement out of the course
try to find out sooner who does not have a computer at home and keep
them from taking the course (for this I will meet with student services person
who recruited the RN students without computers for the course)
switch the course from Front Page to Blackboard. Spread the Calendar
over more pages instead of just using embedded links to sections
5. require everyone to take the pre-course readiness for online learning tool
6. not allow late registration without meeting with the students in person
try to keep the same experts
try to keep the same field trips
make known early to print for reading
make known early to reserve class night and time until after first night of
class is over and schedule for field trips and chats are set to reduce
conflicts with student work schedules
make known early that skipping a required chat or other session is equivalent
to skipping a class
make known the first three weeks in chat will be to introduce selves, to
answer specific questions about assigned readings, to help with using chat, and
understanding computers as tools
Attachments that can be or are made available:
Institute for Higher Education Policy (April, 2000). Quality on the Line: Benchmarks for success in Internet-based distance education. Executive summary p. 1-5. [Available: the Institute for Higher Education Policy , 1320 19th St., NW, suite 400, Washington, DC 20036. Internet: http://www.ihep.com/qualityonline.pdf].