Evaluation of Wells Center High Fidelity Mannequin Simulation

Evaluation of Wells Center
High Fidelity Mannequin
Simulation Trainings
Prepared by Nancy Koester for Colorado Department of Labor and
Employment and the Wells Center
August 31, 2009
This project is supported by the Colorado Trust, Colorado Department of Labor &Employment, Colorado
Workforce Development Council, and WELLS Center partners.
High Fidelity Mannequin Simulation Training Evaluation
Executive Summary
The Work, Education and Lifelong Learning Simulation (WELLS) Center offers state-of-the-art
life like simulation training for building medical clinical knowledge. The WELLS Center received
funds from the Colorado Trust to create and deliver simulation training to increase patient
safety in accordance with the goals of the Institute for Healthcare Improvement’s (IHI) 5 Million
Lives Campaign. The 5 Million Lives Campaign focuses on increasing patient safety through 12
interventions identified in the campaign. The Wells Center received a grant from the Colorado
Trust to create and deliver training for hospitals in meeting four goals of this campaign. Any of
the hospitals in the 5 Million Lives Campaign were eligible to receive this WELLS Center training.
The Colorado Trust award created the opportunity for the Wells Center to provide training to
Colorado hospitals. Hospitals that participate in the Wells Center Training receive customized
development of the training scenario and facilitator training by the WELLS Center. The training
was designed to be completed by nurses, student nurses, emergency medical technicians,
pharmacists, physicians and other hospital personnel who provide direct patient care. There are
three parts to the training: (1) an introduction and orientation about the training and the
mannequin, (2) the simulation, and (3) a debrief session to discuss the simulation with a trained
facilitator.
The WELLS Center training is unique because it includes the use of a high fidelity mannequin in
real time simulation training. The high fidelity mannequin is a lifelike mannequin that can be
programmed to replicate almost any type of medical scenario. All the mannequins breathe
spontaneously, have palpable pulses, heart and lung sounds, and respond to stimuli. WELLS
Center staff facilitated the simulation and controlled the mannequin. Hospital staff facilitated
the debriefing session.
The scenarios created by the Wells Center and participating hospitals were Preventing the spread
of MRSA Infection; Assessment and Prevention of Pressure Ulcers; Rapid Response Team in an
Isolation Room; and Over Sedation Post Colonoscopy.
The evaluation was designed to assess the efficacy of the high fidelity mannequin simulation for
increasing knowledge, skills and abilities to improve patient safety in hospitals.
Findings
Knowledge Increase
Participants indicated that there was an increase in the level of knowledge about how to
improve behavior to increase patient safety.
2
 Respondents indicated that knowledge increased about how and when to activate
the rapid response team.
 Respondents indicated that the simulation was helpful for learning the protocol for
preventing pressure ulcers.
 Respondents indicated that confidence increased about what to do in the event of
narcotic overdose.
 Respondents indicated that confidence increased about preventing over sedation.
 Respondents indicated that the simulation was helpful for learning about high alert
medications.
Job Performance
Respondents indicated there is an improvement in their ability to perform their job.
 Respondents indicated that the training was helpful for learning about a patient’s
risk for pressure ulcers
 Confidence increased about how to conduct a skin assessment and assess a patient’s
risk for pressure ulcers.
 Respondents indicated that the training was helpful for increasing their skills and
abilities for what to do in the event of narcotic overdose.
 The simulation was helpful for increasing my skills in dealing with over sedation.
Training Effectiveness
Respondents indicated that the simulation training was a valuable learning experience.
 Respondents indicated that the high fidelity mannequin simulation offered a
valuable way to learn the information.
 Respondents indicated that the debriefing session was helpful.
 Respondents indicated that they will use what they learned in the training on the
job.
 Respondents indicated that the high fidelity mannequin added to the learning
experience.
Overall, participants in the Wells Center high fidelity mannequin simulation training felt that the
training increased their knowledge, skills and abilities for doing their job. Further research is
necessary to learn more about whether a simulation training like this increases job satisfaction
or reduces staff turnover.
3
Table of Contents
Executive Summary ......................................................................................... 2
Findings ................................................................................................................................. 2
Description of the Project ................................................................................... 7
Need for the Evaluation ...................................................................................... 9
High Fidelity Mannequin Simulation Training ...................................................................... 10
Job Satisfaction vs. Burnout .................................................................................................. 10
Methodology .................................................................................................... 11
Results .............................................................................................................. 12
Survey Respondents ......................................................................................................... 12
Gender ................................................................................................................................... 12
Type of Job ............................................................................................................................ 13
Highest Level of Education .................................................................................................... 14
Previous Training with High Fidelity Mannequin .................................................................. 14
Age, Licensure and Number of Years on Job ......................................................................... 15
Training Evaluation Results .............................................................................................. 15
Perceived Improvement in Knowledge.................................................................................. 16
Perceived Improvement in Ability to Perform Job................................................................. 19
Effect of Training on Job Satisfaction.................................................................................... 23
Training Effectiveness ........................................................................................................... 24
Summary and conclusions................................................................................. 26
References ........................................................................................................ 29
Appendix a: Data Tables ................................................................................... 31
Appendix B: Comments and Feedback .............................................................. 41
4
Table of Tables
Table 1: Objectives of Training Scenarios ....................................................................................... 8
Table 2: Number and Percent of Pre and Post Survey Respondents by Hospital and Scenario .. 31
Table 3: Gender............................................................................................................................. 32
Table 4: Type of Job ...................................................................................................................... 33
Table 5: Highest Level of Education .............................................................................................. 34
Table 6: Previously Participated in High Fidelity Simulation Training? ........................................ 35
Table 7: Perceived Improvement in Knowledge ........................................................................... 36
Table 8: After Training Perceptions of Improvement in Knowledge ............................................ 37
Table 9: Perceived Improvement in Job Performance ................................................................. 38
Table 10: Effect of training on job performance .......................................................................... 38
Table 11: Reduce Spread of Infection ........................................................................................... 39
Table 12: Job Satisfaction Before and After Training ................................................................... 39
Table 13: Job Satisfaction Ratings Before Training ....................................................................... 39
Table 14: Job Satisfaction Ratings After Training ......................................................................... 40
Table 15: Training Effectiveness .................................................................................................. 40
Table 16: Comfort Level with the High Fidelity Mannequin ......................................................... 40
5
Table of Figures
Figure 1: Proportion of Respondents by Training Topic ............................................................... 12
Figure 2: Gender of Respondents by Training Topic..................................................................... 13
Figure 3: Respondent’s Job ........................................................................................................... 13
Figure 4: Respondent’s Level of Education ................................................................................... 14
Figure 5: Previously Participated in High Fidelity Simulation Training ......................................... 15
Figure 6: RRT Perceived Improvement in Knowledge .................................................................. 17
Figure 7: Over Sedation Perceived Improvement in Knowledge.................................................. 17
Figure 8: Pressure Ulcer Perceived Increase in Knowledge .......................................................... 18
Figure 9: Post Simulation Ratings of Knowledge Improvement ................................................... 19
Figure 10: Perceived Improvement in Job Performance .............................................................. 20
Figure 11: Ratings of Effect of RRT Training on Job Performance ................................................ 21
Figure 12: Perceived Effect of Training on Job Performance ....................................................... 22
Figure 13: Reducing the Spread of MRSA Infection ...................................................................... 22
Figure 14: Enjoy Job ...................................................................................................................... 23
Figure 15: Training Effect on Job Satisfaction ............................................................................... 24
Figure 16: Training Effectiveness ................................................................................................. 25
Figure 17: Comfort Level Using High Fidelity Mannequin ............................................................ 25
6
High Fidelity Mannequin Simulation Training Evaluation
DESCRIPTION OF THE PROJECT
The Work, Education and Lifelong Learning Simulation (WELLS) Center offers state-of-the-art
lifelike simulation training for building medical clinical knowledge. The WELLS Center received
funds from the Colorado Trust to create and deliver simulation training to increase patient
safety in accordance with the goals of the Institute for Healthcare Improvement’s (IHI) 5 Million
Lives Campaign. The 5 Million Lives Campaign focuses on increasing patient safety through 12
interventions identified in the campaign. In Colorado, the Colorado Trust committed $3.9
million to assist hospitals in implementing any of the campaign interventions. The Wells Center
received a grant from the Colorado Trust to create and deliver training for hospitals in meeting
four goals of this campaign. Any of the hospitals in the 5 Million Lives Campaign were eligible
to receive this WELLS Center training. The WELLS Center, in collaboration with hospitals,
identified the following four campaign interventions, recommended by the IHI, for the
simulation trainings.
Deploy Rapid Response Teams at the first sign of patient decline
Prevent Harm from High-Alert Medications starting with a focus on anticoagulants,
sedatives, narcotics, and insulin
Prevent Pressure Ulcers by reliably using science-based guidelines for their prevention
Reduce Methicillin-Resistant Staphylococcus aureus (MRSA) infection by reliably
implementing scientifically proven infection control practices
The Colorado Trust award created the opportunity for the Wells Center to provide 30 days of
training to Colorado hospitals. Hospitals that participate in the Wells Center Training receive
customized development of the training scenario and facilitator training by the WELLS Center.
The training was designed to be completed by nurses, student nurses, emergency medical
technicians, pharmacists, physicians and other hospital personnel who provide direct patient
care. The grant covers the Wells Center costs for conducting the training. The only cost incurred
by participating hospitals was staff time to help customize the scenarios, attend facilitator
training and participate in the training. The Wells Center provided the facilitator training to
hospital staff so that those staff could facilitate a debriefing after the training. In many
7
hospitals, individuals who attended the Wells Center facilitator training subsequently trained
other hospital staff to assist with facilitating the simulation debriefing sessions.
The WELLS Center training is unique because it includes the use of a high fidelity mannequin in
real time simulation training. The high fidelity mannequin is a lifelike mannequin that can be
programmed to replicate almost any type of medical scenario. All the mannequins breathe
spontaneously, have palpable pulses, produce heart and lung sounds, and respond to stimuli.
The trainings were held at the hospitals, usually over a one or two day period. The WELLS
Center worked with each hospital to customize the scenario and provided training to hospital
staff on how to conduct a debriefing session after each simulation. There are three parts to the
training: (1) an introduction and orientation about the training and the mannequin, (2) the
simulation, and (3) a debriefing session to discuss the simulation with a trained facilitator. The
number of simulations per day varied, as did the length of time. The simulations lasted between
20 and 40 minutes with a 10 to 20 minute debriefing session after each simulation. The goal
was to have about 6 participants in each simulation session. WELLS Center staff facilitated the
simulation and controlled the mannequin. Hospital staff facilitated the debriefing session.
For many of the WELLS Center participants, the training ends after the debriefing session. This
one-time through model takes approximately 20 minutes. This model allows hospitals to train a
higher number of staff in a short period of time and minimizes the amount of training time on
staff, many of whom must come in on their day off for the training.
The scenarios created by the Wells Center and participating hospitals were (1) Preventing the
spread of MRSA Infection; (2) Assessment and Prevention of Pressure Ulcers; (3) Rapid
Response Team in an Isolation Room; and (4) Over Sedation Post Colonoscopy. A brief
description of the objectives for each scenario is presented in Table 1.
Table 1: Objectives of Training Scenarios
Preventing the Spread of MRSA Infection
The objectives of the scenario are to identify correct room set up for MRSA/contact isolation, demonstrate correct use
of personal protective equipment, demonstrate correct hand hygiene and decontamination, and provide patient
education regarding MRSA/contact isolation.
Assessment and Prevention of Pressure Ulcers
The objectives of the scenario are to identify three evidence-based risk factors that place a patient at risk of
developing a pressure ulcer, calculate the patient’s risk of developing a pressure ulcer using an evidence-based risk
assessment tool (e.g. Norton, Braden, BradenQ, etc), identify nursing interventions to reduce a patient's risk of
developing a pressure ulcer, and identify a deep tissue pressure ulcer and a Stage I pressure ulcer.
Rapid Response Team (RRT) in an Isolation Room
The objectives of the scenario are to recognize the need for Rapid Response Team (RRT) activation, and
demonstrate proper steps to activate RRT.
Over Sedation Post Colonoscopy/Appendicitis
The objectives of the scenario are to recognize the signs and symptoms of over sedation and provide the appropriate
treatment.
8
NEED FOR THE EVALUATION
Medical simulation is increasingly being used as a teaching method for improving patient
safety. Medical simulation is a powerful learning tool because it can be used to teach things
that are not easily learned with other teaching methods (Harrison, et al., 2005). Trainees can
actually go through the experience and use their skills instead of listening to a lecture about
appropriate procedures. Mannequin based simulation training has been used in medicine since
the early 1960s. A recent review of the literature reveals that there are many applications of
mannequin based simulation training but little to no empirical evidence of the efficacy and cost
effectiveness of the training (Fritz et al., 2008).
As interest in applying evidence based approaches in nursing care and the medical profession
has been growing, the need for rigorous evaluation of simulation as a teaching method has
increased. Evidence based nursing/medical practice involves using credible, up to date research
to guide patient care. Simulation training has not been rigorously evaluated to assess its
effectiveness and to identify specific training methods that maximize the effectiveness of
mannequin based simulation training for medical professionals. However, potential advantages
of simulation training over traditional learning methods have been posited in the literature, as
well as disadvantages. Some of the advantages of simulation training identified are listed
below. Simulation training creates:
the ability to go through all the steps of actually delivering care without the danger of
causing harm or injuring a patient (Fritz et al., 2008; Kneebone, 2003);
the potential to increase the speed of acquisition of clinical skills to a defined level of
competence by allowing the opportunity for repetitive practice at the learner’s own
pace (Kneebone, 2003);
the ability to train with a team (Fritz et al., 2008);
the ability to create uncommon, critical scenarios in order to increase exposure to rare
events (Fritz et al., 2008);
the advantage of creating a standardized curriculum so that the same exact scenarios
can be presented to all learners (Fritz et al., 2008);
the component of reflective learning by using facilitated debriefing of scenarios and
video feedback (Fritz et al., 2008); and
the potential to decrease the number and effect of errors through crisis resource
management (CRM) training (Fritz et al., 2008).
The disadvantages associated with simulation training have been identified as cost and negative
transfer (Fritz et al., 2008). The initial set-up costs for simulation training can run into hundreds
of thousands of dollars. “Negative transfer occurs if the student learns something incorrectly
9
owing to imperfect simulation. This most commonly
occurs because the instructor fails to make clear to the
“the belief that wisdom and
students the differences between the training device
clinical experience alone will
and the real situation” (p. 4). This is most likely to occur
produce safe, confident and
in ‘low fidelity’ simulations where the mannequin
effective providers is likely fiction.
cannot replicate real life situations. Simulations using
So many nursing education
high-fidelity mannequins reduce this type of error
programs are now using
because the simulation allows participants to interact
simulation to address issues of
with the mannequin. High fidelity simulation is
patient safety” (Seropian, et. al,
differentiated from low fidelity simulation in the level
2004).
of “realness” of the mannequin. The high fidelity
mannequin is a computer driven mannequin that has realistic breath and bowel sounds, allows
for intubations, giving birth, cardiac arrest, etc.
High Fidelity Mannequin Simulation Training
Scholars argue, however, that simply going through a simulation is not sufficient for effective
learning. Kneebone (2003) suggests that a providing an opportunity for feedback and giving
objective metrics of performance increase the likelihood that the training will be effective. This
added component fits with Kolb’s Experiential Learning Theory (Kolb, 1984) which posits a fourstage learning cycle model. Kolb’s four-stage model is a simple description of a learning cycle
that shows (1) concrete experience, (2) reflective observation, (3) abstract conceptualization,
and (4) active experimentation are needed to really understand what is being taught (Healey &
Jenkins, 2000). “The cycle may be entered at any point, but the stages should be followed in
sequence” (Healey & Jenkins, 2000, p. 186). The WELLS Center training uses an experiential
learning approach with high fidelity mannequins. To incorporate the opportunity for feedback,
the WELLS Center follows the simulation with a debriefing session. In this session, learners
discuss what they learned, what they would change, and ideas for improving performance. The
respondents then provide ideas for how things could have been done differently. Rather than
being told what to do, adult learners do better when they are guided but can draw their own
conclusions (Knowles, 1980).
Job Satisfaction vs. Burnout
In this study, we were asked to measure burnout to determine whether the simulation training
might affect burnout. A review of the literature indicates that job satisfaction is a better
predictor of nurses’ intention to stay in their current job than burnout (Larrabee, Janney, &
Ostrow, 2003; Lu, While, & Barriball, 2004; Tourangeau & Cranley, 2006). In addition, Kalliath,
et al., (2002) demonstrated that job satisfaction is a good indicator of burnout among nurses.
Therefore, measures of job satisfaction were created for this study.
10
METHODOLOGY
The evaluation was designed to assess the efficacy of the high fidelity mannequin simulation for
increasing knowledge, skills and abilities of hospital staff. Specifically, there were five questions
designed to evaluate the training.
1. Is there an increase in the level of knowledge among hospital staff about how to
improve behavior to increase patient safety?
2. Is there an improvement in perceived ability to perform their job after completing the
training?
3. Do trainees rate their perceived job efficacy higher after completing the training?
4. Were the training methods effective?
To address these questions, participants were asked to complete pre and post simulation
survey questionnaires. The pre survey was designed to assess participant’s knowledge,
confidence, and perceived job efficacy before participating in the simulation training.
Participants were then asked to complete a survey questionnaire immediately after the
debriefing session to get their immediate reaction to the training and evaluation of the training.
Finally, participants were asked to complete a follow-up survey questionnaire two months after
participating in the training to find out if they used the information provided in the training and
how useful the training was to their job. The answers for the pre and post survey
questionnaires were compared to measure change. The pre and post surveys at the training
were completed on paper. The follow-up survey was requested by sending an email to the
participants requesting their voluntary participation in an internet based survey. Participants
voluntarily provided their email address when at the simulation training if they were willing to
be contacted to take the follow-up survey.
To increase the strength of the evaluation, the pre surveys were paired with the post surveys by
participant. To pair the pre and post surveys, respondents were asked to create a unique
identification code, based on instructions. The code was designed to keep the data anonymous
while allowing the researchers to link the pre survey responses with the post survey responses
by participant.
This report is a summary of results of the evaluation of the trainings. Individual reports about
each scenario were produced about the Rapid Response Team, Pressure Ulcer and Over
Sedation scenarios1. In this report, data from the individual reports is summarized to address
the research questions. Participants completed pre and post training surveys designed to
address the research questions.
1
Individual reports are available from Nancy Koester, Colorado WIN Partners/University of Colorado Denver;
www.cowinpartners.edu.
11
RESULTS
Survey Respondents
Data for this report was collected at 12 hospitals between September 2008 and May 2009. The
date the training was delivered, the number of respondents for both the pre and immediate
post surveys for each hospital are listed by training scenario in Appendix A, Table 2. A total of
342 participants completed at least a portion of the pre or post survey. There were a total of
238 respondents in the Rapid Response Team (RRT) trainings (69%), with 34 of those
respondents (10%) attended a combined RRT and MRSA training. The Assessment and
Prevention of Pressure Ulcer (PU) trainings had 57 respondents (17%), and the Over Sedation
(OS) trainings had 47 respondents (14%). The proportion of respondents in each training
scenario is displayed in Figure 1.
Figure 1: Proportion of Respondents by Training Topic
RRT & MRSA,
10%
Over Sedation,
14%
Pressure Ulcer,
17%
Rapid Response
Team (RRT),
59%
Gender
The majority of all participants were female (91%), 8% were male, and 1% did not respond to
the question (see Appendix A, Table 3). The ratio of females to males was consistent across
scenarios as displayed in Figure 2.
12
Percent of Respondents
Figure 2: Gender of Respondents by Training Topic
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
92%
91%
87%
Female
Male
Missing
9%
8%
0%
Over Sedation
8%
4%
Pressure Ulcer
1%
Rapid Response
Team
Type of Job
The majority of participants were RNs (79%). The job titles of other participants are listed in
Appendix A, Table 4, and include LPNs (4%), CNAs (5%), EMTs (<1%), RTs (2%), MDs (<1%) and
Other (6%). The percent of respondents in each scenario by job type is displayed in Figure 3.
(Note: CNA is misspelled in the figure because Microsoft Excel changed it to CAN.)
Figure 3: Respondent’s Job
90%
82%81%82%
Percent of Respondents
80%
70%
60%
50%
RRT
40%
Over Sedation
30%
Pressure Ulcer
20%
13%
10%
4% 4% 5%
3%
6%
11% 9%
0%
0%
RN
LPN
CAN
Other
13
Highest Level of Education
The majority of the participant’s highest level of education was a bachelors degree (BA/BS =
50%), about a quarter of the participants had an Associate degree (AA/AS = 29%) and graduate
degrees were held by 6% of the participants (see Appendix A, Table 5). The proportion of
respondents in education level by the training scenario they attended is displayed in Figure 4.
Figure 4: Respondent’s Level of Education
60%
55%
54%
Percent of Respondents
50%
40%
35%
32%
28%
30%
27%
RRT
22%
Over Sedation
20%
10%
Pressure Ulcer
7%
3%
3%
2%
2%
1%0%0%
9%
6%
4%
2% 3%
0%
0%
High
school
AA/AS
BA/BS
MA/MS
M.D.,
Ph.D,
Other
Missing
Previous Training with High Fidelity Mannequin
Participants were asked if they had ever participated in simulation training using a high fidelity
mannequin. The majority (61%) of the participants indicated they had no previous experience
with a high fidelity mannequin, 30% indicated they had attended training using a mannequin
and 9% did not respond to the question (see Figure 5 and Appendix A, Table 6).
14
Figure 5: Previously Participated in High Fidelity Simulation Training
100%
Percent of Respondents
90%
80%
70%
67%
62%
66%
60%
Yes
50%
40%
No
32%
29%
30%
Missing
24%
20%
10%
6%
10%
4%
0%
Over Sedation
Pressure Ulcer
RRT
Age, Licensure and Number of Years on Job
The average age of participants was 39, with ages ranging from age 20 to 70. The average
number of years participants were licensed is 11 years. The average number of years in their
current job was 6 years.
Training Evaluation Results
The purpose of this report is to summarize the results of the evaluation of Wells Center High
Fidelity Mannequin Simulation Trainings designed to improve patient safety in hospitals as
outlined in the Institute for Healthcare Improvement’s 5 Million Lives Campaign. The Wells
Center designed four training scenarios to fit with four of the 5 Million Lives Campaign
interventions for improving patient safety. The four scenarios designed by the Wells Center (in
collaboration with Colorado hospitals) were designed to:
1. improve activation of the rapid response team;
2. prevent harm from high-alert medications;
3. assess and prevent pressure ulcers; and
4. reduce MRSA2 infection by improving infection control practices.
For hospitals interested in reducing MRSA infections, the training scenario was combined with
the rapid response team scenario.
Pre and post survey questionnaires were written for each scenario to address the five research
questions:
2
MRSA – Methicilin-Resistant Staphylococcus aureus
15
1. Is there an increase in the level of knowledge among hospital staff about how to
improve behavior to increase patient safety?
2. Is there an improvement in perceived ability to perform their job after completing the
training?
3. Is there an improvement in the confidence of hospital staff for increasing patient safety?
4. Do trainees rate their perceived job efficacy higher after completing the training?
5. Were the training methods effective?
The following section of this report will summarize the results of the evaluation for each of the
research questions.
Perceived Improvement in Knowledge
Participants were asked to rate their agreement with a series of statements, before and
immediately after the training, to assess whether their knowledge increased about the topic.
The items and participant responses are listed in Table 7. The items about improvement in
knowledge were customized for each scenario. Across all scenarios, participants indicated that
their knowledge increased about the topic.
Rapid Response Team (RRT)
The rapid response team (RRT) trainings increased participant’s confidence about how and
when to activate the RRT, and increased their awareness of RRT protocol. All increases in
knowledge are statistically significant (p<.000 for all three questions)3. The mean scores and
statistical probability for significance are included in Appendix A, Table 7 and shown below in
Figure 6.
3
The p value is the probability that this difference is due to chance. A p value of less than .05 indicates a significant
difference, and that the probability of this difference occurring by chance is less than 5%.
16
Mean Rating
Strongly Disagree to Strongly Agree
Figure 6: RRT Perceived Improvement in Knowledge
5
4.5
4
3.5
3
2.5
2
1.5
1
0.5
0
4.56
4.01
4.5
4.15
3.85
3.3
Pre
Post
How to activate RRT
p < .000
When to activate
RRT
p < .000
Aware of RRT
Protocol
p < .000
Over Sedation (OS)
The majority of those who attended an Over Sedation training (n=47) agreed or strongly agreed
that the simulation was helpful for learning about high alert medications (83%) and how to
prevent over sedation (70%). Respondents were also asked about the helpfulness of the
simulation for learning the protocol for preventing pressure ulcers. Most of the respondents
(56%) rated their agreement as neutral, 33% agreed or strongly agreed with the statement.
Participants indicated that the over sedation (OS) trainings increased their confidence in
knowing what to do in the event of a narcotic overdose, how to prevent over sedation and
what to do in the event of over sedation. All increases are statistically significant in the desired
direction. The data is presented in Appendix A, Table 7 and shown below in Figure 7.
Mean Rating
Strongly Disagree to Strongly Agree
Figure 7: Over Sedation Perceived Improvement in Knowledge
4.5
4
3.5
3
2.5
2
1.5
1
0.5
0
3.94
3.37
3.55
3.98
3.91
3.31
Pre
Post
Know what to do for
narcotic overdose
p < .003
Know about
preventing over
sedation
p < .018
Know what to do for
over sedation
p < .000
17
Pressure Ulcer
The trainings to assess and prevent pressure ulcers (PU) increased participant’s confidence and
knowledge about how to conduct a skin assessment, how to assess a patient’s risk for pressure
ulcers, and knowledge about nutrition and hydration as they relate to pressure ulcers. These
increases in confidence and knowledge were all statistically significant. One item was not
statistically significant: “I am confident that I know the protocol for when to conduct a
Pressure Ulcer”. Most participants rated their agreement with this statement as neutral both
before (53%) and after the training (55%). The data is presented in Appendix A, Table 7, it is
graphically depicted in Figure 8.
Mean Rating
Strongly Disagree to Strongly Agree
Figure 8: Pressure Ulcer Perceived Increase in Knowledge
4.4
4.3
4.3
4.2
4.3
4.2
4.1
4.1
4.1
4.1
4.1
4
3.9
3.9
Pre
3.8
Post
3.7
Know when to Know how to Know how to Know about
assess for
conduct a skin assess patient's nutrition and
pressure ulcer assessment
risk for
hydration
p < .103
p < .023
pressure ulcer
p < .005
p < .005
Additional Questions
In addition to the paired pre and post simulation items, respondents in the Pressure Ulcer and
Over Sedation scenarios were asked to rate their agreement to additional statements about
their increase in knowledge immediately after the training.
Participants responded to two additional items about the Over Sedation training. The majority
of those who attended an Over Sedation training (n=47) agreed or strongly agreed that the
simulation was about high alert medications (83%, mean = 4.13) and how to prevent over
sedation (70%, mean = 3.79). Respondents who attended the Preventing Pressure Ulcers
training were also asked about the helpfulness of the simulation for learning the protocol for
preventing pressure ulcers. Most of the respondents (56%) rated their agreement as neutral,
33% agreed or strongly agreed with the statement. Results of their responses to the items
immediately after the training are shown in Figure 9.
18
Mean Rating
Strongly Disagree to Strongly Agree
Figure 9: Post Simulation Ratings of Knowledge Improvement
5
4.14
4.13
3.79
4
3
2
1
0
High alert medications
How to prevent over
sedation
Protocol for preventing
pressure ulcers
The simulation was helpful for learning about
Perceived Improvement in Ability to Perform Job
Participants were asked to respond to items that assessed whether they thought the simulation
training improved their ability to perform their job. In the Rapid Response Team scenario and
the Pressure Ulcer scenario, they were asked to respond to one item before and after the
training. In the Over Sedation scenario they were only asked to rate items immediately after
the training.
In the Rapid Response Team scenario participants were asked to respond to the item: I am
confident in my ability to use the SBAR form of communication. Responses to this pre and post
item indicated that participants felt the training improved their ability to communicate using
SBAR. The mean ratings increased from 3.42 (pre) to 4.05 (post), significant at p < .000. In the
Pressure Ulcer scenario participants were asked to respond to the item: I am confident with my
skill for identifying a pressure ulcer. Responses to this pre and post item indicated that
participants felt the training increased their confidence with their skill for identifying a pressure
ulcer. The mean ratings increase from 4.00 (pre) to 4.29 (post), significant at p < .021. The pre
and post averages of participant’s ratings of their perceived improvement in their ability to
perform their job are displayed in Figure 10. Additional results are shown in Appendix A, Table
9.
19
Ratings of Perceived Improvement in
Job Performance
Figure 10: Perceived Improvement in Job Performance
5
4.05
4.29
4.00
4
3.42
3
Pre
Post
2
1
I am confident in my ability to
use the SBAR form of
communication
p < .000
I am confident, after the
training, with my skill in
identifying a pressure ulcer
p < .021
The objectives of the Rapid Response Team (RRT) scenario identified job skills to be improved
through the training. The simulation was designed to improve communication and teamwork
skills and hospital staff’s skills and abilities for activating the RRT. Participants responded to four
items immediately after the training to assess the effectiveness of the training. The four items
were:
The simulation training helped improve my communication skills.
I am confident in my ability to effectively communicate with the RRT.
The simulation training helped improve my teamwork skills.
The simulation was helpful for increasing my skills and abilities for activating the RRT?
The majority of respondents agreed or strongly agreed with the statements suggesting that the
scenario improved communication skills, teamwork skills, and their ability to activate the RRT.
Results are displayed in Figure 11. Complete results are provided in Appendix A, Table 9.
20
Strongly Disagree to Strongly Agree
Figure 11: Ratings of Effect of RRT Training on Job Performance
5.00
4.10
4.35
4.19
4.33
4.00
3.00
2.00
1.00
The simulation I am confident in The simulation The simulation
training helped
my ability to
training helped was helpful for
improve my
effectively
improve my
increasing my
communication communicate teamwork skills skills and abilities
skills
with the RRT
for activating the
RRT
Respondents also rated two items about whether the Pressure Ulcer scenario was helpful for
learning skills. The majority of respondents agreed or strongly agreed with the statements:
‘[t]he simulation was helpful for learning about a patient’s risk for pressure ulcers’, and ’[t]he
simulation was helpful for increasing my skills and abilities for assessing a patient’s risk for
pressure ulcers”.. The means for these two items are 4.24 and 4.16 respectively. They are
displayed in Figure 12. Complete results for these two items are provided in Appendix A, Table
10.
Respondents also rated two items about whether the Over Sedation scenario was helpful for
learning job skills. The majority of respondents agreed or strongly agreed with the statements:
‘[t]he simulation was helpful for increasing my skills in dealing with over sedation’, and ’[t]he
simulation was helpful for increasing my skills and abilities for what to do in the event of
narcotic overdose’ The means for these two items are 4.55 and 4.41 respectively. They are
displayed in Figure 12. Complete results for these two items is provided in Appendix A, Table
10.
21
Strongly Disagree to Strongly Agree
Figure 12: Perceived Effect of Training on Job Performance
5
4.55
4.24
4.16
4.41
4
3
2
1
The simulation was The simulation was The simulation was The simulation was
helpful for learning
helpful for
helpful for
helpful for
about a patient’s increasing my skills increasing my skills increasing my skills
risk for pressure and abilities for
in dealing with
and abilities for
ulcers
assessing a
over sedation
what to do in the
patient’s risk for
event of narcotic
pressure ulcers
overdose
Reducing the Spread of MRSA Infection
One of the four scenarios was designed to reduce the spread of MRSA infection. This scenario
was only delivered in conjunction with a Rapid Response Team scenario in three hospitals.
Respondents were asked to rate whether the training had an effect on three behaviors related
to reducing the spread of infection, hand washing before patient contact, hand washing after
removing gloves and frequency of hand washing after contact with patients. Results show no
difference in rating of behavior change between the ratings before the training and
immediately after the training. Results are displayed in Figure 13 and Appendix A, Table 11.
Figure 13: Reducing the Spread of MRSA Infection
5.0
4.33
4.4
4.42
4.52
4.69
4.76
Never to Always
4.0
3.0
PRE
POST
2.0
1.0
Wash hands before patient Wash hands after removing Wash hands after patient
contact
gloves
contact
22
Effect of Training on Job Satisfaction
Job satisfaction has been shown to be related to burnout and nurses intention to leave their
job. Burnout and staff turnover are costly to hospitals and likely to reduce patient safety. We
asked participants to rate their level of satisfaction with their job and to respond to items about
whether this type of training contributes toward increasing job satisfaction. Overall, the
majority of participants agreed or strongly agreed with the statement: ‘[I]n general, I enjoy
most aspects of my job, 94.1% before the training and 94.9% after the training (see Figure 14).
Data is presented in Appendix A, Table 12.
Strongly Disagree to Strongly Agree
Figure 14: Enjoy Job
5
4.41
4.47
Pre
Post
4
3
2
1
In general, I enjoy most aspects of my job
Respondents indicated that they strongly agreed or agreed with the statements: ‘[i]n general I
like working here’ and ‘Most of the time I feel confident in my ability to care for patients’. After
the training participants were asked to rate whether the ‘training increased their level of job
satisfaction’ and whether ‘this type of education opportunity makes them feel valued as an
employee’. The majority agreed or strongly agreed with both statements, 60.2% and 66.8%
respectively (see Figure 15). Data is presented in Appendix A, Table 13 and Table 14.
This project is supported by the Colorado Trust, Colorado Department of Labor &Employment, Colorado
Workforce Development Council, and WELLS Center partners.
Strongly Disagree to Strongly Agree
Figure 15: Training Effect on Job Satisfaction
5
4.49
4.33
4.14
4.27
4
3
2
Pre
1
Post
In general I like
working here
Most of the
This type of
This type of
time I feel
training
education
confident in my increases my opportunity
ability to care level of job makes me feel
for patients
satisfaction
valued as an
employee
Training Effectiveness
Participants were asked to respond to items about the value of the training, the usefulness and
applicability of the high fidelity mannequin, whether they learned the content and information,
and whether the training might affect their job performance and their job satisfaction.
Statements included:
The high fidelity mannequin simulation increased my confidence in my ability to care for
patients.
I clearly understood the purpose and objectives of the simulation.
The debriefing session was helpful.
The high fidelity mannequin simulation offered a valuable way to learn the information.
The use of the high fidelity mannequin added to the learning experience.
I will use what I learned in this training on my current job.
Figure 16 shows the participants’ responses to these items, and the data is presented in
Appendix A, Table 15.
24
Figure 16: Training Effectiveness
Strongly Disagree to Strongly Agree
5
4.17
4.38
4.38
4.42
4.46
4.39
4
3
2
1
Ability to care Understood
for patients the purpose
and
objectives
Debriefing Valuable way Added to the
session was to learn the
learning
helpful
information experience
I will use
what I
learned in
this training
Participants were asked to rate their comfort using a high fidelity mannequin as a method of
training. There was some variability of responses with 52.8% of the respondents rating their
agreement with the comfort of using a high fidelity mannequin between strongly disagree to
neutral and 47.2% agreeing or strongly agreeing. Data is presented in Figure 17, below, and in
Appendix A, Table 16.
Figure 17: Comfort Level Using High Fidelity Mannequin
I feel comfortable using the high fidelity
mannequin simulation as a training method
40%
34%
35%
35%
30%
25%
20%
14%
15%
10%
13%
5%
5%
0%
strongly
disagree
disagree
neutral
agree
strongly agree
25
SUMMARY AND CONCLUSIONS
The evaluation of the Wells Center high fidelity mannequin simulation training was designed to
assess the effectiveness of the training for improving hospital staff knowledge, skills and
abilities in four key areas to increase patient safety. Participants in the training were asked to
complete a survey questionnaire before the training and immediately after the training. The
survey questionnaires were customized for each training. There were some common items
used to assess participant’s reaction to the training. The research questions for the evaluation
are listed below.
Research Questions
1. Is there an increase in the level of knowledge among hospital staff about how to
improve behavior to increase patient safety?
2. Is there an improvement in perceived ability to perform their job after completing the
training?
3. Do trainees rate their perceived job efficacy higher after completing the training?
4. Were the training methods effective?
Increase in Knowledge
Participants were asked to rate their level of agreement with statements about whether the
simulation training increased their knowledge about the training topic. For the rapid response
team (RRT) training they were asked if they knew how to active the RRT, if they were confident
when to activate the RRT and if they were aware of the RRT protocol. The before and after
training responses indicated that the training significantly increased their knowledge about
activating the RRT.
Participants indicated that the over sedation (OS) trainings significantly increased their
confidence in knowing what to do in the event of a narcotic overdose, how to prevent over
sedation and what to do in the event of over sedation.
The trainings to assess and prevent pressure ulcers significantly increased participant’s
confidence and knowledge about how to conduct a skin assessment, how to assess a patient’s
risk for pressure ulcers, and knowledge about nutrition and hydration as they relate to pressure
ulcers.
Perceived Improvement in Job Performance
Participants were asked to respond to items that assess whether they thought the simulation
training improved their ability to perform their job. In before and after comparisons,
participants in the RRT training indicated that the training significantly improved their ability to
use the SBAR form of communication. Pressure ulcer participant’s comparison of before and
after responses indicated that the training significantly increased their confidence with their
skill for identifying a pressure ulcer.
Ratings collected immediately after the training also showed high rates of agreement with
statements about the effectiveness of the training. RRT respondents indicated that the
26
simulation training helped improve their communication skills, teamwork skills and skills and
abilities for activating the RRT. The majority of pressure ulcer participants agreed or strongly
agreed that the simulation was helpful for learning about a patient’s risk for pressure ulcers and
it was helpful for increasing skills and abilities for assessing patient’s risk for pressure ulcers.
Results were similar in for the over sedation scenario, the majority of respondents agreed or
strongly agreed that the simulation was helpful for increasing their skills in dealing with over
sedation and increasing their skills and abilities for what to do in the event of narcotic
overdose.
Reducing the Spread of Infection
The scenario for reducing the spread of infection was included in the RRT training at three
hospitals. Participants were asked about hand washing behavior before and immediately after
the training. There was little difference is their responses to the items between the two
assessments since they were so close together in time. There were not enough respondents to
the two month follow-up assessment to determine if there might be differences over time.
Effect of the Training on Job Satisfaction
Respondents were asked to respond to several items related to job satisfaction. In general,
there were no differences in the responses before the training and after the training. The
majority of the respondents agreed or strongly agreed with statements indicating that in
general they enjoy most aspect of their job, they like working here, and most of the time they
feel confident in their ability to care for patients. After the training the majority of the
participants agreed or strongly agreed that this type of training increases my level of job
satisfaction and this type of educational opportunity makes me feel valued as an employee.
Training Effectiveness
Participants were asked to respond to items about the value of the training, the usefulness and
applicability of the high fidelity mannequin. The majority of participants agreed or strongly
agreed that the high fidelity mannequin simulation increased their confidence in ability to care
for patients, the high fidelity mannequin simulation was a valuable way to learn the
information, the high fidelity mannequin added to the learning experience and they will use
what they learned in the training on the job. The majority also agreed or strongly agreed that
the debriefing session was helpful, they clearly understood the purpose and objectives of the
training and they felt comfortable using the high fidelity mannequin simulation as a training
method.
Overall, results of the evaluation indicate that the use of the high fidelity mannequin simulation
as a training and teaching method is highly effective based on respondents perceptions for
increasing knowledge, skills and abilities. This evaluation did not show strong results in behavior
change for reducing the spread of infection due to the lack of responses to the two month
follow-up survey. The overall lack of responses to the two month follow-up survey was
extremely low. This is likely due to the nature of nurses work and that no incentives were
offered to encourage people to take the time to respond to another survey.
27
There were also not strong results about the effect of the simulation on job satisfaction. Further
research is needed to learn more about whether simulation training has an effect on job
satisfaction. Recent research suggests that confidence is an important factor in patient safety.
Many of the items in this evaluation asked about confidence. The responses to these items
suggests that further research should be directed at looking the effect of the simulation training
on increasing confidence among medical staff.
28
REFERENCES
Caers, R., Du Bois, C., Jegers, M., De Gieter, S., De Cooman, R., & Pepermans, R. (2008).
Measuring community nurses' job satisfaction: literature review. J Adv Nurs, 62(5), 521-529.
Curtis, R.J., Cook, D.J., Wall, R.J., Angus, D.C., Bion, J., Kacmarek, R., Kane-Gill, S.L., Kirchnoff,
K.T., Levy, M., Mitchell, P.H., Moreno, R., Pronovost, P., & Puntillo, K. (2006). Intensive care
unit quality improvement: A “how-to” guide for the interdisciplinary team. Critical Care
Medicine, 34(1), 211-218.
Fritz, P., Gray, T., & Flanagan, B., (2008) Review of mannequin-based high-fidelity simulation in
emergency medicine. Emergency Medicine Australasia, 20, 1-9.
Halbesleben, J. R., Wakefield, B. J., Wakefield, D. S., & Cooper, L. B. (2008). Nurse burnout and
patient safety outcomes: nurse safety perception versus reporting behavior. West J Nurs
Res, 30(5), 560-577.
Harrison, T. K. & Gaba, D.M., (2005). Safe passage – using simulation to teach patient safety.
New Developments in Teaching, 2, 1, 37-41.
Healy, M., & Jenkins, A. (2000). Kolb’s experiential learning theory and its application in
geography in higher education. Journal of Geography, 99, 185-195.
Institute for Healthcare Improvement (2008). Protecting 5 million lives from harm. Retrieved
May 19, 2008, from Institute for Healthcare Improvement Web site:
http://www.ihi.org/IHI/Programs/Campaign/
Jones, C.B. (2005). The costs of nurse turnover, part 2: Application of the nursing turnover cost
calculation methodology. Journal of Nursing Administration, 35, 41-49.
Kalliath, T., & Morris, R. (2002). Job Satisfaction Among Nurses A Predictor of Burnout Levels.
JONA, 32(12), 648–654.
Kneebone, R. (2003). Simulation in surgical training: Educational issues and practical
implications. Medical Implications, 3, 267–277.
Knowles, M.S. (1980). The modern practice of adult education: From pedagogy to andragogy.
Englewood Cliffs, NJ. Cambridge.
Kolb, D. A. (1984). Experiential Learning: Experience as the Source of Learning and
Development. Englewood Cliffs, New Jersey: Prentice-Hall.
29
Larrabee, J. H., Janney, M. A., Ostrow, C. L., Withrow, M. L., Hobbs, G. R., Jr., & Burant, C.
(2003). Predicting registered nurse job satisfaction and intent to leave. J Nurs Adm, 33(5),
271-283.
Long, R.E. (2005). Using simulation to teach resuscitation: an important patient safety tool.
Critical Care Nursing Clinics of North America, 17, 1-8.
Lu, H., While, A. E., & Barriball, K. L. (2005). Job satisfaction among nurses: a literature review.
International Journal of Nursing Studies, 42(2), 211-227.
Taekman, J. M., Hobbs, G., Barber, L., Phillips-Bute, B. G., Wright, M. C., Newman, M. F., et al.
(2004). Preliminary report on the use of high-fidelity simulation in the training of study
coordinators conducting a clinical research protocol. Anesthesia and Analgesia, 99(2), 521527.
Tourangeau, A. E., & Cranley, L. A. (2006). Nurse intention to remain employed: understanding
and strengthening determinants. Journal of Advanced Nursing, 55(4), 497-509.
30
APPENDIX A: DATA TABLES
Table 2: Number and Percent of Pre and Post Survey Respondents by Hospital and Scenario
Hospital
Date of Training
(month/year)
VVH
9/08
TCH
11/08
TCH
4/09
SAC
2/09
RGH
3/09
MMH
2/09
MMH
5/09
MCR
3/09
LMC
3/09
VVMC
5/09
KMH
5/09
YDH
3/09
SMHRMC
2/09
SLVRMC
3/09
TOTAL
342
Over Sedation
Respondents*
Pressure Ulcer
Respondents*
RRT
RRT & MRSA
Respondents* Respondents*
30
15%
9%
93
46%
27%
18
38%
5%
29
14%
9%
10
5%
3%
19
9%
6%
10
21%
3%
7
12%
2%
12
34%
35%
4
11%
1%
3
6%
<1%
22
11%
7%
19
54%
6%
16
28%
5%
34
60%
10%
16
34%
5%
47
14%
57
17%
203*
59%
35
10%
*The top number is the number of respondents, the middle number is the column percentage and the
bottom number is the row percentage or the percent of the total.
This project is supported by the Colorado Trust, Colorado Department of Labor &Employment, Colorado Workforce
Development Council, and WELLS Center partners.
Table 3: Gender
Gender
Female
Male
Missing
Total
Over Sedation
Respondents
n
% of Over
Sedation
% of Gender
43
92%
14%
4
8%
14%
0
0%
0%
47
100%
14%
Pressure Ulcer
Respondents
RRT*
Respondents
n
% of PU
% of Gender
n
% of RRT
% of Gender
48
87%
16%
5
9%
18%
2
4%
50%
55
100%
16%
217
91%
70%
19
8%
68%
2
1%
50%
238
100%
70%
Total
n
% of Gender
308
91%
100%
28
8%
100%
4
1%
100%
340
100%
100%
The top number is the number of respondents, the middle number is the column
percentage and the bottom number is the row percentage or the percent of the total.
* Includes MRSA respondents
32
Table 4: Type of Job
Job Title
RN
LPN
CNA
EMT
PharmD
RT
CCT
SSC
MD
NP
Other:
Missing
Total
Over Sedation
Respondents
Pressure Ulcer
Respondents
RRT
Respondents
Total
n
% of OS
% of Job
38
81%
14%
2
4%
13%
3
6%
17%
0
0%
0%
0
0%
0%
1
2%
13%
0
0%
0%
0
0%
0%
0
0%
0%
0
0%
0%
3
6%
16%
0
0%
0%
47
100%
14%
n
% of PU
% of Job
35
64%
13%
3
5%
20%
7
13%
39%
0
0%
0%
0
0%
0%
0
0%
0%
0
0%
0%
0
0%
0%
0
0%
0%
0
0%
0%
2
10%
11%
8
15%
80%
55
100%
16%
n
% of RRT
% of Job
194
82%
73%
10
4%
67%
8
3%
44%
1
<1%
100%
0
0%
0%
7
3%
88%
0
0%
0%
0
0%
0%
2
1%
100%
0
0%
0%
14
6%
74%
2
1%
20%
238
100%
70%
n
% of Total
% of Job
267
79%
100%
15
4%
100%
18
5%
100%
1
<1%
100%
0
0%
0%
8
2%
100%
0
0%
0%
0
0%
0%
2
<1%
100%
0
0%
0%
19
6%
100%
10
3%
100%
340
100%
100%
33
Table 5: Highest Level of Education
Educational Degree
High school
Associates (AA/AS)
Bachelors (BA/BS)
Masters (MA/M)S
Doctor/Doctorate
(Ph.D., Psy.D., M.D.)
Other
Missing
Total
Over Sedation
Respondents
n
% of Over
Sedation
% of Ed.
3
6%
14%
15
32%
15%
26
55%
15%
1
2%
6%
0
0%
0%
2
4%
10%
0
0%
0%
47
100%
14%
Pressure Ulcer
Respondents
RRT
Respondents
n
% of PU
% of Ed.
n
% of RRT
% of Ed.
n
% of Ed.
12
22%
55%
19
35%
19%
15
27%
9%
1
2%
6%
0
0%
0%
5
9%
24%
3
3%
43%
55
100%
16%
7
3%
32%
66
28%
66%
129
54%
76%
16
7%
89%
2
1%
100%
14
6%
67%
4
2%
57%
238
100%
70%
22
7%
100%
100
29%
100%
170
50%
100%
18
5%
100%
2
1%
100%
21
6%
100%
7
2%
100%
340
100%
100%
Total
34
Table 6: Previously Participated in High Fidelity Simulation Training?
Job Title
Yes
No
Missing
Total
Over Sedation
Respondents
n
% of Over
Sedation
% of Yes/No
15
32%
17%
29
62%
13%
3
6%
10%
47
100%
14%
Pressure Ulcer
Respondents
RRT
Respondents
n
% of PU
% of Yes/No
n
% of RRT
% of Yes/No
n
% of Yes/No
16
29%
18%
37
67%
17%
2
4%
7%
55
100%
16%
58
24%
65%
156
66%
70%
24
10%
83%
238
100%
70%
89
30%
100%
222
61%
100%
29
9%
100%
340
100%
100%
Total
35
Table 7: Perceived Improvement in Knowledge
RAPID RESPONSE TEAM
Strongly
Strongly
Disagree Disagree Neutral Agree Agree
Pre and Post Questions
Pre
Total
Mean p value
4
29
22
79
94
228
4.01
1.8%
12.7%
9.6%
34.6%
41.2%
100%
1
0
6
82
132
221
0.5%
0%
2.7%
37.1%
59.7%
100%
4
22
41
95
64
226
1.8%
9.7%
18.1%
42%
28.3%
100%
1
0
5
95
119
220
0.5%
0%
2.3%
43.2%
54.1%
100%
8
50
67
72
31
228
3.5%
21.9%
29.4%
31.6%
13.6%
100%
2
5
35
94
84
220
0.9%
2.3%
15.9%
42.7%
38.2%
100%
p < .000
Total
Mean p value
*I know how to activate the RRT.
Post
Pre
*I am confident that I know when to
activate the rapid response team.
Post
Pre
4.56
p < .000
3.85
4.50
p < .000
3.30
*I am fully aware of the RRT protocol.
Post
OVER SEDATION
Strongly
Strongly
Disagree Disagree Neutral Agree Agree
Pre and Post Questions
Pre
I am confident that I know what to
do in the event of narcotic overdose.
Post
Pre
I am confident that I know about
preventing over sedation.
Post
Pre
I am confident that I know what to
do in the event of over sedation.
Post
1
9
14
21
4
49
2.0%
18.4%
28.6%
42.9%
8.2%
100%
0
2
8
28
9
47
0%
4.3%
17%
1
3
15
28
2
49
2.0%
6.1%
30.6%
57.1%
4.1%
100%
0
1
9
30
7
47
0%
2.1%
1
6
20
21
1
49
2.0%
12.2%
40.8%
42.9%
2.0%
100%
0
1
11
23
12
47
0%
2.1%
23.4%
48.9%
25.5%
100%
59.6% 19.1%
19.1% 63.8% 14.9%
4.15
3.37
3.94
100%
p < .003
3.55
3.91
100%
p < .018
3.31
3.98
p < .000
36
Table 7: Perceived Improvement in Knowledge continued
PRESSURE ULCER
Strongly
Strongly
Disagree Disagree Neutral Agree Agree
Pre and Post Questions
I am confident that I know the
protocol for when to conduct a
Pressure Ulcer.
Pre
Post
Pre
I am confident that I know how to
conduct a skin assessment.
Post
I am confident that I know how to
assess a patient’s risk for pressure
ulcers.
I am confident that I know about
nutrition and hydration as they
relate to pressure ulcers.
Pre
Post
Pre
Post
Total
0
2
9
32
18
61
0%
3.3%
14.8%
52.5%
29.5%
100%
0
0
5
30
15
50
0%
0%
10%
60%
30%
100%
0
1
9
32
18
60
0%
1.7%
15.0%
53.3%
30.0%
100%
0
0
1
32
16
49
0%
0%
2%
65.3%
32.7%
100%
0
1
11
33
16
61
0%
1.6%
18%
54.1%
26.2%
100%
0
0
0
35
15
50
0%
0%
0%
70%
30%
100%
0
1
17
30
13
61
0%
1.6%
27.9%
49.2%
21.3%
100%
0
1
5
30
14
50
0%
2%
10%
69%
28%
100%
Mean p value
4.1
4.2
p < .103
4.1
4.3
p < .023
4.1
4.3
p < .005
3.9
4.1
p < .005
Table 8: After Training Perceptions of Improvement in Knowledge
Over Sedation
Post Simulation Training
The simulation was helpful for learning about high
alert medications.
The simulation was helpful for learning about
how to prevent over sedation.
Pressure Ulcer
Post Simulation Training
The simulation was helpful for learning the protocol for
preventing pressure ulcers.
Strongly
Disagree Disagree
Neutral
Agree
Strongly
Agree
Total
Mean
4.13
0
1
7
24
15
47
0%
2.1%
14.9%
51.1%
31.9%
100%
0
6
8
24
8
46
0%
13%
17.4%
52.2%
17.4%
100%
Neutral
Agree
Strongly
Agree
Total
Mean
Strongly
Disagree Disagree
3.79
0
0
6
31
13
50
4.14
0%
0%
12%
62%
26%
100%
100%
37
Table 9: Perceived Improvement in Job Performance
RAPID RESPONSE TEAM
Pre and Post Questions
Pre
I am confident in my ability to use the SBAR
form of communication.
Post
Pressure Ulcer
Missing
I am confident with my skill for identifying a
pressure ulcer.
Pre
I am confident, after the training, with my
skill in identifying a pressure ulcer.
Post
Strongly
Disagree
Disagree
Neutral
Agree
Strongly
Agree
Mean
Total p value
12
30
71
75
37
225
5.3%
10.9%
31.6%
33.3%
16.4%
100%
5
7
25
116
64
217
2.3%
3.2%
11.5%
53.5%
29.5%
100% p < .000
Strongly
Disagree
Disagree
Neutral
Agree
Strongly
Agree
Mean
Total p value
0
2
9
33
16
61
0%
3.3%
14.8%
54%
26.2%
100%
0
0
4
34
12
50
0%
0%
8%
68%
24%
p
3.42
4.05
4.00
4.16
100% p < .021
Table 10: Effect of training on job performance
Strongly
Disagree
Disagree
Neutral
Agree
Strongly
Agree
Total
Mean
The simulation training helped improve my
communication skills.
3
1.4%
5
2.3%
42
19.1%
87
39.5%
83
37.7%
220
100%
4.10
I am confident in my ability to effectively
communicate with the RRT.
2
.9%
0
0.0%
13
5.9%
109
49.3%
97
43.9%
221
100%
4.35
The simulation training helped improve my
teamwork skills.
2
0.9%
6
2.7%
27
12.2%
100
45.2%
86
38.9%
221
100%
The simulation was helpful for increasing my skills
and abilities for activating the RRT?
2
0.9%
2
0.9%
18
8.2%
97
44.1%
101
45.9%
220
100%
4.33
Strongly
Disagree
Disagree
Neutral
Agree
Strongly
Agree
Total
Mean
0
0
4
30
16
50
0%
0%
8%
60%
32%
100%
0
0
4
34
12
68
0%
0%
5.9%
50.0%
17.6%
100%
Strongly
Disagree
Disagree
Neutral
Agree
Strongly
Agree
Total
0
0
0
21
26
47
0%
0%
0%
44.7%
55.3%
100%
0
0
2
23
21
46
0
0
4.3%
50%
45.7%
100%
Post Simulation Training - JP
Rapid Response Team
Assessment and Prevention of Pressure Ulcers
The simulation was helpful for learning about a
patient’s risk for pressure ulcers.
The simulation was helpful for increasing my skills and
abilities for assessing a patient’s risk for pressure ulcers.
Over Sedation – JP
The simulation was helpful for increasing my skills in
dealing with over sedation.
The simulation was helpful for increasing my skills and
abilities for what to do in the event of narcotic
overdose.
4.19
4.24
4.16
Mean
4.55
4.41
38
Table 11: Reduce Spread of Infection
Pre and Post Infection Questions
Never
Pre
How often do you wash your hands
before patient contact?
Post
Pre
How often do you wash your hands
after removing gloves?
Post
Pre
How often do you wash your hands
after contact with a patient?
Post
Almost
Never Sometimes
Almost
Always
Always
Total
Mean
4.3
0
0
0
24
12
36
0%
0%
0%
66.7%
33.4%
100%
0
0
0
15
10
25
0%
0%
0%
60%
40%
100%
0
0
1
19
16
36
0%
0%
2.8%
52.8%
44.5%
100%
0
0
1
10
14
25
0%
0%
4%
40%
56%
100%
0
0
0
11
24
35
0%
0%
0%
31.4%
68.6%
100%
0
0
0
6
19
25
0%
0%
0%
24%
76%
100%
4.4
4.4
4.5
4.7
4.8
Table 12: Job Satisfaction Before and After Training
Strongly
Strongly
Disagree Disagree Neutral Agree Agree
Pre and Post Questions
Pre
In general, I enjoy most aspects of
my job
Post
Mean
Total p value
1
4
12
132
141
290
0.3%
1.4%
4.1%
45.5%
48.6%
100.0%
1
3
9
106
138
257
0.4%
1.2%
3.5%
41.2%
53.7%
p value
4.41
4.47
100.0% p < .039
Table 13: Job Satisfaction Ratings Before Training
Job Satisfaction
Pre Simulation Training
In general I like working here.
Most of the time I feel
confident in my ability to care
for patients.
Missing
Strongly
Disagree Disagree
Neutral
Agree
Strongly
Agree
Total
Mean
158
344
4.49
58
2
0
13
113
16.9%
0.6%
0%
3.8%
32.8%
54
2
2
11
159
15.7%
0.6%
0.6%
3.2%
46.2%
45.9% 100.0%
116
344
4.33
33.7% 100.0%
39
Table 14: Job Satisfaction Ratings After Training
Post Simulation Training
Missing
This type of training increases
my level of job satisfaction.
JS
This type of education
opportunity makes me feel
valued as an employee.
JS
Strongly
Disagree Disagree
Neutral
Agree
Strongly
Agree
Total
Mean
4.14
87
2
9
39
107
100
344
25.3%
0.6%
2.6%
11.3%
31.1%
29.1%
100%
78
3
3
30
114
116
344
22.7%
0.9%
0.9%
8.7%
33.1%
33.7%
100%
4.27
Table 15: Training Effectiveness
Post Simulation Training
Strongly
Missing Disagree Disagree
Neutral
Agree
Strongly
Agree Total
The high fidelity mannequin
simulation increased my confidence
in my ability to care for patients.
88
26.6%
5
1.5%
6
1.7%
35
10.2%
104
30.2%
106
344
30.8% 100%
4.17
I clearly understood the purpose
and objectives of the simulation.
78
22.7%
2
0.6%
2
0.6%
12
3.5%
128
37.2%
122
344
35.5% 100%
4.38
The debriefing session was helpful.
90
26.2%
2
0.6%
2
0.6%
19
5.5%
105
30.5%
126
344
36.6% 100%
4.38
The high fidelity mannequin
simulation offered a valuable way
to learn the information
The use of the high fidelity
mannequin added to the learning
experience.
87
25.3%
2
0.6%
1
0.3%
15
4.4%
109
31.7%
130
344
37.8% 100%
4.42
87
25.3%
2
0.6%
3
0.9%
13
3.8%
97
28.2%
142
344
41.3% 100%
4.46
I will use what I learned in this
training, on my current job.
87
25.3%
3
0.9%
1
0.3%
12
3.5%
119
34.6%
122
344
35.5% 100%
4.39
Mean
Table 16: Comfort Level with the High Fidelity Mannequin
Pre Simulation Training
I feel comfortable using the
high fidelity mannequin
simulation as a training
method.
Pre
Strongly
Disagree
Disagree
Neutral
Agree
Strongly
Agree
Total
Mean
13
41
98
100
36
344
3.36
4.51%
14.2%
34.0%
34.7%
12.5%
100%
40
APPENDIX B: COMMENTS AND FEEDBACK
Open Ended Feedback about the Simulation Training
Participants were asked to provide comments or feedback about the simulation training. Below are the
responses organized into categories. They responded to the statement: Please provide any feedback regarding
the simulation, including any ways to make it better for you.
ROOM SET-UP
DID NOT RESEMBLE ANY OF THE ROOM, EQUIPMENT. DISTRACTED BY THE ENVIRONMENT.
I THINK THE SITUATION COULD HAVE RUN SMOOTHER IF A MORE REALISTIC ROOM SETUP WAS USED.
ACTUAL ROOM
NEED PANIC BUTTON.
NEEDS TO BE MORE REALISTIC-EQUIPMENT/ROOM SETUP SAME AS FLOORS.
IT WOULD BE HELPFUL TO HAVE THE ROOMS SET UP LIKE PT. ROOMS SO I KNOW WHERE STUFF IS.
USE A VACANT PTS. ROOM
THE SETTING COULD HAVE BEEN MORE SUITED FOR A "REAL PT" I.E. FULL WALL SETUP WITH O2 GAS.
FEEDBACK ABOUT SIM MAN
I REALLY ENJOYED USING THE SIMULATOR. BEING ABLE TO PRACTICE IN REAL TIME ON SOMETHING OTHER THAN A REAL
PATIENT WAS VERY HELPFUL.
I THOUGHT THAT THIS WAS EXCELLENT! IT WOULD HAVE BEEN A BIT BETTER IF WE COULD HAVE LISTENED TO THE LUNG
SOUNDS BECAUSE I WAS CONCERNED ABOUT THE PT HAVING PNEUMO. OTHER THAN THAT I THOUGHT IT WAS GREAT. IT
WOULD BE FUN TO SEE THE SIM BABY IN A COMPLEX SCENARIO
MOST LIKE LIFE MODEL I HAVE EVER HAD TRAINING ON, VERY INFORMATIVE AND HELPFUL AND USEFUL FOR EXPERIENCE.
IT WAS NEAT HOW MANY THINGS THE SIM MAN CAN DO.
DRAWING DRUGS UP IS WHAT I WOULD REALLY LIKE
VERY REAL SIMULATION, MUCH MORE USEFUL THAN W/ DUMMIES OR VERBALLY TALKING THROUGH.
THE MANNEQUIN IS VERY REALISTIC AND WILL BE VERY HELPFUL FOR TEACHING AND EDUCATION. MORE SCENARIOS
WOULD BE GREAT.
MAKE SURE ALL EQUIPMENT IS FUNCTIONING FOR ACCURATE TROUBLE SHOOTING.
AFTER SUCTIONING, THERE SHOULD BE A RESPONSE; WHETHER INCREASED OR DECREASED OXYGEN STATS, OR CHANGE IN
RESP. RATE.
MANNEQUIN DID NOT FUNCTION PROPERLY. TEAM (TEACHING) SEEMED UNREHEARSED.
MANNEQUIN NEEDS ITS SOUNDS TO BE LOUDER, THE BRIEFING WAS THE MOST BENEFICIAL ASPECT. MAY BE HELPFUL TO
DO SIMULATION IN AN ACTUAL PATIENT ROOM. HELPFUL TO HAVE SOMEONE VERBALIZE CHANGES IN PATIENT'S STATUS
BECAUSE WE CANNOT SEE THESE CHANGES IN THE PATIENT
FEEDBACK ABOUT SCENARIOS/SIMULATIONS
BETTER ORIENTATION TO ROOM AND EQUIPMENT, ABILITY TO ASK QUESTIONS WHEN GETTING REPORT WOULD BE
HELPFUL. IT WAS DIFFICULT TO ASSESS THE PATIENT.
41
PEOPLE PLAYING DIFFERENT ROLES SHOULD ENTER THE ROOM AT DIFFERENT TIMES, MORE CLOSELY RELATING TO REAL
LIFE SCENARIOUS. OVERALL THIS WAS A VERY VALUABLE EXPERIENCE.
SETUP THE SCENARIO WHERE NOT ALL PARTICIPANTS WALK INTO THE ROOM AT THE SAME TIME, AND SET UP THE
WALL/HEAD BOARD OF SCENARIO (I.E. MONITOR AND SUCTION) WHERE THEY WOULD BE IN THE ROOM.
GOOD REAL LIFE AND REAL TIME FEEDBACK. SCENARIO WASN'T DIFFICULT.
MORE STRUCTURE
WOULD HAVE BEEN NICE TO WALK THROUGH THE SCENARIO COMPLETELY 1ST RUN. EVEN THOUGH THE MANNEQUIN IS
LIFE LIKE IT WAS ODD TO WORK WITH AND KNOW WHAT IS EXPECTED.
THE SIMULATION WAS VERY HELPFUL. IT WAS NICE TO GO THROUGH THE SCENARIO AND ACTUALLY PRACTICE BAGGING A
PATIENT AND CHANGING THE TRADE.
IT WAS GREAT. I WOULD HAVE LIKED MORE FEEDBACK FROM PT OR A SEPARATE FAMILY MEMBER. MAYBE TAKE IT
FARTHER SO WE NEED MORE INTERVENTIONS.
IT WOULD HAVE BEEN HELPFUL TO PROCEED TO A FULL CORE SITUATION,
GIVE CARDS ABOUT ROLE TO PLAY. MAKE SURE MANNEQUINS CHEST IS FUNCTIONING PROPERLY. HAVE THE SIMULATION
ENVIRONMENT AND THE HOSPIITAL NEEDS TO HAVE A FULLY FUNCTIONING SIMULATION LAB; NEONATAL, CHILD, ADULT.
ASSIGNED ROLES
MORE SPECIFIC/REALISTIC ROLES FOR PARTICIPANTS (IE. PRIMARY NURSE, RT, MD)
ASSIGN ROLES (RN, CA, CHARGE, ETC…)
PROVIDE STAFF WITH ROLES BEFORE GOING INTO SITUATION.
DISCUSS ROLES BEFORE SITUATION
SCENARIO DESIGN
LONGER TIME
COULD WE DO A CARDIAC RELATED ONE FOR THE CARDIAC FLOORS?
SIMULATION IS SIMILAR TO ALL OTHER SIMULATIONS I'VE BEEN IN.

REAL SIMULATION SITUATIONS FROM OUR FLOOR (ONCOLOGY).

MORE IN DEPTH SCENARIOS
GROUP SIZE
PERHAPS 6 PEOPLE ARE TOO MANY, WAS CROWDED AROUND THE BED, LESS COMMUNICATION BY ALL IN
ATTENDANCE, ONLY A FEW INVOLVED.
LESS PEOPLE
SMALLER GROUPS SO THERE'S MORE PARTICIPATION.
COMMENTS ABOUT DEBRIEFING SESSION
I REALLY LIKED THE FEEDBACK AND DEBRIEFING AFTERWARDS. I THINK THIS EXPERIENCE WILL IMPROVE MY
RESPONSE IN AN ACTUAL PT SITUATION WHERE RRT SHOULD BE CALLED.
LOVED THE DEBRIEFING (INTERACTION, RECOMMENDATIONS, SUGGESTIONS). APPRECIATED THE EMPHASIS ON
PRACTICING GOOD COMMUNICATION SKILLS.
WOULD LIKE MORE SPECIFIC FEEDBACK.
POSITIVE FEEDBACK
EXCELLENT TRAINING.
42
INTERESTING EXPERIENCE. NOT USED TO HAVING INTERACTION, I LIKED IT.
VERY HELPFUL THAT IT'S MORE REALISTIC
I ENJOYED IT AND LEARNED A LOT. THANKS.
EXCELLENT! I ENJOYED THE HANDS ON. I LEARN BETTER WHEN I DO THINGS HANDS ON.
GREAT-THANKS
VERY HELPFUL
GREAT LEARNING TOOL. GOOD TO THINK - HELP - STAFF.
GREAT WAY FOR TRAINING AND SIMULATION LEARNING.
HELPFUL. JUST NEED TO GET USED TO THE MOCK CODE MODEL.
EXCELLENT REVIEW OF SKILLS! THANK YOU!
THE SIMULATION WAS A GREAT LEARNING EXPERIENCE. IT HELPED ME REALIZE AREAS IN WHICH I CAN IMPROVE.
WONDERFUL
I WOULD LIKE TO DO MORE OF THESE SIMULATIONS. AS A NEW GRAD, PATIENTS WHO ARE CRASHING ARE VERY
INTIMIDATING AND I NEED MORE EXPERIENCE WITH IT. THANK YOU!
GREAT-WOULD LIKE MORE HERE AT TCH/DENVER
I THINK WE SHOULD TRAIN ALL EMPLOYEES IN DIRECT PATIENT CARE TO RESPOND QUICKLY TO THESE SITUATIONS.
THIS WAS HELPFUL TO HAVE SUCH A REAL LIFE SIM. THIS WOULD BE GREAT TO HAVE OFFERED ON A REGULAR BASIS
FOR OUR STAFF.
THIS WAS REALLY A GOOD EXPERIENCE TO HELP REINFORCE PROPER CARE AND CHAIN OF COMMAND. EVERYONE WAS
REALLY HELPFUL WHICH LESSENED MY ANXIETY AND HELPED ME TO LEARN. GREAT FEEDBACK.
I REALLY ENJOYED THE SIMULATION OF REAL LIFE EVENT & HAVING TO THINK CRITICALLY ON YOUR FEET IN A SHORT
AMOUNT OF TIME! OVERALL, SCENARIOS APPLICABLE AS WELL!
VERY COOL LEARNING TOOL!
IT WAS A VERY DIFFERENT WAY TO LEARN, RESPOND, AND DEMONSTRATE CRITICAL THINKING SKILLS.
GOOD JOB-MANNEQUIN VERY REALISTIC.
EXPERIENCE WAS GREAT. THANK YOU FOR TRAVELING AND TRAINING. WOULD LOVE TO DO MORE TRAINING, LONGER
PERIODS.
THOUGHT IT WAS EXCELLENT! A COUPLE OF SCENARIOS WOULD HAVE BEEN NICE IF TIME ALLOWED -- GET MORE INTO
DEPTH.
SIM TRAINING WAS VERY EFFECTIVE
IT WAS WONDERFUL!
INCREDIBLY REALISTIC AND WELL WORTH THE TIME SPENT DOING SIMULATIONS. YOU CAN TAKE $ OUT OF MY CHECK
TO FUND THE LAB!!
I THOUGHT THAT THIS EXPERIENCE WAS VERY HELPFUL IN PUTTING US INTO A REAL LIFE SITUATION AND HELPING US
KNOW WHAT TO DO. VERY EDUCATIONAL.
GREAT PRACTICE FOR ME AS A NEW GRAD. EXPERIENCE IN SIMULATIONS ARE ALWAYS HELPFUL.
43