2014IITSECProgram_Full_LowRes

Transcription

NATIONAL TRAINING AND SIMULATION ASSOCIATION
T H E W O R L D ’ S L A R G E S T M O D E L I N G & S I M U L AT I O N E V E N T
I/ITSEC
INTERSERVICE/INDUSTRY TRAINING,
SIMULATION & EDUCATION CONFERENCE
TRAINED AND MISSION READY: DETER • DEFEND • DEFEAT
k
to ch ec
e
r
u
s
Be
fron t
e
h
t
e
d
in si
e rs fo r
v
o
c
k
c
a n d ba
Gu id e
t
e
k
c
o
you r P
.
a n d CD
W W W. I I T S E C . O R G
PROGRAM
GUIDE
u
DECEMBER 1-5, 2014
u
ORLANDO, FLORIDA
TA B L E O F C O N T E N T S
n WELCOME & INTRODUCTIONS
Conference Welcome
2
Keynote Speakers
3
Conference Leadership
4
Interservice Executives
5
Principals & Advisor6
n AGENDA
Pre-Conference Agenda
7
Dress Code
7
Conference Agenda
8
Continuing Education Units/Continuous Learning Points
12
Orange County Convention Center Diagram
13
Hyatt Regency Diagram
14
n TUTORIALS
Tutorial Grid
Tutorial Synopses & Schedule
15
16
n SIGNATURE EVENTS27
n FOCUS EVENTS33
n COMMUNITY OF INTEREST / PROGRAM BRIEFS41
n PAPER SESSIONS
Paper Session Grid
Papers/Authors Presentation Schedule
45
50
n STEM
STEM Workforce Initiative
Future Leaders • Students at I/ITSEC
America’s Teachers at I/ITSEC • Educators (techPATH)
Serious Games Showcase & Challenge
STEM Pavilion: Project Based Learning
I/ITSEC Scholarships
Post I/ITSEC Professional Development Workshops
59
60
61
62
63
64
65
n EXHIBITS
Hall Happenings
2014 Exhibitors
67
70
n COMMITTEES
Conference Committee • Council of Chairs
Program Subcommittees
Special Teams
73
74
76
n CONFERENCE INFORMATION
Registration Information • Parking • Dress Code
77
Lodging78
Getting Around During I/ITSEC
79
Publications & Media
80
Association Sponsors
81
Safety & Security
82
Golf Tournament
83
5K Run
84
n I/ITSEC 2015
I/ITSEC 2015 Save the Date
Call for Papers and Tutorials
Serious Games Showcase & Challenge
Black Swan
85
86
87
88
n ABSTRACT BOOK
89
T H E W O R L D ’ S L A R G E S T M O D E L I N G & S I M U L AT I O N E V E N T 1
WELCOME
ELCOME ATTENDEES OF I/ITSEC 2014:
On behalf of the United States Army, this year’s Lead Service; our sponsoring organization, the National Training
and Simulation Association; the Service Executives and their Principals; and the 200-plus volunteers from the
Army, Navy, Marine Corps, Air Force, Coast Guard, Office of the Secretary of Defense, Industry, and Academia,
it is my distinct honor and great pleasure to welcome you to the 2014 Interservice/Industry Training, Simulation
and Education Conference.
With continuing and emerging threats around the world, our military Services and government organizations—
both here in the US and abroad—are being asked to do more with less. Now more than ever before, modeling and
simulation technologies and methods can help organizations train to high-consequence scenarios, exercise and
verify readiness for an individual or a group, and test countless new ideas and innovations in a shorter amount
of time.
The theme selected for this year’s Conference is “Trained and Mission Ready: Deter, Defend, Defeat,” which
underscores this crucial need for readiness—where people are engaged, equipped, and perfectly positioned to
give their best in performance of the mission. Building on the Conference’s military-focused history, I/ITSEC has
emerged as the predominant cross-industry forum, drawing an increasing number of attendees from industries
including healthcare, energy, transportation and manufacturing, who are in search of new innovations to change
the way people learn and perform, to drive down costs, and increase their ability to compete.
The volunteer members of our six Subcommittees, the Tutorial Board and the Conference Committee have spent
this year finding the best technical papers (149), tutorials (21), educationally-focused special events (12), and
a range of special programs (including Future Leaders Pavilion, Serious Games Showcase and Challenge, and
Warfighters’ Corner). As a result, I am certain you will find the 2014 Program to be rich, diverse, and highly
relevant. All Monday tutorials, Friday professional development workshops, and many of our paper sessions are
available for continuing education units and continuous learning credits. Eligible sessions are identified in the
program guide and additional information is available on the www.iitsec.org website.
Be sure to visit the I/ITSEC exhibit hall, which hosts the largest display of training systems capabilities in the world.
Over 500 exhibitors will present leading-edge technology and innovative concepts. This year, we also welcome
continued growth in the number of international attendees. With nearly 2000 attendees from over 70 countries
expected, we have created even more opportunities for networking to promote dialogue and idea exchange.
Finally, I want to express my sincere thanks and appreciation to our dedicated volunteers and their sponsors.
Their commitment and support has made I/ITSEC 2014 a reality and ensured this Conference remains the premier
professional development event across the globe for the training and simulation professional. As you attend the
events and walk the exhibit floor over the next few days, please take a moment to thank the authors and other
volunteers who have helped make this program a success.
Sincerely,
Janet J. Spruill
2
2 0 1 4 I N T E R S E R V I C E / I N D U S T RY T R A I N I N G , S I M U L AT I O N & E D U C AT I O N C O N F E R E N C E
KEYNOTE SPEAKERS
GENERAL DANIEL B.
ALLYN, USA
35th Vice Chief of Staff
of the Army
Anthony Smeraglinolo
President and Chief
Executive Officer
Engility Corporation
GENERAL DANIEL B. ALLYN assumed duties as the 35th Vice Chief of Staff of the Army, August 15,
2014.
General Allyn is a native of Berwick, Maine, and a graduate of the United States Military Academy
at West Point, New York. He previously served as the Commander of the United States Army Forces
Command, Fort Bragg, NC.
He also served as the Commanding General, XVIII Airborne Corps and Commanding General, 1st
Cavalry Division, “America’s First Team,” including duty as Commanding General, Combined Joint
Task Force-1 and Regional Command East in Afghanistan. General Allyn has also served as the Chief
of Staff, and later, Deputy Commanding General of XVIII Airborne Corps, including duty as Chief of
Staff, Multi-National Corps Iraq. His joint assignments include the Joint Improvised Explosive Device
Defeat Organization and the Joint Operations Directorate, J-3. Prior to his Joint assignments, he
served as Commander, 3rd Brigade Combat Team, 3rd Infantry Division (Mechanized), culminating
with service during Operation Iraqi Freedom. Prior to serving in the “Marne Division,” General Allyn
served two tours of duty with the 82nd Airborne Division, two years with the 2nd Infantry Division,
and three tours of duty with the 75th Ranger Regiment.
General Allyn’s previous duties include command at the platoon through division level and staff
assignments at the battalion through Joint Staff level. He served an overseas assignment in Korea and
operational deployments for Operation Urgent Fury in Grenada, two peacekeeping deployments to the
Sinai Peninsula in Egypt, Operation Just Cause in Panama, Operation Desert Storm in Saudi Arabia,
and Operations Desert Spring and Enduring Freedom in Kuwait, two tours in support of Operation
Iraqi Freedom, and most recently was deployed to Afghanistan in support of Operation Enduring
Freedom XII.
He is a graduate of the Naval War College at Newport, Rhode Island, where he earned a Master of
Arts degree in Strategic and National Security Studies.
General Allyn’s awards and decorations include the Distinguished Service Medal, the Silver Star,
three Defense Superior Service Medals, three Legions of Merit, the Bronze Star Medal, two Defense
Meritorious Service Medals, six Meritorious Service Medals, the Joint Service Commendation Medal,
four Army Commendation Medals, three Army Achievement Medals, the Combat Infantryman Badge
(with Star), the Expert Infantryman Badge, Master Parachutist Badge (with Bronze Star), the Ranger
Tab, the Pathfinder Badge, the Joint Chiefs of Staff Identification Badge.
MR. SMERAGLINOLO has spent more than 30 years in executive positions within the government
services industry. He began his career holding leadership positions in finance, program management,
business development, and operations at Harris Corporation for more than 25 years. From 2005 to
2008, he served as President of the L-3 Services Group’s Intelligence Solutions Division. From 2008 to
2010, Mr. Smeraglinolo was President of the Global Stabilization and Development Solutions division
for Dyncorp International.
In 2010, Mr. Smeraglinolo rejoined L-3 and served as Executive Vice President, L-3 Services Group,
and Acting President, Command & Control Systems and Software division, L-3 Services Group. From
2010 to 2012, he drove the strategy leading to the spin-off of L-3’s six government services business
units to form Engility in July 2012. Mr. Smeraglinolo has been serving as Engility’s President and Chief
Executive Officer since the spin-off.
After the spin-off, Mr. Smeraglinolo spear-headed a major streamlining and cost reduction initiative
to ensure Engility’s customers received the same outstanding service they have come to expect, but
now in a price-disruptive manner. Mr. Smeraglinolo’s vision has enabled Engility to become a market
leader in providing subject matter experts to solve their customers’ most complex problems within
their tightly constrained budgets.
Engility is a leading provider of best-in-class training capabilities, specialized technical consulting,
program and business support services, engineering and technology lifecycle support, and supply
chain and logistics management services.
Engility is ranked 30th in Washington Technology’s Top 100 largest contractors in the Government
marketplace and has been a publicly traded company on the New York Stock Exchange since its spinoff in July 2012.
Mr. Smeraglinolo has a Bachelor’s degree in Business Management from Fairfield University in
Connecticut and a Master of Business Administration from Florida Institute of Technology.
CONFERENCE LEADERSHIP
Conference Chairs
Ron Smits
I/ITSEC 2014
Conference Chair
Janet Spruill
I/ITSEC 2014
Program Chair
RON SMITS is general manager for Training & Readiness Analysis for Engility Corporation and part of
the corporate Center of Excellence where he develops technical solutions that differentiate Engility as an
industry leader in its core capabilities and services. Previously, he led the cross-company organization
responsible for the development and delivery of all external training and readiness analyses solutions.
Responsible for building shareholder value, customer satisfaction, accelerating business growth,
and personnel recruiting and retention, his duties included P&L, budgeting, forecasting, division
administration, business development and operations. A command pilot with over 3000 hours, Ron
served as a Dutch Air Force fighter pilot, holding positions such as Chief, Wing Operations, F-16 Training
Program Manager for the USAF’s Air Combat Command, and various leadership positions at the EuroNATO Joint Jet Pilot Training Program. He transitioned to the training industry when he joined DRC (later
acquired by Engility Corporation) and held assignments as program manager, business unit director,
and division director. Ron has been active in I/ITSEC since 1999, first as an author and exhibitor and
subsequently as subcommittee member and chair, special events coordinator and program chair. He
holds a B.S. from the KMA, an M.A. from Old Dominion University and is a graduate of the Greater
Boston Executive Program at MIT’s Sloan School of Management.
JANET SPRUILL is the senior director of Business Development and the Principal Learning Solutions
Architect for Serco, Inc. where she guides the design and implementation of advanced learning and
simulation services and solutions for the U.S. Department of Homeland Security, the Department of
Defense, and various defense, transportation and health organizations worldwide. Janet specializes in
new concept development for large-scale training deployment, immersive and virtual environments,
performance support systems, knowledge management, and rapid end-user adoption. She brings a
unique insight and unparalleled ability to convey highly technical concepts in relevant and practical
ways. Janet is a Certified Human Capital Strategist (HCS) as granted by the Human Capital Institute
and is an accomplished writer and speaker, presenting on topics that include “Adaptive Leadership,”
“Crowdsourcing Expert Performance,” and “Overcoming Barriers to Virtual World Adoption.” She is a
six-time I/ITSEC author, and has held progressive I/ITSEC leadership positions since 2000. Janet chaired
three I/ITSEC subcommittees and served as special events coordinator in 2012 before assuming the role
of 2014 I/ITSEC Program Chair.
Conference Sponsor
RADM James Robb,
USN (Ret.)
President National
Training and
Simulation
Association
4
Following graduation from Rensselaer
Polytechnic Institute, designation as a
Naval Aviator and training in the F-14
Tomcat, Admiral Robb deployed nine
times across the globe accumulating over
5000 hours and 1000 carrier landings.
Following a tour flying Russian fighters in
the Nevada desert, he commanded Fighter
Squadron Fifty One, Carrier Air Wing
Nine, the Navy Fighter Weapons School
(TOPGUN), and Carrier Strike Group
Seven. As a Flag Officer he managed all
Naval Aviation Programs (N980) and was
the Director of Navy Readiness (N43).
Following 9/11, he joined USCENTCOM
as the Director of Plans (J5) deploying
to the Middle East in support of combat
operations. Retiring in 2006, he built
a successful small consulting business
before joining the National Training and
Simulation Association as President in
June 2012.
Lt Gen Lawrence
P. Farrell, Jr., USAF
(Ret.)
President National
Defense Industrial
Association
Prior to his retirement from the Air
Force in 1998, General Farrell served as
the Deputy Chief of Staff for Plans and
Programs, Headquarters U.S. Air Force,
Washington, D.C. He was responsible for
planning, programming and manpower
activities within the corporate Air Force
and for integrating the Air Force’s future
plans and requirements to support
national security objectives and military
strategy. A command pilot with more than
3,000 flying hours, he flew 196 missions
in Southeast Asia, and commanded the
401st Tactical Fighter Wing, Torrejon Air
Base, Spain. He is a graduate of the Air
Force Academy with a BS in Engineering
and an MBA from Auburn University.
General Farrell became President of the
National Defense Industrial Association
in September of 2001.
EXECUTIVES
MG Jon Maddux, USA
U.S. Army Program
Executive Officer for
Simulation, Training and
Instrumentation
CAPT Wes Naylor, USN
Commanding Officer, Naval
Air Warfare Center, Training
Systems Division
Army Service Executive (Lead Service)
MG JON MADDUX is responsible for simulation, training and test/instrumentation
to support the U.S. Army. PEO STRI annually executes a multi-billion dollar program
with a workforce of 1200 employees. In
addition to his responsibilities as the PEO,
MG Maddux is dual-hatted as the Head of
Contracting Activity authority. He oversees
approximately 1600 contracts valued at
over $28 billion. MG Maddux has held key
positions in the Army, including Assistant
to the Principal Military Deputy, Assistant
Secretary of the Army Acquisition, Logistics and Technology (ASA(ALT)); Deputy
Commanding General, Support, Combined
Security Transition Command-Afghanistan; PEO Ammunition/Commanding
General, Picatinny Arsenal; Chief of Staff,
Office of the ASA(ALT); Director for Army
Evaluation Task Force Integration, Directorate for Program Manager for Future
Combat Systems (Brigade Combat Team);
and Project Manager for Future Combat
Systems Network Systems Integration.
MG Maddux entered the Army as an enlisted Soldier in 1976 and he has earned
several awards, including four Legion of
Merit awards and the Bronze Star Medal. MG Maddux graduated from the U.S.
Army War College and has three Masters
degrees.
Navy Service Executive
CAPT WES NAYLOR, USN: The Naval Air
Warfare Center Training Systems Division
(NAWCTSD) is the Navy’s principal center for modeling, simulation, and training
systems technologies. The command provides training solutions and research for
a wide spectrum of military programs, including aviation, surface & undersea warfare, and other specialized requirements.
CAPT Naylor leads a workforce of more
than 1,000 scientists, evaluators, engineers, technicians, logisticians, contracting specialists, and support personnel. A
native of McLean, Virginia, CAPT Naylor
is a graduate of George Mason University
where he earned his Bachelors of Science
degree in Government and a 2007 graduate
of the National War College, where he was
awarded a Master’s Degree in National Security Strategies. As a Naval Aviator, CAPT
Col (Sel) Walter Yates, USMC
Program Manager, Marine
Corps Systems Command
PM Training Systems
Col Dan Marticello, USAF
Chief, Simulators Division,
Air Force Materiel
Command
Naylor flew the Navy’s P-3 Orion maritime
patrol aircraft. He also has experience
serving in various acquisition and staff
positions. CAPT Naylor served as the Executive Officer of NAWCTSD for two years
prior to assuming command in June 2014.
His awards include the Meritorious Service
Medal, Air Medal, Navy Commendation
Medal, Navy and the Achievement Medal.
Marine Corps Service Executive
LtCol WALTER YATES, Program Manager, Training Systems: As the Marine Corps
Systems Command Program Manager for
Training Systems LtCol Yates is responsible for managing a workforce of over 150
personnel in the acquisition and sustainment of training systems used throughout
the Marine Corps. LtCol Yates graduated
from Texas A&M University in 1990 with
a B.S. in Mechanical Engineering Technology. He graduated from the Naval Postgraduate School in 2004 with a M.S. in the
field of Modeling, Virtual Environments
and Simulation. As a Field Artillery officer he served ashore and deployed afloat
in billets including forward observer, platoon commander, fire direction officer, and
battery commander. LtCol Yates served as
the officer in charge of the Battle Simulation Center for MAGTF Training Command
from 2004 to 2007. From 2007 to 2011 he
served on the Program Manager for Training Systems staff, including a year as the
Marine Corps Systems Command Liaison
Officer to the Command Element of Multinational Force-West, Iraq. In 2011 LtCol
Yates returned to Marine Corps Systems
Command, as the Deputy for Modeling
and Simulation. He completed his assignment as a Secretary of Defense Corporate
Fellow at Norfolk Southern Corporation
prior to reporting for duty as PM TRASYS.
Air Force Service Executive
Col DAN MARTICELLO is Chief of the Simulators Division, AF Life Cycle Management
Center, Wright-Patterson AFB, Ohio. He directs 400 employees in the acquisition and
sustainment of more than 40 USAF and 10
foreign AF training systems. Col Marticello graduated from the USAF Academy and
earned an M.S. degree at the University of
Colorado. He graduated from USAF Under-
T H E W O R L D ’ S L A R G E S T M O D E L I N G & S I M U L AT I O N E V E N T Frank C. DiGiovanni, SES
Acting Deputy Assistant
Secretary of Defense
(Readiness)
graduate Pilot Training, Test Pilot School,
and Air Command and Staff College. His
assignments have included serving as flight
test program manager for the F-15 Joint Helmet-mounted Cueing System and AIM-9X
missile; overseeing F-117A developmental
flight test; working on the HQ AFMC Commander’s Action Group; managing upgrades
to the F-15C/D air superiority fleet; and commanding a squadron executing classified acquisition programs. He attended MIT as an
AF Fellow, and then managed a C-17 capability upgrade program. He deployed with
NATO Training Mission-Afghanistan, where
as Chief of Plans and Operations, he created
the Afghan Defense Acquisition & Resource
Management Institute. He was then assigned
as Chief of the HQ AFMC Capabilities and Requirements Division, managing the AF Agile
Combat Support Core Function portfolio.
Senior Advisor for Readiness and Training
FRANK C. DIGIOVANNI serves as the Acting Deputy Assistant Secretary of Defense
(Readiness). His responsibilities include
policy and oversight of military training
readiness and capability modernization.
He leads the Department’s $4.3B Combatant Commander Exercise and Engagement
and Training Transformation, the sustainment of military training ranges, the development of Live, Virtual and Constructive
Training Standards and Architectures, the
Advanced Distributed Learning Initiative,
the creation of a “virtual world” training
capability, and ensures training is properly incorporated into major acquisition
programs. He also serves as a senior DoD
training member on the Modeling and Simulation Steering Committee and collaborates with interagency partners to develop
training strategy and policy to ensure Government civilians and Service members are
better prepared to conduct reconstruction
and stabilization operations. He oversees
efforts and policies associated with sustaining access to DoD’s land, air and sea
training space and for developing policy,
strategic communication and the research
agenda associated with energy infrastructure and its impact on the ability of the
Department to conduct readiness training
activities.
5
PRINCIPALS & ADVISOR
Service Principals
Traci Jones
Army
G-7 Training and Support Services,
U.S. Army Program Executive
Office, Simulation, Training and
Instrumentation
(PEO STRI)
Diana Teel
Navy
Industry Outreach, Naval Air
Warfare Center Training Systems
Division (NAWCTSD)
Martin Bushika
Marine Corps
Assistant Program Manager for
Program Management, Marine Corps
Systems Command, PM Training
Systems
(MARCORSYSCOM PM TRASYS)
Tony DalSasso
Air Force
Chief Engineer, Simulators Division,
Air Force Materiel Command
(AFMC)
OSD Principal
Education and Training Advisor
Brent Barrow
Deputy Director, Training Readiness
and Strategy ODASD (Readiness)
6
VADM Al Harms, USN (Ret.)
Vice President (Emeritus)
for Strategy, Marketing,
Communications and Admissions,
University of Central Florida
Agenda
PRE-CONFERENCE AGENDA
WEDNESDAY, 26 NOVEMBER 2014
TIME
LOCATION
0730
Exhibitor Registration Open
1700
Exhibitor Registration Close
S220 of the South Concourse
THURSDAY, 27 NOVEMBER 2014
CLOSED FOR THANKSGIVING
FRIDAY, 28 NOVEMBER AND SATURDAY, 29 NOVEMBER 2014
TIME
LOCATION
0730
1800
Exhibitor Registration Close
SUNDAY, 30 NOVEMBER 2014
TIME
LOCATION
0730
1200
Conference Registration Open
1200
All Satellite Registrations Open
Multiple Hotels (see page 78)
1800
All Registrations Close
Dress Code
BRANCH
CONFERENCE AND GENERAL SESSIONS
BANQUET
Army
ACUs or Duty Uniform
Army Blue (Army Evening Mess optional)
Marine Corps
Service “C”
Evening Dress (Dress Blue “B” or Service “A” optional)
Navy
Service Khaki, Navy Service Uniform
Dinner Dress White (Service Dress White optional)
Air Force
Short or Long Service Blues
Service Dress Blue with tie and jacket (Mess Dress optional)
Coast Guard
Tropical Blue Long
Civilian
Business attire
Black tie (optional)
LEGEND (one or more of the following may appear on this page).  Nominated for Best Paper Award  Honorable Mention for Best Paper Award
% Game-related Subject Matter
ñ International Author  Continuing Education Units (see pg. 13) h Medical-related Subject Matter
T H E W O R L D ’ S L A R G E S T M O D E L I N G & S I M U L AT I O N E V E N T Ï Cyber Security
Mobile
7
AGENDA
MONDAY, 1 DECEMBER 2014
TIME
LOCATION
0730
Conference and Exhibit Registration Open
0730
Multiple Hotels
0830 - 1000
TUTORIALS (Synopses begin on page 16) 
Introduction to HLA (1430) ñ
Room S320A
Modeling and Simulation 101: DoD Management (1421)
Room S320GH
Fundamentals of Modeling and Simulation (1416)
Room S310AB
U.S. Export Controls 2014: Continued Change for International Modeling and Simulation
(1441)
Room S320B
Fundamentals of Adaptive Intelligent Tutoring Systems for Self-regulated Learning
(1413)
Room S320C
My Training Talks with Me! Getting Started with Dialogue Agents (1418)
Room S320E
Transmedia Learning in the Wild: Supporting Military Training Through Story-driven
Engagement (1442)  %
Room S320D
1030 - 1200
SIGNATURE EVENT: Congressional Modeling and Simulation Event (page 27)
Room S330ABC
1245 - 1415
TENA/JMETC: Testing and Training for Mission Readiness (1436)
Room S320A
Training Simulation Fidelity: Establishing Preferences, Priorities, and Optimizing
Trade-offs (1402)
Room S320GH
Model Verification and Validation Methods (1407)
Room S310AB
Establishing the Value of Simulation (VoS) for Collective Training: Accent on Methodology Room S320B
(1417) 
Cognitive Neuroscience for Military Education and Training (1412)
Room S320C
Using HTML5 to Develop IMI, Simulations, and Games (1422) %
Room S320E
Scenario Story Development in Military Games/Simulations (1426) %
Room S320D
1400
Break
1400
Exhibits Open
1430 - 1600
Distributed Interactive Simulation 101: The Basics (1439)
Room S320A
Simulation Conceptual Modeling Theory and Application (1409)
Room S320GH
Planning and Execution of a Large Multi-architecture Distributed Event (1420)
Room S310AB
Defense-related Applications of Discrete Event Simulation (1408)
Room S320B
Augmenting Virtual Worlds with Intelligent Tutors (1437)
Room S320C
Speech-based Interaction: Myths, Challenges, and Opportunities (1429) ñ %
Room S320E
Bridging the Gap: How to Build Effective Game-based Training (1425) %
Room S320D
1800
Exhibits Close
1800
All Registration Stations Close
ñ International Author 8
Exhibit Hall
 Continuing Education Units (see pg. 13) h Medical-related Subject Matter
Ï Cyber Security
Mobile
AGENDA
TUESDAY, 2 DECEMBER 2014
TIME
LOCATION
0700
0700
Multiple Hotels
0745
Pre-Ceremony Music
Central Florida VFW Musical Ensemble
Hyatt Regency,
Windermere Ballroom
0830 - 1000
OPENING CEREMONIES
Hyatt Regency/Windermere
Call to Order
Presentation of Colors
National Anthem
Invocation
OPENING REMARKS
Ron Smits, 2014 Conference Chair
KEYNOTE ADDRESSES
General Daniel Allyn, USA, Vice Chief of Staff of the Army
Anthony Smeraglinolo, President and Chief Executive Officer, Engility Corporation
1000 - 1030
Break
1030 - 1200
SIGNATURE EVENT: General/Flag Officer Panel (page 28)
Hyatt Regency/Windermere
1200
Exhibits Open
Exhibit Hall
1200 - 1330
Lunch (Opening of Exhibits and Lunch will occur at 1200 or upon adjournment of the
General/Flag Officer Panel)
South B
1400 - 1530
PAPER SESSIONS
(Title/Author List begins on page 50. Session schedules for this timeframe are on page 45.)
Rooms S320A-F
1400 - 1530
SIGNATURE EVENT: Enabling Combat Readiness: Live, Virtual, Constructive (LVC) Training
(page 29)
Room S320GH
1400 - 1530
COMMUNITY OF INTEREST/PROGRAM BRIEF: Warfighter Human Dynamic Skill: Results
from an Innovative Training and Evaluation Program (page 41)
Room S330C
1530 - 1700
FOCUS EVENT: Warfighters Corner (page 33)
Exhibit Hall, Booth 2681
1530 - 1600
Break
1600 - 1730
PAPER SESSIONS
Rooms S320A-F
1600 - 1730
FOCUS EVENT: Ignite! (page 34)
Room S320GH
1600 - 1730
COMMUNITY OF INTEREST/PROGRAM BRIEF: Early Synthetic Prototyping: Force 2025
Capability Development Driven by Soldiers Using Games (page 41)
Room S330C
1700 - 1830
Exhibitor Networking Event
Exhibit Hall
1800
1830
Exhibits Close
Mobile
9
AGENDA
WEDNESDAY, 3 DECEMBER 2014
TIME
LOCATION
0700
0800 - 1700
COMMUNITY OF INTEREST/PROGRAM BRIEF: Experience Application Program
Interface (xAPI) PlugFest (page 41)
Room S310A
0830 - 1000
PAPER SESSIONS
Rooms S320A-F
0830 - 1000
SIGNATURE EVENT: I/ITSEC 2014 Fellow Paper Presentation: Metasimulation (page 30)
Room 320GH
0830 - 1000
COMMUNITY OF INTEREST/PROGRAM BRIEF: Interoperable Performance Assessment
within Live-Virtual-Constructive Events (page 41)
Room S330C
0930
Exhibits Open
Exhibit Hall
1000 - 1030
Break
1030 - 1200
PAPER SESSIONS
Rooms S320A-F
1030 - 1200
FOCUS EVENT: Improving Patient Safety — Does Simulation Help? (page 35)
Room S320GH
1030 - 1200
COMMUNITY OF INTEREST/PROGRAM BRIEF: Simulation & Training in Europe – the
Government Perspective from a Number of Nations (page 42)
Room S330C
1030 - 1200
1200 - 1330
Lunch South B
1400 - 1530
PAPER SESSIONS
Rooms S320A-F
1400 - 1530
FOCUS EVENT: Mind Over Matter: The Power of Mental Fitness, Emotional Intelligence,
and “Soft Skills” (page 36)
Room S320GH
1400 - 1530
COMMUNITY OF INTEREST/PROGRAM BRIEF: Veterans Health Administration: Next
Generation Simulation-based Training Initiatives (page 42)
Room S330C
1530 - 1600
Break
1600 - 1730
PAPER SESSIONS
Rooms S320A-F
1600 - 1730
FOCUS EVENT: A Virtual World Cup For Collaborating, Understanding & Partnering (page 37)
Room S320GH
1800
1830
Exhibits Close
ñ International Author 10
 Continuing Education Units (see pg. 13) h Medical-related Subject Matter
Ï Cyber Security
Mobile
AGENDA
THURSDAY, 4 DECEMBER 2014
TIME
LOCATION
0700
0830 - 1000
PAPER SESSIONS
Rooms S320A-F
0830 - 1000
FOCUS EVENT: The Quantified Warrior: Enhancing Assessment in the Military (page 38)
Room S320GH
0830 - 1000
COMMUNITY OF INTEREST/PROGRAM BRIEF: Geospatial Environmental Database
Standards Forum (page 42)
Room S330C
0930
Exhibits Open
Exhibit Hall
1000 - 1030
Break
1030 - 1200
PAPER SESSIONS
Rooms S320A-F
1030 - 1200
SIGNATURE EVENT: Training the Human Dimension in a Time of Austerity—
The Opportunity of Technology (page 31)
Room S320GH
1030 - 1200
COMMUNITY OF INTEREST/PROGRAM BRIEF: Trends in Instructional Systems Design
(page 42)
Room S330C
1030 - 1200
1030 - 1200
SPECIAL PAPER SESSION: Presentations from Future Leaders Pavilion (page 60)
Room S330D
1200 - 1330
Lunch
South B
1300
AWARDS CEREMONY: Serious Games Showcase & Challenge
Warfighters Corner Stage
1330 - 1500
PAPER SESSIONS
Rooms S320A-F
1330 - 1500
FOCUS EVENT: The Intelligence Community’s Challenge to Training Innovators (page 39)
Room S320GH
1345
AWARDS CEREMONY: Future Leaders
Warfighters Corner Stage
1500
Exhibit Hall and Registration Close
Exhibit Hall
1800
Hosted Reception sponsored by Lockheed Martin
Hyatt Regency, Windermere Foyer
Hyatt Regency
Windermere Ballroom
1900
&
Reception
Awards
banquet
Conference Awards Banquet
Dinner Music provided by Bob Dehne on the Vibraphones
Best Paper Award Presentation
RADM Fred Lewis Postgraduate I/ITSEC Scholarship
Ceremonial Music provided by Voices of Liberty
Passing of the Flag for I/ITSEC 2015
FRIDAY, 5 DECEMBER 2014
TIME
PROFESSIONAL DEVELOPMENT WORKSHOPS (Synopses can be found on pages 65-66)
0800 – 1200
Certified Modeling & Simulation Professional (CMSP) Exam Preparation
Room S330C
0800 – 1200
Live-Virtual-Constructive (LVC) Interoperability Techniques
Room S330D
0800 – 1200
Modeling & Simulation for Acquisition
Room S330G
0800 – 1200
Back to the Future: Workshop on Applying the UJTL: Linking Missions, METLs, DRRS, and
Lessons Learned to Capabilities
Room S330H
0800 – 1200
Applied Earned Value Management
Room S330F
1300 – 1700
Seamless Mobile Learning and Simulations
Room S330F
0800 – 1700
Serious Game Design Tutorial
Room S330E
Mobile
11
C O N T I N U I N G E D U C AT I O N U N I T S / C O N T I N U O U S L E A R N I N G P O I N T S
Continuing Education Units: An I/ITSEC Opportunity
“CEUs are a convenient and efficient way
to keep track of my participation in
professional development activities.”
Continuing Education Units (CEU) were established in 1970 to
create a unit of measurement to quantify continuing education and
training activities. CEUs apply to technical and educational settings
such as I/ITSEC. The primary focus of I/ITSEC is to highlight innovative implementation of simulation and education technologies
as tools to achieve cost efficient training and increased military
readiness. Therefore CEUs are offered for all Tutorials, selected Paper
Sessions, and the Post-conference Professional Development Workshops.
CEUs are being sponsored and maintained by the University of
Central Florida, Division of Continuing Education.
Why should I earn CEUs at I/ITSEC?
• Participation in the tutorials, papers and/or Post-conference
Workshops for CEU credit reinforces your commitment to remain current in the evolving technologies relating to training
and simulation.
• The CEU transcript indicates your active participation in the
technical program of the conference to your employer.
• Previous attendees have indicated that CEUs have assisted them
in securing approval to attend the conference.
Who may attend the Sessions and who may receive
CEUs at I/ITSEC?
• Tutorials: Open to ALL ATTENDEES; only PAID CONFERENCE
ATTENDEES may receive the CEU credits.
• Papers: PAID CONFERENCE ATTENDEES may attend all Paper
Sessions and may receive the CEU credits for doing so at no
additional charge.
• Professional Development Workshops: Open to ALL. Must register
to attend and PAID CONFERENCE ATTENDEES may receive
CEUs for doing so at no additional charge.
How do I receive CEUs at I/ITSEC?
1. Register as a PAID CONFERENCE ATTENDEE. This automatically triggers your eligibility for CEU.
2. Attend any Tutorial (all are CEU eligible) or any Paper presentation marked by a book symbol ().
3. Attend the Professional Developmental Workshops on Friday.
These are half-day sessions on various subjects. There is no
additional fee to attend, but participants need to register in
advance. All workshops are CEU eligible.
4. Check-in as directed during the CEU eligible sessions.
5. Your CEU transcript will come to you via the University of
Central Florida, Division of Continuing Education. Ten contact
hours equate to one CEU credit.
Contact Maria Cherjovsky at (407) 882-0247 or
mariac@mail.ucf.edu for additional information
Continuous Learning Points (CLPs)
The U.S. Department of Defense (DoD) acquisition workforce members are expected to earn Continuous Learning Points (CLPs) to stay current in leadership and functional
acquisition skills that augment the minimum education, training and experience standards established for certification purposes within their acquisition career fields. It is
each acquisition member’s responsibility to meet the goal of 40 CLPs each year and to meet the mandatory requirement of 80 CLPs every two years. Acquisition Professional
Activities are allowed to count toward CLPs. CLPs are awarded in accordance with DoD-wide guidelines as augmented by Service-specific policies.
I/ITSEC provides an excellent opportunity for the DoD acquisition workforce members to earn mandatory CLPs.
EARNING THE CMSP DESIGNATION WILL:
• Demonstrate expertise in the field of M&S to your
employer and the larger M&S community
• Provide opportunities for professional advancement
Requirements include 3-8 years of work experience
(depending on level of highest collegiate degree), 3
professional letters of reference, and successful
completion of an online examination.
The completion of CEUs has long been used to demonstrate
dedication and career interest to supervisors, employers
or rating officers. CEUs earned at I/ITSEC can be applied
towards CMSP Recertification.
12
CMSP Applicants now have a choice between
CMSP-Technical and CMSP-Management exams.
TO LEARN MORE ABOUT THE REQUIREMENTS AND TO APPLY,
PLEASE VISIT WWW.SIMPROFESSIONAL.ORG OR CONTACT
PATRICK ROWE AT PROWE@NDIA.ORG.
CONVENTION CENTER
South Concourse
Orange County Convention Center, Orlando, Florida
To paraphrase from Oh Brother, Where Art Thou?, the walking distance from the OCCC South Concourse is a “geographical
oddity, it’s five minutes from everywhere”. A five minute walk will take you...
• from the South Concourse to the North Concourse
• from the South Concourse to the Hilton
• from the South Concourse to the Hyatt Regency, formerly Peabody Orlando
• from the South Concourse to the Rosen Center
(In comparison, a walk from Hall A to Hall F in the West Concourse takes about ten minutes.)
Tutorials
S320A-E, G & H
Paper Sessions
S320A-F
Signature Events
S320GH
Focus Events
S320GH
Tutorials
S310AB
Community of Interest/
Program Brief
S310A
International Pavilion
S310E-H
LEVEL 2 (Entry Level)
To Exhibit Floor
Congressional Event
S330ABC
Community of Interest/Program Brief
S330C & S330D
Professional Development Workshops
S330C-H
Practice Rooms (Sun-Thursday)
S330E-H
To Exhibit Floor
First Aid
Onsite Registration
& Property Check
Self-Registration
Hyatt Regency
Hilton
Rosen Centre
H YAT T R E G E N C Y
RECREATION LEVEL
MEZZANINE LEVEL
B-Line Diner
ENTRY LEVEL
Opening Ceremonies
Keynote Speakers
General/Flag Officer Panel
Closing Banquet
CONVENTION LEVEL
Bus Drop Off
Hyatt Regency Satellite
Registration Desk
14
Tutorials
TUTORIALS
M ond a y, 1 De c e m b e r 2014
 All Tutorials are eligible for CEU/CLP credits. (See page 12) 
OPEN TO ALL ATTENDEES
I/ITSEC Tutorials are designed to serve three purposes:
• Provide foundational educational material, including material essential to prepare for Certification as a Modeling and Simulation Professional (CMSP).
• Serve as a refresher and more advanced learning opportunity for those seeking to maintain their
certification.
• Bring topics of special interest in Training, Simulation and Education to I/ITSEC attendees.
FOR YOUR CONVENIENCE,
FOOD STATIONS ARE OPEN
ON LEVEL TWO DURING
MONDAY SESSIONS.
TUTORIALS SCHEDULE
ROOM
TRACK/CHAIR
0830 - 1000
1245 - 1415
1430 - 1600
S320A
Track 1:
Architectures
Introduction to HLA (1430) ñ
TENA/JMETC: Testing and
Training for Mission Readiness
(1436)
Distributed Interactive Simulation
101: The Basics (1439)
Modeling and Simulation 101:
DoD Management (1421)
Training Simulation Fidelity:
Establishing Preferences,
Priorities, and Optimizing
Trade-offs (1402)
Simulation Conceptual Modeling
Theory and Application (1409)
Fundamentals of Modeling
and Simulation (1416)
Model Verification and
Validation Methods (1407)
Planning and Execution
of a Large Multi-architecture
Distributed Event (1420)
U.S. Export Controls 2014:
Continued Change for
International Modeling and
Simulation (1441)
Establishing the Value of
Simulation (VoS) for Collective
Training: Accent on Methodology
(1417) 
Defense-related Applications of
Discrete Event Simulation (1408)
Fundamentals of Adaptive
Intelligent Tutoring Systems
for Self-regulated Learning (1413)
Cognitive Neuroscience for
Military Education and Training
(1412)
Augmenting Virtual Worlds
with Intelligent Tutors (1437)
My Training Talks with Me!
Getting Started with Dialogue
Agents (1418)
Using HTML5 to Develop IMI,
Simulations, and Games (1422) %
Speech-based Interaction: Myths,
Challenges, and Opportunities
(1429) ñ %
Transmedia Learning in the Wild:
Supporting Military Training
Through Story-driven Engagement
(1442)  %
Scenario Story Development
in Military Games/Simulations
(1426) %
Bridging the Gap: How to Build
Effective Game-based Training
(1425)  %
Robert Lutz
Track 2:
S320GH Management Basics
Larry Skapin
Track 3:
S310AB Technical Basics
David Milewski
S320B
Track 4:
Step By Step
James Wall, Ph.D.
S320C
S320E
S320D
Track 5:
Cognitive ‘X’
Denise Nicholson,
Ph.D.
Track 6:
Interactions
Thomas Mastaglio,
Ph.D.
Track 7:
Engaging Learners
Lisa Scott Holt, Ph.D.
LEGEND (one or more of the following may appear on this page). The number in parentheses following Tutorial Title is the ID tracking number.
 Nominated for Best Tutorial Award ñ International Author % Game-related Subject Matter
TUTORIALS
The High-Level Architecture
(HLA) is the leading international standard for simulation
interoperability. It originated
Introduction to HLA
in the defense communities
(1430) ñ
but is increasingly used in
other domains. This tutorial
gives an introduction to the
HLA standard. It describes
the requirements for interoperability, flexibility, composability and reuse and how HLA meets them. It also describes the
new features of the most recent version: HLA Evolved (IEEE1516-2010). Finally it provides some recent experiences of the
use of HLA in NATO M&S groups as well as an overview of recent evolution of Federation Object Models for military platform
simulation. This tutorial is intended for all audiences; however,
some familiarity with basic principles of distributed computing
is recommended.
TRACK 1: ARCHITECTURES
0830 – 1000 • ROOM S320A
TU-1
Presenters:
BJÖRN MÖLLER is the vice president and co-founder of Pitch Technologies, the leading supplier of tools for HLA Evolved, 1516-2000 and HLA
1.3. He leads the strategic development of Pitch HLA products. He serves
on several HLA standards and working groups and has a wide international contact network in simulation interoperability. He has twenty years
of experience in high-tech R&D companies, with an international profile
in areas such as modeling and simulation, artificial intelligence and Webbased collaboration. He is currently serving as the vice chairman of the
SISO HLA Evolved Product Support Group. He also serves as the chairman of the SISO RPR FOM Product Development Group.
ROBERT LUTZ is a principal staff scientist at The Johns Hopkins University Applied Physics Laboratory (JHU/APL). He has over 34 years of experience in the design, implementation, and evaluation of computer modeling and simulation (M&S) systems for military customers. Mr. Lutz joined
JHU/APL in 1992, and currently serves as the Airspace Integration M&S
lead for the Navy’s Triton Program. He has led the development of several
M&S standards within the Simulation Interoperability Standards Organization (SISO) and the Institute of Electrical and Electronics Engineers
(IEEE), including the IEEE 1516.2 (HLA Object Model Template) and IEEE
1730 (Distributed Simulation Engineering and Execution Process) standards. He also serves as the Chairman of the SISO Board of Directors,
serves on the Tutorial Board and Fellows Committee at the Interservice/
Industry Training, Simulation and Education Conference (I/ITSEC), and
is a guest lecturer on M&S-related topics in The Johns Hopkins University
Whiting School of Engineering.
Modeling and Simulation
Fundamentals: Identify key
M&S terms and concepts,
and the budgetary considerModeling and
ations that govern M&S deSimulation 101: DoD
velopment and application,
including requirements clarManagement
ification, and distinguish the
(1421)
defining characteristics and
associated challenges of M&S
applications within the Communities enabled by M&S. Understand the DoD vision for the
future of M&S. Gain a top-level awareness of the various M&S
programs in support of DoD mission requirements. Interoperability: Recognize the nature of HLA, TENA, DIS and their relationship to general purpose architecture for simulation reuse
and interoperability. Representation: Recognize how the Natural
Environment, Systems, and Human and Organizational Behaviors are represented in M&S, and the issues associated with each
of these components of representation as they are employed in
support of M&S requirements. Verification, Validation and Accreditation: Recognize the critical role of VV&A in ensuring that
M&S activities are most effectively organized in support of all
functional area requirements.
This tutorial is an overview of the basics of M&S and how they
are implemented in DoD. It is ideally suited to new members of
the DoD M&S community.
TRACK 2: MANAGEMENT BASICS
0830 – 1000 • ROOM S320GH
TU-2
Presenter:
LARRY HARRIS is the MS 101/Modeling and Simulation Staff Officer
Course (MSSOC) Program Manager/Course Director with his office in Alexandria, VA. He joined Alion Science and Technology in January 2002
after a 20 year career in the Marines and Army. He served in a variety of
troop and staff assignments in the U.S., Philippines, Germany, Panama,
and Korea. Larry has been involved in a variety of projects since coming to work for Alion, such as developing and presenting M&S courses
worldwide, supporting DHS/FEMA emergency response training and exercises, and managing/operating the JPRA simulation center. He provides
program management for the M&S Education program where he updates
M&S education products and coordinates presentations with host organizations. He serves as an instructor for M&S acquisition courses and as an
acquisition consultant. Larry is a lead instructor for the Acquisition M&S
Workshop; M&S in Support of the Research, Development, and Acquisition Process; M&S in Support of Test and Evaluation; M&S Verification,
Validation, and Accreditation; and the Simulation Support Plan. Larry
received a Bachelor of Arts degree in Music and Mass Communications
from Norfolk State University, Norfolk, VA in 1984.
16
TUTORIALS
This tutorial has been designed by a team of subject
matter experts to prepare attendees to understand the
Fundamentals
scope of I/ITSEC presentations and demonstrations. It
of Modeling and
provides definitions of wideSimulation
ly-used technical terms, while
(1416)
explaining the range and
types of models and simulations that are commonly applied in the M&S domain. The
tutorial reviews major simulation architectures (HLA,
TENA, DIS), the basics of instructional design, a description of the major standards and best
practices available for use across the M&S problem space, and
a brief presentation of resources that can provide further information. The tutorial introduces topics that are examined more
extensively in other tutorials.
The tutorial is designed to be more technically focused than
DoD M&S 101 and is not as focused on DoD management and
implementation of M&S.
TRACK 3: TECHNICAL BASICS
0830 – 1000 • ROOM S310AB
TU-3
Presenters:
JAMES E. COOLAHAN, Ph.D., is the Chief Technology Officer of Coolahan Associates, LLC, having retired from full-time employment at the Johns
Hopkins University Applied Physics Laboratory (JHU/APL) in December
2012 after 40 years of service. He currently chairs the M&S Committee
of the Systems Engineering Division of the National Defense Industrial
Association, and teaches courses in M&S for Systems Engineering in the
JHU Engineering for Professionals M.S. program. He holds B.S. and M.S.
degrees in aerospace engineering from the University of Notre Dame and
the Catholic University of America, respectively, and M.S. and Ph.D. degrees in computer science from JHU and the University of Maryland, respectively.
S. K. NUMRICH, Ph.D., CMSP, holds an A.B., M.A. and Ph.D. in physics and worked as a research physicist at the Naval Research Laboratory
plying her trade in a variety of fields including underwater sound in the
Arctic (yes, aboard ship), fluid-structure interactions, parallel processing,
modeling and simulation and virtual reality. Upon leaving government
service, Dr. Numrich has joined IDA.
ROBERT RICHBOURG, Ph.D., is a member of the Research Staff at the
Institute for Defense Analyses. He is a retired Army officer who holds a
B.S. in Mathematics, an M.S. and a Ph.D. in Computer Science. In his last
active duty assignment, he was an Academy Professor and Director of the
Artificial Intelligence Center at the United States Military Academy, West
Point. He is the 2014 Chair of the I/ITSEC Tutorial Board.
This tutorial will focus on
the significant changes implemented in 2014 as a result
of the ongoing Export Control
U.S. Export
Reform initiative that has significantly altered the playing
Controls 2014:
field for U.S. companies with
Continued Change
international business. Refor International
visions to the International
Modeling and
Traffic in Arms Regulations
(U.S. Munitions List Category
Simulation
IX) and the Export Adminis(1441)
tration Regulations (ECCN
0A614 et al.) have dramatically altered the controls on simulation products from software to services. Continuing trends
toward commercial off the shelf solutions for military customers
and greater emphasis on the international market continue to
create challenges for U.S. and foreign companies alike. Participants will understand the scope of the U.S. export laws, anticipated changes and reform, how the U.S. Government applies
them to the simulation industry, including controls on software,
hardware, services and activities at events such as I/ITSEC, as
well as strategies for ensuring compliance in commercial, U.S.
government and foreign contracts.
TRACK 4: STEP BY STEP
0830 – 1000 • ROOM S320B
TU-4
Presenters:
JEREMY HUFFMAN, J.D., is a founding member and partner of Huffman Riley PLLC. Mr. Huffman concentrates his practice advising U.S.
and foreign clients concerning the U.S. export control laws, including the
International Traffic in Arms Regulations (“ITAR”); Export Administration
Regulations (“EAR”); and the Office of Foreign Assets Control (“OFAC”)
regulations. Mr. Huffman has extensive experience assisting high technology, defense industry and other clients to navigate applicable U.S. export
statutes and regulations. Mr. Huffman earned his Juris Doctorate from the
University of Virginia School of Law in 1997.
DARREN RILEY, J.D., is a founding member and partner of Huffman
Riley PLLC. Mr. Riley has extensive experience advising clients on matters involving U.S. export controls and government contracts issues. He
counsels clients on issues related to the International Traffic in Arms Regulations, Export Administration Regulations, the regulations of the Office
of Foreign Assets Control and the Foreign Corrupt Practices Act. Mr. Riley
has extensive experience advising high technology, defense industry and
other clients about U.S. export control laws and other applicable statutes
and regulations. Mr. Riley earned his Juris Doctorate (cum laude) from
the Howard University School of Law in 2000.
TUTORIALS
An emphasis on self-regulated learning in the military
community (U.S. Army Training & Doctrine Command and
Fundamentals of
the Air Education & Training
Command) has highlighted
Adaptive Intelligent
a need for intelligent tutorTutoring Systems
ing systems (ITSs) to support
for Self-regulated
self-regulated learning (SRL)
Learning
at the learner’s point-of-need.
One-to-one human tutoring
(1413)
has been shown to be significantly more effective than traditional classroom instruction, but is impractical for implementation on a large scale. Adaptive ITSs adjust feedback, support,
and challenge level of training scenarios to meet the specific
learning needs of trainees. ITSs offer growing promise to be both
practical and effective solutions for SRL in military training domains. This tutorial is suitable for: beginners (exposure to the
basics of ITSs), journeymen (understanding of how ITS design
influences SRL) and experts.
TRACK 5: COGNITIVE ‘X’
0830 – 1000 • ROOM S320C
TU-5
Presenter:
ROBERT A. SOTTILARE, Ph.D., serves as the Chief Scientist at the Simulation & Training Technology Center (STTC) within the Army Research
Laboratory’s Human Research and Engineering Directorate (ARL-HRED).
He also leads adaptive tutoring research within ARL’s Learning in Intelligent Tutoring Environments (LITE) Laboratory where the focus of his
research is in automated authoring, instructional management, and analysis tools and methods for intelligent tutoring systems. His work is widely
published and includes articles in the Cognitive Technology Journal, the
Educational Technology Journal, and the Journal for Defense Modeling
& Simulation. Dr. Sottilare is a co-creator of the Generalized Intelligent
Framework for Tutoring (GIFT), an open-source tutoring architecture, and
he is the chief editor for the Design Recommendations for Intelligent Tutoring Systems book series. He is a visiting scientist and lecturer at the
United States Military Academy. Dr. Sottilare received his doctorate in
Modeling & Simulation from the University of Central Florida with a focus
in intelligent systems. In January 2012, he was honored as the inaugural
recipient of the U.S. Army Research Development & Engineering Command’s Modeling & Simulation Lifetime Achievement Award.
Adding natural dialogue can
increase engagement in online training, simulation and
performance support enviMy Training Talks
ronments. Although more
commonly associated with
with Me! Getting
non-player characters (NPC’s)
Started with
in games and simulations, avDialogue Agents
atars that give prompts, hints
(1418)
and feedback to learners are
effectively being used in intelligent tutoring systems and
can be incorporated into most
web or mobile environments. Moreover, “chat bots” that process
spoken input and respond in surprisingly natural and intelligent
ways can be built with readily available open source software.
This tutorial is an introduction to capabilities of dialogue agents,
their implementation, and their use in training environments.
Topics include text-to-speech, speech-to-text, chat bots, the AI
behind chat bots, chat web services, dialogue-based avatars,
and incorporating chat into HTML5-based training and simulation. User acceptance issues are explored, including technical, social, and aesthetic challenges. This tutorial provides insights into how these technologies work and demonstrates ways
to leverage open source products and functionality built into
browsers and operating systems to start making your training
talk with your learners.
TRACK 6: INTERACTIONS
0830 – 1000 • ROOM S320E
TU-6
Presenters:
ELAINE KELSEY is a software engineer at Eduworks focusing on development of conversational dialogue agents for intelligent tutoring systems,
natural language processing and semantic analysis. She has developed
multi-cultural interfaces for global software deployments; designed algorithms for improving cost pool allocation in financial planning and analysis software for global nonprofit organizations; and worked on the integration of idiomatic and dialectal forms in natural language translation.
Elaine speaks eight languages and has a B.S. in Cell and Molecular Biology and a B.A. in Scandinavian Languages from the University of Washington, a Masters of Public Health from Tulane University and is currently
finalizing a B.S. in Computer Science from Oregon State University.
ROBBY ROBSON, Ph.D., has been researching and developing innovative
learning technologies for almost twenty years in academia and industry. As Principal Investigator on multiple NSF and DoD research projects
he has contributed to the theory and practice of learning management
systems, digital libraries and adaptive learning systems. He chaired the
IEEE Learning Technology Standards Committee for eight years, helping to develop standards that are used throughout the training industry. He co-founded Eduworks Corporation in 2001, where he is CEO and
Chief Scientist. He is currently a consultant to the IDA, a member of the
IEEE Computer Society Standards Activities Board, an advisor to LanguageTwin, and an IITSEC subcommittee member. He has a doctorate in
mathematics from Stanford University.
18
TUTORIALS
Technology-mediated
solutions for learning, instruction,
and assessment are often
intended for use by schoolTransmedia
houses or formal training programs. Few address informal
Learning in the Wild:
learning or self-paced “learnSupporting Military
ing in the wild.” This tutorial
Training Through
addresses this gap by discussStory-driven
ing transmedia learning design specifically for informal
Engagement
learning and offering open
(1442) %
source software strategies to
unobtrusively track learner
progress. Transmedia learning is story-driven, unfolds across
multiple media, and is designed to promote self-directed engagement and social collaboration. The first section of the tutorial
defines and describes the how and why of transmedia learning. The second section explores the design and development
of a transmedia learning storyworld for a use case shared by
all Services, NATO, industry, and Federal Government: physical
readiness training & nutrition. Strategies for the use of social
media, games, machinima and/or videos, and Web portals are
provided. The third section focuses on connecting open source
software elements for a transmedia learning ecosystem such as
learning record stores & dashboard, experience tracking, competency frameworks, and a game/collaborative 3D virtual environment. No prerequisite knowledge is required. This tutorial is for
all program managers, researchers, designers, and developers
interested in 1) the use of various media to engage learners in a
story while measuring progress, and 2) implementation of audience’s own transmedia learning ecosystem. Participants will
take away practical strategies, resources, tools, and software for
their own use.
TRACK 7: ENGAGING LEARNERS
0830 – 1000 • ROOM S320D
TU-7
Presenter:
ELAINE M. RAYBOURN, Ph.D., is a Principal Member of the Technical
Staff in Cognitive Systems at Sandia National Laboratories. Elaine has
worked on transmedia learning since 2010 and led the development of
an award-winning Government game. She is a Fellow of the European
Research Consortium for Informatics and Mathematics and has worked
in research laboratories in Germany, England, and France. Elaine speaks
regularly on the topic of transmedia learning and serves on several editorial boards including Interactive Technology and Smart Education, Journal of Game-based Learning, and Simulation & Gaming. Elaine was on
the advisory board for the Game Developers Conference (GDC) Serious
Games Summit from 2004-2007, Defense GameTech Program Chair in
2011 and Program Chair Advisor 2012-14. She is a member of I/ITSEC
Serious Games Showcase & Challenge Integrated Project Team and the
Training Subcommittee. She is on assignment to Advanced Distributed
Learning Initiative, Office of the Deputy Assistant Secretary of Defense
(Force Readiness), where she leads research in several areas including
transmedia learning and learner adaptability. Elaine is a recipient of the
Department of the Army Award for Patriotic Civilian Service, awarded to
her by the U.S. Army Special Forces.
The Test and Training Enabling Architecture (TENA)
and the Joint Mission Environment Test Capability
TENA/JMETC:
(JMETC) program provide an
advanced set of interoperabilTesting and
ity software, interfaces, and
Training for Mission
use in joint distributed testing
Readiness
and training. This tutorial will
(1436)
provide information about
how TENA works and why
it is important to the test and
training communities, with
some comparison to other interoperability architectures. TENA provides testers and trainers
software such as the TENA Middleware, a high-performance,
real-time, low-latency communication infrastructure that is used
by training range instrumentation software and tools during execution of a range training event. The standard TENA Object
Model provides data definitions for common range entities and
thus enables semantic interoperability among training range
applications. The TENA tools, utilities, and gateways assist in
creating and managing an integration of range resources. The
current version of the TENA Middleware, Release 6.0.4, is being
used by the range community for testing, training, evaluation,
and feedback and is being used in major exercises in the present.
JMETC has created a persistent test and evaluation capability
throughout the US DOD, connecting many test ranges together,
and including a bridge to the JTEN training network; a set of
TENA-compliant software middleware, interfaces, tools, and databases; and a process for creating large distributed test events.
The combination of TENA and JMETC gives testers and trainers
unprecedented power to craft a joint distributed mission environment that meets testing and training requirements to test and
train for mission readiness.
1245 – 1415 • ROOM S320A
TU-8
Presenter:
EDWARD T. POWELL, Ph.D., is the lead architect for the Test and
Training Enabling Architecture. After receiving his Ph.D. in Astrophysics from Princeton University, he worked for the Lawrence Livermore
National Laboratory performing simulation-based analysis. He moved to
SAIC (now Leidos) in 1994, and participated as lead architect in some
of the most complex distributed simulation programs in DoD, including
the Joint Precision Strike Demonstration (JPSD), the Synthetic Theater of
War(STOW), and the Joint Simulation System (JSIMS). He then worked
in the intelligence community for two years on architectures for integrating large-scale diverse ISR systems. He has been the lead architect for
TENA for ten years now, and is currently working on expanding the applicability of TENA, and integrating multiple interoperability architecture
approaches using ontology-based systems.
TUTORIALS
Fidelity is defined as the degree of correspondence between simulation and reality.
Managers and developers can
Training Simulation
specify the required fidelity
Fidelity: Establishing of media (e.g., software) and
equipment (e.g., hardware)
Preferences,
for a stated training purpose.
Priorities, and
How much is truly necessary
Optimizing
to achieve the training objectives? Although lower-fidelity
Trade-offs
simulation can increase the
(1402)
availability of training solutions, the simulation Fidelity
present is often governed by the financial limitations of the training organization in question, not by requirements. The matter of
Fidelity, as it relates to cost, speaks to why simulators have been
vastly underutilized in civilian training. This tutorial presents
guidelines for identifying a training framework exhibiting an appropriate balance between cost and features. The tutorial overviews the wide range of Fidelity available in simulation-based
training, summarizes techniques for establishing needs based
on Fidelity requirements, and offers guidelines for optimizing
related trade-off decisions for training system acquisition. Finally, three case studies (of increasing complexity) demonstrate the
techniques and help justify the ongoing need for appropriately
specified simulation technology in training.
1245 – 1415 • ROOM S320GH
TU-9
Presenters:
KEVIN HULME, Ph.D., earned his doctorate at the Department of Mechanical and Aerospace Engineering, University at Buffalo in 2000, concentrating on multidisciplinary analysis and optimization of complex
systems. For the past 9 years, Kevin has been technical lead of the Motion Simulation Laboratory at the New York State Center for Engineering
Design and Industrial Innovation (NYSCEDII). His research team focuses
on the custom design and development of ground vehicle simulations for
applications in: clinical research, education and training, and next-generation transportation studies. Recent areas of focus include: standardization of simulators in teen driver safety, fidelity requirements in simulation
system specification, multi-participant civilian driving simulators, serious
gaming and Edutainment in simulation-based training, and Simulator
sickness mitigation.
KEMPER LEWIS, Ph.D., is currently Professor of Competitive Product
and Process Design in the Department of Mechanical and Aerospace Engineering, Executive Director of the New York State Center for Engineering
Design and Industrial Innovation (NYSCEDII), and the Site Director of the
National Center for e-Design as part of the NSF’s Industry/University Cooperative Research Center (I/UCRC) program at the University at Buffalo.
His expertise is in engineering design theory, complex system trade-offs
and optimization, and decision modeling. He is a fellow of ASME and
has received awards from ASEE, SAE, ASME, AIAA, NSF, and the State of
New York for his teaching and research efforts.
1245 – 1415 • ROOM S310AB
TU-10
Model Verification
and Validation
Methods
(1407)
Verification and validation
(V&V) are essential prerequisites to the credible and reliable use of a model. But what
are V&V exactly, and what is
their purpose within a modeling and simulation project?
What types of potential errors
can occur during V&V and
how can they be avoided?
Most importantly, what methods and procedures are available to perform verification
and validation in a rigorous
and effective manner?
The tutorial is intended to answer these questions. It has three
distinct parts. The first part motivates the need for V&V, provides
definitions necessary to their understanding, and explains why all
V&V methods can be understood as comparisons and how this
informs their application. The second part provides guidelines
for selecting V&V methods, introduces a widely used taxonomy
of V&V methods, defines four categories of V&V methods, and
describes two or more methods from each category. Example
applications of the described methods are presented. In the third
part, three detailed case studies of V&V in practice are presented,
showing how V&V methods have been applied in actual modeling
and simulation projects.
This tutorial was recognized as the Best Tutorial at the 2013
I/ITSEC.
Presenter:
MIKEL D. PETTY, Ph.D., CMSP, is Director of the University of Alabama in Huntsville’s Center for Modeling, Simulation, and Analysis,
Associate Professor of Computer Science, and Research Professor of Industrial and Systems Engineering and Engineering Management. Prior
to joining UAH, he was Chief Scientist at Old Dominion University’s
Virginia Modeling, Analysis, and Simulation Center and Assistant Director at the University of Central Florida’s Institute for Simulation and
Training. He received a Ph.D. in Computer Science from the University
of Central Florida in 1997. Dr. Petty has worked in modeling and simulation research and education since 1990 in areas that include verification
and validation methods, simulation interoperability and composability,
human behavior modeling, and applications of theory to simulation. He
has published over 185 research papers and has been awarded over 16
million in research funding. He served on a National Research Council
committee on modeling and simulation, is a Certified Modeling and
Simulation Professional, and is an editor of the journal SIMULATION.
He has graduated five Ph.D. students, including the first and second
students to receive Ph.D.s in Modeling and Simulation at Old Dominion
University and the first student to receive a Ph.D. in Modeling and Simulation at the University of Alabama in Huntsville.
20
TUTORIALS
How should one approach
the problem of assessing the
value of simulation, particularly under circumstances
Establishing the
where neither cost nor valuation data is readily accessible?
Value of Simulation
This tutorial presents general
(VoS) for Collective
approaches to assessing reTraining: Accent on
turn on investment, shows
Methodology
how the problem has been
approached for simulators
(1417) 
and then moves to the assessment of the value of simulation in primary staff training -- a collective training process with
few standards, conditions and metrics usable for assessment.
Although valuation requires both cost and benefit, the tutorial
will focus on establishing the benefit. Attention will be paid to
the development of standards in the absence of usable, official
standards. It will demonstrate how to develop survey instruments using those standards and illustrate the types of results
that can be obtained with an actual test case. Factors such as
internal and external validity will be described. The objective of
the tutorial will be to show the audience a viable approach to
follow in setting up their own assessments. There are no prerequisites and anyone interested in designing means of assessing
simulation-based training results is welcome.
1245 – 1415 • ROOM S320B
TU-11
Presenters:
This tutorial provides a synopsis of key findings and
theoretical advances from
neuroscience directly applicaCognitive
ble to military education and
training. Materials will be preNeuroscience for
sented in a manner that may
Military Education
be readily understood and apand Training
plied by individuals with no
(1412)
formal training in neuroscience. Topics covered will include: perception, attention,
learning and memory, information processing, multi-tasking, conscious awareness, individual differences and social interactions. Particular emphasis will
be placed on methods and techniques to directly apply insights
from research concerning brain science to education and training settings.
1245 – 1415 • ROOM S320C
TU-12
Presenter:
CHRIS FORSYTHE, Ph.D., is a Distinguished Member of the Technical
Staff in the Human Factors organization at Sandia National Laboratories.
His expertise lies in the application of technology to improve human performance and the underlying neurophysiological factors that affect human performance. He holds advanced degrees in Cognitive Psychology
and Biopsychology and has over 20 years of experience in related fields.
Dr. Forsythe has published three books and over 40 papers addressing
human performance in applied settings.
S. K. NUMRICH, Ph.D., CMSP, holds an A.B., M.A. and Ph.D. in physics
and is currently a research staff member at the Institute for Defense Analyses where she undertakes studies in areas related to needs of the operational military including training. Prior to coming to IDA, she worked as
a research physicist at the Naval Research Laboratory where she served
on numerous M&S committees and panels concerned with the value and
capability of simulation for acquisition and training. Her last assignment
as a Navy civilian was as technology director at the Defense Modeling
and Simulation Office.
P.M. PICUCCI, Ph.D., is a Research Staff Member at the Institute for
Defense Analyses. He holds an M.A. in National Security Studies and
a Ph.D. in Political Science. His primary research interests center on
non-traditional conflict (irregular warfare and terrorism) and include the
use of computerized content analysis in the study of Islamic radicalism.
Recent studies have focused on US military deployment and employment
of biometrics, and the integration of socio-cultural knowledge and data
into DoD operations.
TUTORIALS
HTML5 has become the dominant technology for interactive web applications. This is
because of its cross-platform
Using HTML5
and cross-device capabilities, its support from major
to Develop IMI,
web technology companies
Simulations, and
and publishers (e.g. as part
Games
of the ePub3 e-book format),
(1422) %
and its ability to natively deliver “Flash-like” functionality. This tutorial introduces
HTML5 and its applications to developing interactive training,
simulations and games. It explains the capabilities of HTML5
and details its most important features and their uses, illustrating them with real world examples. Emphasis will be placed on
features that are important for simulation and game development and that will help designers and managers motivate their
teams to move to HTML5 if they have not already done so. In response to requests from previous years, this tutorial will provide
an expanded list of tools, code snippets and HTML5 resources.
1245 – 1415 • ROOM S320E
TU-13
Presenters:
ROBBY ROBSON, Ph.D., has been researching and developing innovative
learning technologies for almost twenty years in academia and industry. As Principal Investigator on multiple NSF and DoD research projects
he has contributed to the theory and practice of learning management
systems, digital libraries and adaptive learning systems. He chaired the
IEEE Learning Technology Standards Committee for eight years, helping
to develop standards that are used throughout the training industry. He
co-founded Eduworks Corporation in 2001, where he is CEO and Chief
Scientist. He currently also serves as a consultant to the IDA, as a member
of the IEEE Computer Society Standards Activities Board, as an advisor
to LanguageTwin, and as an IITSEC subcommittee member. He has a
doctorate in mathematics from Stanford University.
AARON VEDEN has developed multiple games and end user applications
in HTML5 and for mobile devices. He has four years experience tutoring
end users in a range of technologies including Windows, Office applications, and a variety of web applications. Aaron has built several commercial players for rendering IMI level 3 and 4 training and is currently
working on two open source HTML5 applications for the U.S. Advanced
Distributed Learning initiative (ADL) involving personalized and adaptive learning. Aaron holds a B.S. in Computer Engineering from Oregon
State University.
SHERRIE VIEIRA is a media specialist who served honorably in the U.S.
Army for four years, receiving a Joint Services Achievement medal for her
work at the Medina Regional SIGINT Operations Center in San Antonio,
Texas. She has a B.S. in digital arts from the University of Oregon and
has worked at Hewlett-Packard and with the State of Oregon as well as
at Eduworks.
With today’s DoD budget
cuts, the need and demand for
military simulations will continue to grow. This is largely
Scenario Story
driven by sheer fiscal necessities since live-exercise costs
Development in
are putting more pressure to
Military Games/
employ simulations. Within
Simulations
the military, there are two
(1426) %
conflicting forces: decreasing resources and increasing
demands for highly trained
and proficient soldiers. However, at the same time, the military
must challenge and stimulate digital learners who have grown
up playing “America’s Army,” “Call of Duty,” and/or “Full Spectrum Warrior.”
Although there is no substitute for live experience, gaming/
simulations provide experiences to soldiers that allow them to
play a character within the game/simulation environment.
Creating compelling game/simulation environments requires trainees to assess situations and make critical decisions.
Design/development teams must create story-based learning
environments (defined from real-world situations) and develop
the storyline, allowing the trainee to move forward through the
simulation while making decisions and choices and causing the
simulation experience to follow a different branching storyline.
This tutorial will discuss using real-world experiences to design
and develop game/simulation storylines. It will cover storytelling
instructional methods currently used. Using a real-world experience, the tutorial will take participants through story development
and how to design the story to include linear/non-linear branching and decision-making. It will also discuss how the storyline
influences the simulation structure and game play and how to
connect completing the mission to how the trainee achieves
mission success or failure.
1245 – 1415 • ROOM S320D
TU-14
Presenter:
MARIE BROYLES, Ed.D., has extensive gaming, training and simulation
design and development experience. She has worked in automotive, banking and financial services, defense, manufacturing and telecommunication industries. At General Dynamics Amphibious Systems she designed/
developed Expeditionary Fighting Vehicle (EFV) operational and maintenance elearning training. She has designed/developed military game
simulations derived from actual situations and has taught game-level and
world-building college undergraduate courses. She holds a B.S. degree in
Art Education from Indiana University, an M.S. in Secondary Education
from Indiana University-Purdue University at Indianapolis and an Ed.D.
in Art Education and Instructional Systems Technology from Indiana University.
22
TUTORIALS
Distributed Interactive Simulation (DIS) is one of the
three major modeling and
simulation standards used in
Distributed
military M&S. This tutorial
will provide an overview of
Interactive
the problem domain for modSimulation 101:
eling and simulation in virtual
The Basics
environments, briefly discuss
(1439)
the standards used by modeling and simulation, and then
discuss the DIS protocol. The
messages that constitute DIS and the techniques used by DIS to
implement a virtual environment are discussed, as are simple
examples for sending and receiving DIS messages.
1430 – 1600 • ROOM S320A
TU-15
Presenters:
DON MCGREGOR is a Research Associate at the Naval Postgraduate
School in Monterey, California. He is the primary author of Open-DIS,
an open source implementation of the Distributed Interactive Simulation
protocol in Java, Javascript, C++, C#, and Objective-C. His research
interests include web-based simulation using Javascript, Websockets, and
WebGL.
DON BRUTZMAN, Ph.D., is Technical Director for 3D Visual Simulation
and Networked Virtual Environments in the Modeling, Virtual Environments, and Simulation (MOVES) Institute. As an Associate Professor at
the Naval Postgraduate School in Monterey, California he is a member
of two academic groups: Undersea Warfare and MOVES. He is an investigator in the NPS Center for Autonomous Underwater Vehicle Research.
His research interests include underwater robotics, real-time 3D computer
graphics, artificial intelligence and high-performance networking.
Simulation conceptual modeling is a critical step in simulation development frequently overlooked in the rush to
Simulation
demonstrate program progress. A simulation concepConceptual
tual model is an abstraction
Modeling Theory
from either the existing or a
and Application
notional physical world that
(1409)
serves as a frame of reference for further simulation
development by documenting
simulation-independent views of important entities and their
key actions and interactions. A simulation conceptual model
describes what the simulation will represent, the assumptions
limiting those representations, and other capabilities needed to
satisfy the stakeholder’s requirements. It bridges between these
requirements, and simulation design.
This tutorial will present the theory and application of simulation conceptual modeling as documented during the research
done by the NATO MSG 058 and SISO SCM SG/SSG/PDG. In
addition, Use Cases that have been drawn from previous conference presentations will be presented to illustrate how conceptual
modeling has been performed. Additional work is necessary to
mature the state-of-the-art of simulation conceptual modeling
before a recommended practices guide could be standardized.
This tutorial has been created to continue the maturation of the
simulation conceptual modeling best practices.
1430 – 1600 • ROOM S320GH
TU-16
Presenter:
JAKE BORAH, CMSP, is a Senior Member of Technical Staff for AEgis
Technologies Group, Inc. He has been assigned as Project Manager or
Technical Lead on several projects that require a high degree of modeling and simulation expertise and a capability to integrate leading edge
technology into ongoing processes. His most recent work has been for
the Air Force Modeling and Simulation Training Toolkit (AFMSTT). He
has frequently supported US and Canadian government sponsored military simulation projects because of his mastery of the M&S technology,
and expertise in High Level Architecture federation development. He is a
Charter Certified Modeling and Simulation Professional (CMSP). He is a
recognized expert and a prominent member of the worldwide M&S community as reflected by his contributions to the Simulation Interoperability
and Standards Organization (SISO) workshops and products. He graduated from the United States Air Force Academy in 1974 and possesses a
Master of Aeronautical Science degree from Embry-Riddle Aeronautical
University.
TUTORIALS
Distributed simulation technologies have changed the way
the DoD does research, development, engineering, training,
Planning and
analysis, and testing. These
technologies and associated
Execution of a Large
standards have been in use for
Multi-architecture
20 years and have been docuDistributed Event
mented in many forums. How(1420)
ever, a critical element that has
not been widely documented
is the processes and tools required to execute a large multi-architecture distributed event. There
are organizations that do these types of events very well, but the
only way to learn these skills is to be a member of one of these
teams.
This tutorial provides a guide to the planning and execution of
a large multi-architecture distributed event. This guide will include
the steps to planning and executing an event including design of
the simulation architecture, planning integration spirals, scenario
development and rehearsal, conduct of the event, and the data
collection and analysis. While the steps described in this tutorial
are applicable to all large distributed events, special emphasis
will be placed on multi-architecture based events.
This tutorial is applicable to anyone involved in the development of a large test event. The material will be applicable to
simulation architects, analysts, scenario developers, simulation
users, and managers. The tutorial will relate the material to the
IEEE 1730 IEEE Recommended Practice for Distributed Simulation Engineering and Execution Process (DSEEP). An example
will be provided.
1430 – 1600 • ROOM S310AB
TU-17
Presenter:
MICHAEL J. O’CONNOR, CMSP, is a Senior Program Manager at Trideum Corporation. Mr. O’Connor has more than 25 years experience in
Modeling and Simulation (M&S). He has been a key participant in the
development of distributed modeling and simulation standards, including
IEEE 1278 and IEEE 1516. He has held many positions in the community, including Chairman of the SISO Standards Activities Committee and
Chairman of the SISO Executive Committee. He served as the chair of
the I/ITSEC Simulation Subcommittee. Mr. O’Connor currently leads the
technical integration of the On-Demand Environment for Networks and
Net-Centric Systems (ODENN) Event 14. He has led the development of
multiple simulations using DIS, HLA, and TENA. Mr. O’Connor has led
the technical integration of several large multi-architecture distributed
events including the Multi-Served Distributed Event (MSDE) in 2005 that
used DIS, HLA, and TENA. MSDE included over 20 sites and included
organizations from the Army, Navy, and Air Force. He holds a B.S. in
Computer Engineering from Auburn University, and as M.S. in Computer
Science from the University of Alabama in Huntsville. Mr. O’Connor is a
Certified Modeling and Simulation Professional (CMSP).
Classic discrete event simulation is a modeling paradigm
that is well known and widely applied for applications in
Defense-related
the industrial, manufacturing, and process optimizaApplications of
tion communities. It has been
Discrete Event
much less frequently used for
Simulation
defense applications, where
(1408)
the continuous time-stepped
modeling paradigm is nearly
ubiquitous.
The tutorial is intended to introduce (or reintroduce) members
of the defense modeling and simulation community to classic
discrete event simulation and show how it may be advantageously
employed for many defense-related applications. The tutorial has
four distinct parts. The first provides key definitions and concepts
that underlie the discrete event simulation paradigm and motivate
its applicability to defense applications. The second part explains
the core elements that make up a discrete event simulation model,
including events, queues, servers, and probability distributions.
The third part demonstrates how to use probability distributions
to model physical phenomena and how to use elementary statistics to draw conclusions from discrete event simulation output.
Finally, the fourth part presents three case studies of actual uses
of classic discrete simulation for defense-related applications
drawn from the research literature.
1430 – 1600• ROOM S320B
TU-18
Presenter:
MIKEL D. PETTY, Ph.D., CMSP, is Director of the University of Alabama in Huntsville’s Center for Modeling, Simulation, and Analysis,
Associate Professor of Computer Science, and Research Professor of Industrial and Systems Engineering and Engineering Management. Prior
to joining UAH, he was Chief Scientist at Old Dominion University’s
Virginia Modeling, Analysis, and Simulation Center and Assistant Director at the University of Central Florida’s Institute for Simulation and
Training. He received a Ph.D. in Computer Science from the University
of Central Florida in 1997. Dr. Petty has worked in modeling and simulation research and education since 1990 in areas that include verification
and validation methods, simulation interoperability and composability,
human behavior modeling, and applications of theory to simulation. He
has published over 185 research papers and has been awarded over 16
million in research funding. He served on a National Research Council
committee on modeling and simulation, is a Certified Modeling and
Simulation Professional, and is an editor of the journal SIMULATION.
He has graduated five Ph.D. students, including the first and second
students to receive Ph.D.s in Modeling and Simulation at Old Dominion
University and the first student to receive a Ph.D. in Modeling and Simulation at the University of Alabama in Huntsville.
24
TUTORIALS
Virtual Worlds are used extensively for multi-user facilitated learning. As the learning experience becomes more
Augmenting
open-ended, facilitators are
necessary to brief before the
Virtual Worlds with
experience, guide during the
Intelligent Tutors
experience, and debrief in af(1437)
ter-action reflection. However,
facilitator-supported learning
does not scale to large numbers of learners. Intelligent
Tutoring Systems (ITS) technology offers an alternative to live
facilitator training in well-prescribed situations, such as triage
of victims in a mass casualty disaster, where a large number of
personnel need to be trained in a short time. Natural language
based ITS’s, in particular, can simulate a dialog with a trained
facilitator.
The tutorial will provide an overview of natural-language based
ITS technology and will show the use of one such ITS to construct
an integrated solution where the intelligent tutor becomes available within the virtual world. The learning situation is medical
triage of victims in an earthquake disaster. The tutoring is about
the triage process, with question-answer dialog, as well as learner
knowledge assessment using increasingly detailed questions.
The tutorial will include the results of preliminary evaluation
comparing the use of the virtual world alone with that using the
intelligent tutor in the virtual world.
1430 – 1600 • ROOM S320C
TU-19
Presenters:
PARVATI DEV, Ph.D., is President and CEO of Innovation in Learning
Inc., where she leads the development and marketing of CliniSpace, IIL’s
web-based virtual medical environment for healthcare training. Parvati
has 35 years of experience developing technology solutions applied to
life sciences learning and research problems. While at CEMAX Inc., she
developed the first commercially available 3D reconstructive imaging system for surgical planning and radiologic imaging. At Stanford University
she founded and ran an internationally recognized learning technologies
lab, SUMMIT, with firsts in multimedia, web and simulation for medical education, and numerous peer-reviewed publications. She worked on
large programs with NIH, DARPA, NSF and foundations. At Innovation in
Learning, she and her team are creating the next generation of immersive
learning environments.
XIANGEN HU, Ph.D., is Dunavant professor in the Department of Psychology at The University of Memphis (UofM), senior researcher at the
Institute for Intelligent Systems (IIS) at the UofM and visiting professor at
Central China Normal University (CCNU). Dr. Hu earned his doctorate in
Cognitive Sciences from the University of California, Irvine. Currently, Dr.
Hu is the director of cognitive psychology at the UofM, the Director of the
Advanced Distributed Learning (ADL) center for Intelligent Tutoring Systems (ITS) Research & Development, and senior researcher in the Chinese
Ministry of Education’s Key Laboratory of Adolescent Cyberpsychology
and Behavior.
Recent developments have
enabled advanced interaction for many applications in
which users can more realisSpeech-based
tically interact with serious
games in virtual environInteraction: Myths,
ments. Unfortunately, mainly
Challenges, and
because of its complex nature,
Opportunities
allowing users to fully interact
(1429) ñ%
through speech is a challenge
for machine processing, particularly in areas where the
task is unconstrained and performed under adverse conditions.
As such, speech has often been neglected as a modality that can
enhance the naturalness of interacting with virtual training systems. Furthermore, user-based evaluations of speech interfaces
are intrinsically difficult. However, recent research brings hope
that, despite these shortcomings, there are several interesting
areas and approaches for research and development that could
lead to improvements in the design and implementation of training systems. This tutorial will explain how Automatic Speech
Recognition and Speech Synthesis work; the challenges in enabling speech as a modality for hands-free interaction; some usability issues in speech-based interaction systems; opportunities
for researchers and developers to enhance system interactivity
by enabling speech, and how to enable speech-based interaction
within immersive, mixed-reality environments. The tutorial is
intended for developers interested in implementing speech recognition in interactive applications, as well as for researchers
dedicated to developing methods and systems that allow humans to naturally interact with technology.
1430 – 1600 • ROOM S320E
TU-20
Presenters:
COSMIN MUNTEANU, Ph.D., is a faculty member of the University
of Toronto, Canada. As a Research Officer with the National Research
Council Canada, he led several research projects exploring speech and
natural language interaction for advanced learning systems and mixed
reality training simulators. His area of expertise is at the intersection of
Automatic Speech Recognition (ASR) and Human-Computer Interaction
(HCI), having extensively studied the human factors of using imperfect
speech recognition systems, and having designed and evaluated systems
that consider humans as an important part of the ASR process. He has
authored numerous publications in HCI, ASR, and Computational Linguistics.
GERALD PENN, Ph.D., is an Associate Professor of Computer Science at
the University of Toronto, Canada, where he is conducting research and
publishing in Speech and Natural Language Processing. His area of expertise is the computational and mathematical study of human languages.
TUTORIALS
A notable trend in modern
training is the dramatic increase of commercial game
production platforms as a baBridging the Gap:
sis to create enriched computer-based training. These proHow to Build
gramming technologies allow
Effective Gamedevelopers to create immerbased Training
sive environments in which
(1425) %
to embed demonstrations,
interactive practice, feedback,
and other instructional strategies to replace traditional text
and pictures that have characterized computer-based training.
These environments also allow for the addition of a variety of
game features. If wisely employed, such features can increase
learner motivation, improve knowledge organization, and support the development of strategic knowledge. Given the newness
of gaming platforms as a basis for instructional systems, documented guidelines and best practices are not widely available.
In fact, instructional designers are rarely informed about the capabilities or limitations associated with these platforms, or the
opportunities they present for optimizing learning. Conversely,
game programmers are rarely aware of sound instructional design principles. The goal of this tutorial is to “bridge the gap”
described above using the context of a large-scale, game-based
learning program to support the Littoral Combat Ship. Specifically, we will describe our experiences in embedding sound
instructional strategies into game-based learning, and how the
application of gaming features can be used to create effective,
motivating learning environments.
1430 – 1600 • ROOM S320D
TU-21
Presenters:
JAN CANNON-BOWERS, Ph.D., has a long history in the simulation and
game-based training area. Her research focuses on optimizing technology-enabled learning. She is now applying what she learned at Cubic Advanced Learning Solutions (CALS), in a large-scale game-based training
effort for the Navy’s new Littoral Combat Ship.
CLINT BOWERS, Ph.D., is a Professor of Psychology and Director of the
RETRO laboratory at the University of Central Florida. His recent research
is in the area of technology-based teaching. Current research projects include the development and evaluation of game-based learning technologies across a variety of domains. He has published numerous refereed
journal articles and book chapters in the area of training and has edited
three books related to training and technology.
KATELYN PROCCI, JENNIFER LOGLIA, SKILAN ORTIZ, and BUDD C.
DARLING III are all Human Performance Engineers for the Mission Support Services branch of Cubic. All have varying levels of experience with
training game research and development.
BILL REBARICK, Ph.D., General Manager of Cubic Advanced Learning
Solutions, is a retired Navy Supply Corp Officer and current organizational executive who has dedicated himself to helping the Navy and Cubic
develop high-performers through innovative learning solutions. His goal
is to extend this work to develop and deploy game-based training to improve learning and work performance across a variety of fields.
26
Signature Events
MONDAY, 1 DECEMBER
1030 – 1200 • ROOM S320ABC
SE-1
Congressional
Modeling and
Simulation Event
Select Members of Congress
will participate in this
I/ITSEC Event
T
S I G N AT U R E E V E N T
his special event continues to excite attendees with its expectations for learning the thinking
of the M&S Caucus Membership. At the same time, and more importantly, it is an opportunity
to reach out to the Congressional Members at I/ITSEC, who are in a listening mode, for clearer
understanding of what M&S is doing for technology based training. This refers not just to military
applications because simulation is central to the national economy, our society and our overall
well being. Therefore, audience attendees should be prepared to inform Caucus Members how the
vitality of the American economy can improve with an expansion of M&S in everyone’s daily life,
and further to mention the promise it holds for the nation’s future. An end to the era of budget
reductions is not in the immediate future so there is no waiting period for action. We tell it as it
is, now, and enhance the conversation with our Congressional Leaders.
T
Ed Markey
Allyson Schwartz*
Massachusetts
Pennsylvania 13th District
Ander Crenshaw*
Jim Matheson*
Bobby Scott*
Alabama 4th District
Florida 4th District
Utah 4th District
Virginia 3rd District
Gus Bilirakis*
Ted Cruz
John Mica
Jeff Sessions
Texas
Alabama
Diane Black*
Susan Davis*
Candice Miller*
Richard Shelby
Tennessee 6th District
California 53rd District
Michigan 10th District
Alabama
Barbara Boxer
Dianne Feinstein
Jeff Miller*
Niki Tsongas*
California
California
Florida 1st District
Massachusetts 3rd District
Jim Bridenstine*
J. Randy Forbes*
Jim Moran*
Tim Walz*
Virginia 4th District
Chair, M&S Caucus
Virginia 8th District
Minnesota 1st District
Bill Nelson
Mark Warner
Alabama 5th District
Virginia Foxx*
Florida
Virginia
Vern Buchanan*
North Carolina 5th District
Richard Nugent*
Elizabeth Warren
Phil Gingrey*
Massachusetts
Ken Calvert*
Georgia 11th District
Bill Posey*
Joe Wilson*
California 42nd District
Richard Hanna*
South Carolina 2nd District
John Carter*
New York 22nd District
Tom Rooney*
Robert Wittman*
Texas 31st District
Tim Kaine
Virginia 1st District
Mike Conaway*
Virginia
Marco Rubio
Texas 11th District
Doug Lamborn*
Florida
John Cornyn
Colorado 5th District
C.A. Dutch Ruppersberger*
he following members of Congress have been invited to
address the M&S Community at I/ITSEC:
Robert Aderholt*
Oklahoma 1st District
Mo Brooks*
Texas
*denotes members of the
Congressional M&S Caucus
Maryland 2nd District
« M&S Caucus Chair addresses the
audience at the 2013 Congressional
M&S Event.
In addition to Congressman Forbes,
we were pleased to welcome
Congressmen Mica, Grayson and
Scott to I/ITSEC 2013.
TUESDAY, 2 DECEMBER
1030 – 1200 • HYATT REGENCY
WINDERMERE BALLROOM
SE-2
General/Flag
Officer Panel
RADM Robb, USN (Ret.)
Mr. DiGiovanni, SES
VADM Grooms, NATO
LTG Brown, USA
VADM Dunaway, USN
Lt Gen Bogdan, USAF
Mr. Thompson, SES
Dr. Brignoni
Moderator
Rear Admiral James A. Robb, USN (Ret.)
Vice Admiral David A. Dunaway, USN
President, National Training and Simulation
Association
Commander, Naval Air Systems Command
Panelists
Frank C. DiGiovanni, SES
Acting Deputy Assistant Secretary of Defense
(Readiness)
Vice Admiral Bruce E. Grooms, NATO (Invited)
Deputy Chief of Staff, Capability Development
at NATO Headquarters Supreme Allied
Commander Transformation
Lieutenant General Christopher C. Bogdan, USAF
Program Executive Officer, F-35 Lightning II
Joint Program Office
Dennis C. Thompson, SES
Executive Deputy of Training and Education
Command, Marine Corps Base Quantico
Gladys Brignoni, Ph.D. (Invited)
Deputy Commander, USCG Force Readiness
Command
Lieutenant General Robert B. Brown, USA
Commanding General, United States Army
Combined Arms Center
W
ith the increasing worldwide terrorism threat and the challenge to international preparedness,
this year’s Senior Officer Panelists will include in their remarks the global security challenge
and how the Services are coordinating training readiness to deter, defend against and defeat a
common enemy. There is no question that a period of great uncertainty continues to threaten our
way of life, particularly in light of sustained financial and personnel pressures. The preparedness
focus, therefore, is on the Services’ ability to have the same or greater capability with a reduced
force structure in the next decade. Technology contributions retain their significance in supporting
the war fighter regardless of the force size. At the same time, innovation and agility propel the now
and future force to a superior edge in the fight to emerge the winner while procuring victory not
just in combat but to overcome an ideology that seeks to destroy us.
28
TUESDAY, 2 DECEMBER
1400 – 1530 • ROOM S320GH
SE-3
Enabling Combat
Readiness:
Live, Virtual,
Constructive (LVC)
Training
Image courtesy of Peggy Frierson, Visual Information Specialist, DMA-Army Production, Fort Meade, Maryland
Moderator
Thomas H. Killion, Ph.D.
Acting Director, Office of Technology, Office of Naval Research
Panelists
Frank C. DiGiovanni
Acting Deputy Assistant Secretary of Defense (Readiness)
Terry Allard, Ph.D.
Head, Warfighter Performance Department, Office of Naval Research
BGen Joseph Shrader, USMC
Commanding General, MARCORSYSCOM
Brigadier General Giovanni K. Tuck, USAF
Director of Operations, Deputy Chief of Staff for Operations, Plans and Requirements,
Headquarters U.S. Air Force
Colonel David S. Cannon, USA
TRADOC Capability Manager Integrated Training Environment (TCM-ITE)
T
his year’s conference theme, “Trained and Mission Ready: Deter, Defend, Defeat” focuses on
the need for readiness for any situation. The I/ITSEC Service Principals are pleased to sponsor this event which will showcase senior training leaders across the U.S. Department of Defense.
The current and future state of Live, Virtual, Constructive (LVC) training will be addressed and
examples of programs will be presented by each Service. The discussion will include the vision
and challenges of using LVC training and the resulting benefits.
Session Chair:
Gerald Dreggors, Northrop
Grumman
WEDNESDAY, 3 DECEMBER
0830 – 1000 • ROOM S320GH
SE-4
The I/ITSEC
Fellows Series:
Metasimulation
I/ITSEC 2014 Fellow
 CEU Eligible
Andy Ceranowicz, Ph.D.
I/ITSEC Fellows is an
annual series of presentations by technical
leaders responsible for
the seminal contributions
that have fundamentally
shaped the simulation and
training capabilities being
delivered today. The paper
authored by Andy Ceranowicz, Ph.D. may be found
on the 2014 Proceedings
CD provided to conference
attendees. The abstract of
his paper is also included
in this Program Guide as
part of the 2014 I/ITSEC
Abstract Section.
W
e recognize Andy Ceranowicz, Ph.D. as the 2014 I/ITSEC Fellow. Dr. Ceranowicz has long
been recognized as a leading technical innovator within the broad simulation community.
While working for DARPA, he inspired and led the development of a line of Semi-Automated
Forces (SAF) systems including SIMNET SAF, ModSAF, and JSAF. Later, he led the integration of
the Millennium Challenge 2002 federation which linked together entity level simulations from all
the Services and served as the basis for the Joint Live Virtual and Constructive simulation used by
Joint Staff J7. At U.S. Joint Forces Command J9, he was responsible for large-scale federation development, including the development of the Urban Resolve federation, which used supercomputer
assets to simulate urban populations with hundreds of thousands of individual entities. Currently,
he is the Navy Continuous Training Environment (NCTE) Architect at the Navy Warfare Development Command (NWDC) and a Chief Scientist at Alion Science and Technology. At NWDC, Dr.
Ceranowicz is responsible for standards development to maintain interoperability in a worldwide
training environment that links live ships, virtual trainers, and constructive JSAF.
In this presentation, Dr. Ceranowicz draws on a uniquely extraordinary experience that started
with the DARPA SIMNET program and has continued through responsibility for today’s most widely
used simulation federations. He postulates a logical basis for the field of simulation as a whole
and applies the resulting concepts to current problems of maintaining fair play and interoperability
during simulation composition.
This event provides a rare opportunity to interact with a visionary leader as he describes a
ground-breaking basis for understanding simulation as a field of endeavor and its implications for
solving the most pressing problems we face today. Come, listen, learn, and share in the inspiration
for that which lies ahead!
Key quotes from Metasimulation by Andy Ceranowicz, Ph.D.:
“Fair play is a particular problem in composition of simulators caused by overlaps in functional
representations. . . The only way to eliminate fair play problems is to eliminate functional
representation overlaps.”
“Making simulators general, defeats reduction and causes an explosion in complexity and
resource requirements. Achievement of full generality can be harder than building real world
systems.”
“Simulation is used in an increasingly large segment of our scientific, social, economic,
entertainment, and government activities. Its expanding and sometimes invisible influence
makes it important to recognize and understand. However, each field specializes simulation
for its problem domain making it difficult to agree on a single definition.”
Session Chair:
Robert Richbourg, Ph.D., Institute
for Defense Analyses
30
“[Reference to the Army Ballistics Lab construction of the ENIAC in 1945.] Using a mechanical
calculator, a human computer could simulate a 60-second flyout in 20 hours, a mechanical
analog computer could complete the simulation in 15 minutes, and the ENIAC could do so
in 30 seconds. It was faster than a speeding bullet.”
THURSDAY, 4 DECEMBER
1030 – 1200 • ROOM S320GH
SE-5
Training the Human
Dimension in a
Time of Austerity –
The Opportunity of
Technology
Leveraging Tech
for Training
Moderator
Frank C. DiGiovanni
Panelists
Lieutenant General Frank Kearney, USA (Ret.)
President, Inside-Solutions, LLC
Doug Lenat, Ph.D.
CEO, Cycorp, Inc.
Colonel Thomas Meyer, USA
Chief, Human Dimension Division, ARCIC
Captain Wes Naylor, USN
Commander, NAWCTSD
N
Session Chair:
Benjamin Bell, Ph.D., Aqru
Research & Technology
ot since WWII has DoD faced funding and force reductions on par with what it is on the
horizon today. Training has historically been hard hit when budgets have fallen. How can
technology help trainers maintain force readiness? This event will discuss what technologies (e.g.,
modeling and simulation, artificial intelligence, virtual reality) could become or enable on-demand,
home station training to enhance or replace cancelled exercises.
Training leaders from both the military and industry will talk about training in today’s environment and how technology has transformed current training. Particular attention will be given
to how the commercial sector is coping in a more competitive environment of smaller defense
budgets; and where it foresees innovative training technologies satisfying greater defense needs.
NOTES
32
Focus Events
FOCUS EVENT
TUESDAY, 2 DECEMBER
1530 – 1700 • EXHIBIT HALL
FE-1
Warfighters Corner
Booth 2681
Moderator
DeLloyd Voorhees, Jr.
General Dynamics Information Technology
A
s combat operations draw down, our warfighters are still expected to perform heroically around
the globe without hesitation. In recognition of their service, Warfighters Corner provides an
opportunity to meet warfighters from across the Services to hear their personal experiences to
better understand the impact education and training has on their job performance.
Warfighters Corner presents multiple events during the conference featuring service men and
women recently returned from deployment. These Soldiers, Sailors, Airmen, Marines, and Coast
Guardsmen derive the great benefit from I/ITSEC and the organizations and industries that support
the conference. Many of the speakers have served multiple tours and will be sharing their stories,
their personal experiences, and their views of what was or was not effective in terms of the training
they received prior to deployment.
All Warfighters Corner sessions will include representatives from each of the Services. The
presenters will discuss operations and also provide insights into the role of Allies, international
organizations and private organizations in theater. The Wednesday morning session will be attended by veterans groups from the local Central Florida area. Don’t miss the opportunity to attend
these sessions!
Dynamic descriptions of how
training experiences benefitted
their service were presented by
members of all Services.
FOCUS EVENT
TUESDAY, 2 DECEMBER
1600 – 1730 • ROOM S320GH
FE-2
Ignite!
Moderator
John Aughey
The Boeing Company
CYBER IN FIVE…
20 SLIDES, 5 MINUTES: WHAT CAN YOU SAY?
I
magine a place where provocative talks by leading experts are delivered in 5 minutes each. If
you’ve been to I/ITSEC the past years, you’ve experienced just that, as well as the impact those
5-minute talks can have.
With data breaches, malware, spy bots, and internet hacks constantly in the news, we asked our
speakers to share their thoughts on Cyber. For this year’s edition of Ignite! at I/ITSEC, we ponder
the question, how does Cyber enable or constrain our ability to be Trained and Mission Ready?
Our distinguished speakers will inspire, stimulate, entertain, educate, and amaze you with their
take on Cyber and it’s impact on our world of simulation and training. They will touch on topics
ranging from the Internet of Everything to the Social Media Revolution. Their expertise and enthusiasm will surprise, compel, provoke, and perhaps even shock you, but you will be energized!
This special event features passionate presenters from industry, academia, and government
using their 5-minutes and 20 slides, auto-advancing every 15 seconds, to jam-pack information
into each talk for this dynamic and fun event.
Speakers
Michael Papay, Ph.D.
Ted Fastert
Ed Cashin
VP and CISO, Northrop
Grumman
M&S Cyber Experimentation
Lead, The Boeing Company
Research Scientist, Georgia
Tech Research Institute
Dylan Schmorrow
Scott Tousley
Patrick Lardieri
Chief Scientist, Soar
Technologies, Inc.
Deputy Director, Cyber
Security Division, Department
of Homeland Security Science
& Technology (DHS S&T)
Fellow for Cyber, Chief
Engineer of Lockheed
Martin’s National Cyber
Range, Lockheed Martin
Tim Aldrich
Tim Bloechl
M&S Cyber Experimentation
Lead, Boeing, SD&E Phantom
Works
Director, Cyber Security
Business Development,
Quantum Research
International
Dave Dampier, Ph.D.
Director, Distributed Analytics
and Security Institute,
Mississippi State University
Jeff Snyder
Vice President of Cyber
Programs, Raytheon
Bora Aytun
Session Chair:
John Dzenutis, The Boeing
Company
34
Co-founder & CEO, Mavi
Interactive, LLC
Chadwick Caison
Technical Director, Advance
Cyber Operations Sector,
KEYW Corporation
Thomas Nedorost, Ed.D.
Lecturer of Computer Science
and Information Technology,
FOCUS EVENT
1030 – 1200 • ROOM S320GH
FE-3
Improving Patient
Safety – Does
Simulation Help?
98,000 die annually. What can simulation do?
Moderator
C. Donald Combs, Ph.D.
Vice President and Dean, School of Health
Professions, Eastern Virginia Medical School
Panelists
George Blike, M.D.
Chief Quality and Value Office, DartmouthHitchcock Health System
Carol Cheney, M.S.
Senior Director, Health Management, Banner
Health System
Stephanie Sudikoff, M.D.
Director, SYN:APSE Center for Learning, Transformation, and Innovation, Yale-New Haven
Health System
A
ccording to the 1999 report “To Err is Human: Building a Safer Health System,” up to 98,000
people die in hospitals annually as a result of preventable medical errors. Even more are harmed,
adding to the negative impact of patient safety errors. Patient safety issues include wrong-site surgery, hospital-acquired infections, falls, hospital readmissions, diagnostic errors, and medication
errors. As expected, these errors add significantly to the cost of healthcare in the U.S., estimated
between $17 billion and $29 billion additionally per year.
The issues of health and patient safety have implications for the DoD and Veterans Health Administration (VHA), as well as the state of our nation’s homeland security. To help fix the problem,
the DoD Patient Safety Program was established to create a culture of patient safety and quality
within the Military Health System. Similarly, the Department of Veterans Affairs established the
National Center for Patient Safety, focused on reducing inadvertent patient harm as a result of their
care. Ensuring patient safety is definitely a priority.
As simulation gains popularity in civilian, DoD, and VHA healthcare education and training processes, it is important to consider simulation’s impact on patient safety and to ascertain where the
healthcare and simulation communities should make changes. Our panel of healthcare simulation
leaders will provide their perspective on how simulations are improving patient safety, and where
there is opportunity for continued improvement and additional development.
Session Chair:
Jen Murphy, Ph.D., Quantum
Improvements Consulting, LLC
FOCUS EVENT
1400 – 1530 • ROOM S320GH
FE-4
The power of mental fitness, emotional
intelligence, and “soft skills”
Mind Over Matter
Moderator
Frank C. DiGiovanni
Panelists
Major General T.J. Jones, USMC (Ret.)
Director, Outdoor Odyssey
Marc Brackett, Ph.D.
Director, Yale Center for Emotional Intelligence, Yale University
Lieutenant Colonel Scott Mann, USA Special Forces (Ret.)
Director, The Stability Institute
T
he Department of Defense must provide a broader set of skills so that Service members are
much more than physically fit. Warfighters need to be “mindful” and resilient — and ready
and able to adapt to unfamiliar operating environments and cultures. The areas of mental fitness,
emotional intelligence and “soft skills” development are becoming increasingly important to military
readiness, and stretching training and education programs in new directions.
This event will introduce this cutting-edge subject and its importance from several unique perspectives. Questions to explore include … What can we learn from the years of combat in Iraq and
Afghanistan and from the experience of those who served? How can mental fitness be fostered,
and what are the benefits for Service members (including those transitioning) and for our evolving
global military mission? What role can DoD and stakeholders play in the years ahead?
Session Chair:
Elaine Raybourn, Sandia National
Laboratories, ADL Initiative
36
FOCUS EVENT
1600 – 1730 • ROOM S320GH
FE-5
A Virtual World Cup
For Collaborating,
Understanding &
Partnering
Moderator/Panelist
Dale Whittaker
Program Manager FMS, U.S. Army PEO STRI
Panelists
Major General Mark McDonald, USA
Commanding General, USASAC
Aida Matta
International Program Director,
NAWCTSD
Brian Kummer
Assistant Director International
Programs Strategic Engagement, MCSC
Robin Toups
FMS New Business Program Manager,
USAF Simulators Division
T
his event will focus on Foreign Military Sales (FMS)
for security cooperation among government and industry
partners, both in the United States and those countries where FMS programs are approved. The
presentations will provide insight on global government organizations, programs with unique
requirements definition, and challenges faced by government leaders and industry in the complex
environment surrounding international training and M&S opportunities. The idea in sharing information is to explain how M&S may be accommodated in light of future business considerations.
In addressing the possibilities for potential future programs, speakers will discuss how modeling,
simulation and training play significant roles in building partner capacity and increasing global
cooperation for increased interoperability. The exchange of information and sharing of ideas, to
include audience participation, will encompass how the presenters can help you help yourself when
attempting procurement or delivery involving international programs’ unique challenges, rules
and regulations. In other words, here are the challenges and opportunities within the international
community of training and M&S, with industry and government expertise we can expedite program
execution and build partner capacity.
Session Chair:
K. Denise Threlfall, Ph.D., Kratos
Defense & Security Solutions
FOCUS EVENT
0830 – 1000 • ROOM S320GH
FE-6
The Quantified
Warrior: Enhancing
Assessment in the
Military
The
Warrior
TheQuantified
Quantified
Warrior
HUMAN
PERFORMANCE
• TECHNOLOGY
• BIG
DATA
• TRAINING
EDUCATION
HUMAN
PERFORMANCE
• TECHNOLOGY
• BIG
DATA
• TRAINING
& &EDUCATION
…for real military applications
…for real military applications
Moderator
This unique
Sae Schatz,
Ph.D. session brings together pundits from the U.S. and British militaries,
academia, and industry to discuss new approaches to comprehensive personnel
his
unique session
(titleintegration
inspired by
Gresham & psychometric
Stone (2012), tools,
Armed Forces
assessment,
including
of Blackhurst,
the latest technologies,
Journal) brings together pundits from the U.S. and U.K. militaries, academia, and industry
and system-design methods. The panel will highlight significant issues in the
to discuss new approaches to comprehensive warfighter assessment, including integration of the
measurement and assessment of human knowledge, skill, and performance in the military, and panelists will
latest technologies, psychometric tools, and system-design methods. The panel will highlight sigrecommend actions to overcome
thoseinobstacles.
In part,and
thisassessment
session is of
designed
help “bridge
nificant issues
the measurement
human to
knowledge,
skill,the
andgap”
performance
between theoretical science
and
military
application.
Although
panelists
will to
discuss
emerging
in the
military,
and
panelists will
recommend
actions
overcome
those concepts,
obstacles. the
In part, this
conversation will be grounded
indesigned
reality and
without
hyperbole.
* Opportunities
forscience
audience
* application.
session is
to help
“bridge
the gap” between
theoretical
andQ&A.
military
T
Although panelists will discuss emerging concepts, the conversation will be grounded in reality
and without hyperbole. * Opportunities for audience Q&A. *
Panelists
MG William Hix
Major General
U.S. ARMY
William
Hix, USA
Deputy
Director, Army
Douglas Hubbard
DougAUTHOR
Hubbard
AND CEO
Val Shute, Ph.D.
Val Shute,
Ph.D.
FLORIDA
STATE UNIVERSITY
Sqn Ldr NR Manders
John Tangney, Ph.D.
SqdLdr
Nigel
Manders OFFICE OF NAVAL
John
Tangney,
BRITISH
ROYAL
AIR FORCE
RESEARCH
Author
and CEO
State
BritishLeader,
Royal SO2
Air Force Director of thePh.D.,
SES
Internationally
recognized Florida
Professor
andUniversity
author of
Squadron
Human
expert
and
author
of
How
Innovative
Assessment
for
Training
Policy
at
HQ
No.
andDirector
Bioengineered
Internationally
Professor and author of
Squadron Leader,
of the Human
to Measure Anything
the 21st Century
22 (Training) Group
Division
(SES)
recognized
expert
Innovative
Assessment
Capabilities Integration
SO2
Training Policy at Systemsand
Bioengineered
and author of How to
for the 21st Century
Center, TRADOC
HQ No. 22 (Training)
Systems Division, ONR
WITH MODERATOR SAE SCHATZ,
PH.D.
Measure
Anything
Group
Capabilities
Integration
Deputy Director,
Army
Center, TRADOC
Session organized by participants from the Office of Naval Research and Joint Staff J7, Aptima, Cognitive Performance Group, and Soar Technology
Session title inspired by Blackhurst, Gresham, & Stone (2012), Armed Forces Journal
Session Chair:
Rene Thomas-Rizzo, ASN RD&A
38
FOCUS EVENT
1330 – 1500 • ROOM S320GH
FE-7
The Intelligence
Community’s
Challenge to
Training Innovators
Trained & mission ready for those who need to know
Moderator
Benjamin Bell, Ph.D.
Aqru Research and Technology, LLC
Panelists
Deborah Melançon
Chief, Regional Expertise and Culture Training, Defense Intelligence Agency
Colonel Dave Paschal, USA (Ret.)
Director, Operations, U.S. Army Training and Doctrine Command (TRADOC), Training Brain
Operations Center (TBOC)
Anthony Cerri
Director, Data Transformation Lab, U.S. Army Training and Doctrine Command (TRADOC),
Training Brain Operations Center (TBOC)
Richard Rennolds
Chief, Learning Development Branch, National Geospatial-Intelligence Agency, National
Geospatial-Intelligence College
T
Session Chair:
Capt. Jon Richardson,
MARCORSYSCOM PM TRASYS
he Intelligence Community (IC) shares with its sister agencies in the DoD continued budget
contraction and a growing array of complex threats. Across the IC, including DoD intelligence
components, organizations are facing unprecedented growth in the volume and diversity of data.
Analysts are contending with dramatic growth in open source information; with insurgent groups
sophisticated about social media and disciplined about OPSEC; and with historical superpower
adversaries whose actions once again are generating urgent needs for immediate and longer-term
intelligence. At the same time, slowed hiring and attrition are creating an acute need for training
in order to ensure a future talent pool of highly-skilled analysts.
This event addresses the question “how can the training and simulation community support
the intelligence analyst?” set against the backdrop of multiple complex and inter-related threats.
Training and education leaders in the IC will provide their insights into what analysts need and
discuss initiatives aimed at sustaining critical skills and enhancing capabilities to overcome emerging challenges. Panelists will also offer observations about the art of the possible and how future
learning technologies could support the intelligence analyst. A moderated Q&A will be followed
by an informal meet and greet.
NOTES
40
Community of Interest/
Program Briefs
COMMUNITY OF INTEREST/PROGRAM BRIEFS
TUESDAY, 2 DECEMBER
1400 – 1530 • ROOM S330C
C/PE-1
Warfighter Human
Dynamic Skills:
Results from An
Innovative Training
and Evaluation
Program
Session Chair: Karen Cooper,
Ph.D., NAWCAD
Moderator
Colonel Jack Guy, USA (Ret.),
Adaptive Execution Office,
DARPA
Participants
Lieutenant Colonel John
Grantz, USA, Ft. Benning;
Ajay Divakaran, SRI/
Princeton; Bruce Roberts,
Raytheon BBN Technologies;
Peter Tu, GE Systems; Aubrey
Logan-Terry, Georgetown
University; Michael van
Lent, SoarTech; Shawn Weil,
Aptima
Warfighters can create positive outcomes by learning to approach
social situations as the dynamic product of thoughts, perceptions,
attitudes, and behaviors. DARPA’s Strategic Social Interaction
Modules (SSIM) program aims to maximize the Warfighter’s ability
to adapt to and successfully manage all interactions – especially
high-risk, high-consequence – on any unfamiliar human terrain.
TUESDAY, 2 DECEMBER
1600 – 1700 • ROOM S330C
C/PE-2
Early Synthetic
Prototyping: Force
2025 Capability
Development Driven
by Warfighters Using
Games
Session Chair: Kristy Murray,
Ed.D., Summit Strategic
Consulting
Moderator/Panelist
Lieutenant Colonel Brian
Vogt, ARCIC
Participants
Chris McGroarty, STTC;
Rob Smith, TARDEC; Ernie
Garcia, ARDEC; Andrew Krug,
CERDEC; Dale Malabarba,
NSRDEC; Rudy Darken and
Major Kate Murray, NPS;
Simon Goerger, ERDC
Early Synthetic Prototyping
(ESP) seeks to elicit feedback
from Warfighters about deep
future capabilities. A distributed game offers a collaborative environment for warfighters, scientists, acquisition professionals,
and decision makers. Instrumented scenarios can be used to collect data from on or off-duty gameplay for evaluation to impact
system development and refinement. This brief will address ESP
efforts to date and plan for future development to bring ESP from
a concept to a reality.
0800 – 1700 • ROOM S310A
C/PE-3
Moderator
Jonathan Poltrack, ADL
Initiative
xAPI PlugFest
The Experience Application Program Interface (xAPI) PlugFest
Session Chair: Paula Durlach,
is provided by the Advanced
Ph.D., OSD
Distributed Learning team. All
registered I/ITSEC participants
are invited to participate in the
PlugFest or to simply observe
throughout the day. For those
not registered as Conference Attendees, there is a $75 charge to
participate in only the PlugFest. Contact karen.murgas.ctr@adlnet.
gov for details.
Spend the morning learning about how the xAPI works and
how it can be used to increase learning and performance effectiveness within your organization. The afternoon will be a unique
hands-on opportunity to engage in xAPI-enabled system design
and construction. ADL’s Technical Team, along with xAPI early
adopters who are using the xAPI specification, will be on hand
to facilitate the PlugFest and answer questions pertaining to the
xAPI. To participate fully, registrants should bring their own wifi-enabled laptop to the afternoon session so they can download
files if necessary.
The xAPI enables a learning environment that takes advantage
of open source, cutting-edge technical learning applications and
delivery systems. The xAPI provides the means to track learning
no matter how the training is accomplished (e.g., from a mobile
device, social media, videos, games, virtual environments, or
full-scale simulators); share tracked learning-data among training systems; and helps instructors, tutors, mentors and administrators establish pathways to faster and more efficient expert
achievement.
0830 – 1000 • ROOM S330C
C/PE-4
Interoperable
Performance
Assessment within
Live-VirtualConstructive Events
Moderator
Mike Hruska, Problem
Solutions
Participants
Rodney Long, ARL; Chuck
Amburn, ARL; Nathan Jones,
PM TRASYS; Ben Goldberg,
Ph.D., ARL STTC; Mike
Hruska, Problem Solutions;
Session Chair: Eric Jarabak,
Tiffany Poeppelman; Matt
Roberts, Aptima; Tara
Kilcullen, Raydon; Brandt
Dargue, Boeing; Nick
Washburn, Riptide; Eric Watz, Lumir
The need for more efficient and effective training continues to
increase across many domains. This event focuses on collecting
standardized human performance data to support analytics and
COMMUNITY OF INTEREST/PROGRAM BRIEFS
adaptive systems in a variety of domains. Thought leaders from
industry and government will cover current efforts, technologies,
challenges, and future focus areas.
1030 – 1200 • ROOM S330C
C/PE-5
Simulation &
Training in Europe
– the Government
Perspective from a
Number of Nations
Moderator
Graham McIntyre, Chairman
of ETSA
Participants
EDA – Jean Marchal;
France – Lionel Khimeche,
Manager Etudes TechnicoOperationnelles; Italy – Lt Col
AF Stefano Giacomozzi, M&S
Session Chair: Kevin Cahill,
Center of Excellence; Sweden
Aero Simulation, Inc.
– Lt Col Arne Norlander,
Ph.D., R&D Program Director,
Policy and Plans, Swedish Armed Forces
Maintaining training in a harsh economic environment is a subject
at the forefront of the minds of the leadership of Government and
Industry throughout the world. The drive to deliver cost effective
and cost efficient training is, and remains, a constant theme. The
flexibility and adaptability of modelling and simulation is increasingly important in helping to meet budget constraints in Europe.
ETSA will offer a European view on Simulation and Training in
Europe, the appeal of I/ITSEC and what more I/ITSEC might offer.
1400 – 1530 • ROOM S330C
C/PE-6
Veterans Health
Administration:
Next Generation
Simulation-based
Training Initiatives
Session Chair: Robert Hester,
Ph.D., University of Mississippi
Medical Center
Moderator
CAPT Harry Robinson,
USN (Ret.), Veterans Health
Administration SimLEARN
Participants
Manny Dominguez, Ph.D.,
Deputy Chief Learning
Officer, Employee Education
System; Haru Okuda, M.D.,
SimLEARN National Medical
Director; Lygia Arcaro,
Ph.D., R.N., B.C., SimLEARN
National Director for Nursing
Programs
0830 – 1000 • ROOM S330C
C/PE-7
Geospatial
Environmental
Database Standards
Forum
Moderator
Bill Hopkinson, JTIEC
Participants
Frank Rhinesmith, Army
SE Core, U.S. Army PEO
STRI; Darryl Steward, Air
Force Common Dataset
(AFCD), USAF Simulators
Session Chair: Roy Scrudder.
Division; Earl Miller, SOCOM
The University of Texas at
Austin, Applied Research
Common Data Base (CDB),
Laboratories
USSOCOM; Bruce Riner, Navy
Portable Source Initiative
(NPSI), NAWCTSD; Rob Cox, Ph.D., Rapid Data Generation
(RDG), U.S. Army PEO STRI; Jean-Louis Gougeat, Reuse and
Interoperation of Environmental Data and Processes (RIEDP),
Sogitec Industries, SA
The Geospatial Environmental Database Standards forum provides
a unique opportunity for major Components’ geospatial database
producers to update DoD dataset consumers and suppliers on
products, processes, and challenges; and explore new capabilities
such as emerging standards and services. The result is reduced
data production times to empower the agile force.
1030 – 1200 • ROOM S330C
C/PE-8
Trends In
Instructional
Systems Design
(ISD)
Session Chair: Fred Fleury,
ZedaSoft, Inc.
Moderator
Nathan Jones,
MARCORSYSCOM PM
TRASYS
Participants
Kelsey Henderson Ph.D.,
NAWCTSD; Nathan Jones,
MARCORSYSCOM PM
TRASYS; Atsusi Hirumi,
Ph.D., UCF
ISD facilitates the acquisition
of knowledge and skill. It spans military training, academia, HR
training, and through all industries. In training systems acquisitions, ISD is a key part of the process for defining training needs,
objectives, tasks, delivery of training, and evaluation of training
effectiveness. This panel will be presenting current trends in
these areas.
VHA, the nation’s largest health care provider system, is implementing revolutionary use of technologies to leverage IT capabilities for information sharing and collaboration that sustain
simulation-based clinical training, education, and research in a
collaborative network for 150 VA Medical Centers and clinical
facilities serving both providers and Veteran patients.
42
I N T E R N AT I O N A L P R O G R A M S
International Pavilion Room S310E-H
International attendees can meet and connect with counterparts from
around the world. Limited private meeting space is available on a firstcome, first-served basis to our international participants and may be
scheduled at the International Pavilion’s Welcome Desk. Additional information about the many international activities throughout
I/ITSEC are readily available in the International Pavilion. Sponsored by
AVT Simulation.
International Registrants should register at the dedicated International Check-in station positioned near
the Main Registration Desk in S220 of the South Concourse. International Conference Attendees’
Meeting Bags will be available for pick-up at the Welcome Desk in the International Pavilion. More
information specific to international attendees will be available at that location.
International Pavilion Hours of Operation
Sunday, 30 November
1400-1800
Monday, 1 December
0800-1800
Tuesday, 2 December
1030-1800
Wednesday, 3 December
0800-1500
Thursday, 4 December
0800-1500
Program Notes of Special Interest for International Attendees
Papers
Explore your Program for the ñ indicating Papers from International Authors.
Tutorials
Monday, 1 December • Room S320A • 0830 – 1000
Introduction to HLA
Monday, 1 December • Room S320E • 1430 – 1600
Speech-based Interaction: Myths, Challenges, and Opportunities
ITEC Best Paper
ME5 Ryan Ng Woon Teck,
Singapore Armed Forces, Chief
Instructor for Air Engineering
Traning Insitute School
A Maintenance Simulator
for AF Engineers: The RSAF
Experience
Events
Wednesday, 3 December • Room S320A • 0830 – 1000
s
SimTecT Best Paper
Jessica Parker, Air Operations
Divison, Defence Science &
Technology Organisation
A Comparison of Visual Display
Systems for a Low-Cost Mission
Training Flight Simulator
Best Papers from Around The Globe
Wednesday, 3 December • Room S330C • 1030 – 1200
Simulation & Training in Europe – the Government Perspective from a Number of Nations
Wednesday, 3 December • Room S320GH • 1600 – 1730
A Virtual World Cup For Collaborating, Understanding & Partnering
Check in the International Pavilion for information about the International Reception on Wednesday evening.
SPECIAL GUESTS
Central Florida Veterans Organizations
The citizens of the United States are more supportive of today’s men and women in Military
Service than they have been since WWII. The Overseas Contingency Operations continue to be
of great concern to all, whether in uniform or not. This I/ITSEC effort is to especially inform
U.S. Veterans about the dramatic changes in training methodologies and systems since their
A veteran…is
someone who, at one
point in his or her
life, wrote a blank
time in Service. Since 2006, I/ITSEC has had the pleasure and honor to have select members of
check made payable
Central Florida Veterans Organizations visit the exhibit floor on Wednesday. The successful
to The United States
experiences in broadening the Veterans’ understanding of today’s training transformation as
of America for an
well as the appreciation of the other I/ITSEC attendees in seeing the Veterans, some in their
amount of “up to and
uniforms and with their decorations, have made this an annual event for I/ITSEC. When you
including my life.”
see these Veterans, thank them for their service to the Nation.
44
Papers
PAPER SESSIONS
TUESDAY, 2 DECEMBER
ROOM
S320A
S320B
S320C
S320D
S320E
S320F
ROOM
S320A
S320B
S320C
S320D
S320E
S320F
SESSION/CHAIR
1400
1430
1500
P-1 Managing Resources at
Home and Abroad
Randy Allen, Ph.D.
Resource Implications of the
Difference between Models and
Simulations (14020) 
COTS to Capability: Lessons Learnt Perspectives on Exportability and
from UK MOD Research Programme Program Protection in Virtual
(14115) ñ
Training Systems (14265) 
S-1 Leveraging Cloud
and High Performance
Computing Environments
Nick Giannias
Embracing The Cloud – Providing
Simulation as a Service (14018) 
Cloud Terrain Generation and
Visualization Using Open
Geospatial Standards (14308) %
Enabling External Player
Connections To Kerberos-secured
Systems (14202)
EC-1 Game On
John Aughey
Considerations on Utilizing a Game
Engine as an Image Generator
(14274)  %
Towards Minimalist Serious Game
Design (14348) %
Human Motion Capture in Natural
Environments (14116) 
T-1 Keepin’ It Real with VR
Susan Coleman, Ph.D.
Declarative Knowledge Acquisition
in Virtual Learning Environments
(14005)
Virtual World Room Clearing: A
Study in Training Effectiveness
(14045) 
Simulating Participant Training
Data to Test Mixed-reality Training
Systems (14252)
ED-1 Designing Educational
Games
Brian Stensrud, Ph.D.
Transmedical and Paramedial
Serious Game Deployment (14375)
%
Effectiveness of Embedded Gamebased Instruction: A Guided
Experiential Approach to Technology-based Training (14198)   %
An Experiment to Evaluate the
Effect of Narrative Delivery in
Military Training (14081) ñ %
H-1 The Body Speaks –
Stress and Workload
Kelly Hale, Ph.D.
Classifying Stress in a Mobile
Environment (14195) 
Investigation of the Sensitivity of
Physiological Performance and
Subjective Measures for Identifying
Changes in Novice Intelligence
Analyst Workload (14035)
Training with Adaptive Systems:
Utility of Baroreflex Sensitivity
(14297)  h
SESSION/CHAIR
1600
1630
1700
P-2 Commonality and
Data Sharing in the LVC
Environment
Robert Matthews
LVC, Translating DoD Policy into
Action (14059)
Data Sharing: The Standard
Specification is Just the Start
(14130)
Establishing Sharing for Geospatial
Environment Data (14255) 
S-2 LVC Interoperability
Bob Kleinhample
Sensor Placement Optimization in
LVC Environments for Training,
Analysis, and Operational
Applications (14314)
Towards Interoperability of
Simulations Systems of Ground
Force: Progress and Challenges
(14082) ñ
Integrating Distributed Virtual
Command and Control Platforms
into Live Training (14318) 
EC-2 Best Simulator
Technologies Ever
Michael O’Connor
The Largest Field of View
Collimated Display Ever Built
(14322)  
High Quality Visual Display
Systems in Physically Constrained
Environments (14248)
Dynamic Flight Simulation: 45
Years of Research & Development
(14019) 
T-2 Nothing Here To See
Brian Cairns
Enhancing the Utility and
Effectiveness of Combat Medic
Simulation (14075)  h
Training Effects for First-responder
Competency in Cholinergic Crisis
Management (14241) h
Simulation Environments for
Offshore Oil and Gas Emergency
Training (14344)  ñ
ED-2 Inspire, Design,
Execute: Simulation in Three
Domains
Nina Deibler
Modeling and Simulation
Challenge Problems in High School
Classrooms and Internships:
Lessons Learned (14103)
Using Unity to Implement a Virtual
Crash Site Investigation Laboratory
in Support of Distance Learning
Objectives (14050)
Developing the Simulator
Instructor’s Pedagogical
Competence (14043)  ñ
H-2 Shooting for Effective
Virtual Training
Martin Bink, Ph.D.
A Novel Approach to Determine
Integrated Training Environment
Effectiveness (14011)  
Pistol Skill Acquisition and
Retention: A 3-Year Longitudinal
Study (14042)   ñ
Effective, Live, Virtual, and
Constructive Training: Personnel
Recovery’s Weeklong Experience
with Integrated Sensor Technology
(14216)
LEGEND (one or more of the following may appear on this page). The number in parentheses following Paper Title is the ID tracking number.
 Nominated for Best Paper Award  Honorable Mention for Best Paper Award
 Continuing Education Units (see pg. 12) % Game-related Subject Matter
ñ International Author h Medical-related Subject Matter
Mobile
45
PAPER SESSIONS
ROOM
S320A
S320B
S320C
S320D
S320F
ROOM
SESSION/CHAIR
0830
0900
0930
ED-7 Best Papers from
Around the Globe
Robert “Buddha” Snyder
SimTecT Best Paper
A Comparison of Visual Display
Systems for a Low-cost Mission
Training Flight Simulator ñ
ITEC Best Paper
A Maintenance Simulator for AF
Engineers: The RSAF Experience ñ
S-3 Simulated Movement
Ron Dionne
Mission Integrated Simulation – A
Case Study (14085)  ñ
An Instructor Operating System
(IOS) Framework for Interactive
Instructor-station (14112)  ñ
Development of a Microscopic
Artificially Intelligent Traffic Model
for Simulation (14003)
EC-3 Emotional Engagement
Anya Andrews, Ph.D.
Design and Evaluation of Surprise
Effects in Simulation –
A Framework (14193) ñ
Validated Development of Stress
Inoculation through Cognitive and
Biofeedback Training (14051) 
Using Virtual Reality as Part of an
Intensive Treatment Program for
PTSD (14079) 
T-3 Comparative Analysis for
Clinical Training
Teresita Sotomayor, Ph.D.
Comparison of the Usability of
Robotic Surgery Simulators
(14168)   h
Forces Applied on Laryngoscope
during Intubation: A Study on
Airway Simulators (14203)  h
Outcomes from Two Forms of
Pediatric and Neonatal Intubation
Training (14240)  h
H-3 A Cornucopia of Human
Behaviors
Robert Sottilare, Ph.D.
In Search of Interoperability
Standards for Human Behavior
Representations (14027)  ñ
Factors Impacting Performance
in Competitive Cyber Exercises
(14108)
Cognitive Processing Considerations
of the Small Unmanned Ground
Vehicle (14134)
SESSION/CHAIR
1030
1100
1130
BP-1 Best Papers from HSE;
PSMA; Training
Karen Pogoloff
Measuring Visual Displays’ Effect
on Novice Performance in Door
Gunnery (14012) (HSE) 
Proficiency Evaluation and Costavoidance Proof of Concept M1A1
Study (14055) (PSMA)
Missing: A Serious Game for the
Mitigation of Cognitive Biases
(14295) (Training) 
S320B
S-4 Culture, Reaction and
Movement: Simulating
Human Behavior
Pete Schrider
Teaching Cross Cultural Social
Competence in a Dynamic,
Synthetic Environment (14289)

Advanced Animation Techniques
in a Dismounted Soldier System
(14136) 
Game-based Simulation for
Philippine Post-typhoon Stability
Operations Training (14329) %
S320C
Microgames for Training Perceptual
EC-4 Training Hard to Train
Skills (14282) %
Skills: Perception,
Sensemaking and Adaptability
Kent Gritton
Beyond Socio-cultural Sensemaking: Developing Effective Adaptive
Observing and Interpreting Patterns Training Systems to Enhance
of Life (14101)
Military Instruction (14140)
S320D
T-4 “Assess”-orizing Your
Training and Performance
Outcomes
Eliot Winer, Ph.D.
Data & Analytics Tools for Agile
Training & Readiness Assessment
(14064) 
Lessons Learned Integrating Mobile
Technology into Two Army Courses
(14128)
Experience API and Team
Evaluation: Evolving Interoperable
Performance Assessment (14157)

S-5 Decision Support
Systems & Methods
Sandy Veautour
A Decision Aid for Optimizing
Experimental Design Involving
LVC Environments (14139) 
A Practitioner’s Approach Using
MBSE in Systems of Systems
(14383) 
Robotic Simulators: A Case for
Return on Investment (14129)
h
H-4 Plane, Train and
Automobile
John Schlott
Leveraging Simulation to Augment
Risky Driving Attitudes and
Behaviors (14004)
Simulation-based Analysis of
the Human Factors Related to
Autonomous Driving (14006)
The Virtuous Circle and
Contextualized Knowledge
Elicitation: Application of a New
Paradigm for Job Analysis (14347)
S320A
S320E
S320F
46
Ï Cyber Security
Mobile
PAPER SESSIONS
ROOM
S320A
S320B
S320C
S320D
S320E
S320F
ROOM
S320B
S320C
S320D
S320E
S320F
S320A
SESSION/CHAIR
BP-2 Best Papers from
Education; ECIT; Simulation
Kelsey Henderson, Ph.D.
1400
1500
From Design to Conception: An
Assessment Device for Robotic
Surgeons (14170) (ECIT)ñ h
Exploration of Soldier Morale Using
Multi-method Simulation Approach
(14215) (Simulation)
Creating a Re-useable Knowledge
S-6 Innovative Approaches
to Environment and Behavior Repository for UK MoD CGF
Behaviours (14080) ñ
Simulation
Scott Hooper
Improving Air-to-Air Combat
Behavior Through Transparent
Machine Learning (14298)  ñ
Improving Material Classification
Quality with Elevation-derived
Metrics (14380) 
EC-5 Perspective on Data
and Services
Eric Weisel, Ph.D.
Future of LVC Simulation: Evolving
Towards The MSaaS Concept
(14072) ñ
Rapid Data Generation: A Flexible
Data Discovery and Access
Architecture (14096) 
Optimizing Supervised Learning
for Pixel Labeling and Material
Classification (14016) 
T-5 Critical Flight Decisions
Bob Johnson
Developing and Evaluating
Performance Measures for MannedUnmanned Teaming (14024)
Using Temporal Occlusion to Assess Distributed Live/Virtual
Carrier Landing Skills (14171) 
Environments to Improve Joint
Fires Performance (14041)
ED-3 Building It Right
Perry McDowell
An Instructional Media Selection
Process for Virtual World Training
Delivery (14369)  
Developing the Human Dimension:
Current Practices and Future
Methods (14161)
Hey, Your E-learning Courses are
Giving Me a Cognitive Overload
(14008) ñ
H-5 See, Hear and Speak –
No Evil Assessments
Jerry Stahl
Serious Game User Data Analysis
and Visualization: Savoring the
Breadcrumbs (14377) %
Sonification: The Sound of Big
Training Data (14261)
Creating a Learning Infrastructure
Where Every Soldier Can Be an
Instructor (14124)
SESSION/CHAIR
Institutionalizing Blended Learning
into Joint Training: A Case Study
and 10 Recommendations (14208)
(Education)
1430
1600
1630
1700
S-7 Improving Healthcare
with Simulation
Gerald Dreggors
Improving and Proving Healthcare
Quality and Value through Physical
Simulation (14144)  h
Employing Modeling and
Simulation to Improve Patient Care
(14034)  h
The Effect of Difficulty Levels
within a Virtual Medical Simulation
(14228) h
EC-6 Simulation: Supporting
Experiment, Acquisition and
Military Planning
James (Josh) Jackson
Early Synthetic Prototyping:
Exploring New Designs and
Concepts Within Games (14133)
%
When Tradespace Analysis Met
Combat Modeling and Simulation
(14264)
Simulation in Support of Course of
Action Development in
Operations (14119)  ñ
T-6 Combating Stress:
Performance Under Fire
Elaine Raybourn, Ph.D.
Accelerating Unit Adaptability: A
Principle-based Approach to Unit
Communication (14038)
Inducing Stress in Warfighters
during Simulation-based Training
(14201) 
ED-4 Practice What You
Preach
Jan Brown
Rediscovering the Eightfold Path:
Some Observations on Using
Simulation for Training and
Evaluation from Afghanistan
(14196) ñ
Mobile Instructional Strategy
Templates for Guided Mobile
Content Development (14194)
P-3 Wave of the Future
Chuck Secard
Agile Program Management on
Simulations in the Cloud – A
Continuous Monitoring of
Software Intensive Training Systems Manager’s Challenge (14104)   Cybersecurity in a Training System
(14311) 
Environment (14121)   Ï
T-11 Simulation &
Game-based Training
Fred Fleury
Research Directions for Future
Simulation-based Training Design
in Defence (14017) ñ
Are They Mission Ready? Using the
Modified Angoff Method to
Set Cut Scores (14060) 
Utilizing Simulation and Gamebased Learning to Enhance Incident
Commander Training (14148)
%
Mobile
47
PAPER SESSIONS
ROOM
SESSION/CHAIR
0830
0900
0930
S320A
EC-7 What to Train
John Dzenutis
Fundamental Competency Sets
(FCS) Definition to Support
Technology Development for Pilot
Training (14126)
Joint Terminal Attack ControllerTraining Rehearsal System:
Competency-based Research
(14097)
Operator Qualification Differences
between Manned and Unmanned
Aerial System (UAS) (14287)
S320B
S-8 Synthetic Environment
Toni Hawkins-Scribner
A Paradigm Shift in the Test and
Evaluation of Terrain Databases
(14200) 
Measuring the Impact of Natural
Environment Representation
on Combat Simulation Outcomes
(14305) 
Implementation of Real-time Snow
Layers in Game-based
Simulation (14361)  %
S320C
EC-8 More Braaiinnss
Jennifer Murphy, Ph.D.
Videogame Design for Cognitive
Enhancement through Micropuzzle Cognitive Profiling (14039)
%
Game-based Training to Mitigate
Three Forms of Cognitive Bias
(14180)  %
Enhancing Intuitive Decision
Making Through Implicit
Learning (14253) 
T-7 Intelligent Tutors: Just
How Smart Are They?
Anne Little, Ph.D.
Developing and Evaluating an
Intelligent Tutoring System for
Advanced Shiphandling (14014) 
Developing Models of Expert
Performance for Support in an
Adaptive Marksmanship Trainer
(14214) 
A Digital Tutor for Accelerating
Technical Expertise (14272) 
ED-5 Is Skill Development
Your Specialty?
Liz Gehr
Retention and Retraining of
Integrated Cognitive and
Psychomotor Skills (14220) h
An Individualized Approach
to Remediating Skill Decay:
Framework and Applications
(14229) h
Assertiveness and Responsiveness
in Teams: Essential for Mission
Command (14197) ñ
S-9 Emerging Visual
Technologies
Randy Crowe, Ph.D.
Integration of Low-cost HMD
Devices in Existing Simulation
Infrastructure (14190) ñ
Simulating Realistic Light Levels in
Next Generation Image Generators
(14233) 
Vergence and Accommodation in
Simulation and Training with 3D
Displays (14147) 
S320D
S320E
S320F
ROOM
S320A
S320B
S320C
S320D
S320E
S320F
S330D
SESSION/CHAIR
1030
1100
1130
EC-9 Medical Matters
Beth Pettitt
Challenges to Upgrading a Mobile
Web Application (14057)  h
Development and Evaluation of
a Humeral Head Intraosseous
Training System (14058) h
Comparative Analysis of
Holographic Display and ThreeDimensional Television (14230)
S-10 IG, Synthetic
Environment and Scalable
Simulation
Carla Cropper
A Distributed Scene Graph
Approach to Scaled Simulation
Based Training Applications
(14033) 
Pseudo-Specific High-resolution
Data Boundary Techniques
(14317) 
Solving the Innovator’s Dilemma
for Simulation and Training Image
Generator Architectures (14373) 
EC-10 Get a Grip on Reality
Scott Ariotti
Live Augmented Reality Based
Weapon Training for Dismounts
(14093)
Augmented Reality Virtual Personal PERLS: An Approach to Pervasive
Assistant for Training, Maintenance, Assistance in Adult Learning
and Repair (14031)
(14335)
T-8 Aviation Training: The
Ups and Downs!
Michael Motko
Evolving Aviation Live Training in
the Future (14078)
Training Fidelity of an Unmanned
Aerial Systems Complementary
Family of Trainers (14135) 
ASOC Training Research: Joint
Theater Air Ground Simulation
System (14166)
T-9 A Variety of Training
Approaches
Cynthia Adams
Fusing Self-Reported and Sensor
Data from Mixed-reality Training
(14158)
A Competency Based Approach to
Marine and Weapons Engineering
Training (14224)  ñ
Scenario-based Training for
Development of Leader —
Subordinate Mental Models and
Cohesion (14333)
S-11 Automation and
Autonomy
Brent Terwilliger, Ph.D.
A Framework for Enabling Virtual
Observer Controllers in Synthetic
Training (14268)
UAV Flight Control Software
Development based on COTS
Product (14206) ñ
Lessons Learned in Creating an
Autonomous Driver for OneSAF
(14106) 
Future Leaders Presentations
Future Leaders Pavilion participants present their projects during this special Paper Session.
48
Ï Cyber Security
Mobile
PAPER SESSIONS
ROOM
S320B
S320C
S320D
S320E
S320F
SESSION/CHAIR
1330
1400
1430
S-12 Engineering-based
Modeling
Mark Soodeen
How the U.S. Navy is Migrating
from Legacy/Large Footprint to
Low Cost/Small Footprint Sonar
Simulation Systems (14090) 
Use of Automated Intelligent
Entities in ASW Simulation (14109)
ñ
A Physics-based Approach to
Simulate Jet Engines (14030) 
EC-11 Networking Cross
Domain Solutions: Cross
Domain Solutions, Cloud,
Social
Susan Sherman
Implementing Stateless Cross
Domain Solutions to Continuously
Maintain Security Assurances
(14301)
Cybersecurity Impacts of a Cloud
Computing Architecture in Live
Training (14120)  Ï
Using Social Network Analysis to
Model the Spread of Misinformation
in Simulated Environments (14205)
T-10 Evaluating Training
Effectiveness
Paul Lyon
Post-fielding Training Assessment
of Dismounted Infantry Simulation
(14022)
Evaluating the Impact of Individual
Training on Units’ Operational
Performance (14123) 
Using LMS Technology for
Kirkpatrick Level 3 Evaluation of
Human Trafficking Training (14162)
Tailoring Multimedia Instruction to
ED-6 Automatic for the
People: Content Alignment for Soldier Needs (14049) 
Instructional Performance
Mark Friedman
Automated Content Alignment for
Adaptive Personalized Learning
(14068)
An Army Learning Model
Implementation: Challenges,
Successes, Future Directions
(14153)
H-6 Shocking Medical
Assessments
Jennifer Arnold
Assessment Instrument Validation
for Critical Clinical Competencies:
Pediatric-neonatal Intubation and
Cholinergic Crisis Management
(14232) h
A Decision Support System
Predicting Imminent Cardiovascular
Shock (14343)  h
&
Quantitative Assessment of Combat
Casualty Skills (14191) h
Reception
Awards
T H U R S D AY E V E N I N G
banquet
AT THE HYATT REGENCY
V O IC E S O F LI B E R TY
(formerly Peabody Orlando)
TRANSPORTATION: BUSES WILL RUN TO AND FROM
ALL I/ITSEC HOTELS (EXCEPT THE HILTON AND ROSEN CENTRE) AND THE HYATT REGENCY FROM 1730-2300.
Mobile
49
PAPERS/AUTHORS
BEST PAPERS
BP-1
WEDNESDAY, DECEMBER 3 • 1030-1200 • S320A
Best Papers from Human Systems Engineering; Policy,
Standards, Management, and Acquisition; Training
Session Chair: Karen Pogoloff, MTS Technologies
Session Deputy: Robert Wallace, USAF 29th Training Systems
Squadron
Measuring Visual Displays’ Effect on Novice Performance in
Door Gunnery (14012) (HSE) Jonathan Stevens, Ph.D.,
Army Research Laboratory (ARL); Peter Kincaid, Ph.D.,
Proficiency Evaluation and Cost-avoidance Proof of Concept
M1A1 Study (14055) (PSMA) Robb Dunn, Ph.D.,
Innovative Reasoning, LLC; Tim Cooley, Ph.D., DynamX
Consulting; Steven Gordon, Ph.D., Georgia Tech Research
Institute
Missing: A Serious Game for the Mitigation of Cognitive Biases
(14295) (Training)Carl Symborski, Meg Barton, Mary M.
Quinn, Ph.D., Leidos, Inc.; Carey K. Morewedge, Ph.D., Karim
S. Kassam, Ph.D., Carnegie Mellon University; James H. Korris,
Creative Technologies, Inc.
BP-2 WEDNESDAY, DECEMBER 3 • 1400-1530 • S320A
Best Papers from Education; Emerging Concepts and
Innovative Technologies; Simulation
Session Chair: Kelsey Henderson, Ph.D., NAWCTSD
Session Deputy: Karen Williams, U.S. Army PEO STRI
Institutionalizing Blended Learning into Joint Training: A Case
Study and 10 Recommendations (14208) (Education)
David Fautua, Ph.D., Joint Staff J7; Sae Schatz, Ph.D., MESH
Solutions, LLC; Emilie Reitz, Alion Science and Technology;
Patricia Bockelman, Ph.D., MESH Solutions, LLC
From Design to Conception: An Assessment Device for Robotic
Surgeons (14170) (ECIT)ñ h Alyssa Tanaka, Gareth
Hearn, Roger Smith, Ph.D., Florida Hospital Nicholson Center;
Manuela Perez, M.D., Ph.D., University Hospital of Nancy,
France; Mireille Truong, M.D., Khara Simpson, M.D., Columbia
University Medical School
Exploration of Soldier Morale Using Multi-method Simulation
Approach (14215) (Simulation) Mariusz Balaban, Thomas
Mastaglio, Ph.D., MYMIC, LLC; John Sokolowski, Ph.D., Barry
Ezell, Ph.D., Old Dominion University
E D U C AT I O N
The Best Paper for this category will be presented on Wednesday in Room S320A at 14
INSTITUTIONALIZING BLENDED LEARNING INTO JOINT TRAINING: A CASE STUDY AND 10 RECOMMENDATIONS (14208)
ED-1
TUESDAY, DECEMBER 2 • 1400-1530 • S320E
Designing Educational Games
Session Chair: Brian Stensrud, Ph.D., Soar Technology, Inc.
Session Deputy: Brian Vogt, U.S. Army TRADOC
Transmedial and Paramedial Serious Game Deployment
(14375) % Brandt Dargue, The Boeing Company; Dov
Jacobson, GamesThatWork; John Sanders, Historical Online
Learning Foundation
Effectiveness of Embedded Game-based Instruction: A Guided
Experiential Approach to Technology-based Training (14198)
  % Heather A. Priest Walker, NAWCTSD; Robert E. Wray,
Soar Technology, Inc.
An Experiment to Evaluate the Effect of Narrative Delivery
in Military Training (14081) ñ % Mark Lewis, Cranfield
University, Defence Academy of the UK; Professor Robert J.
Stone, University of Birmingham, UK
ED-2
TUESDAY, DECEMBER 2 • 1600-1730 • S320E
Inspire, Design, Execute: Simulation in Three Domains
Session Chair: Nina Deibler, Serco, Inc.
Session Deputy: Joe Dalton, Full Sail University
Modeling and Simulation Challenge Problems in High School
Classrooms and Internships: Lessons Learned (14103) Jennifer
Winner, Jerred Holt, Lumir Research Institute, Inc.; Kimberly
Puckett, Leesa Folkerth, Tri-Village Local School District;
Amelia Malone, University of Maryland
Using Unity to Implement a Virtual Crash Site Investigation
Laboratory in Support of Distance Learning Objectives (14050)
Christina Tucker, Jimmy Moore, Pinnacle Solutions, Inc.
Developing the Simulator Instructor’s Pedagogical Competence
(14043)  ñ Peter Sjoestedt, Royal Danish Defence College
ED-3
WEDNESDAY, DECEMBER 3 • 1400-1530 • S320E
Building It Right
Session Chair: Perry McDowell, Naval Postgraduate School
Session Deputy: Tiffany Parrish, NAWCTSD
An Instructional Media Selection Process for Virtual World
Training Delivery (14369)   Leslie A. L. Mazzone,
Submarine Learning Center; Anh Bao Nguyen, NUWC
Developing the Human Dimension: Current Practices and
Future Methods (14161) Michael Prevou, Ph.D., Strategic
Knowledge Solutions, Inc.; Laurie Waisel, Ph.D.
Hey, Your E-learning Courses are Giving Me a Cognitive
Overload (14008) ñ Commander Geir Isaksen, Norwegian
Defense University College
ED-4
Practice What You Preach
Session Chair: Jan Brown, CAE USA
Session Deputy: Adelle Lynch, Rockwell Collins
Rediscovering the Eightfold Path: Some Observations on Using
Simulation for Training and Evaluation from Afghanistan
(14196) ñ Christopher Huffman, Ph.D., Canadian Defense
Academy
Mobile Instructional Strategy Templates for Guided Mobile
Content Development (14194) Peggy Kenyon, Ph.D., Helen
Remily, U.S. Army TRADOC; Dennis Wikoff, Adayana
Are They Mission Ready? Using the Modified Angoff Method to
Set Cut Scores (14060)  Ingrid Mellone, Carol Faben, Camber
Corporation
50
Ï Cyber Security
Mobile
PAPERS/AUTHORS
ED-5
THURSDAY, DECEMBER 4 • 0830-1000 • S320E
Is Skill Development Your Specialty?
Session Chair: Liz Gehr, Ph.D., The Boeing Company
Session Deputy: JoAnn Wesley, MARCORSYSCOM PM TRASYS
Retention and Retraining of Integrated Cognitive and
Psychomotor Skills (14220) h Anna Skinner, AnthroTronix,
Inc.
An Individualized Approach to Remediating Skill Decay:
Framework and Applications (14229) h Roberto K. Champney,
Erin G. Baker, Tarah Daly, Kay M. Stanney, Kelly Hale, Richard
Long, George Chadderdon, Design Interactive; Julie Jacko,
Francois Sainfort, Jit Chan, Andrew Nelson, BioMedical Metrics
LLC
Assertiveness and Responsiveness in Teams: Essential for
Mission Command (14197) ñ Hilde T. A. van Ginkel, Rendel
D. de Jong, Mandy G. van de Velde, Utrecht University; John
W. van Buren, Richard G. Oppelaar, Royal Netherlands Navy
ED-6
Automatic for the People: Content Alignment for
Session Chair:Mark Friedman, Concurrent Technologies
Corporation
Session Deputy: Jan Brown, CAE USA
Tailoring Multimedia Instruction to Soldier Needs (14049) 
Thomas Rhett Graves, Ph.D., U.S. Army Research Institute; Paul
N. Blankenbeckler, Richard L. Wampler, Northrop Grumman
Automated Content Alignment for Adaptive Personalized
Learning (14068) Elliot Robson, Robby Robson, Ph.D.,
Eduworks
An Army Learning Model Implementation: Challenges,
Successes, Future Directions (14153) Camilia Chavez Knott,
Krista Ratwani, Courtney Dean, Fred Diedrich, Aptima, Inc.;
Scott Flanagan, Sophia Speira; William R. Bickley, U.S. Army
Research Institute
ED-7
Best Papers from Around the Globe
Session Chair:Robert “Buddha” Snyder, WBB, Inc.
SimTecT Best Paper
A Comparison of Visual Display Systems for a Low-cost Mission
Training Flight Simulator ñ Jessica Parker, Air Operations
Division, Defence Science & Technology Organisation
ITEC Best Paper
A Maintenance Simulator for AF Engineers: The RSAF
Experience ñ ME5 Ryan Ng Woon Teck, Singapore Armed
Forces Chief Instructor for Air Engineering Training Institute
School
E M E R G I N G C O N C E P T S & I N N OVAT I V E T E C H N O L O G I E S
FROM DESIGN TO CONCEPTION: AN ASSESSMENT DEVICE FOR ROBOTIC SURGEONS (14170)
EC-1
Game On
TUESDAY, DECEMBER 2 • 1400-1530 • S320C
Session Chair: John Aughey, The Boeing Company
Session Deputy:Susan Harkrider, U.S. Army NVESD
Considerations on Utilizing a Game Engine as an Image
Generator (14274)  % Kevin Bland, AVT Simulation;
Stephen Lopez-Couto, U.S. Army PEO STRI; Jelani Vassall,
TAPE, LLC
Towards Minimalist Serious Game Design (14348) % Peter A.
Smith, Ph.D., University of Central Florida; Stuart Armstrong,
QinetiQ Training and Simulation, Inc.
Human Motion Capture in Natural Environments (14116) 
Zhiqing Cheng, Anthony Ligouri, Infoscitex Corporation;
Timothy Webb, Huaining Cheng, Air Force Research Laboratory
EC-2
TUESDAY, DECEMBER 2 • 1600-1730 • S320C
Best Simulator Technologies Ever
Session Chair: Michael O’Connor, Trideum Corporation
Session Deputy: Jeff Grubb, NAVAIRSYSCOM
The Largest Field of View Collimated Display Ever Built
(14322)   Justin Knaplund, Terry Linn, FlightSafety
International-Visual Systems; Dave Fonkalsrud, FlightSafety
International-Simulation
High Quality Visual Display Systems in Physically Constrained
Environments (14248) Reed Moody, Rockwell Collins
Dynamic Flight Simulation: 45 Years of Research &
Development (14019)  Kenneth L. Ginader, Michael
C. Newman, Richard A. Leland, Environmental Tectonics
Company
EC-3
WEDNESDAY, DECEMBER 3 • 0830-1000 • S320C
Emotional Engagement
Session Chair: Anya Andrews, Ph.D., Erudition Corporation
Session Deputy:Sherrie Jones, Ph.D., MARCORSYSCOM
PM TRASYS
Design and Evaluation of Surprise Effects in Simulation –
A Framework (14193) ñ Jelke van der Pal, Konstantinos
Georgiadis, Dutch National Aerospace Laboratory NLR
Validated Development of Stress Inoculation through Cognitive
and Biofeedback Training (14051)  Peter Squire, Ph.D.,
CDR Joseph Cohn, USN, Ph.D.; Elizabeth O’Neill, ONR;
Gershon Weltman, Ph.D., Elan Freedy, Ewart de Visser, Ph.D.,
Perceptronics Solutions, Inc.; Rollin McCraty, Ph.D., Institute
of Heartmath; Donald Chartrand, Ease Interactive, Inc.
Using Virtual Reality as Part of an Intensive Treatment
Program for PTSD (14079)  h Deborah C. Beidel, Ph.D.,
ABPP, Sandra M. Neer Ph.D., Clint Bowers, Ph.D., University
of Central Florida; B. Christopher Frueh, Ph.D., University of
Hawaii; Albert Rizzo, Ph.D., University of Southern California
Mobile
51
PAPERS/AUTHORS
EC-4
Training Hard to Train Skills: Perception, Sensemaking and
Adaptability
EC-7
What to Train
THURSDAY, DECEMBER 4 • 0830-1000 • S320A
Session Chair: Kent Gritton, JTIEC/NAWCTSD
Session Deputy: Jeff Grubb, NAVAIRSYSCOM
Microgames for Training Perceptual Skills (14282) % Sean
Guarino, Ryan Jarvis, Samuel Mahoney, Charles River
Analytics, Inc.; Michael Connell, Ed.D., Institute for Knowledge
Design, LLC
Beyond Socio-Cultural Sensemaking: Observing and
Interpreting Patterns of Life (14101) Tracy St. Benoit, University
of Central Florida; Clarissa Graffeo, MESH Solutions, LLC
Developing Effective Adaptive Training Systems to Enhance
Military Instruction (14140) Fleet Davis, Sandro Scielzo,
Jennifer M. Riley, SA Technologies, Inc.; Heather A. Priest,
NAWCTSD
Session Chair: John Dzenutis, The Boeing Company
Session Deputy: Stuart Armstrong, QinetiQ Training &
Simulation, Inc.
Fundamental Competency Sets (FCS) Definition to Support
Technology Development for Pilot Training (14126) Amanda
Avenoso, AFRL; Jamie Donsbach, The Group for Organizational
Effectiveness
Joint Terminal Attack Controller-Training Rehearsal System:
Competency-based Research (14097) 1Lt. Sean A. Morris,
Christine M. Covas-Smith, Ph.D., Leah J. Rowe, AFRL;
Christina L. Kunkle, Keith W. Westheimer, Leidos, Inc.
Operator Qualification Differences between Manned and
Unmanned Aerial System (UAS) (14287) Jennifer Pagan,
Randy Astwod, Henry Phillips, NAWCTSD
EC-5
EC-8
Perspective on Data and Services
Session Chair: Eric Weisel, Ph.D., Old Dominion University
Session Deputy: Constance Perry, U.S. Army PEO STRI
Future of LVC Simulation: Evolving Towards The MSaaS
Concept (14072) ñ Jose-Ramon Martinez-Salio, Jose-Maria
Lopez-Rodriguez, NADS Group
Rapid Data Generation: A Flexible Data Discovery and Access
Architecture (14096)  Kevin T. Gupton, Bruce Carlton, Roy
Scrudder, Applied Research Laboratories, The University of
Texas; Rob Cox, Ph.D., U.S. Army PEO STRI; Ralph O’Connell,
Joint Staff J6
Optimizing Supervised Learning for Pixel Labeling and
Material Classification (14016)  Mark Rahmes, Ph.D.,
Morris Akbari, Mike McGonagle, Harris Corporation
EC-6
Simulation: Supporting Experiment, Acquisition and Military
Planning
Session Chair: James (Josh) Jackson, SAIC
Session Deputy: Luis E. Velazquez, MARCORSYSCOM
Early Synthetic Prototyping: Exploring New Designs and
Concepts Within Games (14133)  % Maj Kate Murray,
USMC, Rudolph Darken, Naval Postgraduate School; LTC
Brian Vogt, USA, Army Capabilities Integration Center; Simon
Goerger, Army Corp of Engineers
When Tradespace Analysis Met Combat Modeling and
Simulation (14264) Chris Gaughan, Christopher J. Metevier,
ARL HRED STTC; Simon R. Goerger, U.S. Army Corps of
Engineers; Tommer R. Ender, L. Drew Pihera, Georgia Tech
Research Institute; Scott Gallant, Effective Applications
Corporation
Simulation in Support of Course of Action Development in
Operations (14119)  ñ Lt Col Jen Inge Hyndoy, Norwegian
Army; Ole Martin Mevassvik, Karsten Brathen, FFI, Norwegian
Defence Research Establishment
THURSDAY, DECEMBER 4 • 0830-1000 • S320C
More Braaiinnss
Session Chair:Jennifer Murphy, Ph.D., Quantum Improvement
Consulting, Inc.
Session Deputy: Paul Bogard, USAF Simulators Division
Videogame Design for Cognitive Enhancement through MicroPuzzle Cognitive Profiling (14039) % Patrick S. Gallagher,
Ph.D., Serco, Inc. in support of Advanced Distributed Learning;
Shenan Prestwich, Katmai in support of Advanced Distributed
Learning
Game-based Training to Mitigate Three Forms of Cognitive Bias
(14180) % Benjamin A. Clegg, Ph.D., Rosa Mikeal Martey,
Ph.D., James E. Folkestad, Ph.D., Colorado State University;
Jennifer Stromer-Galley, Ph.D., Syracuse University; Kate
Kenski, Ph.D., University of Arizona; Tobi Saulnier, Ph.D.,
Elizabeth McLaren, 1st Playable Productions; Adrienne Shaw,
Ph.D., Temple University; Joanna E. Lewis, University of
Central Florida; John D. Patterson, Binghamton University;
Tomek Strzalkowski, Ph.D., University of Albany
Enhancing Intuitive Decision Making Through Implicit
Learning (14253)  Peter Squire Ph.D., CDR Joseph Cohn,
USN, Ph.D., ONR; Denise Nicholson, Ph.D., Margaret Nolan,
MESH Solutions, LLC; Paul J. Reber, Northwestern University;
James Niehaus, Charles River Analytics; Alexandra Geyer,
Aptima, Inc.; Liz O’Neill, Strategic Analysis, Inc.
EC-9
THURSDAY, DECEMBER 4 • 1030-1200 • S320A
Medical Matters
Session Chair: Beth Pettitt, ARL HRED STTC
Session Deputy: Bob Heinlein, Simbionix USA Corporation
Challenges to Upgrading a Mobile Web Application (14057) 
h Howard Mall, Engineering & Computer Simulations, Inc.;
Teresita Sotomayor, Ph.D., ARL HRED STTC
Development and Evaluation of a Humeral Head Intraosseous
Training System (14058) h Angela M. Salva, Cheryl Coiro,
SIMETRI, Inc.; Teresita Sotomayor, Ph.D., ARL HRED STTC
Comparative Analysis of Holographic Display and ThreeDimensional Television (14230) Matthew Hackett, MAJ Kevin
Fefferman, USA, ARL HRED STTC
52
Ï Cyber Security
Mobile
PAPERS/AUTHORS
EC-10
Get a Grip on Reality
Session Chair: Scott Ariotti, The DiSTI Corporation
Session Deputy: Brian Overy, Diamond Visionics
Live Augmented Reality Based Weapon Training for Dismounts
(14093) Supun Samarasekera, Rakesh Kumar, Ph.D., Zhiwei
Zhu, Ph.D., Vlad Branzoi, Nicholas Vitovitch, Ryan Vilamil, SRI
International; Frank Dean, Pat Garrity, ARL HRED STTC
Augmented Reality Virtual Personal Assistant for Training,
Maintenance, and Repair (14031) Rakesh Kumar, Supun
Samaraskera, Girish Acharya, Louise Yarnall, Zhiwei Zhu,
Michael Wolverton, Vlad Branzoi, Glenn Murray, Nicholas
Vitovitch, Ryan Villamil, Jim Carpenter, SRI International
PERLS: An Approach to Pervasive Assistance in Adult Learning
(14335) Michael Freed, Louise Yarnall, Jason Dinger, Melinda
Gervasio, Adam Overholtzer, Mar Pérez-Sanagustin, Jeremy
Roschelle, Aaron Spaulding, SRI International
EC-11
Networking Cross Domain Solutions: Cross Domain
Solutions, Cloud, Social
Session Chair: Susan Sherman, NAWCTSD
Session Deputy: Luis Pineiro, AFRL
Implementing Stateless Cross Domain Solutions to Continuously
Maintain Security Assurances (14301) Christopher Huey,
Parsons Corporation; Kelly Djahandari, Charles Kristofek,
Northrop Grumman
Cybersecurity Impacts of a Cloud Computing Architecture in
Live Training (14120)  Ï Graham Fleener, U.S. Army PEO
STRI; Cliff Zou, Ph.D., University of Central Florida; Jason
Eddy, AIT Engineering
Using Social Network Analysis to Model the Spread of
Misinformation in Simulated Environments (14205) Paul
Cummings, ICF International; Chalinda Weerasinghe,
Weerasinghe Research Group; Qing Tian, Ph.D., Kraznow
Institute, George Mason University
HUMAN SYSTEMS ENGINEERING
MEASURING VISUAL DISPLAYS EFFECT ON NOVICE PERFORMANCE IN DOOR GUNNERY (14012)
H-1
TUESDAY, DECEMBER 2 • 1400-1530 • S320F
The Body Speaks – Stress and Workload
Session Chair: Kelly Hale, Ph.D., Design Interactive, Inc.
Session Deputy: Phil Brown, D.M., U.S. NORTHCOM
Classifying Stress in a Mobile Environment (14195)  Sara
Dechmerowski, Brent Winslow, Ph.D., George Chadderdon,
Tarah N. Schmidt-Daly, Design Interactive, Inc.
Investigation of the Sensitivity of Physiological, Performance,
and Subjective Measures for Identifying Changes in Novice
Intelligence Analyst Workload (14035) Lisa Tripp, Ph.D.,
Robert Nelson, Elliott Humphrey, Chad Tossell, Ph.D., AFRL;
Jennifer Winner, Jerred Holt, Lumir Research Institute
Training with Adaptive Systems: Utility of Baroreflex Sensitivity
(14297)  h Warren D. Franke, Amanda A. Arens, Nir Keren,
Andrew Lilja, Kevin M. Godby, Iowa State University
H-2
TUESDAY, DECEMBER 2 • 1600-1730 • S320F
Shooting for Effective Virtual Training
Session Chair:Martin Bink, Ph.D., U.S. Army Research
Institute
Session Deputy: Kendy Vierling, Ph.D., USMC TECOM
A Novel Approach to Determine Integrated Training
Environment Effectiveness (14011)   LTC Glenn A. Hodges,
USA, Ph.D., Naval Postgraduate School
Pistol Skill Acquisition and Retention: A 3-Year Longitudinal
Study (14042)  ñ Gregory P. Krätzig, Royal Canadian
Mounted Police
Effective Live, Virtual, and Constructive Training: Personnel
Recovery’s Weeklong Experience with Integrated Sensor
Technology (14216) Curtis Wray, Mark Speed, Timothy
Rodabaugh, Ball Aerospace & Technologies Corp; Kristen
Barrera, AFRL
H-3
WEDNESDAY, DECEMBER 3 • 0830-1000 • S320F
A Cornucopia of Human Behaviors
Session Chair: Robert Sottilare, Ph.D., ARL HRED STTC
Session Deputy: Todd Glenn, FAAC, Inc.
In Search of Interoperability Standards for Human Behavior
Representations (14027)  ñ Glenn Gunzelmann, Ph.D.,
AFRL; Chris Gaughan, ARL HRED STTC; Wim Huiskamp,
Ph.D., Karel van den Bosch, Steven de Jong, Ph.D., TNO;
Thomas Alexander, FKIE Human Factors; Agostino G.
Bruzzone, Ph.D., Alberto Tremori, Ph.D., DIME University of
Genoa
Factors Impacting Performance in Competitive Cyber Exercises
(14108) Ï Austin Silva, Jonathan McClain, Theodore Reed,
Benjamin Anderson, Kevin Nauer, Robert Abbott, Ph.D., Chris
Forsythe, Ph.D., Sandia National Laboratories
Cognitive Processing Considerations of the Small Unmanned
Ground Vehicle (14134) Victor J. Ingurgio, Ph.D., ARI-Fort
Benning; Richard Catrambone, Ph.D., Georgia Institute
of Technology; Richard L. Wampler, Northrop Grumman
Corporation
H-4
Plane, Train and Automobile
Session Chair:John Schlott, L-3 Communications Link
Simulation & Training
Session Deputy: Randy Jensen, Stottler Henke Associates, Inc.
Leveraging Simulation to Augment Risky Driving Attitudes and
Behaviors (14004) Karen L. Morris, Gregory A. Fabiano, Ph.D.,
Kevin F. Hulme, University at Buffalo
Mobile
53
PAPERS/AUTHORS
Simulation-based Analysis of the Human Factors Related to
Autonomous Driving (14006) Yunfei Hou Ph.D., Jingyan Wan,
Ph.D., Yunjie Zhao, Ph.D., Changxu Wu, Ph.D., Adel Sadek,
Ph.D., Chuming Qiao, Ph.D., Kevin F. Hulme, University at
Buffalo
The Virtuous Circle and Contextualized Knowledge Elicitation:
Application of a New Paradigm for Job Analysis (14347)
Sterling Wiggins, Michael J. Keeney, Ph.D., Webb Stacy,
Ph.D., Jeff M. Beaubien, Ph.D., Aptima, Inc.; Jennifer Pagan,
NAWCTSD; Amy Bolton, Ph.D., ONR
H-5
See, Hear and Speak – No Evil Assessments
Session Chair: Jerry Stahl, Cypress International
Session Deputy: Ingrid Mellone, Camber Corporation
Serious Game User Data Analysis and Visualization: Savoring
the Breadcrumbs (14377) % Brandt Dargue, The Boeing
Company; Dov Jacobson, GamesThatWork; John Sanders,
Historical Online Learning Foundation
Sonification: The Sound of Big Training Data (14261) Nat
Napoletano, Lockheed Martin Mission Systems and Training
Creating a Learning Infrastructure Where Every Soldier Can Be
an Instructor (14124) Ed Sims, Ph.D., Irene T. Boland, Ph.D.,
Dan Silvergate, Jeff Cashion, Vcom3D, Inc.; Rodney Long,
Charles Amburn, ARL
H-6
THURSDAY, DECEMBER 4 • 1330-1500 • S320F
Shocking Medical Assessments
Session Chair: Jennifer Arnold, Booz|Allen|Hamilton, Inc.
Session Deputy: Matthew Hackett, ARL HRED STTC
Quantitative Assessment of Combat Casualty Skills (14191) h
Christine Allen, Ph.D., Mark Mazzeo, ARL HRED STTC; Brian
Goldiez, Ph.D., Amanda Romeu, University of Central Florida;
Thomas Pingel, Joint Base Lewis McCord (JBLM) Medical
Simulation Training Center (MSTC)
Assessment Instrument Validation for Critical Clinical
Competencies: Pediatric-neonatal Intubation and Cholinergic
Crisis Management (14232) h Pamela Andreatta, Ed.D., Ph.D.,
Jessica Klotz, University of Minnesota Medical School; COL
James M. Madsen, USA, M.D., COL Charles G. Hurst, USA
(Ret), M.D., U.S. Army Medical Research Institute for Chemical
Defense (USAMRICD); Thomas B. Talbot, M.D., Telemedicine &
Advanced Technology Research Center (TATRC)
A Decision Support System Predicting Imminent Cardiovascular
Shock (14343)  h W. Andrew Pruett, Ph.D., Leland D.
Husband, Robert Hester, Ph.D., University of Mississippi
Medical Center
P O L I C Y, S T A N D A R D S , M A N A G E M E N T & A C Q U I S I T I O N
PROFICIENCY EVALUATION AND COST-AVOIDANCE PROOF OF CONCEPT M1A1 STUDY (14055)
P-1
TUESDAY, DECEMBER 2 • 1400-1530 • S320A
Managing Resources at Home and Abroad
Session Chair: Randy Allen, Ph.D., Lone Star
Session Deputy: Richard Grohs, HQ Air Combat Command
Resource Implications of the Difference between Models
and Simulations (14020)  Thomas J. Yanoschik, SAIC Maneuver Battle Lab
COTS to Capability: Lessons Learnt from UK MOD Research
Programme (14115) ñ John Kent, Amy Stafford, Al Nicholls,
QinetiQ; Caroline Shawl, Dstl
Perspectives on Exportability and Program Protection in Virtual
Training Systems (14265)  Michael Coleman, Ricky Denny,
NAWCTSD
P-2
TUESDAY, DECEMBER 2 • 1600-1730 • S320A
Commonality and Data Sharing in the LVC Environment
Session Chair: Robert Matthews, NAWCTSD
Session Deputy: Jeffery Raver, SAIC
LVC, Translating DoD Policy into Action (14059) LCDR Daniel
Cain, USN, CAPT Robert Snyder, USN (Ret), OPNAV N980T
Data Sharing: The Standard Specification is Just the Start
(14130) Robert F. Richbourg, Ph.D., George E. Lukes, Institute
for Defense Analyses
Establishing Sharing for Geospatial Environment Data (14255)
 Mark Faulk, Cornerstone Software Solutions, Robert Cox,
Ph.D., Bill Reese, U.S. Army PEO STRI
P-3
Wave of the Future
Session Chair: Chuck Secard, Lockheed Martin
Session Deputy: Robby Robson, Ph.D., Eduworks
Agile Program Management on Software Intensive Training
Systems (14311)  CDR Gregory Owens, USN (Ret), Petra
Robinson, Barry Minchey, NAWCTSD
Simulations in the Cloud – A Manager’s Challenge (14104)
  Lawrence A. Rieger, CMSP, U.S. Army Capabilities
Integration Center
Continuous Monitoring of Cybersecurity in a Training System
Environment (14121)   Ï Graham Fleener, Marco Mayor,
U.S. Army PEO STRI, Andrew Maxon, Cybernet Systems
Corporation
54
Ï Cyber Security
Mobile
PAPERS/AUTHORS
S I M U L AT I O N
EXPLORATION OF SOLDIER MORALE USING MULTI-METHOD SIMULATION APPROACH (14215)
S-1
TUESDAY, DECEMBER 2 • 1400-1530 • S320B
Leveraging Cloud and High Performance Computing
Environments
Session Chair: Nick Giannias, CAE
Session Deputy: Mark Soodeen, CAE
Embracing The Cloud – Providing Simulation as a Service
(14018)  Lawrence A. Rieger, CMSP, U.S. Army Capabilities
Integration Center; Daniel Lacks, Ph.D., Cole Engineering
Services, Inc.
Cloud Terrain Generation and Visualization Using Open
Geospatial Standards (14308) % Samuel Chambers, Joint Staff
J7 Environmental Development Division; Jay Freeman, CAE
USA
Enabling External Player Connections To Kerberos-secured
Systems (14202) Peter G. Raeth, Ph.D., Sean B. Ziegler, Ph.D.,
Rhonda Vickery, Ph.D., Engility Corporation
S-2
TUESDAY, DECEMBER 2 • 1600-1730 • S320B
LVC Interoperability
Session Chair: Bob Kleinhample, SAIC
Session Deputy:Capt Jonathan Richardson, USMC,
Sensor Placement Optimization in LVC Environments for
Training, Analysis, and Operational Applications (14314)
Jennifer Lewis, Joyner Livingston, SAIC
Towards Interoperability of Simulations Systems of Ground
Force: Progress and Challenges (14082) ñ Sérgio Simas Lopes
Peres, Jonathan Rosa Moreira, Brazilian Army
Integrating Distributed Virtual Command and Control
Platforms into Live Training (14318)  Ryan McLaughlin,
Orlando Torres, Mike Aldinger, Northrop Grumman
S-3
WEDNESDAY, DECEMBER 3 • 0830-1000 • S320B
Simulated Movement
Session Chair: Ron Dionne, FLETC
Session Deputy: Long Nguyen, Ph.D., NAWCTSD
Mission Integrated Simulation – A Case Study (14085)  ñ
Per Wikberg, Ph.D., Mirko Thorstensson, Lt, Army Res., Peter
Hammer, Ph.D., Gustav Tolt, Ph.D., Swedish Defence Research
Agency (FOI)
An Instructor Operating System (IOS) Framework for
Interactive Instructor-station (14112)  ñ Kim Leng Koh,
Shih Yeong Wah, Singapore Technologies Electronics Training &
Simulation Systems Pte Ltd.
Development of a Microscopic Artificially Intelligent Traffic
Model for Simulation (14003) Viral Raghuwanshi, Sarthak
Salunke, Kevin F. Hulme, NYSCEDII, Yunfei Hou, Ph.D.,
Department of Computer Science and Engineering, University
at Buffalo
S-4
Culture, Reaction and Movement: Simulating Human
Behavior
Session Chair: Pete Schrider
Session Deputy: Lisa Jean Bair, SAIC
Teaching Cross Cultural Social Competence in a Dynamic,
Synthetic Environment (14289)   William Ferguson, Bruce
Roberts, David Diller, Ph.D., Raytheon BBN Technologies; Dan
Shapiro, Ph.D., Michael Mateas, Ph.D., University of California
Santa Cruz
Advanced Animation Techniques in a Dismounted Soldier
System (14136)  Scott M. Johnson, John Carswell, Intelligent
Decisions; Pat Garrity, ARL HRED STTC
Game-based Simulation for Philippine Post-typhoon Stability
Operations Training (14329) %Marjorie Zielke, Ph.D.,
Djakhangir Zakhidov, Gary Hardee, Michael Kaider, University
of Texas at Dallas
S-5
Decision Support Systems & Methods
Session Chair: Sandy Veautour, U.S. Army AMRDEC
Session Deputy:Maj Daniel Loth, USMC, MARCORSYSCOM
PM TRASYS
A Decision Aid for Optimizing Experimental Design Involving
LVC Environments (14139)  Sylvain Bruni, Kenyon Riddle,
Andres Ortiz, Ph.D., Danielle Dumond, Ph.D., Spencer Lynch,
Aptima, Inc.; Henry Marshall, Chris Gaughan, ARL HRED
STTC; Jay Saffold, Research Network, Inc.
A Practitioner’s Approach Using MBSE in Systems of Systems
(14383)  Richard Deakins, Doug Parsons, U.S. Army
AMRDEC
Robotic Simulators: A Case for Return on Investment (14129)
  h Roger D. Smith, Ph.D., Khara M. Simpson, M.D.,
Florida Hospital Nicholson Center
S-6
Innovative Approaches to Environment and Behavior
Simulation
Session Chair: Scott Hooper, Havok
Session Deputy: Greg Sidor, AFRL
Creating a Re-useable Knowledge Repository for UK MoD CGF
Behaviours (14080) ñ Mark Lewis, Cranfield University; Dan
Allison, Discovery Machine, Inc.
Improving Air-to-Air Combat Behavior Through Transparent
Machine Learning (14298)  ñ Armon Toubman, Ph.D., Jan
Joris Roessingh, Ph.D., National Aerospace Laboratory (NLR);
Pieter Spronck, Ph.D., Aske Platt, Ph.D., Tilburg University;
Jaap van den Herik, Ph.D., Leiden University
Improving Material Classification Quality with Elevationderived Metrics (14380)  Christopher Fink, Ph.D., JRM
Technologies, Inc.
Mobile
55
PAPERS/AUTHORS
S-7
Improving Healthcare with Simulation
S-10
THURSDAY, DECEMBER 4 • 1030-1200 • S320B
IG, Synthetic Environment and Scalable Simulation
Session Chair: Gerald Dreggors, Northrop Grumman
Session Deputy:Leslie Dubow, Veterans Health Administration
SimLEARN
Improving and Proving Healthcare Quality and Value through
Physical Simulation (14144)  h Timothy R. Brock, Ph.D.,
The Institute 4 Worthy Performance; Mary Holtschneider, RNBC, US Department of Veterans Affairs
Employing Modeling and Simulation to Improve Patient Care
(14034)  h James Thomas, Allen J. Giannakopoulos, Ph.D.,
Baptist Health South Florida
The Effect of Difficulty Levels within a Virtual Medical
Simulation (14228) h Matthew Hackett, MAJ Kevin
Fefferman, USA, ARL HRED STTC; Steve McIlwain, Bradley
Willson, Applied Research Associates, Inc.
Session Chair:Carla Cropper, Rockwell Collins Simulation &
Training Solutions
Session Deputy: Michael Aldinger, Northrop Grumman
A Distributed Scene Graph Approach to Scaled Simulation
Based Training Applications (14033)  Douglas B. Maxwell,
ARL HRED STTC; Joe Geil, William Rivera, University of
Central Florida; Huaiyu Liu, Ph.D., Intel Research
Pseudo-Specific High-resolution Data Boundary Techniques
(14317)  Daniel J. Lowe, Michael A. Cosman, Rockwell
Collins
Solving the Innovator’s Dilemma for Simulation and Training
Image Generator Architectures (14373)  Bob Grange Michael
Cosman, Nephi Lewis, Brad Southwick, Rockwell Collins
S-8
Automation and Autonomy
Synthetic Environment
Session Chair: Toni Hawkins-Scribner, Air University
Session Deputy: Bradley Ehrhardt, NAWCTSD
A Paradigm Shift in the Test and Evaluation of Terrain
Databases (14200)  Thomas Kehr, Trey Godwin, U.S. Army
PEO STRI; Ryan McIntire, Leidos, Inc.
Measuring the Impact of Natural Environment Representation
on Combat Simulation Outcomes (14305)  Karl D. Pfeiffer,
Atmospheric and Environmental Research; Theresa Tamash,
Dignitas Technologies
Implementation of Real-time Snow Layers in Game-based
Simulation (14361)  % Michael D. Woodman, Ph.D., Peter
Morrison, Bohemia Interactive Simulations
S-9
Emerging Visual Technologies
Session Chair:Randy Crowe, Ph.D., Lockheed Martin Mission
Systems and Training
Session Deputy:Brent Terwilliger, Ph.D., Embry Riddle
Aeronautical University
Integration of Low-cost HMD Devices in Existing Simulation
Infrastructure (14190) ñ Tomer J. Michael, Yaniv Minkov,
Rami Rockah, IDF Ground Forces Command Battle-Lab
Simulating Realistic Light Levels in Next Generation Image
Generators (14233)  Brett Chladny, Kenny Hebert, Brad
Colbert, Renaissance Sciences Corporation
Vergence and Accommodation in Simulation and Training
with 3D Displays (14147)  % David L. Page, Ph.D., C.E.
(Tommy) Thomas, Ph.D., Steve L. Kelley, Paul G. Jones, Third
Dimension Technologies; David A. Miller, AYA Associates, Inc.
S-11
Session Chair:Brent Terwilliger, Ph.D., Embry Riddle
Aeronautical University
Session Deputy:Randy Crowe, Ph.D., Lockheed Martin Mission
A Framework for Enabling Virtual Observer Controllers in
Synthetic Training (14268) Mandira Hegde, Dan Allison, Todd
W. Griffith, Ph.D., Discovery Machine, Inc.
UAV Flight Control Software Development based on COTS
Product (14206) ñ Jung-ho Moon, Da-hyoung Jeon, Korean
Air, R&D Center; Yeong-cheol Kim, Agency for Defense
Development
Lessons Learned in Creating an Autonomous Driver for OneSAF
(14106)  Jonathan Stevens, Ph.D., Latika Eifert, ARL HRED
STTC; Dean Reed, Eugenio Diaz, Institute for Simulation and
Training; Oleg Umanskiy, STILMAN Advanced Strategies
S-12
Engineering-based Modeling
Session Chair: Mark Soodeen, CAE
Session Deputy: Ron Dionne, FLETC
How the U.S. Navy is Migrating from Legacy/Large Footprint to
Low Cost/Small Footprint Sonar Simulation Systems (14090)
 Sean M. Reilly, The AEgis Technologies Group, Inc.;
Jonathan Glass, NAWCTSD
Use of Automated Intelligent Entities in ASW Simulation
(14109)  ñ Morten Kolve, Kongsberg Defence Systems;
Jared Snyder, Discovery Machine, Inc.; Geoff Tompson, Decisive
Encounters Limited
A Physics-based Approach to Simulate Jet Engines (14030) 
Sami S. Mina, Rockwell Collins Simulation and Training
Solutions
56
Ï Cyber Security
Mobile
PAPERS/AUTHORS
TRAINING
MISSING: A SERIOUS GAME FOR THE MITIGATION OF COGNITIVE BIASES (14295)
T-1
TUESDAY, DECEMBER 2 • 1400-1530 • S320D
Keepin’ It Real with VR
Session Chair:Susan Coleman, Ph.D., Intelligent Decision
Systems, Inc.
Session Deputy: Anne Little, Ph.D., Addx Corporation
Declarative Knowledge Acquisition in Virtual Learning
Environments (14005) Rustin Webster, Ph.D., Intuitive
Research and Technology Corporation
Virtual World Room Clearing: A Study in Training Effectiveness
(14045)  Stephanie J. Lackey, Ph.D., Julie N. Salcedo, Gerald
Matthews, Ph.D., Institute for Simulation & Training, University
of Central Florida; Douglas B. Maxwell, ARL HRED STTC
Simulating Participant Training Data to Test Mixed-reality
Training Systems (14252) Ken Kopecky, Ph.D., Eliot Winer,
Ph.D., Iowa State University; Julio de la Cruz, ARL HRED STTC
T-2
TUESDAY, DECEMBER 2 • 1600-1730 • S320D
Nothing Here To See
Session Chair: Brian Cairns, Moulage Sciences & Training
Session Deputy:Robert Wallace, USAF 29th Training Systems
Squadron
Enhancing the Utility and Effectiveness of Combat Medic
Simulation (14075)  h Danielle Julian, John Killilea,
Patricia Bockelman, Ph.D., Margaret Nolan, MESH Solutions;
Training Effects for First-responder Competency in Cholinergic
Crisis Managment (14241) h Pamela Andreatta, Ed.D., Ph.D.,
Jessica Klotz, University of Minnesota Medical School; COL
James Madsen, USA, M.D., COL Charles G. Hurst, USA (Ret),
M.D., U.S. Army Medical Research Institute for Chemical
Defense (USAMRICD); Thomas B. Talbot, M.D., Telemedicine &
Advanced Technology Research Center (TATRC)
Simulation Environments for Offshore Oil and Gas Emergency
Training (14344)  ñ Randy Billard, Virtual Marine
Technology Inc.; Captain Anthony Patterson, Canadian Coast
Guard (Ret), Virtual Marine Technology Inc.
T-3
WEDNESDAY, DECEMBER 3 • 0830-1000 • S320D
Comparative Analysis for Clinical Training
Session Chair: Teresita Sotomayor, Ph.D., ARL HRED STTC
Session Deputy: Kristy Murray, Ed.D.
Comparison of the Usability of Robotic Surgery Simulators
(14168)   h Alyssa D.S. Tanaka, Courtney Graddy, Roger
Smith, Ph.D., Florida Hospital Nicholson Center; Haidar M.
Abdul-Muhsin, M.D., Mayo Clinic
Forces Applied on Laryngoscope during Intubation: A Study on
Airway Simulators (14203)  h Matthew Mui, University of
Central Florida; Christine Allen, Ph.D., ARL HRED STTC; Mojca
R. Konia, M.D., Ph.D., University of Minnesota Department of
Anesthesiology; Jack Stubbs, David Hananel, SimPORTAL &
CREST University of Minnesota
Outcomes from Two Forms of Pediatric and Neonatal
Intubation Training (14240)  h Pamela Andreatta, Ed.D.,
Ph.D., Jessica Klotz, University of Minnesota Medical School;
Suzanne Dooley-Hash, M.D., Joseph House, M.D., University of
Michigan Medical School
T-4
“Assess”-orizing Your Training and Performance Outcomes
Session Chair: Eliot Winer, Ph.D., Iowa State University
Session Deputy: Gene Beauvais, Raytheon Company
Data & Analytics Tools for Agile Training & Readiness
Assessment (14064)  Jared Freeman, Ph.D., Aptima, Inc.;
Denise Nicholson, Ph.D., MESH Solutions, LLC; Peter Squire
Ph.D., Amy Bolton, Ph.D., ONR
Lessons Learned Integrating Mobile Technology into Two
Army Courses (14128) Gregory A. Goodwin, Ph.D., ARL
HRED STTC; Michael Prevou, Ph.D., Holly C. Baxter, Ph.D.,
Mike Hower, Strategic Knowledge Solutions; Heather Wolters,
Ph.D., U.S. Army Research Institute; Linda McGurn, Engility
Corporation
Experience API and Team Evaluation: Evolving Interoperable
Performance Assessment (14157)  Michael Hruska, Problem
Solutions; Charles Amburn, Rodney Long, ARL; Tara Kilcullen,
Raydon; Tiffany R. Poeppelman
T-5
Critical Flight Decisions
Session Chair: Bob Johnson, McKean-Defense Group LLC
Session Deputy:Michael Motko, QinetiQ Training & Simulation,
Inc.
Developing and Evaluating Performance Measures for MannedUnmanned Teaming (14024) John E. Stewart, Ph.D., Scott E.
Graham, Ph.D., Army Research Institute; Courtney R. Dean,
Aptima, Inc.; Troy Zeidman, Imprimis, Inc.
Using Temporal Occlusion to Assess Carrier Landing Skills
(14171)  Webb Stacy, Ph.D., Jeff Beaubien, Ph.D., Sterling
Wiggins, Aptima, Inc.; Melissa Walwanis, NAWCTSD; Amy
Bolton, Ph.D., ONR
Distributed Live/Virtual Environments to Improve Joint Fires
Performance (14041) Emilie A. Reitz, Kevin Seavey, Alion
Science and Technology
T-6
Combating Stress: Performance Under Fire
Session Chair:Elaine Raybourn, Ph.D., Sandia National
Session Deputy: Mary Driskel, NAWCTSD
Accelerating Unit Adaptability: A Principle-based Approach
to Unit Communication (14038) Tara Rench, Zachary Horn,
Ph.D., Alexander Walker, Ph.D., Aptima, Inc.; Steve Zaccaro,
Ph.D., George Mason University
Mobile
57
PAPERS/AUTHORS
Inducing Stress in Warfighters during Simulation-based
Training (14201)  Meredith Carroll Ph.D., Brent Winslow,
Ph.D., Christina Padron, Glenn Surpris, Jennifer Murphy, Ph.D.,
Design Interactive, Inc.; Jason H. Wong Ph.D., Peter Squire,
Ph.D., ONR
T-11
Simulation & Game-based Training
Session Chair: Fred Fleury, ZedaSoft, Inc.
Session Deputy: Felicia Douglis, FRD Solutions, LLC
Research Directions for Future Simulation-based Training
Design in Defence (14017) Luke G. Thiele, Ph.D., Rheinmetall
Simulations Australia Pty, Ltd. ñ
Utilizing Simulation and Game-based Learning to Enhance
Incident Commander Training (14148)  % Ronald W.
Tarr, Eileen Smith, Eric Totten, Michael Carney, Institute for
Simulation and Training; Michael Wajda, Orange County
(Florida) Fire and Rescue Department
T-7
THURSDAY, DECEMBER 4 • 0830-1000 • S320D
Intelligent Tutors: Just How Smart Are They?
Session Chair: Anne Little, Ph.D., Addx Corporation
Session Deputy:Javier “Jeff” Covelli, Ph.D., Computer Sciences
Corporation
Developing and Evaluating an Intelligent Tutoring System for
Advanced Shiphandling (14014)  Jason H. Wong, Ph.D.,
Lauren Ogren, NUWC; Prof. Stanley Peters, Elizabeth O. Bratt,
Ph.D., Stanford University
Developing Models of Expert Performance for Support in
an Adaptive Marksmanship Trainer (14214)  Benjamin
Goldberg, Ph.D., Charles Amburn, Keith Brawner, Marko
Westphal, ARL HRED STTC; Marko Westphal, German Federal
Office of Bundeswehr Equipment
A Digital Tutor for Accelerating Technical Expertise (14272) 
J.D. Fletcher, Ph.D., Institute for Defense Analyses; William
D. Casebeer, Ph.D., Lockheed Martin Advanced Technology
Laboratories
T-8
T-9
A Variety of Training Approaches
Session Chair: Cynthia Adams, Booz|Allen|Hamilton, Inc.
Session Deputy: Karen Cooper, Ph.D., NAWCAD
Fusing Self-Reported and Sensor Data from Mixed-reality
Training (14158) Trevor Richardson, Stephen Gilbert, Ph.D.,
Joseph Holub, Frederick Thompson, Anastacia MacAllister,
Rafael Radkowski, Eliot Winer Ph.D., Iowa State University;
Paul Davies, Scott Terry, The Boeing Company
A Competency Based Approach to Marine and Weapons
Engineering Training (14224)  ñ Commander Richard
Clarke, Royal Navy MoD
Scenario-based Training for Development of LeaderSubordinate Mental Models and Cohesion (14333) Kara L.
Orvis, Ph.D., Aptima, Inc.; Gregory A. Ruark, Ph.D., U.S. Army
Research Institute; Krista L. Ratwani, Ph.D., Aptima, Inc.
T-10
Evaluating Training Effectiveness
Session Chair: Paul Lyon, BARCO Simulation
Session Deputy: Luis Garcia, MARCORSYSCOM PM TRASYS
Post-fielding Training Assessment of Dismounted Infantry
Simulation (14022) Martin L. Bink, Ph.D., Victor J. Ingurgio,
Ph.D., U.S. Army Research Institute; David James, Northrop
Grumman
Evaluating the Impact of Individual Training on Units’
Operational Performance (14123)  Jay Brimstin Ph.D.,
Toumnakone Annie Hester, U.S. Army Maneuver Center of
Excellence
Using LMS Technology for Kirkpatrick Level 3 Evaluation of
Human Trafficking Training (14162) Jill Shepherd, Lewis
Harris, Casey O’Conor, Booz|Allen|Hamilton
Aviation Training: The Ups and Downs!
Session Chair:Michael Motko, QinetiQ Training & Simulation,
Inc.
Session Deputy: Bob Johnson, McKean-Defense Group, LLC
Evolving Aviation Live Training in the Future (14078) Wanda
Fuentes, Anne Dunlap, Jim Grosse, Tien Pham, Patrick
Sincebaugh, U.S. Army PEO STRI
Training Fidelity of an Unmanned Aerial Systems
Complementary Family of Trainers (14135)  Sharon L.
Conwell, Ed.D., Christine M. Covas-Smith, Ph.D., Leah J Rowe,
AFRL; Andrew Shepard, Ph.D., Sinclair Community College;
John B. Bridewell, Ph.D., University of North Dakota
ASOC Training Research: Joint Theater Air Ground Simulation
System (14166) Leah Rowe, Sharon L Conwell, Ed.D., AFRL
58
Ï Cyber Security
Mobile
STEM
S T E M W O R K F O R C E I N I T I AT I V E
Workforce Initiative
STEM supports and promotes activities encouraging students’ interest and
pursuit in Science, Technology, Engineering and Mathematics.
STEM today = Prepared Workforce for Tomorrow
In support of STEM and Workforce Development, I/ITSEC sponsors the following programs:
• Future Leaders Pavilion
• Students at I/ITSEC
• Post Graduate Scholarships (Masters and
Doctorate)
• Simulation Technician Scholarships
• Serious Games Showcase and Challenge
• I/ITSEC Professional Development Workshops
• Central Florida Educators Workshop
• Continuing Education Units
• America’s Teachers at I/ITSEC
• Golf and 5K Fundraiser
We would like to acknowledge the support of the
following STEM Initiative Sponsors:
The AEgis Technologies Group
ASTi
Autodesk
AVT Simulation
Camber Corporation
Design Interactive
General Dynamics IT
Engineering & Computer Simulations
Innovative Reasoning
L-3
Lockheed Martin
TAPE
FUTURE LEADERS/STUDENTS
STOP!
SEE THE FUTURE
Future Leaders
Pavilion
e/Industry T
rvic
ra
rse
i
te
ucation Confe
Ed
re
n
nd
2014
ce
Booth
2467
Simulation
ng
a
ni
Tuesday, 2 December
1200 – 1730
0930 – 1730
0930 – 1500
THURSDAY
SPECIAL SESSION
1030 – 1200 • S330D
AWARDS CEREMONY
1345 • Warfighters’
Corner Sta ge
Students at I/ITSEC
Thursday, 4 December • 0900 – 1400
Over the years, thousands of Central Florida high school students have participated in a unique learning experience by visiting the Exhibitors/Exhibits. The purpose of the I/ITSEC Student
Tours is to allow students to experience first-hand, real-world
Training, Simulation, and Education solutions that will help
bridge the gap between classroom theory and the applied use
of Science, Technology, Engineering and Math (STEM) subjects.
Annually, over 600 students, along with 200 school chaperones
and volunteer I/ITSEC member escorts, are exposed to special
demonstrations and static displays of the Simulation, Training
and Education Industry. Students are able to learn about the basic building blocks required to deliver high fidelity modeling and
simulation products across a broad range of training environments. Participating in the I/ITSEC Student Tours on Thursday,
4 December, 2014, will give students a complete understanding
of how they can apply the STEM related skills they learn in the
classroom to highly successful careers in our Industry. To learn
more about the I/ITSEC Conference and Student Tours, please
contact Cyndi Turner, cyndi.turner@saabtraining.com.
In
Learning and Leadership
are indispensable to each other.
The National Training and Simulation Association and the members of I/ITSEC take great pleasure in welcoming you to the
Ninth Annual Future Leaders Pavilion and Special Session.
We are delighted to host secondary students from such
diverse areas as:
• Columbus, GA
• Latham, NY
• Dayton, OH
• Lexington Park, MD
• Hampton, VA
• Orlando, FL
• Huntsville, AL
The students who participate in the Future Leaders
Pavilion (FLP) are committed to excellence and are enrolled in
engineering, computer sciences, mathematics, or modeling and
simulation tracks. Projects presented this year will continue the
legacy of excellence built by previous Future Leaders.
Please remember to include FLP, located in Booth 2467,
during your visits to the exhibit floor.
On Thursday at 1030, please lend support to our Future
Leaders as they present their projects during their Special
Session – “The Future is Now!”
Join us again at 1345 at the Warfighters Stage, Booth 2681r
for an award ceremony acknowledging the work of our Future
Leaders.
60
E D U C ATO R S
America’s Teachers at I/ITSEC
I/ITSEC has a long history of supporting the education of
students and teachers through visits to the conference. Since
the America’s Teachers at I/ITSEC program began in 2008, we
have hosted teachers from Arizona, California, Florida, Georgia,
Maryland, Montana, New York, Ohio, Rhode Island, Tennessee,
Texas and Virginia. As part of I/ITSEC’s efforts to further
education in Science, Technology, Engineering and Mathematics
(STEM), teachers and administrators from across the country
have been invited to attend the conference. The America’s
Teachers at I/ITSEC Program consists of an orientation session,
attendance at the Modeling and Simulation techPATH, guided
tours of the Exhibit Hall, and attendance at tutorials, paper
sessions, and special events. This program is supported by the
National Training and Simulation Association and its industry
members. We would like to acknowledge, in particular, the
sponsors of the Future Leaders Pavilion and America’s Teachers
at I/ITSEC program.
Educators
Workshop to Introduce Simulation into the Physics Classroom – I/ITSEC 2014
To highlight the thriving Modeling, Simulation & Training
(MS&T) sector in the Corridor, techPATH will be hosting two
special techCAMPs to be held during the I/ITSEC conference
– one for teachers and one for students — to learn about new
technologies in the Modeling, Simulation and Training industry
and the high tech jobs that are involved. This workshop will
feature guided tours of the numerous industry exhibits.
The workshop also offers presentations from well-known
experts in the MS&T field, including representatives from the
Institute for Simulation and Training at the University of Central
Recognizing the need for a high tech workforce, the Florida High
Florida and the National Center for Simulation.
Tech Corridor Council established its educational initiative —
Teachers attending the special Educators techCAMP will
techPATH. Involving representatives from a variety of academic
utilize their knowledge and experiences at I/ITSEC to motivate
affiliations and high tech companies, techPATH is “cultivating
their students to expand their math and science educations.
tomorrow’s workforce” in the region’s 23 counties through a
Students will have the opportunity to experience the simulation
variety of innovative programs, designed to encourage students to
industry up close, through presentations and demonstrations
pursue high tech careers. techPATH supports national objectives
that present concepts in math and physics, and show the
for Science, Technology, Engineering and Mathematics (STEM).
progression from science to real jobs. To explain concepts like
The signature offering of techPATH is the Council’s techCAMP
robotics and computer programming, techPATH takes students
program. techCAMPs are high tech workshops offered to
through an interactive experiment to assign tasks to a full-size
middle and high school math, science, technology and career
“robot.”
education teachers and students, to provide information about
the industrial sectors that make up the Corridor. Since 1998,
For more information, contact Vicki Morelli at
more than 110 techCAMPs have been delivered to more than
vicki.morelli@floridahightech.com
2,090 teachers and 2,100 students.
SERIOUS GAMES
For information, contact Kent Gritton,
john.m.gritton.civ@mail.mil
62
PROJECT BASED LEARNING
STEM – Tomorrow’s
Workforce, Today!
STEM Pavilion, Booth 2755
Project Based Learning (PBL) is a hands-on approach that
engages students in active exploration of real-world problems and
challenges. Though questioning, inquiry, critical thinking and
trial and error, students absorb new knowledge and educational
content in a problem-solving context. PBL necessitates that
students interweave individual learning concepts and ideas
while collaborating and communicating with others. Studies
show that not only do students retain more of what they learn
and for longer, but they are better able to apply the knowledge in
new situations. Just as important, PBL enables students to grasp
the relevance of STEM educational content to their world both
today and into the future. This in-turn helps fill our workforce
pipeline with prospects who are better prepared, more confident
and highly engaged to take on careers in the STEM fields.
Tell me and I forget. Teach me and I remember. Involve me and I learn. Benjamin Franklin
Project Based Learning will be displayed by the following organizations:
I/ITSEC SCHOLARSHIPS
24th Annual RADM Fred Lewis I/ITSEC
Postgraduate Scholarship Recipients
RADM Fred Lewis, USN (Ret.)
President, NTSA • 1995 - 2012
These scholarships have been named
the RADM Fred Lewis Postgraduate
I/ITSEC Scholarship in honor of the former
President of the National Training and
Simulation Association (NTSA).
IMPORTANT DATES FOR 2015
When to Apply
Applications must be postmarked by
22 June 2015. (Don’t Delay!)
How to Apply
See http://www.iitsec.org/Community/
Education/Pages/Scholarships.aspx
for complete application details.
Award Announcement
5 August 2015
Daniel A O’Neil
Doctoral Candidate
Modeling and
Simulation
University of
Alabama in
Huntsville
Mitchell J. Bott
Doctoral Candidate
Modeling and
Simulation
University of
Alabama in
Huntsville
Post Graduate Scholarships
Looking for
Future Leaders in the Simulation, Training and Education Community. Learn more
about the I/ITSEC community at www.iitsec.org
Eligibility
U.S. Citizens
Full-time Masters or Doctoral students
(complete undergraduate work by Spring 2015)
See Study Disciplines at
http://www.iitsec.org/education/studentsandteachers/Pages/Scholarships.aspx
Award Amounts
$10,000 (Doctoral Candidates)
$5,000 (Masters Candidates)
Available for Fall 2015
Be our guest at I/ITSEC November 30 – December 3, 2015
Direct Further Inquiries and Provide Submissions
Lewis-I/ITSEC Scholarship Program
c/o The National Training and Simulation Association
2111 Wilson Boulevard Suite 400
Arlington, VA 22201-3061
(703) 247-2569 or bmcdaniel@ndia.org
Simulator Maintenance Technician Scholarship Programs
To promote the study of simulation technology, I/ITSEC
continues agreements with Daytona State College (DSC)
in Daytona Beach, Florida, and Lake Region State College
(LRSC), Devils Lake, North Dakota, to provide a year's
funding at each school for a student enrolled in the Simulation Technology program. DSC and LRSC have established themselves as leaders with accredited programs in
this field.
In addition to the scholarship programs, both schools are interested in acquiring corporate partners willing to provide used simulators, establish intern positions, or consider
other means of supporting the programs. Contact us at (703) 247-2569 or bmcdaniel@
ndia.org if you are interested in finding out more about scholarship or partnership opportunities.
64
VADM John S. Disher, USN (Ret.)
Executive Director, NTSA • 1991-1995
These scholarships have been named
the Vice Admiral John S. Disher Simulator
Maintenance Technology Scholarship (DSC
and LRSC) in honor of the former Executive
Director of the National Training and
Simulation Association (NTSA).
PROFESSIONAL DEVELOPMENT WORKSHOPS
Professional Development Workshops
Location:
Orange County Convention Center, South Concourse
Date:
Friday, 5 December
Times:
0700 Breakfast and registration • AM Sessions 0800 – 1200 • PM Sessions 1300-1700
Who may attend? All registrants of I/ITSEC are welcome to attend.
Fees:
There is no fee to attend.
CEU/CLP:Paid I/ITSEC Conference registrants are eligible to receive CEU/CLP credits. If not a paid attendee, a
$45 fee will be charged only if you wish to receive the CEU credits.
Registration:Preregister via https://secure2.rhq.com/iitsec/iitsec2014/public/index.cgi?track=workshoponly
Registrations also accepted on-site during I/ITSEC registration hours.
Lunch:
On own.
Coordinated by University of Central Florida’s Institute for Simulation and Training and Division of Continuing Education.
For additional information on these seminars including topical outline and instructor bios, please see: www.ce.ucf.edu/iitsec.
 All Professional Development Workshops are eligible for CEU/CLP credits. (See page 12) 
FW1 H a l f D a y Sess ion • R oom S 330C • 0800 – 1200
Certified Modeling & Simulation Professional
(CMSP) Exam Preparation
David Gross, Ph.D., Deputy Chief Engineer, Lockheed Martin
This workshop will provide an overview of the Certified Modeling & Simulation Professional (CMSP) certification program,
with a particular focus on preparing prospective applicants to
take the CMSP exam. The workshop will cover the application
and examination process (education/ work experience requirements, application fees, how the exam is administered, etc.),
in addition to an in-depth review of the new CMSP Exam Topic
Outline. The CMSP exam has been completely revised and refined over the past two years, and new applicants will now have
a choice of two tracks — Technical and User/Manager — and
will take an entirely new exam. The workshop will be taught by
charter/pioneer CMSPs who have been involved in oversight of
the CMSP program and/or creation/revision of the CMSP exam.
The workshop will not by itself prepare applicants to take the
exam, but will provide a thorough overview of exam content
and a blueprint for further self-study.
F W 2
H a l f D a y S ess ion • R oom S 330D • 0800 – 1200
Live-Virtual-Constructive (LVC)
Interoperability Techniques
Edward Powell, Ph.D., Chief Architect and Program Manager
for TENA, SAIC; Randy Saunders, The Johns Hopkins
University Applied Physics Lab
This workshop will provide an overview of the three major interoperability techniques and the future roadmap for LVC integration. Recognized experts in the use of the Distributed Interactive
Simulation (DIS) standards, the High Level Architecture (HLA)
for Modeling and Simulation, and the Test and Training Enabling
Architecture (TENA) will provide descriptions of their architectures, and discuss some of their use cases. Recent and planned
evolution of each architecture will be explained. A discussion of
how these architectures are actually used in the real world and
the process for integrating disparate systems in a multi-archi-
tecture environment will be discussed. This will include a short
discussion of the Distributed Simulation Engineering and Execution Process (DSEEP) Multi-Architecture Overlay (DMAO) and its
application to LVC interoperability. The format of the workshop
will be part lecture and part informal discussion/question answer.
Participants are encouraged to raise specific topics.
FW 3 H al f D ay Sessi on • Room S330G • 0 8 0 0 – 1 2 0 0
Modeling & Simulation for Acquisition
Rob Lisle, Newport News Shipbuilding; Mike O’Neal,
MARCORSYSCOM SIAT
Modeling and Simulation (M&S) in Department of Defense
(DoD) acquisition programs encompasses a wide variety of
technologies, organizations, processes, and best practices. This
workshop provides a practical overview of M&S for acquisition,
created by professionals experienced in the largest DoD acquisition programs. The workshop begins with a high-level presentation of organizing principles, and then transitions to specific,
real-world examples. A guidebook and slides will be provided as
handouts for this course.
FW 4 H al f D ay Sessi on • Room 330H • 0 8 0 0 – 1 2 0 0
Back to the Future: Workshop on Applying
the UJTL: Linking Missions, METLs, DRRS,
and Lessons Learned to Capabilities
David K. Brown, Ph.D., Naval Warfare Analyst, U.S. Navy,
Navy Warfare Development Command
Achievement starts with a definite sense of Mission. All must
understand the central role played by the power of alignment of
policies and systems. “Interoperability” and “Integrated” have
become common terms in defining requirements. Since the early
1990’s, DoD has employed the Universal Joint Task List (UJTL)
and the Joint Training System (JTS) to align policies and concepts to produce 21st century capabilities across DoD. But we
are still struggling to gain wide understanding and appreciation
of the power of these simple concepts.
PROFESSIONAL DEVELOPMENT WORKSHOPS
Do you understand “mission architectures?” Can you describe
the purpose and potential applications for Mission Essential
Tasks? Can you describe how conditions help us know we are
ready to win? How do we know we are building interoperable
systems–DRRS, JTIMS, etc. for mission planning, preparation,
training, capabilities, and assessment? How do we know we are
learning from “lessons learned”? How do we ensure we remain
focused on excellence, achievement, and reward ever-improving
performance?
Two key concepts of DoD’s Training Transformation—
Training Transparency and Mission Rehearsal—must endure.
Beginning with a “Mission focus,” this Friday Workshop delves
into the execution of the UJTL and the JTS process and their
links to DRRS, Capabilities, and other programs across DoD
to illumine the way ahead including connections across the
“whole of government.” This workshop will equip current and
future leaders and decision makers involved in designing and
developing mission architectures and supporting systems with
insights to capitalize on existing directives and future programs.
Bring a desire to learn to complement a focus on mission
accomplishment.
Audience: People who want to make DoD and the “whole
of government” improve across the wide range of services and
operations. Current and future leaders and decision makers
involved in designing and developing mission architectures
and supporting systems with insights to capitalize on existing
directives. Engineering students studying systems engineering
and operations management.
Pre-requisites: A basic understanding of a sense of mission
and purpose. Bring a desire to learn to complement a focus on
mission accomplishment.
F W 5
H a l f D a y S ess ion • R oom S 330F • 080 0 – 1200
Applied Earned Value Management
Michael Staley, PE, PMP, Dean, School of Engineering, Design
and Construction, Seminole State College
This interactive course will review the EVM-Analysis fundamentals and then apply these fundamentals to a series of EVM
mini-case studies. Each mini-case study increases in complexity
providing insight into the application of EVM to projects. The
mini-case studies will illustrate the following topics:
• How to create Project Measurement Baselines (PMB) to include a scope baseline, schedule baseline, cost baseline and
an integrated PMB.
• A review of Earned Value Analysis to include calculating cost
and schedule performance and understanding the meaning of
these indicators.
• Understanding Project S-Curves: a powerful management tool
for quickly assessing a portfolio of projects.
• Introduction to Earned Schedule Analysis as an extension to
EVM and forecasting in both the cost and time dimensions.
•
Identify project performance issues from the earned value
analysis: using the four basic project performance limit states.
• Uncovering hidden performance issues through earned value
analysis: Calculating a CPI for every performance unit.
66
FW 6 H al f D ay Sessi on • Room S330F • 1 3 0 0 – 1 7 0 0
Seamless Mobile Learning and Simulations
David Metcalf, Ph.D. Director, Mixed Emerging Technology
Integration Lab, UCF Institute for Simulation and Training;
Angela Hamilton, Program Lead, Mixed Emerging Technology
Integration Lab, UCF Institute for Simulation and Training
Participants will discuss how to promote learning and performance within a mobile workforce that is separated by time,
space, and context. The workshop will demonstrate and elaborate on the affordances of mobile simulation for promoting
seamless formal and informal learning experiences and increasing human performance. Focus will be on best practices for design, development, and strategy.
In addition to ubiquitous mobile content delivery and
assessment channels (e-mail, voice, text messages, web, and
mobile apps), emerging technologies and capabilities such
as context-awareness, mobile 3D, and augmented reality are
expanding the potential applications of mobile simulation.
Participants will 1) see current government and military
examples, 2) explore key technological features and design
characteristics unique to mobile, and 3) develop their own
mobile strategy capable of bridging formal and informal contexts.
The workshop will cover information necessary to build and
implement a cohesive design and development strategy for
seamless mobile training and simulation and will also include a
hands-on exercise.
FW 7 Ful l D ay Sessi on • Room S330E • 0 8 0 0 – 1 7 0 0
Serious Game Design Tutorial
Talib Hussain, Ph.D., Senior Scientist, Raytheon BBN
Technologies; Kelly Pounds, Vice President, IDEAS Learning;
Vance Souders, Producer, Janus Research
Participants will be introduced to key concepts, steps and
processes involved in designing a serious game for learning.
Through hands-on activities and working together in groups,
participants will design a learning game. Participants will experience each phase of the design process, including identifying
the training requirements and learning objectives, creating an
effective story, determining instructional and gaming strategies,
and designing key game and instructional mechanics.
Central to our approach will be ensuring that that any key
design decision addresses both gaming and instructional
considerations. During the workshop, participants will be
introduced to key methods to use and issues to consider when
designing a learning game. Groups will share their designs and
discuss their decisions after each phase of design.
The workshop will be held in two sessions, with the same
groups extending from the first session into the second session.
New participants are encouraged to participate for both sessions.
Participants from the 2013 Serious Game Design Professional
Development Workshop are welcome for the whole day, or
may join just the second session, which will cover topics not
addressed last year.
Exhibits
HALL HAPPENINGS
Attendee Luncheon
Lunch will be served Tuesday - Thursday at 1200. You must enter & exit luncheon through the Exhibit Hall. Full Conference
registrants will receive lunch tickets with their registration materials. Exhibitors and Visitors may purchase a ticket for
$25.00 at the main Registration Station. Lunch tickets are dated; you must present the current day’s lunch ticket for entry.
Connections Lounge & Grill
Stop by and relax in the Connections Lounge & Grill for a bite to eat or a refreshing drink and then connect to your
email or review the I/ITSEC program online to plan your next move at the conference. Connections Lounge & Grill will
be located in Booth 100, South A Hall.
Show Management Office
Room S220A • The Show Management Office will be staffed during show hours for all questions regarding booth space,
rules, regulations, exhibitor locators, security and late/early passes. Registration will not be made available at the Show
Management Office.
National Training & Simulation Association (NTSA)
Booth 2481 • The National Training and Simulation Association (NTSA) is America’s premier organization representing
the interests of the modeling and simulation community. As such, it serves as a constant point of contact for government,
academia, industry, research organizations and the military to exchange information, share knowledge, align business
interests, and in general stimulate the growth and overall dynamism of the industry.
Service Booths
U.S. Army PEO STRI
1539
U.S. Marine Corps Systems Command PM TRASYS
1433
U.S. Navy NAWCTSD
1439
USAF Training Systems Product Group
1533
International Pavilions
Canada539
Netherlands559
European Training & Simulation Association
2280
Healthcare Pavilion
Society for Simulation in Healthcare
2649-2751
Recognizing that simulation represents a paradigm shift in health care education, SSH promotes improvements in
simulation technology, educational methods, practitioner assessment, and patient safety that promote better patient
care and can improve patient outcome. Other participants in the Healthcare Pavilion: Laerdal Medical, SonoSim,
TraumaFX, Gaumard, Reynolds.
HALL HAPPENINGS
Innovation Showcase
Exhibit Hall – Booth 2287
P
resentations within the Innovation Showcase are led by cutting-edge exhibiting companies that are knowledgeable on
the various subject matter within the M&S industry. Mark your calendar to stop by one of the 30-minute sessions to
hear what is new and exciting in M&S! Be sure to check out the official I/ITSEC website and onsite signage for updated
participants.
(As of 20 October 2014)
Monday, 1 December
1430
Delivering Affordable & Available Radio Training
Calytrix Technologies
1515
Solid State Light Source Projectors: LED and Laser-Phosphor Choices
Digital Projection
Improving Infantry Squad Maneuver and TTP’s Using Immersive Simulators
Serious Simulations, LLC
1600
1645
Tuesday, 2 December
1230
Using the Unity Game Engine to Produce High-Fidelity 3D Interactive Training
Simulations on a Shoestring Budget
ForgeFX Simulations
1315
Steam-line Training Analysis and Drive Effective Efficient and Timely Solutions with
ADVISOR Enterprise
BNH Expert Software
1400
Unreal Engine Single & Multiplayer Immersive Training in the Browser and the Cloud
Virtual Heroes Division of
ARA, Inc.
1445
MASA Sword – and A.I.-based Simulation Solution Used to Train Logistics Commander
MASA Group
1530
Delivering the Impossible: Simulation and Training Applications to Mobile Devices
Onlive
1615
Advances in Sensor Equipped Unmanned Vehicles
Presagis
1700
Accessible Mental Health Awareness with Virtual Reality
Cubicle Ninjas LLC
1000
GreenTargets: Sound Environmental
Sterling Global Operationis, Inc.
1045
VAS Training and Simulation, Present and Future
Frost & Sullivan
1130
The Emergence of Web & Mobile Technologies in Modeling, Simulation and Training
VT MAK
1230
New Innovative Products in Relative Navigation and Timing
Geodetics
1315
CyberSecurity Edge – A Lifecycle Solution
SAIC
1400
Rapid Development of Adaptive Training Using Automated Analysis
Eduworks Corporation
1445
Simulating Dynamic Environments: Vehicles, Earthmoving and Cable Dynamics with
the Vortex Platform
CM Labs Simulations
1530
Live-Fire Gamification
Open Fire
1615
Measuring and Monitoring the Intangible Risk, Team Communication Skills
Crewfactors
1700
StallBox: A Flexible Math Model Upgrade Solution for Existing Flight Simulators
Bihrle Applied Research, Inc.
1000
Modest3D – An Evolution in 3D Content Development
AAADA
1045
Network Solutions for Simulation & Training
Brocade
1130
Combating SSR (Survival Stress Reaction)
MJ Impulse
1215
1300
1345
68
HALL HAPPENINGS
Exhibitor Networking Event
Tu e s d a y, 2 De c e m b e r • 1700 - 1830 • E x hib it H a lls
B
e sure to kick off I/ITSEC 2014 with a stop by one of the participating booths at the I/ITSEC Exhibitor Networking
Event. What a great way to view the latest technology while networking with exhibitors and your fellow attendees.
Be sure to check out the official I/ITSEC website and onsite signage for updated participants.
Booth #
Company
429
AMSEC, subsidiary of Huntington Ingalls Industries
517
Oakwood Worldwide
635
Aptima, Inc.
721Cubic
835
Alion Science and Technology
1225
VT MÄK
1421
JVC Visual
1471
1723
Q4 Services
2115
Adayana Government Group
2221Thales
2280ETSA
2281NTSA
2357SAIC
2419 The DiSTI Corporation
2427
RUAG Defence
2436BARCO
2014 EXHIBITORS
NTSA Sustaining Member • NTSA Regular Member
3D Perception
865
3D Systems, Simbionix
516
4C Strategies
873
8LAS2085
AAADA539
Acme Worldwide Enterprises, Inc.
1074
Adacel Systems, Inc.
448
Adayana, Inc.
2115
Advanced Simulation Technology, Inc. (ASTi)
1149
Aechelon Technology, Inc.
1722
AEgis Technologies
2411
Aero Simulation, Inc.
1113
Aerotronics417
Air National Guard Trainer Development
1769
Alelo Inc.
2122
835
American Apex Corporation
349
AMSEC a Subsidiary of Huntington Ingalls Industries
429
Applied Research Associates Inc.
872
Aptima, Inc.
635
AQT Solutions
1170
Argon Electronics
449
Arrington Research, Inc.
781
Ascension Technology Corporation
560
AVA Direct
2257
AVT Simulation
1906
BAE Systems
1863
Bagira Systems Ltd.
2173
Barco/projectiondesign2435
Battle Space
1249
B-Design3D880
BEC1758
Beijing Sunheart Simulation Technology Co., Ltd
1788
BGI, LLC
681
Bihrle Applied Research, Inc.
1139
BIONATICS2022
BNH Expert Software, Inc.
348
Boeing Company
1700
Bohemia Interactive Simulations
2235
Bosch Rexroth
1933
Brazilian Defense and Security Industries Association
1080
Brocade Communications Systems
439
Bugeye Technologies
717
C2 Technologies, Inc.
801
CAE1734
CALIBRE403
Calytrix2233
Camber Corporation
813
Canon USA
2073
Capstone Corporation
513
Carley Corporation
2049
CAST Navigation LLC
1258
Central Florida STEM Education Council
2852
CGI2039
70
Charles F. Day & Associates LLC
2273
Christie Digital Systems
2248
CM Labs Simulations
622
Concurrent Real-Time
2033
Connections Café and Lounge
100
Control Products Corporation
1157
Corsair Engineering
1234
Cranfield Aerospace Ltd.
1973
Crew Training International
2164
CrewFactors Ltd.
2170
CSE Software Inc.
2751
Cubic Defense Applications
721
Cubicle Ninjas
2850
Cybernet Systems Corporation
1932
D2 TEAM-Sim
2115
Da-Lite Screen Company
2469
DAQRI659
David Clark Company Incorporated
2179
D-BOX Technologies Inc.
613
Dedicated Computing
2068
Delaware Resource Group of Oklahoma, LLC
664
Design Interactive, Inc.
2069
Diamond Visionics
2239
Diginext2726
Digital Projection
1280
Discovery Machine, Inc.
806
Displays & Optical Technologies, Inc.
1926
DiSTI Corporation
2419
Doron Precision Systems, Inc.
665
Draper, Inc.
761
Drew Defense GmbH
2160
Driven Technologies, Inc.
670
Dytecna2288
E2M Technologies B.V
2107
Eagle Support Services Corp
2734
EDM Ltd.
1072
Eduworks Corporation
2079
Elbit Systems, Ltd.
1581
Electro-Optical Imaging, Inc.
521
Elite Finesse
1070
eMDee Technology, Inc.
780
Engility2457
Engineering & Computer Simulations, Inc.
1465
Enovative Technologies
409
Envitia, Inc.
620
ESP, Inc.
1914
ESRI301
ETC2127
E-Tech529
ETSA2280
Extron Electronics
2268
eyevis GmbH
758
F2Si808
FAAC, Inc.
1873
2014 EXHIBITORS
Fain Models, Simulation Systems
1973
Fedex Institute of Technology
2079
Fidelity Technologies
1564
FIRST Robotics
2862
FlightSafety International
1401
ForgeFX Simulations
2088
Forth Dimension Displays
554
Frasca International, Inc.
672
Full Sail University
514
Future Leaders Pavilion
2467
Gaumard Scientific
2748
General Dynamics
1413
General Dynamics IT
1220
Geodetics, Inc.
515
George Mason University Serious Games Institute
421
Georgia Tech Research Institute
1009
Geoweb3d713
Global Business Solutions, Inc. (GBSI)
641
Hampden Engineering Corporation
407
Harris Corporation
534
Hatalom Systems, LLC
401
Havok
1900, 1901
IDS International
2712
IEEE Xplore Digital Library
612
IHS423
Illogic S.r.l.
538
Immersive Display Solutions, Inc.
1180
IMMY, Inc.
2256
Indra1727
Industrial Smoke & Mirrors
1038
Inert Products LLC
2463
Inertial Labs, Inc.
2271
Innovation Showcase
2287
Institute for Simulation and Training
2763
Intelligent Decisions, Inc.
1882
Inter-Coastal Electronics Inc.
834
Intevac Photonics
632
IPKeys Technologies
870
Iris Dynamics Ltd.
616
Israel Aerospace Industries Ltd.
1219
ITEC2182
J.F. Taylor, Inc.
2225
JANUS Research Group
2740
JHT, Inc.
1624
JRL Ventures, Inc.
2120
JRM Technologies
1915
JVC Professional Products Company
1421
Katz Computer Service, Inc.
2363
Kentucky Trailer Technologies
1163
Kongsberg Maritime Simulation, Inc.
1000
Kratos Technology & Training Solutions
2161
Krauss-Maffei Wegmann GmbH & Co. KG
2401
L-3 Communications
1748
Laerdal Medical
2657
Laser Ammo
542
Laser Shot
1882
Leidos1012
Lockheed Martin
1449
LSI, Inc.
840
M3 Touch, Inc.
425
Marathon Targets Pty Ltd.
2741
Marine Corps System Command (PM TRASYS)
1433
MASA Group
2227
MDI443
Mechanical Simulation
735
Meggitt Training Systems
1712
Merlin Simulation, Inc.
523
MetaVR1249
Metters Incorporated
2700
Military Training Technology
2157
MIL-SIM-FX International, Inc.
2089
MJ Impulse Inc.
2365
Moench Publishing Group
773
Moog1513
Motion Analysis
895
MSE1249
MYMIC, LLC
2200
NASA PLACE
2769, 2864
National Center for Simulation
2007
National Training & Simulation Association (NTSA)
2481
NATO1907
nCASE – Materials World Modules
2761
NCMA Mid-Florida Chapter
413
NCS/Orlando Tech
2753
Netherlands Simulation Pavilion
559
Newport News Shipbuilding
1620
Next Limit Technologies
1172
Nida Corporation
501
Northrop Grumman
1949
Nova Technologies
2619
NSC2483
NVIS, Inc.
1158
Oak Grove Technologies
765
Oakwood Worldwide
517
ODU (MSVE Department)
2765
Onlive1121
Openfire Systems
552
OptiTrack1481
Orlando Science Center Hands-On STEM Activities
2759
Oshkosh Specialty Vehicles
2710
Otto Trading
533
Panel Products, Inc.
707
Paramount Panels, Inc.
575
Parsons1213
PatchPlus Consulting, Inc.
525
Pathfinder Systems, Inc.
329
Patriot Products LLC
1260
Pitch Technologies
1927
2014 EXHIBITORS
PLEXSYS Interface Products, Inc.
1058
PLW Modelworks
548
Polhemus1812
Power Innovations Int’l Inc.
1961
Powersource Transportation, Inc.
512
Pragmatics, Inc.
520
Presagis1920
Project Lead the Way
2856
Promo Coins
2282
Pulau Corporation
2214
Q4 Services
1723
QinetiQ1238
Quadrant Simulation Systems, Inc.
1027
Qualisys Motion Systems
333
Quantum 3D, Inc.
2000
Questionmark508
Rapid Prototyping Services
524
Rave Computer
2206
RAYDON Corporation
1048
Raytheon2149
REALTIMEVISUAL2057
Reynolds Advanced Materials
2749
RGB Spectrum
2260
Rheinmetall Defence
2213
RightEye, LLC
775
Rockwell Collins
2201
RPA Electronic Solutions, Inc.
2220
RSI Visual Systems
1921
RUAG Defence
2427
SA Photonics
618
Saab Defense and Security
1939
Safety Training Systems, Inc.
861
SAIC2281
Santoku Corporation
881
SCALABLE Network Technologies
1213
SDS International
807
SEKRI2275
Sensics1159
SensoMotoric Instruments, Inc.
564
Senspex, Inc.
709
Serco, Inc.
2471
Serious Games Challenge
2663
Serious Simulations LLC
2704
SGB Enterprises, Inc.
2735
Shephard Media
540
Shooting Range Industries, LLC
675
Sierra Nevada Corporation
433
SimiGon, Inc.
2101
SIMmersion LLC
2224
SimPhonics, Inc.
2015
Simtek, Inc.
535
Simthetiq871
Simulation and Control Technologies
607
SMART EYE AB
680
72
1471
Society for Simulation in Healthcare
2750
Sonalysts1008
SonoSim, Inc.
2653
Sony Electronics, Inc.
848
Sterling Global Operations, Inc.
2269
Stirling Dynamics
621
Stottler Henke Associates
1800
Surgical Science, Inc.
532
Symbolic Displays, Inc.
2020
SYNERCO SA
1969
Synertial Motion Werx
580
Tactical Communications Group
2185
Tactical Micro
1161
TAPE/Strong Point Research Division
1135
The Tatitlek Corporation
2449
Tech Wizards, Inc.
2848
Technical Sales & Applications/Pelican Products
2278
Ternion Corporation
701
Thales2221
Theissen Training Systems, Inc.
1026
ThyssenKrupp Marine Systems GmbH
772
Tobii Technology, Inc.
415
Trailer Transit, Inc.
634
TraumaFX-KGS2649
TrianGraphics871
TRU Simulation & Training
1101
UCF Foundation, Inc.
571
UCF RESTORES
1256
UFA, Inc.
601
United Electronic Industries (UEI)
1806
URS507
U.S. Army PEO STRI
1539
U.S. Jaclean
771
U.S. Navy / NAWCTSD
1439
USAA2183
USAF Training Systems Product Group
1533
Valkyrie Enterprises, LLC
2713
VDC Display Systems
1032
Vencore2021
Veraxx Engineering Corporation
1127
VirTra Systems, Inc.
1762
Virtuix Technologies LLC
2461
VPixx Technologies
581
VT MÄK
1225
Wacom558
Warfighters Corner
2681
Westar Display Technologies, Inc.
1078
WITTENSTEIN Aerospace & Simulation
1780
WorldViz1162
Worldwide Technology
454
ZedaSoft, Inc.
1058
Zel Technologies, LLC
2027
Committees
COMMITTEES
Conference Committee
Service Executives
MG Jon Maddux, USA, Program Executive Officer for PEO STRI
CAPT Wes Naylor, USN, Commanding Officer, NAWCTSD
Col (Sel) Walter Yates, USMC, Program Manager, MARCORSYSCOM
PM TRASYS
Col Dan Marticello, USAF, Director, Simulators Division, Air Force
Materiel Command
OSD/Joint ExecutiveFrank C. DiGiovanni, Acting Deputy Assistant Secretary of Defense
(Readiness)
Service Principals
Traci Jones, U.S. Army PEO STRI
Diana Teel, NAWCTSD
Martin Bushika, MARCORSYSCOM PM TRASYS
Tony DalSasso, USAF Simulators Division
OSD PrincipalBrent Barrow, Office of the Deputy Assistant Secretary of Defense
for Readiness
Conference Chair
Ron Smits, Engility Corporation
Deputy Conference Chair Brent Smith, Engineering & Computer Simulations, Inc.
Program ChairJanet Spruill, Serco, Inc.
Deputy Program Chair
David Hutchings, Raydon Corporation
Subcommittee Chairs
Education
Ramona Shires, Education Programs and Support, Inc.
Emerging Concepts and Innovative Technologies
Stacy Pierce, Rockwell Collins
Human Systems Engineering
Jim Threlfall, C2 Technologies, Inc.
Policy, Standards, Management and AcquisitionRoy Scrudder, The University of Texas at Austin, Applied Research
Laboratories
Simulation
Brian Holmes, The Aegis Technologies Group, Inc.
Training
Fred Fleury, ZedaSoft, Inc.
Best Paper Committee Chair Karen Williams, U.S. Army PEO STRI
Tutorial Board Chair
Robert Richbourg, Ph.D., Institute for Defense Analyses
Best Tutorial Committee Chair
Katrina Ricci, Ph.D., NAWCTSD
Education and Training Advisor
VADM Al Harms, USN (Ret.), UCF, VP Emeritus
Scholarship Committee Chair
Cyndi Turner, Saab Defence and Security
Director for International ProgramsK. Denise Threlfall, Ph.D., Kratos Defense & Security Solutions
Strategic Planning and STEM Committee ChairLinda Brent, Ed.D., The ASTA Group, Inc.
Special Event Coordinator
Jennifer McNamara, BreakAway Games
Website and Social Media Advisor
Sae Schatz, Ph.D.
Conference Sponsor
National Training and Simulation Association
President
RADM James Robb, USN (Ret.)
Coordinator Barbara McDaniel
Exhibits and Sponsorships
Debbie Langelier, CEM
Advisor
CAPT Nelson P. Jackson, USN (Ret.)
Media Relations/Communications
John Williams
Operations
Len Kravitz, LRK Associates, Inc.
Protocol Coordinator
Steve Detro, Lockheed Martin
Historians
Carol Denton and Allen Collier
Veterans Coordinator
Earle Denton
Council of Chairs
The Council of Chairs is a special advisory group to the NTSA Sponsor and to the I/ITSEC Committee organization. The exclusive
membership comprises the previous I/ITSEC Conference chairs. Drawing on their cumulative experience, these leaders provide a
unique perspective and advice for the ongoing mission of I/ITSEC.
1979A.W. Herzog (Deceased)
and G.V. (Vince) Amico
1980 Robert W. Layne
1981 Kurt Merl
1982 James A. Gardner, Ph.D.
1983 John Todd (Deceased)
1984Ralph T. Davis (Deceased)
1985 John W. Hammond
1986 Rodney S. Rougelot
1987
1988
1989
1990
1991
1992
1993
1994
1995 David P. Crane (Deceased)
Thomas E. Sitterley, Ph.D.
Arthur L. Banman
Steve Selcho
Donald M. Campbell
Jerry Jerome
J.D. (Jack) Drewett
G.P. (Pres) McGee
Judith Riess, Ph.D.
1996
1997 1998
1999
2000
2001
2002
2003
2004
Ed Ward
Dennis Shockley
Jim Cooksey
Stan Aronberg (Deceased)
Ron Johnson (Deceased)
Debbie L. Berry
Paul Bernhardt
Bill Walsh
Buck Leahy
2006 2007
2008
2009
2010
2011
2012
2013
Steve Swaine
Steve Detro
Amy Henninger, Ph.D.
Don Currie
DeLloyd Voorhees, Jr.
Jim Wall, Ph.D.
Mike Genetti, Ph.D.
Amy Motko
Cyndi Turner
73
COMMITTEES
Education
Chair:
Ramona Shires
Education Programs
and Support, Inc.
Deputy Chair:
Christopher Bryant
Operation Smile
Emerging
Concepts &
Innovative
Technologies
Chair:
Stacy Pierce
Rockwell Collins
Deputy Chair:
Steve Gordon, Ph.D.
Georgia Tech Research
Institute
Human Systems
Engineering
Chair:
Jim Threlfall
C2 Technologies, Inc.
Deputy Chair:
Elizabeth Biddle, Ph.D.
The Boeing Company
74
Mike Armstrong, Pulau Corporation
Benjamin Bell, Ph.D., Aqru Research and
Technology, LLC
Jan Brown, CAE USA
Kevin Cahill, Aero Simulation, Inc.
Joe Dalton, Full Sail University
Nina Deibler, Serco, Inc.
Robert Dixon, U.S. Army PEO STRI
Mark Friedman, Concurrent Technologies
Corporation
Liz Gehr, Ph.D., The Boeing Company
Michael Genetti, Ph.D., Rockwell Collins Simulation
& Training Solutions
Michael Hagen, Squadron Officer College
Cheryl Johnson, NAWCTSD
Adelle Lynch, Rockwell Collins
Perry McDowell, Naval Postgraduate School
Ellen Menaker, Ph.D., MTP Associates
Koren Odermann, MARCORSYSCOM PM TRASYS
Kara Orvis, Pacific Science
Tiffany Parrish, NAWCTSD
Kelly Sauls, General Dynamics Information
Technology
Robert “Buddha” Snyder, WBB, Inc.
Brian Stensrud, Ph.D., Soar Technology, Inc.
Suzy Sutton, HQ Air Education & Training
Command
Skip Vibert
Brian Vogt, U.S. Army TRADOC
JoAnn Wesley, MARCORSYSCOM PM TRASYS
Jay White, Department of Homeland Security
Anya Andrews, Ph.D., Erudition Corporation
Scott Ariotti, The DiSTI Corporation
Stu Armstrong, QinetiQ Training & Simulation, Inc.
John Aughey, The Boeing Company
Paul Bogard, USAF Simulators Division
Harold Bowlin, USSOCOM J7-TM
Ba Duong, MARCORSYSCOM PM TRASYS
John Dzenutis, The Boeing Company
Charles Frye, Camber Corporation
Jim Godwin, The Tolliver Group, Inc.
Kent Gritton, JTIEC/NAWCTSD
Jeff Grubb, NAVAIRSYSCOM
Susan Harkrider, Night Vision & Electronic Sensor
Directorate
Chess Harris, Capstone Corporation
Bob Heinlein, Symbionix
James (Josh) Jackson, SAIC
Sherrie Jones, Ph.D., MARCORSYSCOM PM
TRASYS
Jennifer Murphy, Ph.D., Quantum Improvements
Consulting, LLC
Michael O’Connor, Trideum Corporation
Brian Overy, Diamond Visionics
Constance Perry, U.S. Army PEO STRI
Beth Pettitt, ARL HRED STTC
Mark Phillips, Lockheed Martin Mission Systems
and Training
Luis Pineiro, AFRL
Rebecca Sampson, Eaker Center USAF PPDS
Susan Sherman, NAWCTSD
Dennis Shockley, Motion Analysis Corporation
Luis E. Velazquez, MARCORSYCOM SIAT
Eric Weisel, Ph.D., Old Dominion University
Jennifer Arnold, Booz|Allen|Hamilton, Inc.
Lee Barnes, ProActive Technologies, LLC
Kristen Barrera, Air Force Research Laboratory
Maureen Bergondy-Wilhelm, NAWCTSD
Martin Bink, Ph.D., U.S. Army Research Institute
Phil Brown, D.M., U.S. NORTHCOM
Bill Gerber, Ph.D., WJ Gerber Consulting
Todd Glenn, FAAC, Inc.
Mike Haas, Air Force Institute of Technology
Matthew Hackett, Ph.D., ARL HRED STTC
Kelly Hale, Ph.D., Design Interactive, Inc.
Kelsey Henderson, Ph.D., NAWCTSD
Robert Hester, Ph.D., University of Mississippi
Medical Center
Eric Jarabak, MARCORSYSCOM PM TRASYS
Randy Jensen, Stottler Henke Associates, Inc.
Jennifer Law, AFAMS
Pres McGee, ZedaSoft, Inc.
Ingrid Mellone, Camber Corporation
Barron Mills, MARCORSYCOM PM TRASYS
Steve Monson, The Boeing Company
Susan Myers, Ph.D., ManTech International
Corporation
Tiffany Poeppelman
John Schlott, L-3 Communications Link Simulation
& Training
Don Sine, Ph.D., Dickieson Projects, Inc.
Robert Sottilare, Ph.D., ARL HRED STTC
Jerry Stahl, Cypress International
Kendy Vierling, Ph.D., USMC TECOM
COMMITTEES
Policy, Standards,
Management,
and Acquisition
Randy Allen, Ph.D., Lone Star Analysis
Keith Anderson, Paltech, Inc.
Paul Bernhardt, P&S Partners
Jan Drabczuk, JD Defense Solutions, LLC.
Sam Fragapane, AFAMS
Richard Grohs, HQ Air Combat Command
Bill Hopkinson, JTIEC
Bill Hornsby, A. Harold & Associates, LLC
Steve Husak, Steve Husak & Associates
Tara Kilcullen, Raydon Corporation
Pete Marion, TMST Consultants
Rob Matthews, NAWCTSD
Annie Patenaude, AMP Analytics
Karen Pogoloff, MTS Technologies, Inc.
Jeff Raver, SAIC
James Reynolds, USMC TECOM
Lawrence A. Rieger, TRADOC ARCIC
Robby Robson, Ph.D., Eduworks
Elizabeth Root, Isis Solutions and Results, LLC
Mark Russell, Mark Russell Consulting
Robert Scott, NSWCCD
Chuck Secard, Lockheed Martin Mission Systems
and Training
Craig Siefert, USAF Simulators Division
Harry Sotomayor, U.S. Army PEO STRI
Brett Telford, MCMSMO
Rene Thomas-Rizzo, ASN RD&A
Mary Trier, Capital Communications & Consulting
Gloria Tuck, MARCORSYSCOM PM TRASYS
Paul Watson, U.S. Army PEO STRI
Michael Aldinger, Northrop Grumman
Lisa Jean Bair, SAIC
Keith Biggers, Ph.D., Texas A&M Engineering
Experiment Station
Richard Boyd, Szl.it, Inc.
Carla Cropper, Rockwell Collins Simulation &
Training Solutions
Randy Crowe, Ph.D., Lockheed Martin Mission
Ron Dionne, FLETC
Gerald Dreggors, Northrop Grumman
Leslie Dubow, Veterans Health Administration
SimLEARN
Bradley Ehrhardt, NAWCTSD
Nick Giannias, CAE
Toni Hawkins-Scribner, Air University
Scott Hooper, Havok
Bob Kleinhample, SAIC
Ed Kulakowski, OT Training Solutions, Inc.
Timothy Lincourt, USAF Simulators Division
Daniel Loth, MARCORSYSCOM PM TRASYS
Long Nguyen, Ph.D., NAWCTSD
Michael O’Neal, MARCORSYSCOM SIAT
Jonathon Richardson, MARCORSYSCOM PM
TRASYS
Pete Schrider
Gregory Sidor, AFRL
Mark Soodeen, CAE
Brent Subramanian, Daedalus Technologies
Brent Terwilliger, Ph.D., Embry Riddle Aeronautical
University
Sandy Veautour, U.S. Army AMRDEC
Karen Williams, U.S. Army PEO STRI
Cynthia Adams, Booz|Allen|Hamilton, Inc.
James Allen, USAF Simulators Division
Jan Baka, Electronic Consulting Services, Inc.
Gene Beauvais, Raytheon Company
Amy Bolton, Ph.D., Office of Naval Research
Brian Cairns, Moulage Sciences & Training
Susan Coleman, Ph.D., Intelligent Decision
Systems, Inc.
Curtis Conkey, Ph.D., U.S. Army AMRDEC
Karen Cooper, Ph.D., NAWCAD
Javier "Jeff" Covelli, Ph.D., CMSP, PMP, Computer
Sciences Corporation
Gabriel Diaz, USMC Warfighting Laboratory
Mary Driskel, NAWCTSD
Catherine Emerick, QinetiQ Training & Simulation,
Inc.
Graham Fleener, U.S. Army PEO STRI
Luis Garcia, MARCORSYSCOM, PM TRASYS
Bob Johnson, McKean-Defense Group, LLC
Nathan Jones, MARCORSYSCOM PM TRASYS
Anne Little, Ph.D., Addx Corporation
Paul Lyon, BARCO Simulation
Michael Motko, QinetiQ Training & Simulation, Inc.
Kristy Murray, Ed.D., Summit Strategic Consulting
Elaine Raybourn, Ph.D., Sandia National
Bill “Roto” Reuter, R-Squared Solutions, LLC
John Stratis, Jacobs Technology, Inc.
Lisa Tripp, USAF 711 Human Performance Wing
Robert Wallace, USAF 29th Training Systems
Squadron
Eliot Winer, Ph.D., Iowa State University
Chair:
Roy Scrudder
The University of
Texas at Austin,
Applied Research
Laboratories
Deputy Chair: Tom Yanoschik
SAIC
Simulation
Chair:
Brian Holmes
The AEgis
Technologies Group
Inc.
Deputy Chair:
Matt Spruill
Engineering &
Computer Simulations,
Inc.
Training
Chair:
Fred Fleury
ZedaSoft, Inc.
Deputy Chair:
Felicia Douglis
FRD Solutions, LLC
SPECIAL TEAMS
International Programs
Director
K. Denise Threlfall, Ph.D., Kratos Defense
& Security Solutions
Deputy Coordinator
Cathy Matthews, Matthews Systems
Engineering, Inc.
Member
Michael Weber, Camber Corporation
Operations
Chair
Deputy Chairs
Jim Pohlen, Pulau Corporation
Bruce Schwanda, B.A.S. Associates, LLC
Members
Mike Armstrong, Pulau Corporation
Jan Baka, Electronic Consulting Services,
Inc.
Lee Barnes
Richard Boyd, Szl.it, Inc.
Catherine Emerick, QinetiQ Training and
Simulation, Inc.
Charlie Frye, Camber Corporation
Jim Godwin, The Tolliver Group, Inc.
Steve Golberg
Bill Hornsby, A. Harold & Associates, LLC
Zach Johnson, Booz|Allen|Hamilton, Inc.
Ed Kulakowski, OT Training Solutions, Inc.
Annie Patenaude, AMP Analytics
Skip Vibert
Serious Games Showcase
and Challenge IPT
Chair
Stu Armstrong, QinetiQ Training and
Simulation, Inc.
Deputy Chair
Leslie Dubow, Veterans Health
Administration SimLEARN
Members
Jennie Ablanedo, STTC, UCF IST
Roger Caldwell, USAF Simulators Division
Eric Church, BreakAway Games
Karen Cooper, Ph.D., NAWCAD
Mark Friedman, Concurrent Technologies
Corporation
Jason Haag, ADL Initiative
Shawn Hart, ADL Initiative
Lisa Scott Holt, Ph.D., Intelligent
Automation, Inc.
Gregg Lagnese, Autodesk
76
Matthew Morris, NAWCTSD
Elaine Raybourn, Ph.D., Sandia National
Laboratories/ADL
Trey Reyher, Booz|Allen|Hamilton, Inc.
Pete Schrider
Scott Shiffert, Hewlett-Packard
Steve Slosser, JTIEC/NAWCTSD
Peter Smith, Ph.D., UCF
Vance Souders, JANUS Research Group
Matt Spruill, Engineering & Computer
Simulations, Inc.
Stephen Stewart, Evviva Games
Shane Taber, Engineering & Computer
Simulations Inc.
K. Denise Threlfall, Ph.D., Kratos Defense
& Security Solutions
Mary Trier, Capital Communications &
Consulting
Roberto Vargas, UCF IST
Wendy Williams, NAWCTSD
Michael Woodman, Ph.D., Bohemia
Interactive Simulations
Special Events Committee
Coordinator
Deputy Chair
Benjamin Bell, Ph.D., Aqru Research and
Technology, LLC
Council of Chairs
Denny Shockley, Motion Analysis
Corporation
Members
Warfighters’ Corner
DeLloyd Voorhees, General Dynamics
Information Technology
I/ITSEC Fellows
Robert Richbourg, Ph.D., Institute for
Defense Analyses
Margaret Loper, Ph.D., Georgia Tech
Research Institute
Robert Lutz, The Johns Hopkins
University/Applied Physics Lab
Operations Liaison
STEM Committee
Chair
Linda Brent, Ed.D., The ASTA Group, LLC;
NTSA, Strategic Planning
Members
Serious Games
Students at I/ITSEC
Scholarships
CEU/Professional Development Workshops
Debbie Berry, Lockheed Martin
Maria Cherjovksy, University of Central
Florida Continuing Education
America’s Teachers at I/ITSEC
Margaret Loper, Ph.D., Georgia Tech
Research Institute
STEM Pavilion Project Based Learning Exhibits
Robert Seltzer, NAWCTSD
Teacher Tours and Training
Benn Aaronson, STEM Outreach, Team
Orlando
TechPATH
Lynn Sand, Skybridge Tactical
Eileen Smith, University of Central Florida
Tutorial Board
Chair
Robert Richbourg, Ph.D., Institute for
Defense Analyses
Deputy Chair
David Milewski, Booz|Allen|Hamilton
Members
Charles Cohen, Ph.D., Cybernet Systems
Corporation
James Coolahan, Ph.D., Coolahan
Associates, LLC
Luis Miguel Encarnação, Ph.D., ACT, Inc.
Michael Freeman, Ed.D., Adayana
Government Group
Lisa Scott Holt, Ph.D., Intelligent
Automation, Inc.
Zach Johnson, Booz|Allen|Hamilton, Inc.
Robert Lutz, The Johns Hopkins
University/Applied Physics Lab
Thomas Mastaglio, Ph.D., MYMIC
Denise Nicholson, Ph.D., Soar Technology
Inc.
S. K. Numrich, Ph.D., Institute for Defense
Analyses
Katrina Ricci, Ph.D., NAWCTSD
Leah Rowe, AFRL
Larry Skapin, The Boeing Company
Jim Wall, Ph.D., Texas A&M Engineering
Experiment Station
Future Leaders Pavilion
Ann Friel
Conference Information
R E G I S T R AT I O N I N F O R M AT I O N
About Registration
WHAT DO THE REGISTRATION FEES COVER?
Registration fees cover Continuing Education Units (CEUs), Lunches
(T-W-Th), Coffee Breaks (T-W PM, W-Th AM), Continental Breakfasts (W-Th), and the Thursday Banquet. A meeting bag with
conference materials (including an Abstract book and CD ROM of
the current papers) is included. The fees also cover administrative
VIPs, Speakers (including Paper Presenters), Media, and International registrants will have special registration stations. More
details will be provided to each group, but be sure and watch for
signage pointing to these areas.
Registration outside of the Orange County Convention Center. Attendees staying at the Hyatt Regency, formerly Peabody Orlando,
expenses incurred.
should look for the I/ITSEC Full Service Registration located
I/ITSEC Registration Services for 2014
OCCC). From Sunday noon through Tuesday, staff will assist
in the Convention Lobby (lower level, walkway from Hyatt to
We strive to minimize the time spent in line so you can move on
to the conference events or the exhibit floor. Our goal is to make
your I/ITSEC experience a pleasant one even before you enter
whether you just need to pick up a bag and badgeholder, are
starting from scratch, or need to complete any stage of the
registration process.
the OCCC. Avoid that line and move on to what you came to
Parking
I/ITSEC to do!
EXHIBITOR PARKING:
Traditional Registration Stations. Located in S220 of the South Con-
$15 per Day – For regular vehicles with re-entry privileges each
course Registration area, traditional walk-up registration will be
day. Exhibitor must show badge and receipt for repeat entries.
available for Full Service Registration, on-site payments, changes/
edits to name badges, multiple badge pick-ups, or just because
you prefer dealing one-to-one with a real person.
Alternate Registration Stations within the Orange County Convention
Center. Specific stations at the Main Registration Station will be
open Friday and Saturday to handle, especially, Exhibitor Registration. Conference Attendees are encouraged to wait until Sunday
$25 per Day – For oversized vehicles with re-entry privileges
each day. Exhibitor must show badge and receipt for repeat
entries.
ATTENDEE PARKING:
$15 per Day – For regular vehicles per entry.
$25 per Day – For oversized vehicles per entry.
afternoon or use the Self Badging/Self Registration kiosks.
Self-badging printing stations will be available for those who
pre-registered and received a confirmation number. To complete
your registration at this station, you must be paid in full with
no outstanding balance or questions remaining about your registration.
Self-Registration will be available for those who want to walk
up to a station and register on-site. Credit card payments are
required at these stations.
AFTER 5PM:
$9 per Day – For regular vehicles. Same stipulations as above.
$15 per Day – For oversized vehicles. Same stipulations as
above.
ACCEPTED PAYMENT METHODS:
Cash, Traveler’s Checks, American Express, MasterCard & Visa
Dress Code
BRANCH
CONFERENCE AND GENERAL SESSIONS
BANQUET
Army
ACUs or Duty Uniform
Army Blue (Army Evening Mess optional)
Marine Corps
Service “C”
Evening Dress (Dress Blue “B” or Service “A” optional)
Navy
Service Khaki, Navy Service Uniform
Air Force
Short or Long Service Blues
Service Dress Blue with tie and jacket (Mess Dress optional)
Coast Guard
Tropical Blue Long
Civilian
Business attire
Black tie (optional)
LODGING
The National Training and Simulation Association has blocked
rooms with the Orlando hotels listed below. Make your lodging
arrangements either on-line or by phone through Travel Planners,
our official Housing Partner, through 25 November 2014. Hotel
phone numbers will be posted on the I/ITSEC web at that time for
your convenience in making last minute changes or arrangements.
(Current room rates may apply after 25 November.) Travel Planners, Inc. is our official housing partner and the only company
authorized to represent I/ITSEC and NTSA. If you are contacted
by other companies who present themselves as representing the
Conference or Association, please report to bmcdaniel@ndia.org.
On-Line: Go to http://www.iitsec.org/attendees/Pages/Local Accommodations.aspx, select Lodging, select whether you are a corporate
or government attendee and the program attached will lead you
through the process from location, to hotel selection, to special
needs, to payment and confirmation.
By Phone: If you prefer to book via telephone, friendly and knowledgeable agents are ready to take your calls Monday through
Friday from 9:00AM – 7:00PM ET at 800-221-3531 or 212-532-1660.
More Information about Lodging Arrangements:
• Some Room Rates are subject to change, based on the govern-
ment per diem rate. Those listed with an * are the most likely
to change.
• Government Rate Room Reservations: Rooms shown in the
“Gov’t Rate” column are to be assigned to those with appropriate
ID, to be presented at the hotel desk upon check-in. Please do
not reserve unless you are eligible to do so.
• Be aware that some hotels may charge an additional Resort Fee
as well as applicable taxes.
• Additional hotels may be added at a later date.
• The individual hotels are not authorized to accept reservations
directly for this conference. You may state your hotel preference
when making your reservations.
• Attendees must identify themselves as being with the I/ITSEC
to receive the rates shown.
• The Conference is being held at the Orange County Convention
Center, located between the Hyatt Regency and the Hilton Hotels. The majority of the I/ITSEC 2014 activities will be located
in the South Concourse.
• Shuttle buses/vans will be available throughout the conference
(including following the closing banquet).
• To help defray conference management costs, an assessment
is included in the room rates shown with these hotels. We
encourage you to make your lodging arrangements within the
designated housing package established.
Hyatt Regency
u Homewood Suites
(Conference Headquarters)
8745 International Drive
(407) 248-2232
(407) 352-4000
Industry: $136 • Government: $115*
iHyatt Place Convention Center
q Castle Hotel, Autograph Collection
(407) 370-4720
(407) 317-5753
One Rate: $126
One Rate: $115*
o Rosen Centre Hotel
w Days Inn Convention Center
(407) 996-9840
(407) 352-8700
One Rate: $71
aRosen Inn at the Pointe Orlando 9000 International Drive
eDoubletree by Hilton at Sea World
(407) 996-8505
(407) 352-1100
One Rate: $81
sRosen Plaza Hotel
r Embassy Suites I-Drive
(407) 996-9700
(407) 352-1400
Industry: $184 • Government: $115
dWestin Orlando Universal Boulevard
9501 Universal Boulevard
tHampton Inn Convention Center
8900 Universal Boulevard
(407) 233-2200
(407) 354-4447
yHilton Orlando
78
6001 Destination Parkway
(407) 313-4300
GETTING AROUND DURING I/ITSEC
To get from your hotel to the South Concourse of the OCCC, you have several choices of transportation.
• I/ITSEC will provide Shuttle Bus service to all properties listed. (Schedules will be available at the hotels and at the entrance to
the conference registration area.)
• Very reasonable Public Transportation is available on the I-Ride trolley bus along International Drive.
Check http://www.iridetrolley.com or your hotel for schedules.
• Your own or a rented vehicle. Limited parking available, $15.00 per passenger car per day. Pay for each re-entry. See page 86 for
more detailed parking information. Oversize vehicles $25.00. Prices are subject to change.
• Most of the hotels are within walking distance (wear comfortable shoes).
The National Training and Simulation Association has arranged for the Hertz Company to be the
official car rental agency for I/ITSEC with the special rates below. You can also make your reservations on-line through the I/ITSEC website (Lodging/Travel). Vehicles may be returned to any
Hertz location in Florida at no additional charge.
CAR
CLASS
DAILY
WEEKEND
WEEKLY
TO RECEIVE SPECIAL MEETING RATES
A
Economy
$46.49
$27.49
$189.49
Call Hertz at 1-800-654-2240 or 405-749-4434
B
Compact
$48.49
$29.49
$199.49
or your nearest Hertz reservation center, your
C
Midsize
$54.49
$31.49
$219.49
D
Standard 2/4-Door
$57.49
$34.49
$229.49
F
Full Size 4-Door
$59.49
$39.49
$249.49
December 12, 2014 subject to car availability.
G
Premium
$63.49
$44.49
$267.49
Government surcharges, taxes, tax reimbursment,
I
Luxury
$85.49
$85.49
$599.49
title and license fee reimbursement and optional
Q4
Midsize SUV
$64.49
$59.49
$329.49
L
Standard SUV
$64.49
$74.49
$369.49
R
Minivan 2WD
$69.49
$79.49
$399.49
differential for age 20-24 applies). Standard rental
U
Convertible
$66.49
$74.49
$349.49
conditions and qualifications qualify.
Make all the difference in your
trip to Orlando by filling your
down time with magical moments. Whether it’s spending
a truly unforgettable evening
with an old friend or sharing a
dazzling nighttime sky with a
new contact, magical experiences
reign supreme in the Walt Disney
World® Theme Parks. To get additional information and order tickets go online at:
http://www.iitsec.org/attendees/planningyourstay/
Pages/OrlandoConnections.aspx
or call 407-566-5600.
corporate travel department, or your travel agent
and give the agent CV#04860007.
Rates are guaranteed from November 26 -
items such as refueling or additional driver fees,
are extra. Advance reservations are (strongly)
recommended. Minimum rental age is 20 (age
Client Events & Discounts to Dining,
Nightlife, Attractions, Golf, etc.!
Orlando Convention Aid has partnered with I/ITSEC to help you arrange for the perfect restaurant
for your client or staff dinner/event. We have a relationship with 60 local
venues to provide this service on a complimentary basis, and we will provide
you with availability, pricing, and options, normally at a discount, within 24
hours! Please also visit our website by clicking on the golden ticket graphic
at http://www.iitsec.org/attendees/planningyourstay/Pages/OrlandoConnections.aspx to make dinner reservations, buy discounted attraction tickets,
book tee times, and so much more! This web site will help you plan your time
in Orlando and SAVE MONEY! Be sure to check out the coupons available
and get your coupon book at registration — jam packed with thousands of
dollars of savings!
P U B L I C AT I O N S & M E D I A
Advertising Opportunities:
Official Publications of I/ITSEC
I/ITSEC Proceedings (Three ways to purchase)
ONLINE REPOSITORY (PAPERS FROM 1966 - 2014)
Now more than ever, with the increased
challenges facing the defense and
security marketplace, you need to
keep your organization’s message
in front of its target audience. Reach
the leading decision-makers at the
world’s largest simulation, training and modeling event of the year
by advertising your products and
services in the Official Publications
of I/ITSEC.
We are starting a new era and a new service with the I/ITSEC
Papers. All papers will soon be available, and all at no charge for
the first year. Watch for the details on the I/ITSEC website at
http://www.iitsec.org/Pages/default.aspx.
Advertising in these publications is an excellent way to stand out in the crowd and
invite the attendees to visit your exhibit,
product demonstration and/or website.
Then after the event has ended, these publications are used
by many as desk-references, so your advertisement will reach the
decision-makers long after the conference is over.
Great news for students, librarians, researchers! The full record
of papers published from1966 through 2000 is available for order
(or at I/ITSEC) as a two-CD set for $300.00. You may place an
order through the NTSA office (703) 247-9471, or on the I/ITSEC
registration form. Papers from the pre-electronic era have been
reviewed, scanned and provided with keywords, making ALL
papers searchable electronically. Orders can be placed by calling
(703) 247-9471. Limited supply.
The National Training and Simulation Association’s
Annual Simulation & Training Trends and Technology
Review – I/ITSEC Exhibitor Directory
This publication will be available to all the attendees, exhibitors,
and exhibit visitors at I/ITSEC. It will be placed in the attendees’
conference bags and available at registration, and other locations
at the convention center. As an added bonus, your ad will also
appear in the December Issue of National Defense Magazine —
exposure beyond the walls of the convention center. National
Defense is sent to over 83,500 BPA audited readers, including the
members of NTSA. (Directory section will not appear in National
Defense Magazine).
YEARLY PROCEEDINGS (PAPERS FROM 1995 - 2013)
Individual CDs for the years shown above are available for $30.00
each. Each CD includes all accepted papers from that year. Orders
can be placed by calling (703) 247-9471. (2002, 2004, 2005, 2007,
2009 sold out)
I/ITSEC COMPENDIUM (PAPERS FROM 1966 - 2000)
Stay in Touch
Free Wireless hot spots. E-mail/Internet Kiosks.
In various locations throughout the conference area, I/ITSEC
attendees will have complimentary internet and e-mail. Internet
Access is also available at no charge at the Connections Lounge
and Grill located inside South Exhibit Hall 100 Aisle. (Additionally,
all of OCCC is now Wi-Fi enabled for a modest user fee.)
I/ITSEC is the premier annual event of its kind, attendance by
the mainstream and specialist trade press is heavy, resulting in
The I/ITSEC Show Daily
coverage that reaches your key marketing targets. Our media
Advertise in this year’s Daily and be noticed by your customers and
potential partners who are attending I/ITSEC. The I/ITSEC Show
Daily informs the simulation & training community on breaking
events & happenings on-site at I/ITSEC. It is printed overnight
and distributed daily at the conference center, choice hotels, and
uploaded to the I/ITSEC website. The daily has evolved into a vital
part of I/ITSEC; a “must read” while attending the conference.
staff stands ready to assist you in achieving maximum exposure
Use both to give your company Unequalled Exposure
persons. We strongly recommend early bookings for this room,
during your time at I/ITSEC. Corporate representatives are invited to bring their marketing materials to the Media Room for
distribution as early as possible after the opening of registration.
The Media Room area will also include a separate facility for
briefings/ presentations with a capacity of approximately 30
Special packages have been created so your organization can take
advantage of both opportunities!
which will be in demand. Additional exhibitor presentations
Web Banners
Showcase, booth 2287.
will be made available inside the exhibit hall at the Innovation
A limited number of banner ad spaces are available on the
I/ITSEC website.
Prior to the conference, contact John Williams at (703) 362-7005
For more information on advertising in these publications, contact Dino
Pignotti at (703) 247-2541 or dpignotti@ndia.org.
News page of http://www.iitsec.org. The I/ITSEC Media Room is
80
or jwilliams@ndia.org; check out more details on the I/ITSEC
S210E, phone (407) 685-6107.
A S S O C I AT I O N S P O N S O R S
National Training and Simulation Association
The NTSA, an affiliate of NDIA, represents and promotes the
business interests of companies in the simulation, training, mission planning/rehearsal, and support services industry. NTSA’s
200 corporate and 500 individual members enjoy reduced fees on all NTSA events and
services, as well as a bi-monthly newsletter (Training Industry News) and National
Defense magazine. Sustaining and Regular Corporate members receive early space
selection and discounts on exhibit space at I/ITSEC. Individual memberships are also
available. For membership information, call (703) 247-9471 or visit the NTSA website: at
http://www.trainingsystems.org.
Exhibit Information
Debbie Langelier, CEM
Director of Exhibits, NTSA
Phone: (703) 247-9480
FAX: (703) 243-1659
E-mail: dlangelier@ndia.org
Visit http://exhibits.iitsec.org
National Defense Industrial Association
Based in Arlington, Virginia, the National Defense Industrial Association (NDIA) is a non-profit, educational association representing
industry, government, and all the military services. About 1,600
companies and 90,000 individuals rely on NDIA for networking, knowledge, and business
development opportunities. As the nation’s leading defense industry association promoting national security, NDIA advocates cutting-edge technology and superior weapons,
equipment, training, and support for the warfighter and first responder. Through events,
working divisions, local chapters, and four affiliate organizations, NDIA connects the
government and defense industry to create a vigorous and ethical forum of information
exchange leading to greater support for national security.
For NDIA membership information visit www.ndia.org or contact Mike Kibler at
mkibler@ndia.org.
EARNING THE CMSP
DESIGNATION WILL:
TO LEARN MORE ABOUT THE
REQUIREMENTS AND TO APPLY,
PLEASE VISIT
WWW.SIMPROFESSIONAL.ORG
OR CONTACT PATRICK ROWE AT
PROWE@NDIA.ORG.
• Demonstrate expertise in the field
of M&S to your employer and the
larger M&S community
• Provide opportunities for
professional advancement
Requirements include 3-8 years
of work experience (depending on
level of highest collegiate degree), 3
professional letters of reference, and
successful completion of an online
examination.
CMSP Applicants now have a choice between
CMSP-Technical and CMSP-Management exams.
T H E W O R L D ’ S L A R G E S T M O D E L I N G & S I M U L AT I O N E V E N T to view the current floor plan, exhibitor list, and
sponsorship opportunities.
SAVE THE DATE FOR
I/ITSEC 2015!
NOVEMBER 30 DECEMBER 4, 2015
Orange County Convention
Center • South Concourse
Orlando, FL
Women In Defense,
A NATIONAL SECURITY ORGANIZATION
Cultivating and supporting the advancement and
recognition of women in
all aspects of national security is the mission of
Women In Defense. An affiliate of NDIA,
this non-profit professional organization
provides women professional growth
through networking, education and career development at both the national
and chapter levels. Members, including
men and women, have careers related to
the defense of the United States and national security. Details and membership:
http://wid.ndia.org.
81
SAFETY AND SECURITY
SECURITY HOTLINE DURING I/ITSEC: (407) 685-6111
MEDICAL EMERGENCIES, DIAL 5-7119 FROM ANY CONVENTION CENTER PHONE
Security Training Before The Conference
Technology collection directives contain mandates requiring exhibitors and presenters to receive a Counterintelligence
(CI) briefing from their CI support staff prior to I/ITSEC. Contractors with classified contracts may contact their Defense
Security Service Special Agents. To avoid security breaches, I/ITSEC presenters and exhibitors should ensure that the required briefing has been received. A list of CI support agencies follows. Please contact your security officer/manager and
ensure that an appropriate briefing for yourself and your colleagues is arranged. Providers of the briefings are:
902 Military Intelligence
Army
Navy, USMC, Coast Guard
Naval Criminal Investigative Service
Air Force Office of Special Investigation
Air Force
Defense Security Service
Contractors
Personal Security
The most important thing to protect, of course, is yourself. Pay attention to your surroundings. Report suspicious behavior
or security breaches to a security person or NTSA staff. Familiarize yourself with emergency procedures and exits at your
hotel and the Convention Center. Conference Security Office will be located in the South Lobby Registration Area and
inside the Exhibit Hall.
Emergency Medical Services
EMT and/or paramedics will be on-site during I/ITSEC (including hall build-up and tear-down). During
I/ITSEC 2014, they will be located on the same level as Registration, near the escalators between S220 and
S230. See the layout on page 13 for the exact location. Within the Convention Center dial 5-9809 or contact
any security or I/ITSEC staff member with a radio. If outside the Center or on your cell, dial (407) 685-9809.
Bags and Briefcases
Bags and Briefcases may be carried in by those wearing Conference Attendee or Exhibitor badges. Exhibit Visitors
(those who are only visiting the exhibits) WILL NOT be allowed to carry in bags or briefcases. A check room
will be available in the main registration area. A small purse or fanny pack is allowed, but is subject to search.
Additional security restrictions may be posted on http://www.iitsec.org and on signage at the conference.
Conference Management reserves the right to adjust security levels as deemed necessary during the conference.
Presentations
Recording devices will not be permitted in the presentation rooms, unless authorized by the conference
management. Presenters and Exhibitors should review their company’s policy documents and those of the
government agencies with whom you contract regarding open distribution, limited distribution, restricted
distribution and sharing limitations.
Cameras
Exhibitors have the right to limit photographs and videos of their displays. Please respect this right by asking
before photographing or videotaping. Participants found taking photos or videos without the consent of the
subject presentors or exhibitors will be dealt with according to security procedures, to possibly include confiscation of materials and removal from the premises.
Inquiries (before the conference)
Registration (702) 798-8340 • Exhibit/Sponsorship (703) 247-9480 • All other inquiries (703) 247-2569
82
GOLF TOURNAMENT
Central Florida Chapter
Scholarship* I/ITSEC Golf Tournament
Sunday,
30 November
or
Monday,
1 December
(A DAVID HARTMAN DESIGN)
Lunch Provided
Deadlines
Two-Day Sponsorships†:
Entry Fee
Sponsorship
Tournament Times
DAY
Sunday Monday (EST) 20 November to receive 50% refund.
No refunds after 20 November. Substitu24 November Details available at www.iitsec.org
†
Tee
Box
$400
ea.
tions allowed on-site, no extra charge.
20 November
Putting Green or Driving Range
$400 ea.†
GPS
$600 ea.† Register and Pay On-Line!!
Beverage Cart, Par-take or
$2,500 ea.† Cut-off date: 24 November
SHOTGUN Hole-in-One
†
SIGN-IN
1100
0630
Register and pay for green fees and sponsorships while registering for I/ITSEC at
www.iitsec.org. Through the on-line form
(The Par-Take Snack Stand is open at 0630)
(www.iitsec.org), you may register one to
Sponsors
four players and select the desired sponsor
Point of Contact
Send your logos (hi-res jpeg) via e-mail to opportunity.
Debbie Berry
407-306-4487
debbie.berry@lmco.com (final will be color,
debbie.berry@lmco.com
on white background) by close of business To complete the registration for your
20 November. Do not bring your own sign. group or team, be ready to provide each
1230
0730
Need not be golfer to sponsor.
Payable in advance with on-line registration.
Cost
player’s handicap, phone & e-mail.
$90 per Player
(includes Green Fees, Cart, Range Balls, and Lunch)
Payable in advance with on-line registration.
Mulligans available onsite.
Golf Format
144 per Start
Pairings & Starting Holes
Assignments and pairings will be made by
tournament coordinator. Priority is based
upon receipt of payment.
Requested team pairings can only be guaranteed
if all players are registered at the same time.
Requests noted under comments when
registering will be considered but CANNOT BE
GUARANTEED.
West on Highway 528
(Beach Line) (toll road)
approx. 7 miles. Take exit
2, Universal Blvd. Right on
Universal to main entrance
on right.
From 1-4:
E a s t o n H i g h way 5 2 8
(Beach Line) (toll road)
past International Drive.
Take exit 2, Universal Blvd.
Right on Universal to main
entrance on right.
1/2 mile east of the Orange County Convention
Center (OCCC) North/South (New) Complex
From the Orange County Convention Center:
Prizes (For each start)
First Place Team
Second Place Team
Closest to Pin
Longest Drive
50/50 Cash Jack Pot
*For full list of scholarships funded by
Cancellations must be received via e-mail to the Central Florida Chapter, please visit
debbie.berry@lmco.com by close of business http://www.ndia-cfl.org.
From Orlando
International Airport:
(Limit 1 per player)
Captain’s Choice/Scramble
Max Number of Players
Cancellations
Low Gross
Low Gross
(M/W)
(M/W)
East (right) onto International Boulevard. North (left) onto Convention Way.
(Convention Way is the street that runs between the OCCC and the Rosen Centre.) East
(right) onto Universal Boulevard. Shingle Creek entrance will be on the left.
SHINGLE CREEK GOLF COURSE • 9939 UNIVERSAL BLVD • ORLANDO, FL 32819
407-996-9933 OR 866-996-9933 • WWW.SHINGLECREEKGOLF.COM
5K RUN
ANNUAL I/ITSEC 5K
RUN/WALK/ROLL 2014
When: Wednesday, December 3, 2014
Where: Orange County Convention Center
Time: 5 :30AM Packet Pickup
6:45AM Start Time
WEBSITE: http://www.iitsec.org/attendees/planningyourstay
www.facebook.com/iitsec5k
All registered runners will receive custom race tech shirt, custom race metal, swag bag,
race bib and official timing by Fleet Feet Orlando, pre- and post-race refreshments.
Tax deductible registration.
REGISTER EARLY!!
Registration
25
$
after November 1
registration will be
$30.
CHARITIES THE
5K WILL SUPPORT
TITLE SPONSOR
Operation Give Back
IITSEC STEM
Initiative
Email Sean Osmond for Race Information at iitsec5k@gmail.com
or Debbie Langelier for Sponsorship information at dlangelier@ndia.org
84
I/ITSEC 2015
Save the date!
RTEH GE IWSOTR RL DA’ TS I LOANR G IENS TF OM RO DME AL ITN IGO &N S I M U L A T I O N
I/ITSEC
EVENT
INTERSERVICE/INDUSTRY TRAINING,
SIMULATION & EDUCATION CONFERENCE
F O R G I N G T H E F U T U R E T H R O U G H I N N O VAT I O N
e!
Sa v e th e Da t
No v emb e r 30015
Dec emb e r 4, 2
w w w.ii ts ec.o r g
DECEMBER
1-4,
2014
u
WWW.IITSEC.ORG
u
ORLANDO, FLORIDA
THE WORLD’S LARGEST MODELING & SIMULATION EVENT
CALL FOR PAPERS
AND TUTORIALS
I/ITSEC 2015
ABSTRACT DEADLINE: 22 FEBRUARY 2015
ON-LINE ABSTRACT SUBMITTAL
SUBCOMMITTEES/CATEGORIES
• Education
• Emerging Concepts & Innovative Technologies
• Human Systems Engineering
• Policy, Standards, Management & Acquisition
• Simulation
• Training
TUTORIALS
Information on core M&S, training, and education
topics suitable for management and technical
personnel.
The submission process for the I/ITSEC Papers
and Tutorials coincide. Submittal details will vary
slightly, but the milestones will match.
Follow the
Papers/Tutorials Completion Process
for 2015 Abstract Submittal which
will be posted in December.
http://www.iitsec.org/authors
I/ITSEC 2015 Program Chair
David Hutchings
Raydon Corporation
Phone: (407) 346-6852
E-mail: dhutchings@raydon.com
I/ITSEC 2015 Tutorial Board Chair
Robert Richbourg, Ph.D.
Institute for Defense Analyses
Phone: (703) 845-2158
E-mail: rrichbou@ida.org
NOVEMBER 30 – DECEMBER 4, 2015
u
WWW.IITSEC.ORG
u
ORLANDO, FLORIDA
A NEW I/ITSEC EVENT
We all love good thrillers, but the best of these are based on reality, or at least potential reality: those intriguing catastrophic events, either manmade or natural, that are too daunting
for most of us to imagine. Black Swan (as in “as rare as a black swan”) is a term coined
to describe a high-impact, hard-to-predict event that has a major effect on our world, and
in hind-sight seems as though it should have been considered/predicted. Black Swans can
be good or bad and sometimes both.
At I/ITSEC 2015, NTSA and the I/ITSEC Committee invite participation in a showcase
of Black Swan candidates, demonstrating the use of modeling and simulation (M&S) to
predict, plan for, and potentially prevent such events or using M&S to respond, recover
or mitigate its effects. Due to their extreme nature, most Black Swan events can only be
studied in a modeling and simulation environment which is at the heart of the strength of
the NTSA membership.
Terrorists, pandemics, transportation, electric grids are among potential subjects. What
challenge can you imagine, and what M&S analysis or capability do you want to highlight at
I/ITSEC 2015? Participating exhibits will be acknowledged with “Black Swan Participant”
signage and highlighted during I/ITSEC week. More details to come, but let your M&S fed
imaginations start processing!
Questions, to volunteer or offer ideas contact:
RADM James Robb, USN (Ret)
Barbara McDaniel
President, NTSA
I/ITSEC Coordinator
703 247 2567 or jrobb@ndia.org
703 247 2539 or bmcdaniel@ndia.org
Abstracts
2014 I/ITSEC Abstracts
FELLOWS
2014 FELLOWS PAPER: METASIMULATION ..................................................................................... 8
BEST PAPERS
MEASURING VISUAL DISPLAYS' EFFECT ON NOVICE PERFORMANCE IN DOOR
GUNNERY .................................................................................................................................................... 9
PROFICIENCY EVALUATION AND COST-AVOIDANCE PROOF OF CONCEPT M1A1 STUDY
RESULTS ...................................................................................................................................................... 9
MISSING: A SERIOUS GAME FOR THE MITIGATION OF COGNITIVE BIASES......................10
INSTITUTIONALIZING BLENDED LEARNING INTO JOINT TRAINING: A CASE STUDY
AND TEN RECOMMENDATIONS .........................................................................................................11
FROM DESIGN TO CONCEPTION: AN ASSESSMENT DEVICE FOR ROBOTIC SURGEONS…
.......................................................................................................................................................................11
EXPLORATION OF SOLDIER MORALE USING MULTI-METHOD SIMULATION
APPROACH ................................................................................................................................................12
EDUCATION
TRANSMEDIAL AND PARAMEDIAL SERIOUS GAME DEPLOYMENT ......................................13
EFFECTIVENESS OF EMBEDDED GAME-BASED INSTRUCTION: A GUIDED
EXPERIENTIAL APPROACH TO TECHNOLOGY BASED TRAINING .........................................13
AN EXPERIMENT TO EVALUATE THE EFFECT OF NARRATIVE DELIVERY IN MILITARY
TRAINING...................................................................................................................................................14
MODELING AND SIMULATION CHALLENGE PROBLEMS IN HIGH SCHOOL
CLASSROOMS AND INTERNSHIPS: LESSONS LEARNED .............................................................15
USING UNITY TO IMPLEMENT A VIRTUAL CRASH SITE INVESTIGATION LABORATORY
IN SUPPORT OF DISTANCE LEARNING OBJECTIVES ..................................................................15
DEVELOPING THE SIMULATOR INSTRUCTOR’S PEDAGOGICAL COMPETENCE ..............16
AN INSTRUCTIONAL MEDIA SELECTION PROCESS FOR VIRTUAL WORLD
DELIVERY ……………………………………………………………………………………………..…17
DEVELOPING THE HUMAN DIMENSION: CURRENT PRACTICES AND FUTURE
METHODS...................................................................................................................................................17
HEY, YOUR E-LEARNING COURSES ARE GIVING ME A COGNITIVE OVERLOAD ..............18
REDISCOVERING THE EIGHTFOLD PATH: LESSONS LEARNED IN EMPLOYING
SIMULATION FOR TRAINING AND ASSESSMENT IN AFGHANISTAN ......................................19
MOBILE INSTRUCTIONAL STRATEGY TEMPLATES FOR GUIDED MOBILE CONTENT
DEVELOPMENT ........................................................................................................................................19
ARE THEY MISSION READY? USING THE MODIFIED ANGOFF METHOD TO SET CUT
SCORES .......................................................................................................................................................20
RETENTION AND RETRAINING OF INTEGRATED COGNITIVE AND PSYCHOMOTOR
SKILLS ........................................................................................................................................................21
AN INDIVIDUALIZED APPROACH TO REMEDIATING SKILL DECAY: FRAMEWORK AND
APPLICATIONS .........................................................................................................................................21
Papers are available on the 2014 I/ITSEC CD ROM included in the Conference Attendee meeting bag, or visit the
I/ITSEC Website (www.iitsec.org) for ordering information. (Limited numbers of CDs from 1998-2014 are also
available.) All papers from 1966 through 2000 are available in the I/ITSEC Compendium. Individual papers from
1966 through 2014 may also be ordered through the www.iitsec.org portal.
1
ASSERTIVENESS AND RESPONSIVENESS IN TEAMS; ESSENTIAL FOR MISSION
COMMAND .................................................................................................................................................22
TAILORING MULTIMEDIA INSTRUCTION TO SOLDIER NEEDS ...............................................23
AUTOMATED CONTENT ALIGNMENT FOR ADAPTIVE PERSONALIZED LEARNING ........23
AN ARMY LEARNING MODEL IMPLEMENTATION: CHALLENGES, SUCCESSES,
FUTURE DIRECTIONS ............................................................................................................................24
EMERGING CONCEPTS & INNOVATIVE TECHNOLOGIES
CONSIDERATIONS ON UTILIZING A GAME ENGINE AS AN IMAGE GENERATOR .............25
TOWARDS MINIMALIST SERIOUS GAME DESIGN ........................................................................25
HUMAN MOTION CAPTURE IN NATURAL ENVIRONMENTS .....................................................26
THE LARGEST FIELD OF VIEW COLLIMATED DISPLAY EVER BUILT...................................27
HIGH QUALITY VISUAL DISPLAY SYSTEMS IN PHYSICALLY CONSTRAINED
ENVIRONMENTS ......................................................................................................................................27
DYNAMIC FLIGHT SIMULATION: 45 YEARS OF RESEARCH & DEVELOPMENT .................28
DESIGN AND EVALUATION OF SURPRISE EFFECTS IN SIMULATION - A FRAMEWORK
.......................................................................................................................................................................29
VALIDATED DEVELOPMENT OF STRESS INOCULATION THROUGH COGNITIVE AND
BIOFEEDBACK TRAINING ....................................................................................................................29
USING VIRTUAL REALITY AS PART OF AN INTENSIVE TREATMENT PROGRAM FOR
PTSD .............................................................................................................................................................30
MICROGAMES FOR TRAINING PERCEPTUAL SKILLS ................................................................31
BEYOND SOCIO-CULTURAL SENSEMAKING: OBSERVING AND INTERPRETING
PATTERNS OF LIFE .................................................................................................................................31
DEVELOPING EFFECTIVE ADAPTIVE TRAINING SYSTEMS TO ENHANCE MILITARY
INSTRUCTION ...........................................................................................................................................32
FUTURE OF LVC SIMULATION: EVOLVING TOWARDS THE MSAAS CONCEPT..................33
RAPID DATA GENERATION: A FLEXIBLE DATA DISCOVERY AND ACCESS
ARCHITECTURE ......................................................................................................................................33
OPTIMIZING SUPERVISED LEARNING FOR PIXEL LABELING AND MATERIAL
CLASSIFICATION .....................................................................................................................................34
EARLY SYNTHETIC PROTOTYPING: EXPLORING NEW DESIGNS AND CONCEPTS
WITHIN GAMES........................................................................................................................................35
WHEN TRADESPACE ANALYSIS MET COMBAT MODELING AND SIMULATION ................35
SIMULATION IN SUPPORT OF COURSE OF ACTION DEVELOPMENT IN OPERATIONS ....36
FUNDAMENTAL COMPETENCY SETS (FCS) DEFINITION TO SUPPORT TECHNOLOGY
DEVELOPMENT FOR PILOT TRAINING ............................................................................................37
JOINT TERMINAL ATTACK CONTROLLER-TRAINING REHEARSAL SYSTEM:
COMPETENCY-BASED RESEARCH.....................................................................................................37
OPERATOR QUALIFICATION DIFFERENCES BETWEEN MANNED AND UNMANNED
AERIAL SYSTEM (UAS) ..........................................................................................................................38
2
Papers are available on the 2014 I/ITSEC CD ROM included in the Conference Attendee meeting bag, or visit
the I/ITSEC Website (www.iitsec.org) for ordering information. (Limited numbers of CDs from 1998-2014 are
also available.) All papers from 1966 through 2000 are available in the I/ITSEC Compendium. Individual papers
from 1966 through 2014 may also be ordered through the www.iitsec.org portal.
VIDEOGAME DESIGN FOR COGNITIVE ENHANCEMENT THROUGH MICRO-PUZZLE
COGNITIVE PROFILING ........................................................................................................................39
GAME-BASED TRAINING TO MITIGATE THREE FORMS OF COGNITIVE BIAS ...................39
TOWARDS ENHANCING INTUITIVE DECISION MAKING THROUGH IMPLICIT
TRAINING...................................................................................................................................................40
CHALLENGES TO UPGRADING A MOBILE WEB APPLICATION ...............................................41
DEVELOPMENT AND EVALUATION OF A HUMERAL HEAD INTRAOSSEOUS TRAINING
SYSTEM ………………………………………………………………………………………………..…41
COMPARATIVE ANALYSIS OF HOLOGRAPHIC DISPLAY AND THREE-DIMENSIONAL
TELEVISION ..............................................................................................................................................42
LIVE AUGMENTED REALITY BASED WEAPON TRAINING FOR DISMOUNTS ......................43
AUGMENTED REALITY VIRTUAL PERSONAL ASSISTANT FOR TRAINING,
MAINTENANCE, AND REPAIR..............................................................................................................43
PERLS: AN APPROACH TO PERVASIVE PERSONAL ASSISTANCE IN ADULT
LEARNING….. ...........................................................................................................................................44
IMPLEMENTING STATELESS CROSS DOMAIN SOLUTIONS TO CONTINUOUSLY
MAINTAIN ..................................................................................................................................................45
CYBERSECURITY IMPACTS OF A CLOUD COMPUTING ARCHITECTURE IN LIVE
TRAINING...................................................................................................................................................45
USING SOCIAL NETWORK ANALYSIS TO MODEL THE SPREAD OF MISINFORMATION IN
SIMULATED ENVIRONMENTS .............................................................................................................46
HUMAN SYSTEMS ENGINEERING
CLASSIFYING STRESS IN A MOBILE ENVIRONMENT ..................................................................47
INVESTIGATION OF THE SENSITIVITY OF PHYSIOLOGICAL, PERFORMANCE, AND
SUBJECTIVE MEASURES FOR IDENTIFYING CHANGES IN NOVICE INTELLIGENCE
ANALYST WORKLOAD ..........................................................................................................................47
TRAINING WITH ADAPTIVE SYSTEMS: UTILITY OF BAROREFLEX SENSITIVITY ............48
A NOVEL APPROACH TO DETERMINE INTEGRATED TRAINING ENVIRONMENT
EFFECTIVENESS ......................................................................................................................................49
PISTOL SKILL ACQUISITION AND RETENTION: A 3-YEAR LONGITUDINAL STUDY .........49
EFFECTIVE LVC TRAINING: PERSONNEL RECOVERY’S EXPERIENCE WITH
INTEGRATED SENSOR TECHNOLOGY .............................................................................................50
IN SEARCH OF INTEROPERABILITY STANDARDS FOR HUMAN BEHAVIOUR
REPRESENTATIONS ................................................................................................................................51
FACTORS IMPACTING PERFORMANCE IN COMPETITIVE CYBER EXERCISES ..................51
COGNITIVE PROCESSING CONSIDERATIONS OF THE SMALL UNMANNED GROUND
VEHICLE.....................................................................................................................................................52
LEVERAGING SIMULATION TO AUGMENT RISKY DRIVING ATTITUDES AND
BEHAVIORS ...............................................................................................................................................53
A VALIDATED AND INTEGRATED SIMULATION FRAMEWORK FOR HUMAN FACTORS
ANALYSES ..................................................................................................................................................53
3
THE VIRTUOUS CIRCLE AND CONTEXTUALIZED KNOWLEDGE ELICITATION:
APPLICATION OF A NEW PARADIGM FOR JOB ANALYSIS ........................................................54
SERIOUS GAME USER DATA ANALYSIS AND VISUALIZATION: SAVORING THE
BREADCRUMBS ........................................................................................................................................55
SONIFICATION: THE SOUND OF BIG TRAINING DATA ...............................................................55
CREATING A LEARNING INFRASTRUCTURE WHERE EVERY SOLDIER CAN BE AN
INSTRUCTOR ............................................................................................................................................56
QUANTITATIVE ASSESSMENT OF COMBAT CASUALTY SKILLS .............................................57
ASSESSMENT INSTRUMENT VALIDATION FOR CRITICAL CLINICAL COMPETENCIES:
PEDIATRICNEONATAL INTUBATION AND CHOLINERGIC CRISIS MANAGEMENT...........57
A DECISION SUPPORT SYSTEM PREDICTING IMMINENT CARDIOVASCULAR
SHOCK….....................................................................................................................................................58
POLICY, STANDARDS, MANAGEMENT & ACQUISITION
RESOURCE IMPLICATIONS OF THE DIFFERENCE BETWEEN MODELS AND
SIMULATIONS...........................................................................................................................................59
COTS TO CAPABILITY: LESSONS LEARNT FROM UK MOD RESEARCH
PROGRAMME... ........................................................................................................................................59
PERSPECTIVES ON EXPORTABILITY AND PROGRAM PROTECTION IN VIRTUAL
TRAINING SYSTEMS ...............................................................................................................................60
LVC, TRANSLATING DOD POLICY INTO ACTION .........................................................................61
DATA SHARING: THE STANDARD SPECIFICATION IS JUST THE START ...............................61
ESTABLISHING SHARING FOR GEOSPATIAL ENVIRONMENT DATA .....................................62
AGILE PROGRAM MANAGEMENT ON SOFTWARE INTENSIVE TRAINING SYSTEMS .......63
SIMULATIONS IN THE CLOUD – A MANAGER’S CHALLENGE ..................................................63
CONTINUOUS MONITORING OF CYBERSECURITY IN A TRAINING SYSTEM
ENVIRONMENT ........................................................................................................................................64
SIMULATION
EMBRACING THE CLOUD – PROVIDING SIMULATION AS A SERVICE ..................................65
CLOUD TERRAIN GENERATION AND VISUALIZATION USING OPEN GEOSPATIAL
STANDARDS...............................................................................................................................................65
ENABLING EXTERNAL PLAYER CONNECTIONS TO KERBEROS-SECURED
SYSTEMS ....................................................................................................................................................66
SENSOR PLACEMENT OPTIMIZATION IN LVC ENVIRONMENTS FOR TRAINING,
ANALYSIS, AND OPERATIONAL APPLICATIONS ...........................................................................67
TOWARD INTEROPERABILITY OF SIMULATION SYSTEMS FOR GROUND FORCES:
PROGRESS AND CHALLENGES ...........................................................................................................67
INTEGRATING DISTRIBUTED VIRTUAL COMMAND AND CONTROL PLATFORMS INTO
LIVE TRAINING ........................................................................................................................................68
MISSION INTEGRATED SIMULATION – A CASE STUDY ..............................................................69
AN INSTRUCTOR OPERATING SYSTEM FRAMEWORK FOR INTERACTIVE INSTRUCTORSTATION .....................................................................................................................................................69
4
DEVELOPMENT OF A MICROSCOPIC ARTIFICIALLY INTELLIGENT TRAFFIC MODEL
FOR SIMULATION ...................................................................................................................................70
TEACHING CROSS CULTURAL SOCIAL COMPETENCE IN A DYNAMIC, SYNTHETIC
ENVIRONMENT ........................................................................................................................................71
ADVANCED ANIMATION TECHNIQUES IN A DISMOUNTED SOLDIER SYSTEM ..................71
GAME-BASED SIMULATION FOR PHILIPPINE POST-TYPHOON STABILITY OPERATIONS
TRAINING...................................................................................................................................................72
A DECISION AID FOR OPTIMIZING EXPERIMENTAL DESIGN INVOLVING LVC
ENVIRONMENTS ......................................................................................................................................73
A PRACTITIONER’S APPROACH USING MBSE IN SYSTEMS OF SYSTEMS ............................73
ROBOTIC SIMULATORS: A CASE FOR RETURN ON INVESTMENT ..........................................74
CREATING A RE-USEABLE KNOWLEDGE REPOSITORY FOR UK MOD CGF
BEHAVIOURS ............................................................................................................................................75
IMPROVING AIR-TO-AIR COMBAT BEHAVIOR THROUGH TRANSPARENT MACHINE
LEARNING .................................................................................................................................................75
IMPROVING MATERIAL CLASSIFICATION QUALITY WITH ELEVATION-DERIVED
METRICS ....................................................................................................................................................76
IMPROVING AND PROVING HEALTHCARE QUALITY AND VALUE THROUGH PHYSICAL
SIMULATION .............................................................................................................................................77
EMPLOYING MODELING AND SIMULATION TO IMPROVE PATIENT CARE ........................77
THE EFFECT OF DIFFICULTY LEVELS WITHIN A VIRTUAL MEDICAL SIMULATION ......78
A PARADIGM SHIFT IN THE TEST AND EVALUATION OF TERRAIN DATABASES ..............79
MEASURING THE IMPACT OF NATURAL ENVIRONMENT REPRESENTATION ON
COMBAT SIMULATION OUTCOMES ..................................................................................................79
IMPLEMENTATION OF REAL-TIME SNOW LAYERS IN GAME-BASED SIMULATION ........80
INTEGRATION OF LOW-COST HMD DEVICES IN EXISTING SIMULATION
INFRASTRUCTURE ..................................................................................................................................81
SIMULATING REALISTIC LIGHT LEVELS IN NEXT GENERATION IMAGE
GENERATORS ...........................................................................................................................................81
VERGENCE AND ACCOMMODATION IN SIMULATION AND TRAINING WITH 3D
DISPLAYS ...................................................................................................................................................82
A DISTRIBUTED SCENE GRAPH APPROACH TO SCALED SIMULATION-BASED TRAINING
APPLICATIONS .........................................................................................................................................83
PSUEDO-SPECIFIC HIGH-RESOLUTION DATA BOUNDARY TECHNIQUES ............................83
SOLVING THE INNOVATOR’S DILEMMA FOR SIMULATION AND TRAINING IMAGE
GENERATOR ARCHITECTURES ..........................................................................................................84
A FRAMEWORK FOR ENABLING VIRTUAL OBSERVER CONTROLLERS IN SYNTHETIC
TRAINING...................................................................................................................................................85
UAV FLIGHT CONTROL SOFTWARE DEVELOPMENT BASED ON COTS PRODUCT ...........85
LESSONS LEARNED IN CREATING AN AUTONOMOUS DRIVER FOR ONESAF .....................86
HOW THE U.S NAVY IS MIGRATING FROM LEGACY/LARGE FOOTPRINT TO LOW
COST/SMALL FOOTPRINT SONAR SIMULATION SYSTEMS .......................................................87
5
USE OF AUTOMATED INTELLIGENT ENTITIES IN ASW SIMULATION ..................................87
A PHYSICS-BASED APPROACH TO SIMULATE JET ENGINES ....................................................88
TRAINING
DECLARATIVE KNOWLEDGE ACQUISITION IN IMMERSIVE VIRTUAL LEARNING
ENVIRONMENTS ......................................................................................................................................89
VIRTUAL WORLD ROOM CLEARING: A STUDY IN TRAINING EFFECTIVENESS ................89
SIMULATING PARTICIPANT TRAINING DATA TO TEST MIXED-REALITY TRAINING
SYSTEMS ....................................................................................................................................................90
ENHANCING THE UTILITY AND EFFECTIVENESS OF COMBAT MEDIC
SIMULATION .............................................................................................................................................91
TRAINING EFFECTS FOR FIRST-RESPONDER COMPETENCY IN CHOLINERGIC CRISIS
MANAGEMENT .........................................................................................................................................91
SIMULATION ENVIRONMENTS FOR OFFSHORE OIL AND GAS EMERGENCY
TRAINING...................................................................................................................................................92
COMPARISON OF THE USABILITY OF ROBOTIC SURGERY SIMULATORS ..........................93
FORCES APPLIED ON LARYNGOSCOPE DURING INTUBATION: A STUDY ON AIRWAY
SIMULATORS ............................................................................................................................................93
OUTCOMES FROM TWO FORMS OF PEDIATRIC AND NEONATAL INTUBATION
TRAINING...................................................................................................................................................94
DATA & ANALYTICS TOOLS FOR AGILE TRAINING & READINESS ASSESSMENT.............95
LESSONS LEARNED INTEGRATING MOBILE TECHNOLOGY INTO TWO ARMY
COURSES ....................................................................................................................................................95
EXPERIENCE API AND TEAM EVALUATION: EVOLVING INTEROPERABLE
PERFORMANCE ASSESSMENT ............................................................................................................96
DEVELOPING AND EVALUATING PERFORMANCE MEASURES FOR MANNEDUNMANNED TEAMING ...........................................................................................................................97
USING TEMPORAL OCCLUSION TO ASSESS CARRIER LANDING SKILLS .............................97
DISTRIBUTED LIVE/VIRTUAL ENVIRONMENTS TO IMPROVE JOINT FIRES
PERFORMANCE........................................................................................................................................98
ACCELERATING UNIT ADAPTABILITY: A PRINCIPLE-BASED APPROACH TO UNIT
COMMUNICATION ..................................................................................................................................99
UTILIZING SIMULATION AND GAME-BASED LEARNING TO ENHANCE INCIDENT
COMMANDER TRAINING ......................................................................................................................99
INDUCING STRESS IN WARFIGHTERS DURING SIMULATION-BASED TRAINING ............100
DEVELOPING AND EVALUATING AN INTELLIGENT TUTORING SYSTEM FOR
ADVANCED SHIPHANDLING ..............................................................................................................101
DEVELOPING MODELS OF EXPERT PERFORMANCE FOR SUPPORT IN AN ADAPTIVE
MARKSMANSHIP TRAINER ................................................................................................................101
A DIGITAL TUTOR FOR ACCELERATING TECHNICAL EXPERTISE .....................................102
EVOLVING AVIATION LIVE TRAINING IN THE FUTURE ..........................................................103
TRAINING FIDELITY OF A UNMANNED AERIAL SYSTEMS COMPLEMENTARY FAMILY
OF TRAINERS ..........................................................................................................................................103
6
ASOC TRAINING RESEARCH: JOINT THEATER AIR GROUND SIMULATION
SYSTEM ....................................................................................................................................................104
FUSING SELF-REPORTED AND SENSOR DATA FROM MIXED-REALITY TRAINING ........105
A COMPETENCY BASED APPROACH TO MARINE AND WEAPONS ENGINEERING
TRAINING.................................................................................................................................................105
SCENARIO-BASED TRAINING FOR DEVELOPMENT OF LEADER-SUBORDINATE MENTAL
MODELS AND COHESION ....................................................................................................................106
POST-FIELDING TRAINING ASSESSMENT OF DISMOUNTED INFANTRY
SIMULATION ...........................................................................................................................................107
EVALUATING THE IMPACT OF INDIVIDUAL TRAINING ON UNIT'S OPERATIONAL
PERFORMANCE......................................................................................................................................107
USING LMS TECHNOLOGY FOR KIRKPATRICK LEVEL 3 EVALUATION OF HUMAN
TRAFFICKING TRAINING ...................................................................................................................108
TABLE OF AUTHORS ............................................................................................................................109
PLEASE NOTE: TUTORIALS ARE INCLUDED ON THE CD
7
2014 FELLOWS PAPER: METASIMULATION
2014 IITSEC Paper No. IF1401
Andy Ceranowicz
Harvard, MA
Simulation is used in an increasingly large segment of our scientific, economic, entertainment, and
government activities. Its expanding influence makes it important to understand its strengths and limitations.
However, each field specializes simulation for its problem domain making it difficult to agree on a common
definition. In this paper, I survey the fundamental mechanisms underlying simulation and attempt to come
closer to such a definition. In this quest, I have borrowed heavily from metaphysics, especially the concept
of possible worlds from modal logic. I define a simulator as a device that uses deduction and sampling to
incrementally create possible worlds. I define simulation analysis as the use of analogical reasoning to map
actual or hypothetical target worlds to a simulator and the use of induction on the possible worlds produced
by the simulator to make claims about the target world. The analogs that simulators are built on can be
physical, human, symbolic, analog (computer), digital or hybrid. The construction of simulation analogs
requires approximating the target world with a finite model bounded by means of inputs, state, and objects.
Integration of small changes and sampling are the magic bullets that allow simulation to tackle problems that
are impossible to solve by analytic means. Small changes decouple complex systems and sampling replaces
the complexity of the general with the simplicity of the concrete. Parallel simulator design and its realization
as a posteriori composition of simulators are reviewed. I find that advancing simulation time is not a
necessary or sufficient criterion for identifying a simulator and that simulation is used widely by the human
mind and probably by animals. It may even be the foundation of consciousness. Finally I examine a
posteriori simulator composition and interoperability concluding that once it becomes technically viable, a
priori composition will be a better approach.
WEDNESDAY, 3 DECEMBER, 2014 ROOM S320A
BP-1 - HSE, PSMA & Training
1030
Measuring Visual Displays’ Effect
on Novice Performance in
Door Gunnery (14012) (HSE)
1100
Proficiency Evaluation and Costavoidance Proof of Concept
M1A1 Study (14055) (PSMA)
1130
Missing: A Serious Game for the
Mitigation of Cognitive Biases
(14295) (Training)
Notes
8
MEASURING VISUAL DISPLAYS' EFFECT ON NOVICE PERFORMANCE IN DOOR
GUNNERY
2014 IITSEC Paper No. 14012
Dr. Jonathan Stevens
Army Research Laboratory (ARL)
Orlando, FL
Dr. Peter Kincaid
University of Central Florida (UCF)
Orlando, FL
The purpose of this paper is to present the results of our recent experimentation involving a novice
population performing aerial door gunnery training in a mixed reality simulation. Specifically, we examined
the effect that different visual displays had on novice soldier performance; qualified infantrymen with
machine gun experience. The results of this study differed from the findings of our first study, which
utilized an expert population of qualified helicopter crew members.
The U.S. Army continues to develop new and effective ways to use simulation for training. One example is
the Non- Rated Crew Member Manned Module (NCM3), a simulator designed to train helicopter
crewmembers in critical, high risk tasks. Novice participants were randomly assigned to one of two visual
display treatments (flat screen or Head-Mounted Display) and executed three aerial door gunnery training
scenarios in the NCM3. Independent variables were visual display, trial, immersive tendency and simulator
sickness questionnaire scores. Dependent variables included performance, presence and simulator sickness
change scores. The performance results of this study differed from our first study and indicated there was a
main effect of visual display on performance. However, both visual treatment groups experienced the same
degree of presence and simulator sickness. Results of this study indicate that higher immersive simulation
may lead to better performance for a novice population.
PROFICIENCY EVALUATION AND COST-AVOIDANCE PROOF OF CONCEPT M1A1
STUDY RESULTS
Robb Dunne
Innovative Reasoning LLC
Orlando, FL
Dr. Tim Cooley
DynamX Consulting
Larkspur, CO
Dr. Steven Gordon
Georgia Tech Research Institute
Orlando, FL
Evaluation of simulation-based training systems to determine their contributions to trainee proficiency and
to determine the level of cost avoidance vice live training is essential to plan the future live-virtualconstructive training environment for the United States Marine Corps (USMC). This need is reinforced in a
recent Government Accountability Office report (GAO 13-698, August 2013) on Army and Marine Corps
Training titled Better Performance and Cost Data Needed to More Fully Assess Simulation-Based Efforts,
which states that the Services “lack key performance and cost information that would enhance their ability
to determine the optimal mix of training and prioritize related investments.” USMC Program Manager
Training Systems (PM TRASYS) has conducted cost avoidance studies on USMC simulation-based training
systems for the past 2 years, and these studies are being refined to capture improved cost information. A
related study, begun in June 2013, evaluates the effects of USMC simulation-based training programs on
proficiency.This paper presents the process, results, and recommendations of the recent PM TRASYS Proof
of Concept (POC) study of measuring proficiency changes and cost avoidance due to use of the M1A1
Advanced Gunnery Training System (AGTS) simulator. For the POC, a group of consistent crews in initial
AGTS training are monitored (without interference) through a sequence of 10 gunnery table tasks, with a
total of 500+ task instances in the AGTS simulator, to the culminating live-fire tasks. Early session scores
are compared to “Gate-To-Live-Fire” scores in the simulator, and these results are compared to the live-fire
M1A1 qualification scores for these crews. Results of the POC are promising. The study finds that with
performance-oriented metrics and measures, tied to doctrine and captured automatically, it is possible to
determine both proficiency trending and cost avoidance. This paper also discusses lessons learned and
provides recommendations and implications of findings for training system design.
9
MISSING: A SERIOUS GAME FOR THE MITIGATION OF COGNITIVE BIASES
Carl Symborski, Meg Barton & Mary
Quinn
Leidos, Inc.
Arlington, VA
Carey K. Morewedge & Karim S.
Kassam
Carnegie Mellon University
Pittsburg, PA
James H. Korris
Creative Technologies Inc.
Hollywood, CA
The current study was designed to address the following research question: Can a computer game provide an
effective mechanism for training adults to identify and mitigate their cognitive biases? Human decision making
relies on a variety of simple heuristic decision rules that can be quick and effective mental shortcuts when
making judgments. However, these heuristics can also lead to irrational thinking and problem-solving in ways
that produce errors or illogicality, known as cognitive biases. Though knowledge of cognitive biases and bias
mitigation strategies can help to reduce the potential impact of cognitive biases on human reasoning, such
deeply ingrained cognitive strategies are difficult to alter. The current study was designed to leverage the
virtual learning environment of a serious game to take on this training challenge. To that end, a training game
– Missing: The Pursuit of Terry Hughes (Missing) – was developed. Missing was created for an audience of
educated adults, and the described instructional design is based on current research on effective andragogical
learning theory. The Missing game design immerses the user into bias-invoking situations which provide direct
experience with cognitive bias identification and mitigation strategies. In this paper, details of the game
instructional design are presented, including a cognitive framework based on dual-process systems of
reasoning which relates multiple biases, their causes, and mitigation techniques. An external test campaign was
conducted to determine whether the game had a positive transfer of in-game experiential learning about biases
to real world skills and behavior change. Results are presented that suggest this novel serious game both
engages and trains players, resulting in measurable reductions in cognitive biases.
BP-2 Education, ECIT & Simulation
1400
Institutionalizing Blended Learning
into Joint Training: A Case
Study and 10 Recommendations
(14208) (Education)
1430
From Design to Conception: An
Assessment Device for Robotic
Surgeons (14170) (ECIT)
1500
Exploration of Soldier Morale Using
Multi-method Simulation
Approach (14215) (Simulation)
Notes
10
INSTITUTIONALIZING BLENDED LEARNING INTO JOINT TRAINING: A CASE STUDY
AND TEN RECOMMENDATIONS
David Fautua, Ph.D.
Joint Staff J7, Joint Training
Suffolk, VA
Sae Schatz, Ph.D. & Patricia Bockelman
MESH Solutions, LLC™ (a DSCI® Company)
Orlando, FL
Emilie Reitz
Suffolk, VA
In 2011, the Joint Staff J7 (Joint Training) directorate initiated the Continuum of eLearning project in order to
integrate blended learning into joint exercises. This three-year research and development effort included
construction of both the blended learning instructional materials (e.g., best practices for online instructional
delivery [andragogy] within Joint Knowledge Online) and the processes required to implement blended learning
within the existing joint training enterprise. Although the capacity for blended learning has existed for decades,
such large-scale institutionalization of it presented unique challenges, which have previously limited its use
within the joint training community.
Joint Training personnel built the blended learning system iteratively, concurrently, and incrementally over the
three-year project. We also systematically measured the effectiveness of implemented components. This paper
presents an overview of this process as a case study for others, and it summarizes the results of the empirical
testing. The paper builds upon two previous I/ITSEC presentations, each of which detailed separate portions of
the ongoing project (i.e., effectiveness of blended online courses in 2012 and integration of a part-task teamtraining simulation in 2013). This paper adds to those earlier articles by presenting holistic project outcomes,
along with previously unpublished data from the empirical trials. For instance, some notable results included
21% higher learning outcomes (knowledge) when complementary e-learning courses preceded an exercise, and
when an additional team-training simulation was added to the pre-exercise preparation, 62.9% of participants
indicated that they felt more confident performing their assigned tasks as a result. Finally, this paper includes ten
recommendations for other organizations seeking to formally implement a blended learning system, including
don’t assume that trainers know how to “blend” from an instructional perspective and to truly implement a
robust blended learning system the organizational culture and its shared narrative about training must evolve.
FROM DESIGN TO CONCEPTION: AN ASSESSMENT DEVICE FOR ROBOTIC SURGEONS
Alyssa Tanaka, M.S.
Florida Hospital Nicholson
Center
Celebration, FL
Manuela Perez, M.D.
University Hospital of
Nancy
Nancy , FR
Mireille Truong M.D. &
Khara Simpson M.D.
Columbia University
Medical School
New York, NY
Gareth Hearn & Roger Smith,
Ph.D.
Florida Hospital ISA, Florida
Hospital NC
Orlando, FL, Celebration, FL
The daVinci Surgical System offers surgeons improved capabilities for performing complex minimally invasive
procedures; however, there is no standardized assessment of robotic surgeons and a need exists to ensure that a
minimal standard of care is provided to all patients. The Department of Defense and governing surgical societies
convened consensus conferences to develop a national initiative, resulting in a curriculum called the
Fundamentals of Robotic Surgery (FRS). FRS is comprised of an online curriculum and a psychomotor skills
dome.
This paper describes the production process used to create a psychomotor skills assessment device - the FRS
Dome. The device was designed to measure the essential skills that are required of any robotic surgeon and to
provide a basis upon which to grant or deny privileging with the robot. It was constructed to test seven tasks of
manual dexterity: Docking, Ring Tower Transfer, Knot Tying, Suturing, 4th Arm Cutting, Puzzle Piece
Dissection, and Energy Dissection.
The initial design of the device was created by a committee of experienced minimally invasive surgeons, with a
background in testing protocols and materials. The design was rendered in computer animation, which
kickstarted a prototyping effort with physical materials. These included platinum cure silicone approximating
human tissue and a 3D polyjet printer for the structural framework. Usability testing was conducted and iterative
modifications were made to improve ergonomics, standardization, and cost requirements. Final CAD diagrams
and specifications were created and distributed to medical and simulation companies for both physical and
digital manufacturing. This development process demonstrates the evolution of a simulation and a physical
testing device based on international expert consensus. The specifications are open source, allowing competitive
production and future iterations. The goal of this paper is to discuss how this device evolved from an idea to a
manufactured product and a digital simulation.
11
EXPLORATION OF SOLDIER MORALE USING MULTI-METHOD SIMULATION
APPROACH
Mariusz Balaban, Dr. Thomas Mastaglio
MYMIC LLC
Portsmouth, VA
Dr. John Sokolowski, Dr. Barry Ezell
Old Dominion University
Norfolk, VA
Soldier morale is a complex social construct influenced by factors at multiple levels and integrates many
research domains. Performance of soldiers can be influenced by morale, but the degree of this effect is
difficult to assess and map back to strategic, operational, and/or tactical decisions based on current
research approaches. Physiological and psychological factors related to low morale can lead to undesirable
behaviors like suicides, substance abuse, and accidents. The current approach to measuring soldier morale
is often based on a single point estimation limiting dynamic perspectives on this phenomenon. This may
also impact similar exploratory studies - due to insufficient data for validation of model causalities. The
simulation-based work found in the literature focuses on a single level of analysis and uses the System
Dynamics method, which overly reduces scope and detail necessary for capturing related dependencies
even if more data were available. This research seeks to explore, model, and simulate soldier morale at
multiple levels of analysis. The developed sample case scenario pertains to US security efforts against
insurgency in Afghanistan in 2007. The scenario includes representation of the patrol base operations
serving as a platform for calibration and exploration of factors affecting soldier morale and its relationship
with sample performance measures. The developed proof-of-concept simulation model was used for
testing the effect of both size of patrol base and violence level on soldier morale. The model can serve as a
platform for evaluation of decisions during surges and deployments that pertain to system structure,
scheduling, and policies.
TUESDAY, 2 DECEMBER, 2014 ROOM S320E
ED-1 Designing Educational Games
1400
Transmedial and Paramedial Serious
Game Deployment
(14375)
1430
Effectiveness of Embedded GameBased Instruction: A Guided
Experiential Approach to
Technology Based Training (14198)
1500
An Experiment to Evaluate the
Effect of Narrative Delivery
in Military Training (14081)
NOTES
12
TRANSMEDIAL AND PARAMEDIAL SERIOUS GAME DEPLOYMENT
Brandt Dargue
Boeing Research & Technology
St. Louis, MO
Dov Jacobson
GamesThatWork
Atlanta, GA
John Sanders
Louisville, KY
Despite extensive research, training to overcome cognitive biases has proven largely ineffective. Critical
decision-making in the face of uncertainty is difficult because participants employ heuristics that are
unconscious and subtle, but which can produce very serious impacts. It has often been demonstrated that
even cognitive bias experts make judgment errors by falling prey to the very biases they study (Heuer,
1999). Our team designed and developed a video game to teach cognitive bias recognition and mitigation
as an alternative to current classroom methods. Then we performed an empirical study of that game’s
learning and training efficacy. The results indicated that the game was effective for learning, but impacts
on biased behavior were inconsistent for the different cognitive biases addressed by the game. Additional
training aids to reinforce the game learning may be critical in order for it to reliably supplant higher
education courses in cognitive thinking. This paper provides a high-level overview of the project and the
particular cognitive biases taught in the game. We discuss the blend of instructional theories, techniques,
and media used in the game and the results of our effectiveness study. The paper describes the transmedia
training package we created to provide a variety of out-of-game experiences which show promise for
increasing the learner’s ability to mitigate these cognitive biases. This training package provides: a) an ongoing reminder to apply their new knowledge and skills; b) additional practice honing their skills; and c)
refresher training to reduce knowledge and skill decay. The use and ratios of the in-game transmedia types
and the use of out-of-game transmedia has not been studied yet. The paper concludes with suggested
research of the effectiveness of post-learning transmedia.
EFFECTIVENESS OF EMBEDDED GAME-BASED INSTRUCTION: A GUIDED
EXPERIENTIAL APPROACH TO TECHNOLOGY BASED TRAINING
Heather A. Priest Walker
Naval Air Warfare Center Training Systems
Orlando, FL
Robert E. Wray
Ann Arbor, MI
Game-based technologies are being increasingly leveraged by the military for training purposes. However,
despite their perceived contribution, games are rarely empirically evaluated for their training effectiveness
nor used to their full potential. Instead games are often implemented as practice environments alone,
lacking instructional capabilities that are known to lead to more effective training (Clark, Yates, Early, &
Moulton, 2007). In an effort to integrate evidence-based instructional design into serious games,
effectively “bringing the classroom into training games,” the U.S. Army Research Institute collaborated
with Soar Technology to develop a training game for course-of-action analysis (COAA) based on the
Guided Experiential Learning (GEL) Model (Clark et al., 2010). Course of Action Analysis
(“Wargaming”) involves step-by-step human simulation and evaluation of a course of action. The gamebased instruction (GBI) COAA prototype includes part/whole task practice, field-based problems,
demonstrations, and guided practice. These methods are prescribed by GEL, which is based on Merrill’s
principles of instruction (Merrill, 2006). To explore the effectiveness of this approach, Soldiers were run
through a study to compare the use of the GBI COAA prototype to a control that approximates how games
are typically used in simulation centers (i.e., upfront instruction, practice without guided feedback,
AAR/post-training feedback). Assessments included usability, attitudes toward technology, and training
effectiveness/efficiency (e.g., number of errors). The goal of this evaluation was to determine whether the
method used in the COAA tool is more effective and efficient when compared to current training practices.
Findings support the use of embedded instruction in games; a significant decrease in the number of errors
was found for Soldiers exposed to game play that featured the GEL model of instruction compared to
Soldiers who had traditional training. These results will help guide future development of game-based
instruction environments and provides support for the continuing use of games in training.
13
AN EXPERIMENT TO EVALUATE THE EFFECT OF NARRATIVE DELIVERY IN MILITARY
TRAINING
Mark Lewis
Centre for Simulation and Analytics, Cranfield University, Defence Academy of the UK
United Kingdom
Prof Robert J. Stone
University of Birmingham
United Kingdom
The use of serious games in military training has now become commonplace. However, unlike most
commercial entertainment games, explicit “storylines” (as portrayed in, for example, cut scene inserts) are
rarely used to introduce games-based training simulations. A storyline constitutes an information-rich
framework for the structuring of learning content delivery, networked episodes, tasks and activities. It
follows a narrative and pedagogical outline with reference to key questions, learning tasks, activities,
resources, media and cooperative interactions. A game-based simulation study was undertaken using
undergraduate Defence Technical Officer and Engineer Entry Scheme students from the UK universities of
Birmingham and Aston. Based around cultural awareness training, a core scenario and narrative were
developed to teach participants how to interact with a fictional civilian population.
The experimental aim was to investigate if the method of narrative delivery had a direct impact on the
student’s learning performance. Using one of three delivery styles (“passive”, “semi-active” and “active”),
the narrative took the form of (a) a paper brief, (b) a (military) instructor-led presentation, (c) a short paper
brief followed by an interactive but predominantly scripted first-person (FP) simulation scenario, and (d) a
fully interactive FP scenario. Participants were then exposed to a similar FP scenario in which they had 20
minutes to achieve the objective of intercepting an insurgent. In addition, subjective measures of
participant learning style were obtained to investigate whether or not different styles might have some
bearing on the impact of specific narrative delivery method.
This paper presents the early findings of this study, which is sponsored by the UK’s Defence Science and
Technology Laboratory (dstl).
TUESDAY, 2 DECEMBER, 2014 ROOM S320E
ED-2 Inspire, Design, Execute: Simulation in Three Domains
1600
Modeling and Simulation Challenge
Problems in High School
Classrooms and Internships: Lessons
Learned (14103)
1630
Using Unity to Implement a Virtual
Crash Site Investigation
Laboratory in Support of Distance
Learning Objectives (14050)
1700
Developing the Simulator
Instructor’s Pedagogical
Competence
(14043)
Notes
14
MODELING AND SIMULATION CHALLENGE PROBLEMS IN HIGH SCHOOL
CLASSROOMS AND INTERNSHIPS: LESSONS LEARNED
Jennifer Winner
Lumir Research
Institute, Inc.
Dayton, OH
Kimberly Puckett & Leesa
Amelia Malone
Folkerth
University of Maryland
Tri-Village Local School District
College Park, MD
New Madison, OH
Jerred Holt
Lumir Research
Institute, Inc.
Dayton, OH
The use of modeling and simulation is widespread across scientific and engineering disciplines and all
branches of the United States military utilize modeling and simulation for training, testing, and developing
next generation capabilities. Despite this reality, modeling and simulation is largely absent from high
school classrooms. Through the use of game-based technology challenges, our team has implemented three
years of internship experiences and developed high school M&S content as a way to get learner buy-in and
engage students. The Air Force Research Laboratory’s Gaming Research Integration for Learning
Laboratory has hosted educators, student interns, and mentors since 2011. In this paper we describe the
history and evolution of this program which introduces high school students to modeling and simulation,
problem-based learning, and provides models of the types of problem-solving capabilities required for
working in the defense industry. We review and discuss observations from summer internships and high
school classrooms and present anecdotal evidence on student outcomes. Further, we discuss lessons
learned with regard to student motivation and teacher education and training as well as future measurement
of the program’s effectiveness. Observations made to-date suggest that for students who have already
achieved academic excellence in high school, substantial gains may be made through short-term
internships such as a summer appointment. For students at risk of excluding themselves from science and
technology-related career paths, modeling and simulation content has potential to motivate students to
address any gaps in their completion of prerequisite courses they will need to move forward. Successful
integration of modeling and simulation content within a classroom requires a substantial time investment
in the teaching staff but through consistent support, teachers are able to develop the baseline level of
comfort and proficiency with the content to support the students in their exploration of the technologies.
USING UNITY TO IMPLEMENT A VIRTUAL CRASH SITE INVESTIGATION LABORATORY
IN SUPPORT OF DISTANCE LEARNING OBJECTIVES
Christina Tucker & Jimmy Moore
Pinnacle Solutions, Inc.
Huntsville, AL
The purpose of this paper is to describe the use of Unity to develop an interactive virtual environment to support
distance learning in a higher education curriculum. Online and distance learning has become a viable delivery
method for course content in higher education and is continuing to gain acceptance and popularity as students
and faculty become more comfortable with this medium. The demand for online learning and the emerging
requirement to have students bring their own devices to the classroom is also driving a need for innovative
methods of instruction. This paper discusses the need to utilize a virtual crash site for the Embry Riddle
Aeronautical University (ERAU) – Worldwide Master’s-level Crash Site Investigation course and the decision
to utilize the Unity gaming engine for its implementation. Many obstacles had to be overcome during the
development and implementation of the virtual laboratory environment to ensure that it provided an effective
learning environment for students, and that it is engaging, useful, and intuitive enough for students who are not
“gamers.” A set of course objectives was provided by staff members of the College of Aeronautics from the
school’s eLearning sector. These objectives were then transformed into a concrete set of requirements to be used
as the basis of development. Derived requirements were developed to define how to meet these objectives in the
virtual world. This paper will address why Unity was chosen for the development environment; it will discuss
the different capabilities of the lab desired to meet the course objectives (including tasks like taking photographs
and measurements, diagramming the crash site, and interviewing witnesses); and how the course is being made
available to students. The Virtual Crash Site Investigation Laboratory is currently in its pilot course; therefore,
data showing the effectiveness of the course is not available. However, feedback from staff and technology
personnel has been positive. Once the course is complete, student surveys and instructor feedback will be
collected to determine the aspects of the class that were well received, as well as any issues that need to be
addressed.
15
DEVELOPING THE SIMULATOR INSTRUCTOR’S PEDAGOGICAL COMPETENCE
Peter Sjoestedt
Royal Danish Defence College
Copenhagen, Denmark
The full benefits of investment in simulator-based training are achieved only when development of the
simulator instructor‟s pedagogical competence accompanies the acquisition of advanced technology. In
spite of this, the instructor‟s pedagogical competence often is not well developed. Courses for simulator
instructors that address their experiential instructional skills and their understanding of adult participants'
learning processes are rare. Lessons learned about the instructor‟s role that focus on the relationship
between instruction and learning are seldom reported. Simulator instructors‟ pedagogical competence
appears to be an area afforded insufficient attention from the simulation community.
Based on lessons learned from our four-day pedagogical course for naval simulator instructors in the
Danish armed forces, this paper presents a framework for instructors‟ pedagogical competence
development. It explains why the subjects of the course are 1) the instructor‟s various feedback tasks, 2)
knowledge about how to ask questions, 3) the likelihood of conflicts and how they can be managed and 4)
cooperation between the learners and the instructor as a leader and facilitator of learning. The paper
addresses how this course – based on the particularly experiential nature of simulator-based training –
differs from general courses on teaching and instruction. The participants' reactions are reported and
discussed and finally, possible adjustments and perspectives for the future of the course are outlined.
The purpose of this paper is to make those teachers and leaders responsible for simulator-based training
aware of the value of pedagogical competence development and to provide inspiration for how a course
can be planned in their academies and organizations.
WEDNESDAY, 3 DECEMBER, 2014 ROOM S320E
ED-3 Building It Right
1400
An Instructional Media Selection
Process for Virtual World
Training Delivery (14369)
1430
Developing the Human Dimension:
Current Practices and
Future Methods (14161)
1500
Hey, Your E-learning Courses are
Giving Me a Cognitive
Overload (14008)
Notes
16
AN INSTRUCTIONAL MEDIA SELECTION PROCESS FOR VIRTUAL WORLD DELIVERY
Leslie A. L. Mazzone
Submarine Learning Center
Groton, Connecticut
Anh Bao Nguyen
Navy Undersea Warfare Center
Newport, Rhode Island
The U.S. Navy has recently focused on improvements to training techniques with reduced resources. Strategies
to meet this goal include evaluating the necessity of each course, determining if each learning objective is
required, and examining if efficiencies can be met through a reduction in training time or through the use of
advanced training technologies. Within the last few years, Submarine Learning Center (SLC) and the Naval
Undersea Warfare Center (NUWC) have been exploring the use of virtual world technology to cope with
declining resources. The primary focus has been on the technological capabilities within a restrictive
environment in which the protection of data networks drives training technology acquisition. Having addressed
the technological issues, SLC is prepared to consider virtual world technology a viable alternative to delivering
some of the 300 submarine courses. These courses are unique in that most are classified and rely on use of actual
training hardware at full fidelity. A downside to current training hardware is that the submariners must go to the
physical location to obtain training. The virtual world technology that SLC plans to implement allows the
submariners to use these trainers without having to travel to their physical location. This paper presents a
process that facilitates the selection of the best course candidates for conversion to virtual world technology.
General factors considered include course length, site distribution, and the characteristics of the course. For
courses that include the use of training hardware, factors considered include the degree of virtual world
equipment fidelity, environment fidelity, and sensory fidelity required. Purposeful media selection is an
important part to reducing training resources while improving training. The instructional media selection process
presented in this paper can be implemented by a larger community faced with deciding which courses are the
best candidates for virtual world conversion.
DEVELOPING THE HUMAN DIMENSION: CURRENT PRACTICES AND FUTURE
METHODS
Michael Prevou, Ph.D. & Laurie B. Waisel, Ph.D.
Strategic Knowledge Solutions, Inc.
Leavenworth, KS
The strategic white paper “Winning the Clash of Wills” declared that better integration of the human
domain into military plans and operations is essential to achieving overarching national security objectives.
The human dimension reaches beyond simply equipping soldiers with language, culture, moral, and
physical skills and must focus more on the subtle cognitive components. While training develops skills and
techniques through practice and observation, educating leaders must emphasize the development of new
competencies that apply critical thinking and reasoning skills, strategies for problem solving under
pressure, and cooperative leadership styles. But Army doctrine does not tell us how to achieve this; it tells
us only that we must develop the triad of moral, physical, and cognitive components of the human
dimension. Meanwhile, the emerging Army and Air Force learning models call for continuous, adaptive
learning using operationally relevant training scenarios to provide outcome-oriented instruction and
leveraging informal and formal training opportunities. Exactly how this can be done is the subject of this
paper.
This paper describes one of two case studies that provide insights into how to achieve improved learning
outcomes using an adult learning model that emphasizes Critical Moment Strategy and experiential
learning as espoused by Army Learning Model (ALM) 2015 and emerging Human Dimension doctrine.
The paper then describes a future state methodology for course design structured to develop self-aware and
adaptive leaders. The case studies use simulations, games, and vignette-based exercises that put the learner
“in the moment,” continuously changing variables to teach soldiers to think critically and out-of-the-box.
This methodology is easy to replicate and uses a set of thinking exercises (cognitive battle drills) to
reinforce effective thinking habits in tactical situations by applying deliberate practice to leaders’ thinking
skills. The paper also outlines an Experiential Learning Model that addresses different learning styles and
turns traditional classroom methodologies upside down.
17
HEY, YOUR E-LEARNING COURSES ARE GIVING ME A COGNITIVE OVERLOAD
Commander Geir Isaksen,
Norwegian Defense University College
Oslo, Akershus
Like many large organizations, the Norwegian Armed Forces rely more and more on e-learning to deliver
cost-effective and high-quality learning to their employees. The main distribution method is through
learning management systems and varies from low-cost courses, based on text and graphics, to more
expensive productions containing video and advanced interactivity.
A continuous drawback however is the fact that many of the students do not complete all of the courses
they have registered for, which is often called “dropout.” The dropout rate varies from below 10% to over
50% in some courses. In a typical e-learning environment, the learner will go through the course alone, on
a personal computer, with no contact with a teacher or a tutor.
This paper presents the findings from a first years’ thesis conducted as a part of master’s program in
information computer technology & learning. It examines how the use of multimedia in e-learning courses
might influence learner dropout. Research suggests that used incorrectly, multimedia can contribute to a
so-called cognitive overload for the learner. This again can influence learner retention and decrease
motivation, ultimately leading to dropout.
Recommendations from leading theories and research are used to analyze four different e-learning courses
from the Norwegian Armed Forces and discuss any relevance between the theories and the dropout rate of
each course. This paper summarizes the findings and delivers concrete recommendations on how to build
e-learning course in the future, to minimize student dropout.
ED-4 Practice What You Preach
1600
Rediscovering the Eightfold Path:
Some Observations on using
Simulation for Training and
Evaluation from Afghanistan
(14196)
1630
Mobile Instructional Strategy
Templates for Guided Mobile
Content Development (14194)
1700
Are They Mission Ready? Using the
Modified Angoff Method to
Set Cut Scores (14060)
Notes
18
REDISCOVERING THE EIGHTFOLD PATH: LESSONS LEARNED IN EMPLOYING
SIMULATION FOR TRAINING AND ASSESSMENT IN AFGHANISTAN
Christopher Huffam PhD
Canadian Defence Academy
Kingston, Ontario
In developed countries, selection of simulation technologies is based on intended use, planned content and resources available. In these settings, choices of simulation for training or competency assessment are only limited
by time, expertise and resource availability. In failed state settings such as Afghanistan, the use of specific technologies, the availability of support infrastructure, and cultural considerations (including baseline educational
skills) are factors in selection and use, as the available approaches to the delivery of content and use of simulation to support learning and assessment may be limited by the circumstances of place. This situation results in
unique challenges to the provision of education and the rebuilding of training and educational institutions, with
occasional novel solutions to defined obstacles to effective training and competency assessment. Examples discussed range from purely cultural (such as Dari having one word and related concept for the English equivalents
of Task, Job and Occupation), a consideration which has significant impact on initial training and subsequent
assessment of individual capability, to more technically complex issues. This paper will discuss a sample of the
lessons learned in training delivery and assessment of student capabilities for the Afghan National Police (ANP).
The information included is drawn from a combination of onsite personal observation in Afghanistan by the
author between July 2013 and March 2014, Training Assessment Team reports for training conducted before
and after handover to local authorities by the NATO Training Mission – Afghanistan (NTM-A) for the 13
enduring Provincial Training Centers. It includes information drawn from interviews with staff from NTM-A,
the Europe-an Union Police Mission in Afghanistan (EUPOL), German Police Project Team (GPPT), the
International Po-lice Coordination Board (IPCB), and relevant findings from the NATO lessons learned
repository for that same period. This information was collected during the final nine months of the Canadian
contribution to the International Security Assistance Force (ISAF).
MOBILE INSTRUCTIONAL STRATEGY TEMPLATES FOR GUIDED MOBILE CONTENT
DEVELOPMENT
Peggy Kenyon & Helen Remily
U. S. Training and Doctrine Command
Ft. Eustis, VA
Dennis Wikoff
Adayana Vice President of DoD Programs
Falls Church, VA
Mobile learning (mLearning) is an evolving field that introduces exciting capabilities and challenging
complexities into the learning-design process with limited guidelines to steer mLearning designers. The promise
of true anytime, anyplace learning compels decision makers to make learning content available on mobile
platforms immediately. However, without the appropriate research to rethink and redesign learning
methodologies targeted for mobile platforms, many training practitioners are simply shrinking the screen size of
learning content for mobile devices. Such conversions do not actualize the full potential of the mobile platform
and may result in learners’ rejecting the content.
The Advanced Distributed Learning (ADL) MoTIF project focuses on a practical understanding and
applicability to real-world design projects for mLearning. At present, the early stage of the ADL research is
examining the issue at the macro level of general learning theory and a mLearning framework. The available
research provides “how-to” guidance that instructional systems designers (ISD) can use to develop their own
instructional strategy approaches to mLearning, but the budgets and schedules of most mLearning development
projects usually constrain ISDs from completely following the how-to guides through analysis, design, and
development of instructional strategies that consider how users interact with mobile devices and take advantage
of technology available in those devices. Improving fielded mLearning requires the application of the existing
research on effective mLearning models to create a library of reusable mobile instructional strategy templates.
This paper proposes how the ADL macro research findings can be coalesced into a library of useable
instructional strategies or instructional interactivities that can be used by mLearning developers to create
effective learning modules. Strategies will be coded as reusable templates and incorporated into the Army
Enterprise Content Development Capability authoring framework for rapid prototyping and distribution to the
Army user community.
19
ARE THEY MISSION READY? USING THE MODIFIED ANGOFF METHOD TO SET CUT
SCORES
Ingrid Mellone, Carol Faben
Camber Corporation
Orlando, Florida
Formal assessment is well established in the military and government for applications such as initial
selection, promotion, and end-of-course training. For end-of-course assessments of lengthy and/or critical
training, it is particularly important for leadership to be confident in the passing score required. Qualified
people must not be excluded from passing, and unqualified people should not pass. Yet currently, required
passing scores for criterion referenced tests are often set using arbitrary methods. Although such methods
may take into account the criticality of the content overall, they do not use a detailed enough description of
job performance requirements to establish “minimally acceptable levels.”
This paper describes the importance of establishing a rational passing score, or cut score, and several ways
of establishing cut scores, focusing on the Modified Angoff (MA) method. This widely used conjectural
method has been adjudicated in the courts and is therefore considered defensible. The MA method features
a group of informed judges independently estimating what proportion of minimally qualified test takers
will correctly answer each test question. Advantages and disadvantages of the method are discussed, as
well as factors in its successful application. The authors have employed the MA method for several years
on behalf of the Veterans Benefits Administration (VBA) Skills Certification program, a system of
Congressionally mandated, high-stakes certification tests. Although the MA method may be applied to a
variety of assessment tests and formats, the VBA tests are comprised of multiple choice and similar test
item formats, and are delivered online. The process used to collect judges’ estimates is discussed,
including the frame-of-reference training provided, the technology supporting the intake of ratings, and the
computation of cut scores for these tests. Compared with arbitrary methods, the MA method provides
greater assurance that those who pass are, indeed, qualified to pass.
THURSDAY, 4 DECEMBER, 2014 ROOM S320E
ED-5 Is Skill Development Your Specialty?
0830
Psychomotor Skills (14220)
0900
Psychomotor Skills (14220)
0930
Assertiveness and Responsiveness in
Teams: Essential for
Mission Command (14197)
Notes
20
RETENTION AND RETRAINING OF INTEGRATED COGNITIVE AND PSYCHOMOTOR
SKILLS
Anna Skinner
AnthroTronix, Inc,
Silver Spring, MD
Maintenance of specialized skills during periods of nonuse presents a significant challenge across multiple
domains, and is most relevant within applications in which skill degradation is common and has significant
negative consequences, as is the case within high-risk task environments typical of the military and medical
domains. Few studies have examined retention of complex skills that integrate multiple skill components
involving cognitive, psychomotor, and perceptual subskills. As a result, little is known about the nature of skill
decay for performance of many military and medical tasks, and the military medical/surgical domain presents an
especially problematic construct with respect to skill decay due to the nature of deployment cycles. For example,
specialized medical skills such as laparoscopic surgery (LS) skills, which involve integration of underlying
psychomotor and cognitive/perceptual skill components, often are subject to decay during deployments in which
military surgeons primarily practice open procedures. A need has been identified for research and development
to support prevention of LS skill attrition, as well as subsequent retraining. This experiment empirically assessed
the relative retention of psychomotor and cognitive skill components within an integrated task relevant to LS, as
well as in isolation, following a 3-week retention period. Results demonstrated significantly greater skill decay
for the integrated task than the cognitive task, and no significant decay for the psychomotor task, suggesting
greater decay overall for the constituent skills within an integrated context. This study also assessed the
comparative effectiveness of video-based retraining to hands on retraining of the integrated skill. Results
indicated significant skill recovery for both the video-based and hands-on retraining groups based on mean trial
time and cognitive subtask component accuracy, with no between groups differences detected. Results are
discussed within the context of training strategies to detect and reduce skill decay within LS as well as other
complex task domains.
AN INDIVIDUALIZED APPROACH TO REMEDIATING SKILL DECAY: FRAMEWORK AND
APPLICATIONS
Roberto K. Champney, Erin G. Baker,
Tarah N. Schmidt-Daly, Kay M. Stanney, Kelly Hale,
Richard Long, George Chadderdon
Design Interactive
Oviedo, FL
Julie Jacko, François Sainfort, Jit Chan, Andrew Nelson
BioMedical Metrics, LLC
Golden Valley, MN
Physicians predominantly use self-monitoring to assess and maintain skill proficiency, and to determine
when refresher training is required. However, strikingly low correlations exist between physician selfassessments and observer-expert ratings. In addition, in many military and civilian positions, training and
education schedules are often standardized and rigid, potentially leading to wasted resources on training
that is not needed for those that remain proficient at needed skills. In order to optimize training, there is a
critical need for adaptive learning systems that can objectively measure, and preemptively or timely
refresh knowledge and support skill maintenance. This paper outlines challenges associated with
objectively quantifying skill decay within the medical domain. Requirements for a skill decay framework
are summarized based on identified challenges, and a preliminary Skill- DETECT (Degradation Evaluation
Toolkit for Eliminating Competency-loss Trends) framework is presented. This Skill-DETECT framework
uses objective data to tailor an education and training program to a user’s specific needs. The current
application of the Skill-DETECT framework is developed within a medical environment, and utilizes
electronic medical records generated by a physician, as well as real-time cognitive assessment data to
suggest recommendations on individualized, optimized retraining regimens to reduce the likelihood of skill
decay.
21
ASSERTIVENESS AND RESPONSIVENESS IN TEAMS; ESSENTIAL FOR MISSION
COMMAND
Hilde T.A. van Ginkel, Rendel D. de Jong, Mandy G. van de Velde
Utrecht University
Utrecht, The Netherlands
John W. van Buren, Richard G. Oppelaar
Royal Netherlands Navy
Den Helder, The Netherlands
Though differences in rank within military teams help to define responsibility, they may hamper necessary bottom up
communication, jeopardizing performance and safety. In aviation, lack of bottom up communication was attributed to
lack of assertiveness in junior team members, providing them with assertiveness training as a consequence. However,
failing upward communication may not only be related to junior member’s characteristics but also to lack of
responsiveness of the senior member. It was the purpose of this study to examine the connection between rank,
assertiveness, responsiveness and team potency as indicator of effectiveness in teams.
Data were provided by 67 military crews, consisting of pilot and observer, during training-missions in a Naval
Helicopter high fidelity simulator. In a post-flight questionnaire, both crewmembers provided ratings of the other
member’s assertiveness and responsiveness, and gave their own rating of team potency.
Results show a negative impact of pilot’s rank on observer’s assertiveness, while observer’s rank was not related to
pilot’s assertiveness.
The higher the pilot’s rank, the higher the observer’s responsiveness, and the lower his own responsiveness, both
irrespective of observer’s rank. The higher the observer’s rank, the more responsive the pilot was, again, irrespective
of the pilot’s rank. Both crewmembers’ rank was related positively to their own rating of team potency, but not to the
rating by the other member.
A consistent positive connection was found between responsiveness and team potency; pilot’s responsiveness was
related positively to team potency as indicated by the pilot as well as the observer. The same pattern was found for the
observer’s attributed responsiveness.
These results confirm that responsiveness is essential in multi-rank military teams, with important consequences for
Mission Command. Therefore, it is incorporated into existing initial and advanced RNL Navy Crew Resource
Management training, to enhance mission readiness in our Navy and Marines.
ED-6 Automatic for the People: Content Alignment for
1330
Tailoring Multimedia Instruction to
Soldier Needs (14049)
1400
Automated Content Alignment for
Adaptive Personalized
Learning (14068)
1430
An Army Learning Model
Implementation: Challenges,
Successes, Future Directions
(14153)
Notes
22
TAILORING MULTIMEDIA INSTRUCTION TO SOLDIER NEEDS
Thomas Rhett Graves, Ph.D.
Paul N. Blankenbeckler, Richard L. Wampler
U.S. Army Research Institute
Northrop Grumman Technical Services
Fort Benning, GA
Columbus, GA
To address particular learning needs, instruction should be designed to provide the right information to the
right learner at the right time. While interactive multimedia instruction can reach a large audience,
different learners have different learning needs. In this research, three questions were addressed: (a) how
could existing Army interactive multimedia instruction (IMI) be modified to a needs-based format, (b)
what types of instructional design techniques could be applied to design IMI for learners’ specific needs,
and (c) what outcomes are associated with different types of needs-focused IMI? Following a survey of
existing Combat Arms IMI, it appeared that most would require modification for reuse in a needs-based
context. Six IMI exemplars were developed for two topics (i.e., Adjust Indirect Fire and Conduct a
Defense by a Squad) targeting the learning needs of new squad or team leaders. The IMI were developed
for three needs-focused conditions to test whether learners with differing levels of prior knowledge and
experience performed better with different types of IMI. One condition was tailored training, and the other
two were not tailored, and were designated as familiarization and core/refresher training. The exemplars
were tested with Soldiers attending the Warrior Leader Course at Fort Benning. In all conditions, Soldiers
showed higher test scores after training. However, for the less familiar topic, the greatest impact was found
for the tailored training condition over non-tailored familiarization and core/refresher training. The
familiar topic showed no differences among needs-based conditions. These results indicated that structure
is necessary for novel material to mimic what learners naturally do with familiar material. To effectively
modify IMI to a needs-based format, one needs to define the learning needs of the specific audience up
front and to structure the IMI to support individual choice and flexibility.
AUTOMATED CONTENT ALIGNMENT FOR ADAPTIVE PERSONALIZED LEARNING
Elliot Robson
Eduworks
New York, NY
Robby Robson
Eduworks
Corvallis, OR
Effective learning interventions (online courses, SIMS, live instruction, and self-directed activities) must
be strongly aligned with instructional goals. Programs such as the Personal Assistant for Learning (PAL)
being developed by the US Advanced Distributed Learning initiative and the Generalized Intelligent
Framework for Tutoring (GIFT) developed by the Army Research Lab (ARL) emphasize the
Government’s investment in learning interventions that adapt to learner goals and preferences. To be
practical, such systems must automatically detect and align digital content and other learning intervention
with learning goals.
The research reported here addresses one step in this process. It is part of the larger integration effort
between GIFT and Tools for the Rapid Development of Expert Models (TRADEM), supporting the efforts
and goals of the Army Research Lab (ARL). This paper presents techniques that automatically use a set of
text-based features to detect pedagogically appropriate topics. These techniques are part of an attempt to
automate portions of the front-end anal-ysis and design steps in the tradition “ADDIE” (analysis, design,
development, implementation, and evaluation) [Branson et. al., 1975] approach to content creation. This
paper sets the context for this work, describes the tech-niques and algorithms used, and provides data that
shows that auto-detection performs well when reviewed by and compared to hand-generated mappings by
instructional design experts.
23
AN ARMY LEARNING MODEL IMPLEMENTATION: CHALLENGES, SUCCESSES, FUTURE
DIRECTIONS
Camilla Chavez
Knott
Aptima, Inc.
Washington, DC
Scott Flanagan
William R. Bickley
Krista Ratwani
Sophia Speira
Army Research
Aptima, Inc
Washington, DC
Institute
Washington, DC.
Ft. Benning, GA
Courtney Dean, Fred
Diedrich
Aptima, Inc.
Woburn, MA
The Army Learning Model (ALM) focuses on developing Soldiers over time using a variety of “Soldiercentered” methods in a range of settings that foster 21st Century Soldier Competencies (e.g., initiative,
critical thinking). Such competencies are critical for enabling a high degree of operational effectiveness in
the context of Unified Action/Full Spectrum Operations. Achieving the ALM’s vision requires changes to
Army training including instructional approaches and technology applications. Accordingly, one
instructional approach suggested in response to this challenge is “Adaptive Soldier and Leader Training
and Education” (ASLTE). The ASLTE approach, rooted in Outcomes Based Training and Education,
promotes instructional principles focused on developing desired leader competencies through instructorstudent interactions over time. It requires that instructors (1) possess a reasonable level of expertise and
leverage their capability to adapt to new challenges, (2) are capable of maintaining a positive attitude
conducive to building trust and confidence in their students, and (3) purposefully and systematically move
student learning activity towards a threshold of failure to challenge them while ensuring success to build
confidence and initiative. This paper discusses the theoretical basis of the ASLTE approach and findings
concerning successes, challenges, and opportunities based on observations during various applications
(i.e., marksmanship training in Initial Entry Training, the Army Reconnaissance Course, and the Infantry
Advanced Leader Course). Additionally, we summarize workshop findings from a cross section of courses
and organizations at Ft. Benning regarding challenges and best practices. Collectively, these analyses
indicated that much progress has been made in implementation and ASLTE holds great promise, but key
challenges remain. We propose recommendations for further ASLTE implementation focused on issues
including but not limited to instructor development (e.g., building expertise), instructional methods (e.g.,
scaffolding techniques), and reliable assessment of competencies across and within courses (e.g., methods
to consistently assess leader attributes such as initiative).
TUESDAY, 2 DECEMBER, 2014 ROOM S320C
EC-1 Game On
1400
Considerations on Utilizing a Game
Engine as an Image
Generator (14274)
1430
Towards Minimalist Serious Game
Design (14348)
1500
Human Motion Capture in Natural
Notes
24
CONSIDERATIONS ON UTILIZING A GAME ENGINE AS AN IMAGE GENERATOR
Kevin Bland
AVT Simulation
Orlando, FL
Stephen Lopez-Couto
US Army PEO STRI
Orlando, FL
Jelani Vassall
TAPE, LLC
Orlando, FL
Modern commercial game engines enable the rendering of visual scenes that are a stark improvement over
what was feasible just a few years ago. Powering these engines are graphics processing units (GPUs) that
continue to rapidly advance in capability. Game developers keep pace with the hardware vendors by
regularly incorporating software modifications that take advantage of the new hardware. The Department
of Defense (DOD) has already taken steps to incorporate these commercial capabilities into its training
portfolio, primarily in ways that are complementary to the typical gaming use case. The utilization of
gaming technology in DOD Virtual (man in the loop) Training Systems is still far behind specialized,
requirement-intensive Image Generator (IG) technology. The US Army’s Close Combat Tactical Trainer
(CCTT) program is in the process of integrating gaming technology used for the Army’s Games for
Training (GFT) program to perform as a multi-channel IG with legacy virtual environments. This paper
discusses the technical and programmatic considerations that are recommended when evaluating the
inclusion of a gaming application into traditional virtual simulator architecture. The focus will be on
considerations related to incorporating a game engine as an IG in a heterogeneous environment, integrating
gaming into a legacy training system with an established architecture, and planning for the long term
sustainability of the full system.
TOWARDS MINIMALIST SERIOUS GAME DESIGN
Dr. Peter A Smith
Stuart Armstrong
QinetiQ
Orlando, FL
Orlando, FL
The last ten years the military has seen large success in the use of games and game technologies within the
tactical training community. First person shooters dominate this area with significant investment around
the world, primarily increasing the realism and fidelity to meet perceived training requirements.
Unfortunately, not all training objectives can be easily met through the current military approach to
gaming. In an effort to meet these needs some organizations are investing in completely new high end
game based training systems or worse, attempting to shoehorn in functionality that is not a good fit for
current systems. Both strategies are leading to greater cost, system complexity, and user confusion. One
methodology that has been applied to entertainment games is minimalist game design. Minimalist games
designs generally define small games with compact but rich rule sets that provide narrow decision spaces
and often abstract world representations without diminishing the perceived depth of play. These minimalist
designs create micro worlds in which a player can experience and experiment with a distinct subset of
rules to train on requirements that would not necessitate a large simulation game. Further, they meet the
promise of replay ability, self-regulation, and motivation that other gaming solutions often lack. This paper
looks at how minimalist game design can be leveraged in serious games in order to focus on delivering a
particular learning or training outcome. Using case studies from around the world, and supported by data
collected from the Serious Games Showcase & Challenge this paper aims to set out the fundamental game
design principles that support minimalist serious games design, and provide examples of how they can be
leveraged to synergistically support existing enterprise solutions to gaming in the field
25
HUMAN MOTION CAPTURE IN NATURAL ENVIRONMENTS
Zhiqing Cheng and Anthony Ligouri
Infoscitex Corporation
Dayton, Ohio, USA
Timothy Webb and Huaining Cheng
Air Force Research Laboratory
Dayton, Ohio, USA
In this paper, the problem of capturing human motion in a natural environment is discussed from the
perspective of needs, significance, scenarios, and technical challenges. The technologies that can be
potentially used to capture human motion and activity in a natural environment are discussed, which
include electromagnetic sensors, LED lights, inertial measurement units, range sensors, and computer
vision-based markerless motion capture technology.
Two markerless motion capture methods for capturing human motion from video imagery are investigated
and implemented in this paper. The first method uses a silhouette shape descriptor to describe silhouette
shape and maps the silhouette shape descriptor (input vector) to joint angles (output vector) through a
mapping matrix which is determined using relevance vector machine. The second method performs pose
estimation by fitting a 3D human model to the silhouette through an iterative optimization. By minimizing
the distance between the silhouette and the template skeleton-surface model that is embedded inside the
silhouette, joint angles are estimated and thus pose is identified. The silhouettes extracted from human
animation data are used for training the methods. The initial results of the two methods are presented and
analyzed.
TUESDAY, 2 DECEMBER, 2014 ROOM S320C
EC-2 Best Simulator Technologies Ever
1600
The Largest Field of View
Collimated Display Ever Built
(14322)
1630
High Quality Visual Display
Systems in Physically Constrained
1700
Dynamic Flight Simulation: 45
Years of Research &
Development (14019)
Notes
26
THE LARGEST FIELD OF VIEW COLLIMATED DISPLAY EVER BUILT
Justin Knaplund
FlightSafety International Visual
Systems
Austin, TX
Dave Fonkalsrud
FlightSafety International –
Simulation
Broken Arrow, OK
Terry Linn
FlightSafety International Visual
Systems
St. Louis, MO
As flight simulators increase in fidelity and performance, more training tasks can be transferred to the
simulator, freeing up aircraft time for other tasks. For some training missions, there are tasks that can
currently only be performed in the aircraft due to limitations in the simulator Field of View (FOV). In
addition, a large horizontal FOV aides in the pilot’s peripheral cues for aircraft attitude, speed and height
above terrain, and the addition of lower chin and side displays are required for helicopter pilots to perform
hover and landing tasks, especially in “brown out” conditions. Since Mylar displays are not typically able
to extend beyond 65° vertically x 225° horizontal, the customer would have to add supplemental real
image or collimated displays located outboard of the Mylar mirror plenum, resulting in a large
discontinuity in the image. A better option is to use glass mirrors for the Out the Window collimated
display, extending the FOV by adding glass mirror segments to achieve a 300º horizontal FOV.
Supplemental chin and side displays can be tucked under the edge of the mirror to eliminate gaps between
the displays and extend the vertical FOV down to -65º. However, designing and building such a large FOV
display has its own challenges, including engineering a single piece Back Projection Screen (BPS) to cover
the full FOV, manufacturing a matched array of glass mirrors, designing a projector turret that locates the
array of projectors across the top of the BPS, and fitting the cockpit and Instructor Operating Station
within the wedge-shaped gap left between the ends of the mirrors and/or BPS. This paper will focus on the
unique challenges our team overcame to build the largest collimated system ever designed, the 300° x 85°
FOV display for the US Marine Corps UH-1Y Flight Training Device, and how these lessons can apply to
enhancing the FOV of other flight simulators.
HIGH QUALITY VISUAL DISPLAY SYSTEMS IN PHYSICALLY CONSTRAINED
ENVIRONMENTS
Reed Moody
Rockwell Collins Inc.
Salt Lake City, UT
Realistic training for pilots seated side by side is, and always has been, a top priority for all simulation
customers. Traditional collimated displays meet this need with relatively large and delicate systems. These
displays are housed in specially-designed facilities that can accommodate the complete visual system and
its associated structures.
In recent years, the need for high quality collimated display training has grown and the desire to bring this
capability nearer to the battlefield has intensified. To ensure pilots are trained and mission ready, high
quality simulation must become available in the theatre of operation. An innovative solution to this
problem, currently being deployed on the US Army’s Transportable Blackhawk Operations Simulator
(TBOS) program, applies the basic concepts of large collimated displays reconfigured into smaller
transportable configurations.
The approach described by this paper meets the following objectives: 1) reduce overall size and weight of
the display. 2) Separate the mirror into manageable and easily-transportable units that can be safely stored
and also accurately installed and aligned in the field. 3) Ruggedize each component of this typically
delicate and fragile visual system, thereby enabling its reliable use in a variety of harsh environments. This
paper describes how technical challenges were overcome to meet these three objectives. Using examples
from the TBOS program this paper will also help explain how timely and mission-critical training can
occur in the theater of operation anywhere in the world.
27
DYNAMIC FLIGHT SIMULATION: 45 YEARS OF RESEARCH & DEVELOPMENT
Kenneth L. Ginader, Michael C. Newman, Richard A. Leland
Environmental Tectonics Company (ETC)
Southampton, PA
This paper will present the development of technologies available for the next generation of flight
simulation that would provide realistic training, combined with physical and physiological stressors, and
recent results on research conducted on their effectiveness. Included will be information highlighting the
development of motion for flight simulators, the distinct differences between transient motion cueing and
G on Demand motion, the misnomer that motion cueing provides G forces, and that transient motion
cueing in hexapod motion-based simulator cannot provide realistic motion sufficient for tactical flight
training.
In the early 1980s both the U.S. Navy and Air Force concluded (hexapod) motion-based simulators did not
justify their cost and complexity in transfer of training. However, by 2004 both manufacturing and
software technology advanced to the point where it was possible to combine high fidelity flight simulator
training with acceleration stressors for realistic tactical flight training.
Conventional wisdom has held that Coriolis Cross-Coupling (CCC) artifacts in centrifuge-based systems
prevent any useful motion-based tactical training. This paper will present research establishing sufficient
management of motion artifacts that allows not only for adaptation but also increased mitigation of CCC
induced by cockpit and head movements in a high performance motion system. This research further
establishes that real motion for tactical flight simulation is achievable and replicates an experience just like
flying an aircraft. Information will additionally be presented on the technology of high performance
motion, software technology that can provide realistic G on Demand and sustained G motion, and
technology that provides the “real feel” of an aircraft in flight simulation.
Finally, high performance motion in flight simulation has the added benefit for advanced research on the
effects of air combat accelerations on pilot training, G tolerance refreshment, and protective modalities that
will provide significant cost savings when applied to realistic air combat training.
WEDNESDAY, 3 DECEMBER, 2014 ROOM S320C
EC-3 Emotional Engagement
0830
Design and Evaluation of Surprise
Effects in Simulation –
A Framework (14193)
0900
Validated Development of Stress
Inoculation through Cognitive
and Biofeedback Training (14051)
0930
Using Virtual Reality as Part of an
Intensive Treatment
Program for PTSD (14079)
Notes
28
DESIGN AND EVALUATION OF SURPRISE EFFECTS IN SIMULATION - A FRAMEWORK
Jelke van der Pal, Konstantinos Georgiadis
National Aerospace Laboratory NLR
Amsterdam, The Netherlands
While a certain level of surprise is required for nearly any type of learning, it is a challenge to provide for
surprises in an effective way. Simulation enables the training designer with powerful options to provide for
surprising experiences, either to engage students, to stimulate thinking, or to learn to deal with them.
Dealing with emergencies or replanning for example are explicit training objectives in many simulator
sessions – although the students often already expect the surprising events. Alternatively, surprises in
simulation sessions can be instrumental to achieve a context in which other training objectives can be
achieved, such as leadership, decision making, and coordination. This study explores the nature of
surprises and provides suggestions for designing surprises in training and subsequently for assessing its
effectiveness. The framework for designing and evaluating surprises relates to the capabilities that cause
the surprise (this may be cue based, narrative based or personal-based) as well as a human (surprise)
information processing model. Assessing the effects of surprises is relevant during the design of the
training scenario to tailor the effects to the target audience, and may also have the potential to guide the
instructor during the training to inject weaker or stronger events. The use of electro encephalogram (EEG)
is a promising technique for assessing mental state levels of relaxation, attention, or agitation/confusion. In
this study EEG is applied to analyze brainwave patterns and investigate the potential for assessing the
effects of a variety of surprise types in a VBS training scenario. Preliminary results indicate that EEG is
sufficiently sensitive to measure mental state effects of surprising events. More study is required to
determine the validity of the measurements and whether it can be used as the single technique or that a
toolkit using a variety of techniques are needed.
VALIDATED DEVELOPMENT OF STRESS INOCULATION THROUGH COGNITIVE AND
BIOFEEDBACK TRAINING
Dr. Peter Squire
Elizabeth O’Neill
ONR
Washington DC
CDR Joseph
Cohn
US Navy MSC
Washington DC
Dr. Gershon Weltman, Elan
Freedy, Dr. Ewart de Visser
Perceptronics Solutions, Inc.
Washington, DC
Dr. Rollin
Donald Chartrand
McCraty
Ease Interactive,
Institute of
Inc.
Heartmath
San Diego, CA
Boulder Creek, CA
A key military priority is building resilience to stress, because of the near- and long-term effects of stress on mission
performance, personal well-being, and relationships. Most stress treatment and prevention research focuses on the
adverse effects of stress, neglecting to build on its potential positive effects on performance. The efforts discussed in
this paper demonstrate that a software training app can provide an effective individualized method for mitigating the
negative effects of stress while emphasizing its positive effects on performance. The Stress Resilience Training
System (SRTS) iPad app blends cognitive training with advanced biofeedback to teach individuals to understand,
regulate and exploit their stress responses, using a game-based learning framework. SRTS provides information on
stress and its effects, techniques for self-regulation based on heart rate variability (HRV) biofeedback, HRVcontrolled
simulations, narrative self-tests, and adaptive coaching. The paper describes the SRTS methodology and positive
results from multiple evaluations, including usability and efficacy studies on military, para-military and non-military
samples. The usability study showed the app is intuitive and easy to use. The efficacy studies showed that participants
improved in a number of key resilience-related measures including perceived PTSD symptoms, stress and depression,
and also reported improvements in on-the-job performance, personal, and familial relations. The empirical results
strongly support the ability of game/simulation-based biofeedback training to build stress resilience in a variety of
different populations. Evaluation results suggest additional enhancements to the methodology including: a web-based
version for multi-platform delivery; increased use of biometric data for better feedback of progress; incorporation of
social networking for supportive interaction; and virtual mentoring to replicate the benefits of personal mentoring. The
enhancements will make the capability a more valuable addition to initial and sustainment resilience training over a
range of military and non-military applications.
29
USING VIRTUAL REALITY AS PART OF AN INTENSIVE TREATMENT PROGRAM FOR
PTSD
Deborah C. Beidel, Ph.D., ABPP
Sandra M. Neer, Ph.D. Clint Bowers, Ph.D.
Orlando, FL
B. Christopher Frueh, Ph.D.
University of Hawai’i Hilo
Hilo, Hawai’i
Albert Rizzo, Ph.D.
Institute for Creative Technologies
University of Southern California
Up to 18.5% of veterans returning from OIF/OEF are diagnosed with posttraumatic stress disorder
(PTSD). In addition to symptoms of anxiety (intrusive thoughts, re-experiencing, hyperarousal, and
avoidance), PTSD can result in social maladjustment, poor quality of life, and medical problems. Other
emotional problems include guilt, anger, and unemployment, impulsive or violent behavior, and family
discord. Many veterans seeking treatment for PTSD also seek disability compensation for debilitating
occupational impairment. There are few administrative or research data to indicate veterans are recovering
from PTSD. Exposure therapy, a form of behavior therapy, alleviates anxiety symptoms, but may not
address the anger, depression and social impairment that accompanies this disorder. In this presentation,
we will discuss an intensive treatment program, known as Trauma Management Therapy (TMT), which
combines individual virtual reality (VR) assisted exposure therapy with group social and emotional
rehabilitation skills training, delivered in a 3 week format. The presentation will demonstrate the VR
environment (Virtual Iraq), will discuss how often/successfully various VR elements are integrated into a
comprehensive treatment program, and the adaptability of the program for active duty military personnel,
as well as veterans. We will discuss the format of the intensive program as well as factors such as
compliance and drop-out rates, comparing these important clinical variables to more traditional outpatient
treatment programs. Additionally, we will address common clinical concerns regarding the use of VR
exposure therapy for individuals suffering from PTSD.
WEDESDAY, 3 DECEMBER, 2014 ROOM S320C
EC-4 Training Hard to Train Skills: Perception, Sensemaking and Adaptability
1030
Microgames for Training Perceptual
Skills (14282)
1100
Beyond Socio-Cultural
Sensemaking: Observing and
Interpreting Patterns of Life (14101)
1130
Developing Effective Adaptive
Training Systems to Enhance
Military Instruction (14140)
Notes
30
MICROGAMES FOR TRAINING PERCEPTUAL SKILLS
Sean Guarino, Ryan Jarvis, Samuel Mahoney
Charles River Analytics Inc.
Cambridge, MA
Michael Connell
Institute for Knowledge Design, LLC
Arlington, MA
A common challenge in military training lies in motivating personnel to practice and retain lessons in the
limited time they have available. A training methodology that can exploit the moments of “microboredom” that lie between missions and other duties can help to address this challenge. However,
exploiting those moments requires training methods that are self-motivating, can be completed in isolation
(e.g., without trainer interaction or forcing factors), and of a duration that can easily fit in these available
time slices. To address this need, we are exploring the application of microgame-based training tools—
casual games that provide critical lessons in periods lasting no more than a few minutes—across a number
of domains. Here, we describe our ongoing effort with the Office of Naval Research (ONR) to adapt
microgames to augment perceptual training. Specifically, in this work we adapted microgames to support
training objectives for the Marines’ Combat Hunter program, a program that focuses on training perceptual
skills needed to recognize threats in urban environments. Designed to augment existing training, our
engaging microgames assist Marines to prepare for in-classroom training, provide practice lessons during
class, and rehearse lessons learnt after training is complete. In this paper, we describe our work designing
these games, and some of the lessons we learned in adapting microgames to perceptual training objectives.
In future work, we plan to further evaluate and test these games, and to extend our game library to address
other Combat Hunter training objectives.
BEYOND SOCIO-CULTURAL SENSEMAKING: OBSERVING AND INTERPRETING
PATTERNS OF LIFE
Tracy St. Benoit.
Orlando, FL
Clarissa Graffeo
MESH Solutions, LLC
Orlando, FL
Military leaders have identified a need for socio-cultural sensemaking capabilities to support operations in
irregular conflicts. However, training programs lack practical applied techniques for such sensemaking.
For example, observational training programs such as Combat Hunter instruct warfighters to set sociocultural baselines, but provide little specific instruction on relevant sensemaking processes; furthermore,
little of the existing or proposed socio-cultural training robustly integrates field-tested methodologies and
concepts from anthropology or other social sciences. A related issue involves the overemphasis that
training and policy recommendations place on “culture” as a rigid concept. Warfighters may
overcompensate by focusing too much on culture over other relevant factors, and often treat culture as a
fixed entity that can be read at a superficial level. Culture, however, is a fluid construct without fixed
boundaries that constantly interacts with other factors and situational exigencies; the complexity of social
systems alongside cultural mixing and shifting within operational environments demands a more holistic
model.
In a 2012 I/ITSEC paper, our team outlined a concept of archetypal, cross-cultural Patterns of Life for
training in virtual environments. In this paper we propose a revised concept of Patterns of Life as a critical
thinking framework that extends beyond culture to incorporate human and non-human actors, practices,
functions, environmental interactions, and temporal, cultural, and situational contexts that better reflect
social science theories. We also draw on prior perceptual training and ethnographic methodologies to
define an Ethnographically-informed Sensemaking Protocol consisting of a nested, iterative process of
framing and baseline construction that supports both individual encounters and the entirety of a
warfighter’s deployment; this will improve sensemaking and framing baselines in complex, uncertain
environments, and allow applicability across operational environments. We discuss the theoretical
foundations of this revised approach, and then provide a brief summary of the current state of the
framework and protocol.
31
DEVELOPING EFFECTIVE ADAPTIVE TRAINING SYSTEMS TO ENHANCE MILITARY
INSTRUCTION
Fleet Davis, Sandro Scielzo, Jennifer M. Riley
SA Technologies, Inc.
Marietta, Georgia
Heather A. Priest
Navy Air Warfare Center Training Systems Division
Orlando, Florida
Today’s military training environment poses many challenges to instructors, developers, and support
personnel. One obstacle is the limited amount of time instructors have to interact with trainees during live
training, leading to an increase in simulation in the school house with often high instructor-to-student
ratios, dividing an instructor’s efforts across many students. To address this issue, the U.S. Army Research
Institute for the Behavioral and Social Sciences (ARI) and SA Technologies, Inc. developed an adaptive
training system prototype that provides automated support for instructor-led training through trainee selfguided learning. Using predefined instructor input, the system actively monitors trainees’ activities within
a simulation and automatically provides targeted feedback and coaching through metacognitive prompts.
These prompts mimic the essential input a live instructor would normally provide, allowing instructors to
provide consistent, valuable input to all students while lowering their workload. However, despite a wealth
of evidence promoting the efficacy of feedback during practice, such system-based interventions are often
regarded as intrusive. The purpose of this paper is to provide an overall review of the constraints and
considerations associated with developing and implementing such training systems. This includes a
summary of our system evaluation, conducted at the United States Military Academy (USMA) at West
Point during incoming cadets’ normal land navigation training, which included classroom, simulation, and
live training exercises. Results demonstrated the efficacy of the system for enhancing training with
improved task performance both in the simulation and the subsequent live exercise. Additionally,
subjective measures yielded positive evaluations for perceived effectiveness of the training intervention,
usability of the system, and subjective workload associated with trainees’ interaction with the tool.
Together, these findings suggest that the training system and intervention it provides may be a viable
approach to enhancing instructor-led, classroom training and provide guidance for the development of
future adaptive training tools.
WEDNESDAY, 3 DECEMBER, 2014 ROOM S320C
EC-5 Perspective on Data and Services
1400
Future of LVC Simulation: Evolving
Towards The MSaaS
Concept (14072)
1430
Rapid Data Generation: A Flexible
Data Discovery and Access
Architecture (14096)
1500
Optimizing Supervised Learning for
Pixel Labeling and
Material Classification (14016)
Notes
32
FUTURE OF LVC SIMULATION: EVOLVING TOWARDS THE MSAAS CONCEPT
Jose-Ramon Martinez-Salio & Jose-Maria Lopez-Rodriguez
NADS
Madrid, Spain
Live, Virtual, and Constructive (LVC) simulation has been one of the main topics of discussion in
Modeling and Simulation (M&S) community in last decade. Reports like “Live Virtual Constructive
Architecture Roadmap” (LVCAR) established a baseline to start planning next steps to improve LVC
simulations. Based on LVCAR insights, new SISO workgroups, like LSA (Layered Simulation
Architecture) or WebLVC have been born. While improvements in performance, usability and scalability
of the LVC federations are still a hot topic for discussion a new demand is standing-out; the use of
simulations assets as Services. Initiatives in US DoD (JLVC2020) and NATO -Modeling and Simulation
as a Service (MSaaS) and Distributed Networked Battle Labs (DNBL)- are trying to change the paradigm
of how simulations are developed, deployed and used, looking for a Cloud-based publisher-consumer
service paradigm for the assets. This paper analyzes where we are in this quest, pinpointing gaps and main
challenges we need to address to be able to do a fluent transition from LVC simulations to MSaaS clouds.
RAPID DATA GENERATION: A FLEXIBLE DATA DISCOVERY AND ACCESS
ARCHITECTURE
Kevin T. Gupton, Bruce Carlton, Roy Scrudder
Applied Research Laboratories
The University of Texas at Austin
Austin, Texas
Dr. Rob Cox
PEO Simulation, Training, and
Instrumentation
Orlando, Florida
Ralph O’Connell
Joint Staff J6
Suffolk, Virginia
Obtaining and preparing the right data for M&S-based activities is a huge consumer of resources,
regardless of the activity supported by M&S (training, testing, etc.). The Rapid Data Generation (RDG)
project, sponsored by the US Department of Defense (DoD) M&S Coordination Office, on behalf of the
Under Secretary of Defense for Acquisition, Technology and Logistics, was therefore established to
improve the visibility and accessibility of data, as well as to reduce the time and effort necessary to
integrate the necessary data for an M&S event.
This paper presents the RDG Common Data Production Environment (CDPE) system architecture. This
architecture defines a service-oriented design that specifies how data provider and data consumer systems
integrate to enable net-centric discovery, assessment, and retrieval of M&S-relevant data. The architecture
has been implemented in “order of battle” data capabilities in addition to the “environmental
representation” data capabilities. These two capability releases focus on the sharing of military force
structure datasets, such as orders of battle, scenarios, and entity-type enumeration data, as well as
geospatial imagery, elevation, feature, and weather effects datasets and 3D models. The CDPE system
architecture design makes use of DoD enterprise standards with industry best practices and design patterns
to achieve a solution that is agnostic to the types of data exchanged. Through the use of reference
architectures with implementation-independent and -specific designs, the design is resilient and adaptive to
evolving technologies. The architecture also incorporates design alternatives that mitigate the variety in
data producer and consumer system architectures. As a result, the architecture can be applied by others to
develop capabilities for data discovery and sharing across diverse, loosely connected communities. The
CDPE system architecture enables the rapid use and improved reuse of the data necessary for simulationenabled training and mission readiness exercises for multiple tiers of training, all while incorporating and
enabling data sharing with peer communities.
33
OPTIMIZING SUPERVISED LEARNING FOR PIXEL LABELING AND MATERIAL
CLASSIFICATION
Mark Rahmes, Morris Akbari, Mike McGonagle
Harris Corporation, Government Communications Systems
Melbourne, Florida
The visualization and simulation industry has a demonstrated interest in classification products for sensor
simulation. The challenge lies in providing highly accurate material classification of remotely sensed
imagery while significantly reducing the time and cost to create products. Visualization and simulation
products for material classification are created by merging and mosaicking multi-source satellite and aerial
imagery of different resolutions on an elevation surface to provide realistic, geo-specific terrain features.
This requires that all image data is orthorectified, seamlessly co-registered, tonally balanced and feather
blended into mosaics from source data of different resolution. To achieve highest accuracy at faster speed
and lower cost, we apply an innovative, optimal pixel-labeling process to the mosaic imagery. This process
is based on artificial intelligence (AI) algorithms using Nash Equilibrium and game theoretic analyses to
help solve the problem of feature extraction through supervised classification. This can be viewed as a
constant sum game, whereby the players are pixel data points that take part in the game to decide their
class memberships. A player's land cover classification strategies are based on four different supervised
learning algorithms: k-Nearest Neighbors (KNN), Decision trees using a classification and regression tree
(CART), Normal/Naïve Bayes probabilistic graphical model, and support vector machine (SVM). Within
this formulation, we used a weighted reward matrix for consistent labeling of feature pixels and
classification factors, resulting in higher accuracy and precision when compared to the individual machine
learning algorithms alone.
EC-6 Simulation: Supporting Experiment, Acquisition and Military
Planning
1600
Early Synthetic Prototyping:
Exploring New Designs and
Concepts Within Games (14133)
1630
When Tradespace Analysis Met
Combat Modeling and
Simulation (14264)
1700
Simulation in Support of Course of
Action Development in
Operations (14119)
Notes
34
EARLY SYNTHETIC PROTOTYPING: EXPLORING NEW DESIGNS AND CONCEPTS
WITHIN GAMES
Kate Murray, Rudolph Darken
Naval Postgraduate School
Monterey, CA
Brian Vogt
Army Capabilities Integration
Fort Eustis, VA
Simon R. Goerger
Army Corp of Engineers ERDC
Vicksburg, MS
Early Synthetic Prototyping (ESP) is a new concept the Army is exploring that will use game
environments to assess novel system designs and concepts early in the acquisition cycle. ESP is a process
and tools that enable Soldiers to assess emerging technologies within scenarios to provide feedback that
will inform decisions. Acquisition, science and technology, and industry partners develop scenarios and
models of interest to serve on the ESP network for Soldiers to play. ESP allows an unbounded increase in
potentially disruptive ideas to be explored at minimal cost. The goal is to engage the whole Army in
defining the future of the Army and to ensure that the Soldier remains the centerpiece of future
development. To this end, we completed a study to explore an unmanned vehicle concept called Wingman.
Groups of military officers of all services played red versus blue in three scenarios: chase/recon, attack,
and defend. The study asked (1) What feedback could we gather from game players that is useful to the
Research Development and Engineering Centers (RDECs) and the Army Capabilities Integration Center
(ARCIC), (2) Would the organization value that feedback? Using a game environment to explore design
concepts early in the acquisition process is valid and can be applied to early requirement refinement and
rudimentary tradeoff analysis. Through the game sessions, players expressed ideas, both creative and
surprising, towards a preferred interface and how to best employ Wingman. The encouraging results of this
preliminary work clearly demonstrated a strong potential to leverage game environments to explore
revolutionary concepts to efficiently and effectively shape the future of the Army.
WHEN TRADESPACE ANALYSIS MET COMBAT MODELING AND SIMULATION
Chris Gaughan, Christopher J.
Metevier
Army Research Laboratory, Human
Research and Engineering
Directorate, Simulation and Training
Technology Center
Orlando, FL
Simon Goerger
U.S. Army Corps of
Engineers – Engineer
Research and Development
Center
Vicksburg, MS
Tommer R.
Ender, L. Drew
Pihera
Georgia Tech
Research
Institute
Atlanta, GA
Scott Gallant
Effective
Applications
Corporation
Orlando, FL
The Department of Defense (DoD)’s Science & Technology (S&T) priority for Engineered Resilient
Systems (ERS) calls for adaptable designs with diverse system models that can easily be modified and reused, the ability to iterate designs quickly and a clear linkage to mission needs. Towards this end,
tradespace analysis is of great importance. The Georgia Tech Research Institute (GTRI) has been
developing web-based, collaborative modeling and simulation tools that use a Model-Based Systems
Engineering approach to address the analysis of alternatives for acquisition programs to assess cost,
schedule and performance risk; of particular note is the United States Marines Corps (USMC) funded
Framework for Assessing Cost and Technology (FACT). In parallel, the United States (U.S.) Army
Research Laboratory (ARL) has been pursuing the Executable Architecture Systems Engineering (EASE)
research project, which links analytical, experimental and training objectives with the technical complexity
of modeling and simulation in an easy to use, scalable tool. This paper details an effort to develop a formal
Application Programming Interface (API) between FACT and EASE, which creates the ability to develop
system concepts and assess Measures of Performance (in FACT), and then send those system concepts to a
combat simulation to assess Measures of Effectiveness (through EASE), and finally back to FACT for a
high-level trade study. It further describes a proof-of-concept demonstration using a Force Protection use
case that allows a user to tune parameters of detection on an unmanned platform that is then simulated in
an operational scenario to collect performance data. This effort effectively lays the framework for future
simulation-enabled tradespace analysis that will be a pillar of ERS and can be adapted by other simulation
efforts.
35
SIMULATION IN SUPPORT OF COURSE OF ACTION DEVELOPMENT IN OPERATIONS
Lt Col Jens Inge Hyndoy
Ole Martin Mevassvik, Karsten Brathen
Norwegian Army, Land Warfare Centre
FFI (Norwegian Defence Research Establishment)
Rena, Norway
Kjeller, Norway
This paper outlines investigations and suggests use cases to where simulation could be utilized to assist
planners in developing better plans. In order to introduce simulations to headquarters and command posts
one first has to convince operational personnel that the additional equipment and training are acceptable
considering the contribution these systems offer. On-going work in Norway focuses on developing a
demonstrator for allowing simulations to interoperate with command and control (C2) systems using
Coalition Battle Management Language (C-BML) and Military Scenario Definition Language (MSDL).
The purpose is to demonstrate that even a small planning group without simulation specialist training can
draw benefit from autonomous or semi-autonomous simulations. We believe that important factors to
success are a simple user interface and commonality in “look and feel” between the C2 system and
simulations.
The main hypothesis is that simulation can assist wargaming for better understanding of the planned
sequence of events. This will allow planners to draft a more precise synchronization matrix and more
efficiently determine the use of combat support and combat service support assets. One can envision
analysis functionality where different Courses of Action (COAs) are compared based on quantitative
measures rather than personal preferences safeguarding that the potential for operational success is
maximized. Experimentation conducted in an international context, the NATO Modelling and Simulation
Group 085 C2-SIM Interoperability, indicate that such a capability allows for parallel planning and better
synchronization between coalition forces.
The pedagogical aspect of this combination of systems should not be forgotten. Review of simulations
could greatly enhance the common understanding during orders meetings and briefings. A further
distribution and viewing of simulations would also allow subordinate commanders and staff insight to a
commanders’ intent. This research is conducted in close co-operation between the Norwegian Army and
FFI and is characterized by development and testing executed in an iterative pattern. This ensures that
operational personnel are comfortable using the system, and cumbersome and marginal functionality is
discarded.
This paper reviews on-going research on a digital COA capability for the Norwegian Army C2 system and
experiences from an autonomous land warfare simulation demonstrator developed by FFI.
THURSDAY, 4 DECEMBER, 2014 ROOM S320A
EC-7 What to Train
0830
Fundamental Competency Sets
(FCS) Definition to Support
Technology Development for Pilot
Training (14126)
0900
Joint Terminal Attack ControllerTraining Rehearsal System:
Competency-based Research
(14097)
0930
Operator Qualification Differences
between Manned and
Unmanned Aerial System (UAS)
(14287)
Notes
36
FUNDAMENTAL COMPETENCY SETS (FCS) DEFINITION TO SUPPORT TECHNOLOGY
DEVELOPMENT FOR PILOT TRAINING
Amanda Avenoso
Jamie Donsbach
The Group for Organizational Effectiveness
Wright Patterson AFB, OH
Albany, NY
As the Air Force looks toward the future of preparing highly competent warfighters, there is a need to
articulate the capability required to develop next generation pilots. The challenges associated with this are
non-trivial. Not only must training approaches address emerging knowledge/skill sets, they must do so in a
way that efficiently leverages training technologies and resources. This paper describes the Training
Enhancement Study (TES), a collaborative effort between 711 HPW/RHA and AETC that can provide a
foundation for optimizing the Advanced Pilot Training (APT) syllabus and evaluating the utility of various
training technology/media within the pilot training program. The TES centers on the Fundamental
Competency Set (FCS) approach that defines knowledge/skills required for APT students to successfully
transition to their Formal Training Unit (FTU). This approach is based on a proven methodology for
identifying Mission Essential Competencies (MECs), which focuses on mission execution in a nonpermissive or combat environment, and is also used to identify Initial Competency Sets (ICS) – the
building blocks in the transition from FTU to combat readiness. The FCS, ICS, and MEC continuum
represents a unique opportunity for analytic alignment across all parts of the training and readiness
pipeline. This paper discusses the application of the FCS process for Fighter, Bomber, and Mobility APT,
based on SME input and quantitative data from 1051 Instructor Pilots (IPs) to identify training
requirements, gaps, and priorities for the APT syllabus. It shares potential innovative applications of the
FCS data, such as using the APT testbed to review training technologies and determine the best
environment for addressing training gaps. Finally, we provide implications of our capability to integrate
the FCS, ICS, and MEC work and use a consistent analytic approach to support the operator throughout
the full training life cycle.
JOINT TERMINAL ATTACK CONTROLLER-TRAINING REHEARSAL SYSTEM:
COMPETENCY-BASED RESEARCH
1Lt. Sean A. Morris, Dr. Christine M. Covas-Smith,
Leah J. Rowe
Wright Patterson Air Force Base, OH
Christina L. Kunkle, Keith W. Westheimer
Leidos, Inc.
Dayton, OH
The Joint Terminal Attack Controller (JTAC) warfighter is responsible for supporting the Army Maneuver
commanders by controlling aircraft and weapons employment in Close Air Support (CAS) environments.
JTACs are typically co-located with Army units, however, they are required to communicate and
collaborate with a number of personnel operating external to their locations. JTACs are required to
maintain a significant level of proficiency by regularly training in both live and simulation environments.
Due to increasing reductions of aircraft to aide in live training events and limitations in simulator
technology, trainings gaps have arisen that hinder the opportunities for JTACs to achieve required levels of
proficiency. This paper will introduce an ongoing effort to create a robust JTAC training environment ‒
the Joint Terminal Attack Controller-Training Rehearsal System (JTAC-TRS). The JTAC training gaps are
being assessed and explored within the JTAC-TRS using problem-based learning approaches by analyzing
the Mission Essential CompetenciesSM (MEC). Using MECs, we have identified the primary and
supporting competencies, knowledge, skills, and developmental experiences that a JTAC must have to
effectively execute the mission. Preliminary evaluations of this system demonstrate that the JTAC-TRS
has reduced 50% of these training gaps. We will present data regarding the identified training gaps and
how they are addressed in this unique training environment.
37
OPERATOR QUALIFICATION DIFFERENCES BETWEEN MANNED AND UNMANNED
AERIAL SYSTEM (UAS)
Jennifer Pagan, Randy Astwood &Henry Phillips
Naval Air Warfare Center Training Systems Division (NAWCTSD)
Orlando, FL
Currently, no empirically validated qualification standard exists for selecting Naval Unmanned Aerial
System (UAS) operators (Howse, 2011). Some UAS platforms (e.g., Triton, Fire Scout, and Predator)
require their operators be winged aviators. This involves a $1 million investment per pilot and years of
pilot training, in addition to mandatory, UAS platform-specific training (Cohn, 2012). The Shadow UAS
program, on the other hand, uses junior to mid-grade enlisted personnel with no aviation experience. The
training program for Shadow pilots is 10 weeks long and approximately a third of the investment (about
$347,000) of manned aviators (Cohn, 2012). While adapting a Shadow-like selection/training model could
yield significant cost avoidance, thorough research is necessary to develop qualification and training
standards that support identification of the most qualified people to operate UAS and who will be most
likely to succeed in training and operations (i.e., select the right individuals capable of acquiring these
UAS specific skill sets). These differences in standards may be driven more by the relative size and cost of
different UAS platforms rather than by empirical comparison of the Knowledge, Skills, Abilities, and
Other personal characteristics (KSAOs) underlying performance in each (Howse, 2011). This paper
describes differences between KSAOs required to operate manned and unmanned platforms, possible
reasons underlying those observed differences, and implications of the observed trends for selection
criteria, training requirements, and system design.
THURSDAY, 4 DECEMBER, 2014 ROOM S320C
EC-8 More Braaiinnss
0830
Videogame Design for Cognitive
Enhancement through MicroPuzzle Cognitive Profiling (14039)
0900
Game-based Training to Mitigate
Three Forms of Cognitive Bias
(14180)
0930
Enhancing Intuitive Decision
Making Through Implicit
Learning (14253)
Notes
38
VIDEOGAME DESIGN FOR COGNITIVE ENHANCEMENT THROUGH MICRO-PUZZLE
COGNITIVE PROFILING
Patrick S. Gallagher, PhD
Serco in support of Advanced Distributed Learning
Alexandria, VA
Shenan Prestwich
Katmai in support of Advanced Distributed Learning
Alexandria, VA
The Advanced Distributed Learning (ADL) Initiative’s Next Generation Learner researchers previously
investigated whether five video game design features hypothesized to be contained within Portal 2 might
increase cognitive adaptability (CA). Their results highlighted a lack of understanding of the cognitive elements
of video games within the literature. Subsequently, a protocol for applying cognitive task analysis (CTA) to
video games was developed and a CTA was performed on Portal 2 to understand the cognitive components,
decisions, and knowledge needed for successful gameplay, as well as to gain a detailed understanding of its
design. As a result of the CTA, a compendium of within-level tasks and puzzles the player must complete,
referred to as “micro-puzzles,” was compiled, and mapped to the five design features for CA. Results from the
initial study showed that certain measures of CA were increased in those playing Portal 2; however, the design
of Portal 2 was treated as a “black box.” Through performing a CTA, the presence of the five design
characteristics for adaptability was validated by location and by micro-puzzle. Although precisely identified and
mapped by game location, there were no specific alignments identified between cognitive measures and micropuzzle attributes, or between micro-puzzle typology and design feature support. For this reason, the researchers
are cognitively codifying micro-puzzles in Portal 2 by type according to their measurable cognitive attributes.
This involves defining the micro-puzzles and mapping them to cognitive skills, measurable by the CANTAB
battery of tests for CA, followed by empirical testing in the game environment. This paper details this
codification and mapping, as well as efforts to build levels in Portal 2 based upon this information in order to
cultivate specific cognitive skills, empirically validate the correlation of puzzle type in-game to cognitive gains,
and further validate hypothesized game design features to improve cognitive functioning.
GAME-BASED TRAINING TO MITIGATE THREE FORMS OF COGNITIVE BIAS
Benjamin A. Clegg, Rosa Mikeal
Martey, James E. Folkestad
Colorado State University
Fort Collins, CO
Adrienne Shaw
Temple University
Philadelphia, PA
Jennifer Stromer-Galley
Kate Kenski
Syracuse University
University of
Syracuse, NY
Arizona
Tucson, AZ
Joanna E. Lewis
University of
John D. Patterson
Central Florida
Binghamton University
Orland, FL
Binghamton, NY
Tobi Saulnier
Elizabeth McLaren
1st Playable Productions
Troy, NY
Tomek Strzalkowski
University at Albany
Albany, NY
Cognitive biases are systematic errors that result from reliance on heuristics in decision-making. Such biases are
typically automatic and unconscious influences on behavior, and can occur in a wide range of situations and
contexts. Cognitive biases are generally resistant to mitigation training. This project adopted a novel approach to
develop computer game-based training to attempt to mitigate three forms of cognitive bias: fundamental
attribution error, the tendency to assume dispositional rather than situational influences account for behavior of
others; confirmation bias, the tendency to seek and remember information that matches or supports one’s view;
and bias blind spot, the tendency to regard one’s own decisions as being free from cognitive bias, even where
one can recognize that bias in others. Participants were randomly assigned to play the training game once, or
repeated twice with a 7-10 day delay between sessions (mean duration first play=43 minutes; second play=34
minutes), or to a control condition that employed a 30-minute professionally developed training video. Effects of
training were measured on external questionnaire-based items, both immediately post-exposure, and at an 8week retention interval. The game was intended to develop conceptual understanding of these biases, and
recognition of circumstances within which they might occur. Using notional “tools” presented within the game,
participants learned and practiced strategies to avoid decision-making influenced by the cognitive biases.
Results showed that the training game successfully reduced bias on the assessment instrument, and
outperformed the video both immediately post-training and at the retention test. Repetition of the training game
did not further advantage immediate post-test performance but significantly improved retention. Validation of
the key findings was confirmed by an independent group who used the training game with their own novel bias
assessment instruments (to which the researchers and game-developers had no access or content information).
39
TOWARDS ENHANCING INTUITIVE DECISION MAKING THROUGH IMPLICIT TRAINING
Peter Squire,
Joseph Cohn
Office of Naval
Research
Arlington, VA
Denise Nicholson,
Margaret Nolan
MESH Solutions,
LLC
Orlando, FL
Paul J. Reber,
Delphine Oudiette
Northwestern
University
Evanston, IL
James
Niehaus
Charles River
Analytics
Boston, MA
Alexandra
Geyer
Aptima, Inc.
Boston, MA
Liz O’Neill
Strategic
Analysis, Inc.
Arlington, VA
A recent study published by the National Academies focuses on improving decision making (DM) abilities
of small unit leaders, underscoring the significant weight that senior military leadership assigns to the art
of training effective DM. DM training is often based on an analytical model which requires a methodical,
step-by-step, time consuming approach to sequentially process data. While this model is appropriate for
many military decisions, an interesting outcome from military operations in Iraq and Afghanistan has been
the degree to which intuitive decision making (IDM), which uses a more holistic approach to processing
information at a subconscious level, has been cited as playing a critical role in saving lives and enabling
mission success.
IDM offers distinct advantages during ambiguous military missions. For example: a leader may be forced
to make a time-critical decision for which he can neither afford to wait for detailed, quantitative data, nor
analyze new information without risking the tactical initiative. Nevertheless, the processes underlying
analytical DM have traditionally been viewed as more amenable to training than those which underlie
IDM. Yet, a growing body of results, ranging from biological to cognitive, suggests that IDM uses some of
the same underlying neurocognitive structures that are affected by implicit learning, a type of nonconscious learning that occurs through repeated interactions with an environment.
In this paper we propose that IDM may be enhanced through a novel regimen that enables acquiring
domain knowledge implicitly. We motivate the theory that targeted, implicit training automatically
strengthens, at the neural, cognitive, and behavioral levels, the same capabilities that are needed for
effective IDM. We also provide a framework for testing and implementing this theory. The results from
this work will advance the body of research in understanding IDM processes and inform and direct
successful training strategies to develop IDM training for military leaders.
THURSDAY, 4 DECEMBER, 2014 ROOM S320A
EC-9 Medical Matters
1030
Challenges to Upgrading a Mobile
Web Application (14057)
1100
Development and Evaluation of a
Humeral Head Intraosseous
Training System (14058)
1130
Comparative Analysis of
Holographic Display and ThreeDimensional Television (14230)
Notes
40
CHALLENGES TO UPGRADING A MOBILE WEB APPLICATION
Howard Mall
ECS, Inc
Orlando, FL
Teresita Sotomayor, PhD
Army Research Lab
Orlando, FL
The Army Research Laboratory (ARL), Human Research and Engineering Directorate, (HRED),
Simulation and Training Technology Center (STTC) developed a mobile web application for conducting
research in applying emerging mobile capabilities to the U.S. Army's Combat Medic curriculum. The
mobile application used early Apple iOS devices and their native web browsers to deliver highly
interactive training content. It consisted of a web server that delivered the application to mobile devices via
Wi-Fi wireless internet connections. Students would play group trivia games or answer questions about
emerging medical scenarios that included a visual synthetic casualty. A training effectiveness evaluation
was conducted to assess how introducing this system into a program of instruction would improve
individual learning outcomes. Lessons learned identified the need to update the application. This paper
outlines the challenges and solutions that were addressed in updating the mobile application to take
advantage of the strides made in mobile web capabilities. Application server technologies and web client
development libraries have matured and become highly capable in terms of visual fidelity and usability.
The mobile devices themselves now support multi-core Central Processing Units, Graphic Processing
Units for rendering, highly optimized web browsers, and greater resolution screens that sometimes eclipse
their desktop counterparts. We will describe our strategy for upgrading the mobile application to take
advantage of the new technologies especially with regard to the simulation and visualization of the
synthetic casualties in the scenario exercise portion of the mobile application. We will walk through our
decision process and describe the lessons learned during the upgrade. We conclude with a set of guidelines
for other groups taking on the task of upgrading an older mobile web application to take advantage of the
myriad and ever-expanding possibilities that mobile devices afford in delivering important simulationbased curriculum to our warfighter and to education in general.
DEVELOPMENT AND EVALUATION OF A HUMERAL HEAD INTRAOSSEOUS TRAINING
SYSTEM
Angela M. Alban, Cheryl Coiro
Teresita M. Sotomayor, Ph.D.
SIMETRI, Inc.
U.S. Army Research Laboratory HRED-STTC
Winter Park, Florida
Orlando, Florida
Over the past few years, the British Medical Emergency Response Team (MERT) and U.S. Air Force
Search and Rescue Unit (also known as PEDRO) have been administering fluids to patients at point of
injury and en route through the use of intraosseous (IO) devices in the humeral head. The MERT includes
an Emergency Medicine residency trained physician. The PEDRO includes pararescue trained medical
providers who are afforded the opportunity to train on cadavers prior to deployment. The U.S. Army
Center for Predeployment Medicine (CPDM) at Fort Sam Houston, Texas provides medical training to
providers of all levels. CPDM currently does not have an adequate training model for the humeral head
intraosseous device. The U.S. Army Research Laboratory (ARL) executed a Small Business Innovative
Research (SBIR) initiative to analyze the scientific, technical, and commercial merit, and feasibility of
using a low-cost medical simulator for training medical personnel in Army Combat Training Schools. As
part the initial phase, ARL conducted research and developed a capability to fill the gap in training this
procedure. The research focused on identifying innovative technologies, technical risks of the approach,
costs, and benefits associated with development and demonstration of the prototype. Additionally, a
usability study was conducted with emergency medicine residents to gather feedback and assess whether
the initial prototype met training requirements. This paper will discuss in detail how training requirements
impacted the design of the humeral head intraosseous training system. It will also explore the criteria used
to develop the overall design, as well as the identification of specific capabilities. In addition, it will
explain how subject matter expertise was utilized to develop requirements and performance metrics used to
evaluate the feasibility of the concept. Finally, it will review results from usability evaluations and lessons
learned from the development and implementation of this project.
41
COMPARATIVE ANALYSIS OF HOLOGRAPHIC DISPLAY AND THREE-DIMENSIONAL
TELEVISION
Matthew Hackett; Kevin Fefferman
ARL-HRED STTC
Orlando, FL
Data visualization is a key component in a variety of high-impact fields: medicine, engineering,
architectural design, intelligence, and many others. Current sensors used in these fields record multidimensional data sets, such as light detection and ranging (LiDAR) sensors, magnetic resonance imaging
systems (MRI), and three-dimensional (3D) cameras. While these communities have a plethora of sensors
to create data sets, the visualization of these data sets is lacking. The most common display modality is
two-dimensional (2D), despite having data sets representing 3D geometries. Furthermore, additional
dimensions such as time or a force measurement must be displayed in many situations. When using a 2D
display, these additional dimensions must be compressed, or they are simply not displayed. The use of a
3D display alleviates many of these issues, by presenting the additional dimension naturally. A number of
3D display modalities are present in the market, with various strengths and weaknesses inherent in their
designs. In this study, we compare a commercial 3D television which is a time-multiplexed stereoscopic
display and an autostereoscopic holographic display. Participants in the study completed two tasks: a
medical task and a tactical task. The tasks required them to identify certain landmarks in each data set,
such as the tallest building or a particular anatomical structure. After the tasks, researchers gathered data
on usability, visual perception, and cognitive load using the displays. Performance metrics for the medical
and tactical task were also collected. The paper reports the study results and discusses the merits of the 3D
display modalities, including recommendations of suitable use cases for both.
EC-10 Get a Grip on Reality
1030
Live Augmented Reality Based
Weapon Training for Dismounts
(14093)
1100
Augmented Reality Virtual Personal
Assistant for Training,
Maintenance, and Repair (14031)
1130
PERLS: An Approach to Pervasive
Assistance in Adult Learning
(14335)
Notes
42
LIVE AUGMENTED REALITY BASED WEAPON TRAINING FOR DISMOUNTS
Supun Samarasekera, Rakesh Kumar, Zhiwei Zhu,
Frank Dean, Pat Garrity
Vlad Branzoi, Nicholas Vitovitch, Ryan Villamil
U.S. Army Research Laboratory, Human Research and
SRI International
Engineering Directorate, Simulation and Training
Princeton, NJ
Technology Center
Orlando, FL
Current small arms training is limited to either live ammunition training, which is both expensive and
limited to specific ranges and times, or laser based Tactical Engagement Simulation System (TESS)
training which has some potentially negative training implications, since it has no visual blast effects, does
not simulate time of flight and leading of moving targets and does not work for Non-Line Of Sight
(NLOS) scenarios. Additionally, live ammunition training is limited to Force on Target training with
extremely limited scenarios (no movers, same old targets); while laser based TESS training does allow
Force on Force training it is limited by the scheduling of exercises, range time availability and limitation of
scenarios possible with live forces. In this paper, we present technical algorithms, system description and
experiment results for a prototype Augmented Reality (AR) based system that addresses the limitations of
both live ammo and laser TESS training. The AR based dismount weapon training system provides fully
geo-located 6-degrees of freedom orientation and location of the weapon and of the trainee operating it,
thus allowing the weapon to fire simulated projectiles for both direct fire and NLOS during live training.
Using this tracking and a terrain model of the environment, the fully virtual projectiles and synthetic
enemies are displayed on the trainee’s head mounted display overlaid on top of the real world and full blast
effects and simulated damage are displayed allowing the soldier to adjust fire accordingly. Since the
projectiles, weapon characteristics and enemy combatants are all simulated they can easily be changed to
vary scenarios, new projectile types and future weapons. Additionally, turn-around time on scenarios is
very quick allowing more training in shorter amount of time in any available live environment, without the
need to schedule range time, Opposition Forces, training ammo or any other logistical requirements.
AUGMENTED REALITY VIRTUAL PERSONAL ASSISTANT FOR TRAINING,
MAINTENANCE, AND REPAIR
Rakesh (Teddy) Kumar, Supun Samarasekera, Girish Acharya, Louise Yarnall, Zhiwei Zhu,
Michael Wolverton, Vlad Branzoi, Glenn Murray, Nicholas Vitovitch, Ryan Villamil, Jim Carpenter
SRI International
Princeton, NJ
The military trains a large pool of personnel skilled in maintaining and repairing a variety of complex
equipment. The U.S. Army itself requires personnel for more than 130 different Military Operational Skills.
Often these trained personnel are not available for repair and maintenance of critical low density equipment in
deployed locations. Augmented Reality and Virtual Personal Assistance are technologies that can supplement
live training to address the challenge of affordably training personnel.
In this paper, we present the system design, hardware, algorithms and initial field results for a prototype training
system AR-Mentor. The system is designed to act as a personal mentor to a user, providing human-like
understanding and guidance. It provides a Heads-up and Hands-free experience. The user can train anywhere
and also use the system for providing guidance during actual maintenance of the equipment.
The experimental system consists of a compact computer, head worn cameras, microphone, ear-buds and
eyewear. Virtual Personal Assistant technology is used to provide a real-time dialog and reasoning system that
supports human-like interaction using spoken natural language. The reasoning system aims to recognize the
user’s intent and provides feedback to the user. The feedback and interaction occurs both verbally and by
engaging the Augmented Reality system to display icons and instructions visually on the user’s eye-glasses. The
inserted visual objects appear as part of the live scene and are precisely aligned to the equipment.
A formative evaluation indicated that the AR-Mentor system permitted individual learners to focus on their
learning needs and reduced the perceived mental demand of learning the procedure. Checks into understanding
showed no difference between learning with the AR-Mentor system, as compared to learning from an instructor,
or a technical manual. The evaluation also indicated the need for alternative ways to design the AR-Mentor
representations around complex procedural steps.
43
PERLS: AN APPROACH TO PERVASIVE PERSONAL ASSISTANCE IN ADULT LEARNING
Michael Freed, Louise Yarnall, Jason Dinger, Melinda Gervasio, Adam Overholtzer, Mar Pérez-Sanagustin,
Jeremy Roschelle, Aaron Spaulding
SRI International
Menlo Park, CA
Adult learners in both military and civilian settings increasingly use mobile devices for “Pervasive
Learning” (Banavar et al., 2000; Thomas, 2007), which occurs without classrooms, instructors, and
training facilities. By expanding options for what, when, and how we learn, Pervasive Learning has the
potential to remedy stubborn deficiencies of traditional instruction. The central feature of PERLS is a
virtual personal assistant that supports selflearning by recommending specific content, general topics, and
various learning actions based on learners’ interests, available time, and location. PERLS is intended to
guide learners to resources located in both formal (closed corpus) and informal (open corpus) repositories.
In this paper, we present the pedagogical design, user interface, system architecture, initial concept
validation results, and field test goals for PERLS, a prototype PERvasive Learning System. The concept
validation and field-testing take place in one civilian corporate context. The concept validation indicated
that adult learners in the corporate setting favored limited use of “push” reminders to engage in learning
and broader use of adaptive lists of content that have been intelligently informed by contextual data about
their interests and available time for learning. Planned field tests will examine system functionality,
usability, and impacts on self-learning habits around corporate onboarding content for new hires.
EC-11 Networking Cross Domain Solutions: Cross Domain Solutions, Cloud
1330
Implementing Stateless Cross
Domain Solutions to Continuously
Maintain Security Assurances
(14301)
1400
Cybersecurity Impacts of a Cloud
Computing Architecture in
Live Training (14120)
1430
Using Social Network Analysis to
Model the Spread of
Misinformation in Simulated
Notes
44
IMPLEMENTING STATELESS CROSS DOMAIN SOLUTIONS TO CONTINUOUSLY
MAINTAIN
Christopher Huey, CISSP, OSCP
Kelly Djahandari, CISSP, Charles Kristofek
Parsons Corporation
Northrop Grumman Information Systems
Orlando, Florida
Orlando, Florida
The Combat Air Force (CAF) Distributed Mission Operations (DMO) Network uses cross domain
solutions to interconnect networks operating at different security domains during scheduled training
events. Implementation of cross domain solutions on the DMO Network helps achieve the CAF DMO
vision of "train like we fight” to keep our air crews mission-ready. There is an increased risk that
networked systems function with degraded security assurances caused by the system's continuous exposure
to harmful actions such as inadvertent installation of malware (viruses, rootkits, spyware, worms),
unauthorized changes made to the system security configurations, and unintended introduction of
exploitable vulnerabilities caused by system users. Although cross domain solutions are designed to be
resistant to attacks, it is prudent to ensure the system is operating in a known secure state to reduce the risk
of exploitation of unidentified vulnerabilities. Maintaining a high level of security assurance is critical
since cross domain solutions must be trusted to correctly and consistently adjudicate the release of data
between two different security domains according to established security policies. A cross domain solution
that consists of a read-only disk, no hard drive, and lacks writable non-volatile storage (stateless system)
will provide an innovative and cost-effective approach to greatly reduce the risk of degraded security
assurances. This paper discusses the need for evolution to a stateless cross domain solution and explains
how a stateless cross domain solution can be implemented to maintain the required security assurances
throughout system operations. This paper provides the technical and accreditation challenges associated
with implementing a stateless cross domain solution. Finally, this paper describes potential solutions for
mitigating security risks associated with stateless system implementations.
CYBERSECURITY IMPACTS OF A CLOUD COMPUTING ARCHITECTURE IN LIVE
TRAINING
Graham Fleener
U.S. Army PEO STRI
Orlando, FL
Dr. Cliff Zou
Orlando, FL
Jason Eddy
AIT Engineering
Orlando, FL
Today’s live training environment is comprised of many systems in various states of configurations with a
limited ability to leverage shared services. The future of live training systems will evolve to a Training as a
Service (TaaS) state to reduce overall operating costs, implement new technologies to improve the training
experience, and centrally manage the training exercise of distributed training systems. With a TaaS
approach to system architecture, a number of new cybersecurity and DoD Information Assurance
requirements will need to be implemented in order to ensure the Confidentiality, Integrity, and Availability
of DoD information Systems. Previous papers (Lanman and Linos, 2012) have outlined in greater detail
the motivation and migration strategy for a pilot study on implementing TaaS within the Common
Training Instrumentation Architecture (CTIA) used by the Army’s Live Training Transformation (LT2)
Product Line. This paper will present a number of cybersecurity threats, challenges, requirements, and
commercial best practices for secure operations as well as Certification and Accreditation (C&A)
requirements of a TaaS approach. Threats not previously present in isolated system architectures will now
need to be countered with appropriate defense mechanisms across physical and logical boundaries. This
paper will describe and discuss cloud computing guidance for cybersecurity from the U.S. Army Chief
Information Officer/G-6 guidance, National Institute of Standards and Technology (NIST), and the
Defense Information Systems Agency (DISA). This paper will present a strategy for implementing
commercial best practices to facilitate secure operations of a cloud computing approach to live training.
Finally, the purpose of this paper is to provide an overview of the security requirements associated with
cloud computing, document the certification process necessary to achieve an Authorization To Operate
(ATO) for a cloud implementation, and discuss unique best practices associated with a PM TRADE
implementation of a TaaS architecture.
45
USING SOCIAL NETWORK ANALYSIS TO MODEL THE SPREAD OF
MISINFORMATION IN SIMULATED ENVIRONMENTS
Paul Cummings
ICF International
Fairfax, VA
Chalinda Weerasinghe
Weerasinghe Research Group
Clearwater, FL
A central question for social interaction is to recognize the circumstances under which exchange of
information will lead to the spread of misinformation (incorrect information) and how misinformation
spread can be stopped. What is unclear is the importance of variables within networks in curtailing the
spread of misinformation. Specifically, if we were trying to stop the spread of misinformation within
certain network types (i.e. clustered, small world, scale-free) what network elements should we consider
most important, given that we may not know where the misinformation is arising from? We pose this
research question: what are the relationships between network types and misinformation spread inventions
types? Using simulated models we find that only in the small world network setting do we see a statistical
difference in the misinformation spread rate among the four intervention types (random placement, and
targeting based on degree centrality, betweenness centrality and closeness centrality). We also find that the
misinformation spread rate for the three network settings is different only in the case of the closeness
centrality targeted intervention type and not in the others types. Next, we apply this model to a virtual
world training scenario under which basic social network principles are taught to help soldiers recognize
how to infiltrate networks that may cause misinformation spread.
TUESDAY, 2 DECEMBER, 2014 ROOM S320F
H-1 The Body Speaks – Stress and Workload
1400
Classifying Stress in a Mobile
Environment (14195)
1430
Investigation of the Sensitivity of
Physiological Performance
and Subjective Measures for Identifying
Changes in Novice
Intelligence Analyst Workload (14035)
1500
Training with Adaptive Systems:
Utility of Baroreflex Sensitivity
(14297)
Notes
46
CLASSIFYING STRESS IN A MOBILE ENVIRONMENT
Sara Dechmerowski, Brent Winslow, George Chadderdon, Tarah N. Schmidt-Daly, David Jones
Design Interactive Inc.
Oviedo, FL
Over half of all Veterans suffer from stress-related illnesses; of particular concern is PTSD. In addition to
supporting post-deployment stress treatment, it is critical to integrate stress inoculation training predeployment to teach proper coping mechanisms and prevent the PTSD cycle from starting. A challenge
with developing such training is the objective, real-time monitoring of stress across trainees. Current
methods for stress monitoring are laboratory-based (not mobile), and episodic in nature (e.g. self report).
Wearable physiological sensors provide a quantitative assessment of stress (such as heart rate variability
and electrodermal activity); however, the main challenge with these technologies is the lack of robust
algorithms to classify stress in a mobile environment in real time. Physiological sensors are often activated
by other inputs such as temperature and physical activity, and individual differences (e.g. age, gender,
health status) and daily activities (e.g., physical movements, environmental changes, caffeine intake) pose
a complex problem in achieving an accurate classifier. A review of several stress monitoring algorithms
published in literature has been conducted and applied to a study designed to collect the high quality data
necessary for modeling and development of a classifier that accurately detects stress in a mobile
environment in real-time. The study procedures, results, and development of this algorithm are outlined,
including use of unobtrusive hardware and robust logic to disseminate between psychological stress and
physical activity. Although the main objective of developing a mobile classifier using non-invasive sensors
to classify stress with over 85% accuracy was achieved, further refinement is needed to maintain the high
level of accuracy across a variety of users and environmental conditions. Future research will include
further accuracy refining through reduction in environmental noise and a smart algorithm to learn
individual user stress thresholds. Applications for this research within the military and others are
discussed.
INVESTIGATION OF THE SENSITIVITY OF PHYSIOLOGICAL, PERFORMANCE, AND
SUBJECTIVE MEASURES FOR IDENTIFYING CHANGES IN NOVICE INTELLIGENCE
ANALYST WORKLOAD
Lisa Tripp, Robert Nelson, Elliot Humphrey, Chad Tossell
WPAFB, OH
Jennifer Winner, Jerred Holt
Lumir Research Institute
WPAFB, OH
The United States Air Force has a vested interest in advancing intelligence, surveillance, and
reconnaissance technologies. Although software and hardware testing is performed for these technologies
to demonstrate functionality, only limited research has investigated the effect of these tools on human
performance. This research describes a process for the identification of suitable metrics to assess the
effectiveness of new ISR technologies. We used several factors to determine the potential suitability of
candidate measures including their relative sensitivity, reliability, content validity, and task intrusiveness.
Additionally, the sensitivity of several measures, including performance-based, physiological and
subjective measures, for the discrimination between levels of difficulty of imagery analyst tasking were
compared. Twenty participants from a school for training intelligence analysts volunteered. Real recorded
footage from two imagery types, wide area motion imagery and full motion video, was presented to
analysts in short video clips. Tasking for each clip was provided prior to viewing. Tasking was developed
by a subject matter expert and validated by five career analysts who independently rated the tasking in
terms of difficulty. Performance data showed a significant difference based on difficulty of tasking as
predicted (F(1,19) = 220.32, p < .001), as did subjective difficulty ratings assessed by the NASA-Task
Load Index (F(1,19) = 12.84, p < .01). The sensitivity of physiological data to difficulty was mixed.
Significant differences based on difficulty rating were identified for fixation duration (F(1, 14) = 5.30, p =
.037) and saccade duration (F(1, 14) = 15.13, p < .01). However, no significant differences were identified
in heart rate or heart rate variability (p > .05).
There were also no significant differences in indices of workload across imagery types. The suitability and
applications of these measures for assessing intelligence analyst performance in simulated analyst
operational
environments
is discussed.
47
TRAINING WITH ADAPTIVE SYSTEMS: UTILITY OF BAROREFLEX SENSITIVITY
Warren D. Franke, Amanda A. Anderson, Nir Keren, Andrew F. Lilja, Kevin M. Godby
Iowa State University
Ames, IA
Significant resources have been invested toward the development of systems that adapt to user functional
state in real-time and based on users’ physiological responses, where the user may be in a wide array of
stressful situations. These adaptive systems are promising as platforms to enhance training effectiveness,
yet progress to date has been somewhat limited.
The physiological responses to a stressful situation have been characterized as “fight-or-flight” or
“challenge vs. threat” responses. The cardiovascular changes associated with these responses are mediated
by the autonomic nervous system and include both central (e.g., heart rate, stroke volume) and peripheral
(e.g., blood pressure, total peripheral resistance) changes. Blood pressure (BP) is modulated acutely by the
baroreflexes. Baroreceptors are stretch-sensitive mechanoreceptors located in the vasculature which
provide negative feedback to the brain; changes in BP change this stretch and ultimately lead to changes in
BP and heart rate (HR). Both physical exercise and mental stress can increase HR and BP. However,
baroreflex sensitivity is unchanged with physical exercise and limited evidence suggests it is altered with
mental stress. Changes in baroreflex sensitivity may therefore provide an objective marker for mental
stress that HR- and BP-based markers cannot. Thus, real-time monitoring of baroreflex sensitivity may be
the missing component for bridging the gap in developing an effective adaptive system.
Consequently, the purpose of this study was to assess the extent to which baroreceptor sensitivity changes
during acute physical stress (cold pressor test), laboratory-based mental stress (Stroop test, mental
arithmetic, anagrams) and using a virtual reality environment, stressful occupationally-relevant “real-life”
simulations.
We will then propose a framework for the utilization of baroreflex sensitivity measures as a tool for
assessing laboratory and occupational stressors in real-time.
TUESDAY, 2 DECEMBER, 2014 ROOM S320F
H-2 Shooting for Effective Virtual Training
1600
A Novel Approach to Determine
Integrated Training
Environment Effectiveness (14011)
1630
Pistol Skill Acquisition and
Retention: A 3-Year Longitudinal
Study (14042)
1700
Effective, Live, Virtual, and
Constructive Training: Personnel
Recovery’s Weeklong Experience
with Integrated Sensor Technology
(14216)
Notes
48
A NOVEL APPROACH TO DETERMINE INTEGRATED TRAINING ENVIRONMENT
EFFECTIVENESS
LTC Glenn A. Hodges
Naval Postgraduate School
Monterey, California
This paper discusses the development and use of an analytical assessment methodology anchored in
systems engineering principles, affordance theory, and human abilities, to measure the potential of
integrated training environments (ITE) to effectively support training. An integrated training environment
is defined here as any human in-the-loop training system that includes live, virtual, constructive or gamebased training aids, devices, simulators, or simulations (TADSS) alone or in combination, used to support
the deliberate practice of skills for defined mission tasks. Empirical investigation of ITE is costly, lacks
formal guidance, and is therefore often unreliable. Ad hoc studies, commissioned by individual
organizations, constitute the current state of Army ITE evaluation. These assessments are often entirely
based on subjective opinions gained through surveys, which produce results that are linked indirectly and
loosely to the ITE. What is required is a repeatable, inexpensive, analytical approach to ITE assessment
that bounds the potential of a given system to the support it provides to the deliberate practice of specific
tasks. The results of this research include the development and use of the integrated training environment
assessment methodology (ITEAM). ITEAM was used to evaluate the ability of several ITE to support the
deliberate practice of specific tasks during training. During application, ITEAM consistently predicted
where training was supported by an ITE and generally how well. ITEAM is offered as a tool to be used
early in the material acquisition process to affordably define and verify the requirements of candidate ITE
solutions for Department of Defense needs.
PISTOL SKILL ACQUISITION AND RETENTION: A 3-YEAR LONGITUDINAL STUDY
Gregory P. Krätzig
Royal Canadian Mounted Police
Regina, Saskatchewan Canada
This paper will build on previous research designed to investigate the effects that pistol training, in a norecoil synthetic environment, has on skill acquisition, the transferability of this skill in situ (Krätzig, Parker
& Hyde, 2011), and the long term implications on skill retention. Krätzig et al (2011) found that live-fire
training was not necessary for pistol skills acquisition nor was it needed when testing occurred in a livefire setting. However, two areas of investigation was missing. The first was to measure performance after
introducing live-fire before each Benchmark test. The second was to investigate the long term effects that
this type of training has on pistol shooting skill retention (e.g., annual firearms recertification). This paper
will present three follow-up lines of research. Twohundred and fifty-six Cadets [i.e., 128 Cadets (control
group) vs. 128 Cadets (experimental group)] were used for this study, with ninety-six Cadets being trained
in the synthetic environment. The three lines of research are: 1) The effects of this training on skill
retention in the field for three consecutive years. 2) Will adding live-fire training before each test improve
performance and what effect does this have on skill retention in the field? 3) Results of a replication study.
Although evidence was found that skills transfer from a synthetic training environment to a live-fire setting
(Krätzig, et al., 2011), unknown were the long term effects on skill retention, and as such results of a 3year longitudinal study will be reported. Additionally a potential confound was identified following the
Final Benchmark Test from the Krätzig et al (2011) study. Krätzig argued that because the first time these
Cadets fired a live-fire weapon was during their Benchmark tests, that the unfamiliarity of the recoil,
concussive blast, noise, etc resulted in lower scores and an increase in the failure rate during the
Benchmark tests. In an effort to determine if overall performance could be improved, it was decided that
live-fire training would occur preceding each Benchmark test. These results will be discussed in full. There
is increasing evidence that supports integrating technology into the basic 24-week training program;
however, it was important to replicate the Krätzig et al., (2011) study. The third area of discussion will
present the results of the replication study.
49
EFFECTIVE LVC TRAINING: PERSONNEL RECOVERY’S EXPERIENCE WITH
INTEGRATED SENSOR TECHNOLOGY
Curtis Wray, Mark Speed and Timothy Rodabaugh
Ball Aerospace & Technologies Corp
Fairborn, Ohio
Ms. Kristen Barrera
Warfighter Readiness Research Division 711 Human
Performance Wing Wright-Patterson AFB, Ohio
Special operations training presents a unique challenge for exploring the application of live, virtual, and
constructive (LVC) training opportunities. Personnel Recovery (PR) operators (Pararescue, Combat
Rescue Officers, and Survival, Evasion, Resistance and Escape) have specialized and widely varying
training needs placing unique requirements on the training system. The objective is to carry out meaningful
training and therefore the training system itself must be flexible and minimally invasive to the trainee.
Present training of PR operators typically does not exploit modern training systems. By augmenting the PR
operators training with LVC technology and rich time-synchronized after action review capabilities,
training can be far more effective. The Air Force Research Laboratory has extensive experience in research
and development (R&D) for fast jet LVC. This paper discusses AFRL’s efforts to extend LVC
development to the PR domain. The LVC Sensor Integration for Data Fusion in Operations and Training
(SIDFOT) effort presents unique challenges for instrumenting live participants, collecting physiological
metrics, observing and collecting data for the training exercise and robustly supporting after action review.
This paper summarizes the recent efforts in integrating PR training into an LVC research infrastructure.
The paper is divided into sections that cover newly integrated LVC technology, after action review, the PR
training scenario and method, and concludes with results from the final PR demonstration. Each training
scenario was performed in two parts, one with and one without the technology augmentation. By
comparing the results, the authors were able to assess the overall value of LVC technology augmentation
to training. Effectiveness results, with an emphasis on after action review, are further broken down into
categories exposing which LVC capabilities and technologies yield the greatest gains in training
effectiveness.
WEDNESDAY, 3 DECEMBER, 2014 ROOM S320F
H-3 A Cornucopia of Human Behaviors
0830
In Search of Interoperability
Standards for Human Behavior
Representations (14027)
0900
Factors Impacting Performance in
Competitive Cyber Exercises
(14108)
0930
Cognitive Processing Considerations
of the Small Unmanned
Ground Vehicle (14134)
Notes
50
IN SEARCH OF INTEROPERABILITY STANDARDS FOR HUMAN BEHAVIOUR
REPRESENTATIONS
Glenn Gunzelmann,
Chris Gaughan
U.S. Air Force/Army
Research Laboratory, USA
Wim Huiskamp,
Karel van den Bosch
Steven de Jong
TNO, The Netherlands
Thomas Alexander
FKIE Human Factors,
Germany
Agostino G. Bruzzone,
Alberto Tremori
DIME University of
Genoa, Italy
There is a long history of research to create capabilities that address the need for human behaviour
representations in training simulations and other M&S application domains. In training, human behaviour
models have applications as synthetic teammates and adversaries, but can also be used as a representation
of the state of the trainee and as synthetic instructors to increase the effectiveness of the training enterprise
as a whole. They are essential components for achieving the goals for training simulations and for Live,
Virtual, Constructive (LVC) training, including affordability, availability, and credibility. Over the last two
decades, numerous formalisms and architectures for modelling cognition, performance, and other relevant
characteristics of the human being have emerged, and the capabilities and applications have expanded
dramatically. However, models vary along many dimensions, including fidelity, application domain,
underlying modelling formalisms, and behavioural repertoire. This diversity leads to critical challenges
with respect to interoperability and reuse, in particular the integration of component models into a
comprehensive behaviour model, and the integration of behaviour models into simulation environments.
The challenges are further complicated by a lack of standards for human behaviour modelling, leading to
brittle models, lack of reusability, and increased costs driven by the requirements of model integration and
reengineering. In this paper, we discuss the need for human behaviour modelling, its role in supporting
affordable, available, and credible training experiences in simulation and LVC environments, and propose
a reference architecture to enable interoperability standards that support a variety of models serving a
diverse set of purposes, both within and beyond the training domain. The authors represent a North
Atlantic Treaty Organization (NATO) Modelling and Simulation Group (NMSG) activity focused on
developing a baseline reference architecture and interoperability standards for human behaviour modelling
to facilitate the creation and integration of human behaviour representations into simulation.
FACTORS IMPACTING PERFORMANCE IN COMPETITIVE CYBER EXERCISES
Austin Silva, Jonathan McClain, Theodore Reed, Benjamin Anderson, Kevin Nauer, Robert Abbott
& Chris Forsythe
Sandia National Laboratories
Albuquerque, NM
Many opportunities are available for training that involves participation as either individuals or teams in
competitive events. Cyber security has proven conducive to this form of training. In competitive cyber
security exercises, participants are usually provided with standardized hardware and software, including
various software tools for cyber forensic analysis. Generally, performance is assessed on the basis of
points awarded for completing challenges presented to the participants. Ideally, through thorough
instrumentation of the software environment, instructors and test coordinators would be provided with
detailed data concerning the performance of individual students, as well as their unique training needs. The
research described here provides an illustration of such instrumentation implemented within the context of
a competition-based cyber security exercise (Tracer FIRE). The study considered factors that contributed
to successful performance within the competition. Emphasis was placed on the use of software tools by
participants, including tools provided by the exercise coordinators and tools acquired online by participants
during the event. Resulting findings provide the basis for recommendations to competition coordinators
regarding key facets of the software environment and cues that individual participants are struggling and
there is need for training intervention.
51
COGNITIVE PROCESSING CONSIDERATIONS OF THE SMALL UNMANNED GROUND
VEHICLE
Victor J. Ingurgio
ARI-Fort Benning
Fort Benning, GA
Richard Catrambone
Georgia Institute of Technology
Atlanta, GA
Richard L. Wampler
Northrop-Grumman Corporation
Columbus, GA
Handheld applications (apps), such as those run on Android and iPhone devices, hold the possibility of
revolutionizing military training by increasing the availability and engagement of training material. This
paper describes progress on software design and development towards a general framework for deploying
Android training apps. A primary objective is to allow nonprogrammers to reuse existing content to create
training apps that make full use of the capabilities offered by mobile devices. The described prototype
implementation includes a web page where the end user fills out a form, uploads content, and receives an
email with a link that they can follow (and share with others) to download their app directly to their device.
The main contributions of this paper are: The requirements that led to the framework design, the
description of the implemented framework, and a summary of qualitative feedback received from targeted
demonstrations. While this framework has been developed with a focus on military training, it is broadly
applicable in a civilian educational setting as well.
H-4 Plane, Train and Automobile
1030
Leveraging Simulation to Augment
Risky Driving Attitudes
and Behaviors (14004)
1100
Simulation-based Analysis of the
Human Factors related to
Autonomous Driving (14006)
1130
The Virtuous Circle and
Contextualized Knowledge
Elicitation: Application of a New
Paradigm for Job Analysis (14347)
Notes
52
LEVERAGING SIMULATION TO AUGMENT RISKY DRIVING ATTITUDES AND
BEHAVIORS
Karen L. Morris
Gregory A. Fabiano
Kevin F. Hulme
Center for Children and Families
Counseling, School, and Educational Psychology
NYSC EDII
Buffalo, NY
Buffalo, NY
Buffalo, NY
Young, novice drivers continue to be responsible for a disproportionate amount of negative driving outcomes. A
novice driver’s lack of: i) exposure, ii) semantic knowledge of driving situations, and iii) understanding of risky
situations make them particularly vulnerable to costly mistakes while driving. Texting while driving (TWD) is a
behavior commonly engaged in by novice drivers that greatly increases the risk for accidents, injuries and mortality.
Cell phone use while driving causes deficits in performance (e.g., impaired attention to signs; braking and lane
positioning deficits due to visual, motor, and cognitive distraction). While many drivers recognize that TWD is a
serious problem, many admit to engaging in the behavior frequently. Studies have demonstrated that drivers perceive
their own distracted driving performance to be better than their actual performance. This suggests that drivers may be
engaging in dangerous behavior because they believe it affects the driving performance of others, but not their own.
This is a major ongoing public health concern. Past studies have suggested that receiving concrete performance
feedback can correct perceptions of risk of driving while engaging in a distracting task, and improve subsequent
driving performance. In this regard, Simulation can serve an effective tool for Education and Training. Accordingly,
the current study leverages a high fidelity driving simulator to provide performance feedback for a pilot cohort of
novice adult drivers while driving distracted. The primary goal is to change attitudes towards and subsequently reduce
TWD behavior with the use of the performance feedback during the simulated TWD exercise. Along with the
simulator-acquired data and graphs (e.g., speed, lane position), TWD behaviors are measured objectively with a
performance monitoring “Car Chip” device installed within each participant’s vehicle during the study observation
period. Car Chip records including the dates and times of each participant’s drives are compared against each
participant’s text messaging records (containing dates and times of texts sent or received) for any overlap, objectively
measuring in-vehicle TWD behavior. The current study presents a novel approach for evaluation and intervention to
reduce distracted driving behaviors specifically for the most at-risk driving population.
A VALIDATED AND INTEGRATED SIMULATION FRAMEWORK FOR HUMAN FACTORS
ANALYSES
Yunfei Hou, Jingyan Wan, Yunjie
Zhao
University at Buffalo, Buffalo, NY
Kevin F. Hulme
NYSCEDII
Changxu Wu, Adel Sadek,
Chunming Qiao
Transportation simulation researchers commonly institute two distinct simulation platforms that are often
implemented independent of one another. Traffic Simulation models emulate the macroscopic or mesoscopic
behavior of ground vehicles, while Driving Simulators are used to examine microscopic driver behavior within a
virtual environment. This research sees the integration of these heterogeneous simulation platforms, which
broadens the range of applications for which both simulator types are applicable. The integrated simulation
framework has been validated by having several human subjects drive a segment of a signalized arterial in both
the artificial environment and on the corresponding real-world roads, during (simulated and actual) rush hour
traffic. Various data is collected within the integrated simulation framework, including timestamp, position,
velocity, and accelerations, and comparable data is collected (and compared) when the human subjects drive the
actual roads. The described framework is then deployed to focus on Human Factors (e.g., driver acceptance and
preference) associated with autonomous control features anticipated in next-generation vehicles. In our
experiments, participants were asked to assign the headway to a minimum value that they could “tolerate” (i.e.,
based on workload, confidence, comfort, safety and acceptance). The results demonstrate that most drivers
prefer spacing between vehicles by relying on their judgment on distance, rather than headway (time). Future
technology will be able to support autonomous vehicle operations, most likely with an evolving trajectory of
acceptance, and the human factors element of accepting the technology may lag the deployment of the
technology itself. Accordingly, simulator-based efforts to identify human tolerances on the roads have the
potential to help to accelerate the adoption of these advanced autonomous technologies. This is the primary
motivation for this study, which will help to inform the design of future autonomous vehicle applications, and
will serve as a reference point for optimizing the route capacity of next-generation transportation systems.
53
THE VIRTUOUS CIRCLE AND CONTEXTUALIZED KNOWLEDGE ELICITATION:
APPLICATION OF A NEW PARADIGM FOR JOB ANALYSIS
Sterling Wiggins, Michael J. Keeney, Webb
Jennifer Pagan
Stacy, Jeffrey M. Beaubien
Naval Air Warfare Center,
Aptima, Inc.
Training System Division
Woburn, MA
Orlando, FL
Amy Bolton
Office of Naval Research
Arlington, VA
Landing on the deck of an aircraft carrier is one of the most difficult and dangerous tasks that a Naval
aviator must perform. Maritime Augmented Guidance with Integrated Controls for Carrier Approach and
Recovery Precision Enabling Technologies (MAGIC CARPET) is a technology intended to make carrier
landing performance better and safer, and to reduce pilot workload. The workshop described in this report
included a set of contextualized knowledge elicitation sessions that provided preliminary multi-faceted
evidence that MAGIC CARPET leads to improved performance during landings and to lower participant
workload. Contextualized knowledge elicitation collects a combination of system-generated data and selfreports together in real-time. The combination of system-generated data and self-reports can provide cues
to guide investigations about perceptual-cognitive skills required for successful task performance in highconsequence environments. Self-reports can highlight where to look within voluminous system data to
gain insights about operator or system performance. Similarly, system data can indicate where verbal
follow-ups can provide additional context about the objective data that were collected. The workshop was
held at the Manned Flight Simulation operation at Naval Air Station Patuxent River. Six Naval Aviator
participants, representing various F/A-18 experience levels, plus two non-pilot engineers used the
simulator to conduct carrier landings using both conventional and MAGIC CARPET technology.
Participant and Landing Signal Officer (LSO) reports as well as physiological and simulator-based
measures showed a strong, noticeable, positive effect of MAGIC CARPET on landing performance.
Participants reported dramatically reduced workload and LSOs reported that MAGIC CARPET made lessexperienced participants perform as well as experienced pilots. These subjective participant observations
were borne out by objective measures. Finally, the workshop provided rich information about a future
more formal experiment to assess the impact of MAGIC CARPET on pilots and on the pilot training
pipeline.
H-5 See, Hear and Speak – No Evil Assessments
1400
Serious Game User Data Analysis
and Visualization: Savoring
the Breadcrumbs (14377)
1430
Sonification: The Sound of Big
Training Data (14261)
1500
Creating a Learning Infrastructure
Where Every Soldier Can Be
an Instructor (14124)
Notes
54
SERIOUS GAME USER DATA ANALYSIS AND VISUALIZATION: SAVORING THE
BREADCRUMBS
Brandt Dargue
Boeing Research & Technologies
St. Louis, MO
Dov Jacobson
John Sanders
GamesThatWork
Atlanta, GA
Louisville, KY
Following the evidence of research in critical thinking and cognitive bias training, we developed training
designed to demonstrate people’s decisions and actions affected by cognitive bias or elicit a bias in the
player. Ideas were collected from and refined by experts with diverse backgrounds distributed
geographically using an innovative solution. We then developed a video game that used multiple scenarios
to teach cognitive bias recognition and mitigation. A learning and training effectiveness study indicated
that the game was effective for learning although results were inconsistent for the different cognitive
biases addressed by the game. The game recorded detailed data about scenes, scenarios and decisions each
player made in the game. Called “breadcrumbs”, this data detailed the path every player took. Traditional
statistical analytic techniques tend to be clumsy instruments for breadcrumb analysis. Additionally, early
aggregation and dimension reduction make the data more tractable but less meaningful. This paper details
specific examples of how the breadcrumbs – paired with the study data – provided valuable answers in
pinpointing areas to improve the game’s learning effectiveness. The paper provides enough background
information on the subject to enable the audience to appreciate the difficulty in cognitive bias training
effectiveness and understand the examples shown. The majority of the paper discusses the data, the
analysis, and the innovative data visualization techniques used. We discuss approaches that may prove
more appropriate to extracting useful information from breadcrumb trails than traditional statistical
analytic techniques. The audience will gain an understanding of the value of testing, data collection, and
data visualization in training, education, simulations, and serious games. The paper will conclude with a
discussion on using the techniques to improve the small batch testing in serious game development.
SONIFICATION: THE SOUND OF BIG TRAINING DATA
Nat Napoletano
Lockheed Martin, MST
Akron, OH
Instructor assisted warfighter training requires a system that allows the instructor to monitor individual and
team progress while guiding the students to preparedness. Traditional instructor stations rely heavily on
visualizations of the team’s geographical position and status, while confining the use of sound to voice
communications and warning messages. Our experience of sound is unlike any of our other senses in its
immediate temporal nature and speedy presentation of massive information. Since sound can be used to
create a unique cognitive view of a team’s situation, by underusing sound, we may be discarding an
opportunity to inform the instructor about student performance in numerous domains.
The emerging technology of “Big Data Analytics” offers the promise of hastening and deepening the
warfighter’s training progress while reducing training costs. Static data is most familiar to analysts who are
developing sophisticated tools to extract, analyze and visualize important features. Streaming data is
different in that it emerges in real time, eventually expires and demands immediate attention to be of
optimal use. It’s temporal in a way that resembles sound and the human auditory system is uniquely
positioned to analyze streaming data. Sonification is the technique of turning data into sound.
This paper presents the results of a study examining three computer based sonification tools developed by
the author and designed to render mission activity into sound. The sonifications represent diverse
techniques designed to sonify individual and team metrics for mortalities, damage, weapons skills and
accuracy, fuel status, emergencies and communications usage. This paper demonstrates that the concept is
practical, potentially effective and notes what techniques were superior. This paper reviews the knowledge
gained from this evaluation and presents a framework for future experimentation designed to collect
statistical data from a larger community of users.
55
CREATING A LEARNING INFRASTRUCTURE WHERE EVERY SOLDIER CAN BE AN
INSTRUCTOR
Ed Sims, PhD, Irene T. Boland, PhD
Dan Silverglate, Jeff Cashion
Vcom3D, Inc.
Orlando, FL
Rodney Long, Charles Amburn
US Army Research Laboratory, Human Research and Engineering
Directorate, Simulation and Training Technology Center
Orlando, FL
In recent years, web sites featuring user-created content have become some of the most popular sources of
information for the general population. Users consult Angie's List and Yelp! to locate vendors and
services, DIY Network to learn how to fix or create something, Wikipedia for information on more than
four million topics and numerous LinkedIn communities of practice to enhance their professional skills
and networks. Fourteen of the twenty top web sites are populated in part or entirely by content created by
users. By contrast, the collaborative support networks for warfighters are often fragmented, difficult to
access and navigate and limited in useful content. And when experienced warfighters report lessons
learned in the field, it can take two years or more for these lessons to be reflected in formal training. This
often results in gaps between official training and best practice.
Under the Army Research Laboratory's Soldier-Centered Army Learning Environment (SCALE) program,
a social media test bed was developed to identify crowd-sourcing strategies that can be adopted to ignite a
similar revolution in military knowledge management. Our research has identified technologies,
affordances, incentives and user attitudes that have helped to create the wealth of accessible user-generated
knowledge available on the World Wide Web. In this paper, we will discuss our research and propose
changes to Army technology and organizational culture that could improve the speed and effectiveness of
managing and disseminating crowd-vetted knowledge for performance support.
THURSDAY, 4 DECEMBER, 2014 ROOM S320F
H-6 Shocking Medical Assessments
1330
Quantitative Assessment of
Combat Casualty Skills
(14191)
1400
Assessment Instrument Validation for
Critical Clinical Competencies: Pediatricneonatal Intubation and Cholinergic
Crisis Management (14232)
1430
A Decision Support System
Predicting Imminent Cardiovascular
Shock (14343)
Notes
56
QUANTITATIVE ASSESSMENT OF COMBAT CASUALTY SKILLS
Christine Allen, Ph.D. & Mark Mazzeo
Brian Goldiez, Ph.D. & Amanda
Army Research Laboratory (ARL) Human
Romeu
Research and Engineering Directorate (HRED)
University of Central Florida (UCF)
SFC Paul Ray Smith Simulation and Training
Technology Center (STTC)
(IST)
Thomas Pingel
Joint Base Lewis
McCord (JBLM)
Medical Simulation
Training Center (MSTC)
Evaluating proficiency in combat casualty training includes the assessment of hands-on training with
mannequins through instructor observation. The evaluation process can suffer due to the subjective nature of the
assessment: differences between instructor rating schemas, student to instructor ratios, and time to observe
individual student performance. Because combat casualty care requires timely and accurate assessment for
medical interventions, evaluators can look at the trainees’ physical actions (e.g., hand motion) to assess
proficiency, as seen in suturing literature. The Lempel Ziv (LZ) complexity index is then used to assess
proficiency. The LZ algorithm reduces complex strings of data (i.e., hand motion) to a string of 1’s and 0’s. The
string is then broken into small “unique” strings that are grouped together. The pattern formed is a measure of
performance with more complex patterns per unit of time indicating expertise. Expanding the current state of
the art, experimentation occurs using several different precision tracking devices that are unobtrusive and
require limited setup. During this effort, student hand motion is tracked and digitally stored as participants
complete multiple tasks part of a cricothyroidotomy (emergency airway procedure in the neck). Motion data is
subsequently processed using an algorithm adapted for text compression (LZ algorithm). Data has been gathered
from nearly 100 military combat medic trainees at Joint Base Lewis McChord (JBLM) Medical Simulation
Training Center (MSTC). Participant hand acceleration data from an emergency surgical cricothyroidotomy
reveals a statistically significant difference in ability among different expertise levels. The higher the LZ score
and self reported expertise level, the better the participant performed. The results show that when presented with
demographic and video performance-based data, it is possible to gauge experience using LZ scores.
ASSESSMENT INSTRUMENT VALIDATION FOR CRITICAL CLINICAL COMPETENCIES:
PEDIATRICNEONATAL INTUBATION AND CHOLINERGIC CRISIS MANAGEMENT
Madsen JM, Hurst CG
Talbot TB
U.S. Army Medical Research
Telemedicine & Advanced
Institute for Chemical Defense
Technology Research Center
(USAMRICD)
(TATRC)
Aberdeen, MD
Fort Detrick, MD
Background: Military and civilian first-responders must be able to recognize and effectively manage casualties that
necessitate immediate application of critical clinical competencies. Two examples of these critical competencies are
the clinical management of injuries resulting from nerve agents and difficult intubation, especially for pediatric or
neonatal patients. The opportunity to learn and practice the necessary skills for these rare, but urgent, situations is
complicated by the limited ability to replicate essential situational factors that influence performance in the applied
clinical environment. Simulation-based training may resolve some of these challenges, however it is imperative that
evidence be captured to document the achievement of performance competencies in the training environment that
transfer to applied clinical care. The purpose of this study was to establish psychometric characteristics for
competency assessment instruments associated with two such critical competencies: management of cholinergic crisis
and pediatric-neonatal intubation. Methods: To inform the development of assessment instruments, we conducted
comprehensive task analyses across each performance domain (knowledge, performance). Expert review confirmed
content validity. Construct validity was established using the instruments to differentiate between the performance
abilities of practitioners with variable experience (novice through expert). Purposively selected first responder subjects
for pediatric-neonatal intubation (N=214) and cholinergic crisis management (N=123) were stratified by level of
experience performing the requisite clinical competencies. All subjects completed knowledge and performance
assessments. Reliability was established using test-retest (Pearson correlation) and internal consistency (Cronbach’s
alpha) for knowledge and performance assessments. Results: Significantly higher scores for subjects with greater
levels of experience, compared to those with less experience established construct validity for each assessment
instrument (p < .01). Significant correlations between test-retest outcomes indicated measurement reliability p < .01.
Cronbach’s alpha for knowledge and performance scores demonstrated excellent internal consistency. Conclusions:
Psychometric evidence establishes the value of assessment for identifying and remedying critical competency
performance gaps.
Andreatta P, Klotz J
University of Minnesota Medical
School
Minneapolis, MN
57
A DECISION SUPPORT SYSTEM PREDICTING IMMINENT CARDIOVASCULAR SHOCK
W. Andrew Pruett, Leland D Husband & Robert Hester
University of Mississippi Medical Center
Jackson, MS
The human dynamical response to hemorrhage is complex and difficult to predict. For medical personnel
involved in a triage situation, correct prioritization of patient is of paramount importance. In this paper, we
present a decision support system (DSS) whose purpose is to provide warning of imminent
shock/decompensation. As inputs, the system utilizes only rudimentary clinical measurements: heart rate,
blood pressure, respiratory rate, skin temperature.
A human experiences three phases of hemorrhagic shock: an initial tachycardic, a compensatory phase,
which is followed by a bradycardic decompensating phase and finally a tachycardic irreversible
decompensation that leads shortly to death. Recognizing that most triage situations will involve individuals
in the second phase, we used the progression into the final tachycardia as the observational endpoint.
The DSS is a function constructed through machine learning techniques (support vector machines). The
function takes serial readings of standard clinical measurements as inputs that are transformed into a
simple signal that indicates whether decompensation is imminent. The measurements in this case are
drawn from an in silico population. The population is generated using HumMod, an integrative model of
human physiology. HumMod, as originally created, is a deterministic model. We have converted HumMod
to a population model by allowing model parameters to vary uniformly. This proof of concept is described
in a recent paper (PLoS One, PMID:24058546).
The constructed model classifies a data set as decompensating within two minutes or not with an accuracy
of 93%, and a 70% accuracy on these patients that will decompensate. These facts indicate that, despite
requiring minimal inputs, the model is a potent tool for predicting imminent irreversible cardiovascular
shock in a manner that allows medical professionals to offer appropriate intervention. In a triage situation,
such a model would allow for improved prioritization of patient needs.
TUESDAY, 2 DECEMBER, 2014 ROOM S320A
P-1 Managing Resources at Home and Abroad
1400
Resource Implications of the
Difference between Models
and Simulations (14020)
1430
COTS to Capability: Lessons Learnt
from UK MOD Research
Programme (14115)
1500
Perspectives on Exportability and
Program Protection in Virtual
Training Systems (14265)
Notes
58
RESOURCE IMPLICATIONS OF THE DIFFERENCE BETWEEN MODELS AND
SIMULATIONS
Thomas J. Yanoschik
SAIC / Maneuver Battle Lab
Fort Benning, Georgia
The purpose of this paper is to argue that given the constrained fiscal environment that the Department of
Defense (DoD) is facing, mid-level policymakers must consider whether training or experimentation
objectives can be met with low fidelity models rather than high fidelity simulations. In order to do this,
they must understand perhaps the most basic lesson of modeling and simulation (M&S)—the difference
between a model and a simulation. The paper will begin with a discussion of the difference between the
terms and show they are often (incorrectly) used interchangeably. It will then transition to a case study
where the Maneuver Battle Lab (MBL) and the Combat Developments Division (CDD) of the U.S. Army's
Maneuver Center of Excellence (MCoE) conducted a short suspense wargame for which a specific
simulation was requested (out of a lack of knowledge of other available tools) but the desired endstate was
achieved through the use of a low fidelity model. The conclusion is, for some training and experimentation
cases, the DoD save significant resources through the use of low fidelity models while still achieving their
objectives to standard. On the surface, a simple discussion (tutorial) of the differences between models and
simulations would not be worthy of discussion, but under fiscal constraints it is imperative that M&S
professionals ensure that policymakers understand the differences and how differentiating between the two
may result in a significant savings of resources. The paper will also emphasize the point that selection of
the proper tool, be it a model or a simulation, should be based on the experiment or training objectives
rather than selecting the tool and then determining which objectives can be achieved. The paper will end
with areas for continued research.
COTS TO CAPABILITY: LESSONS LEARNT FROM UK MOD RESEARCH PROGRAMME
Kent, J.R., Stafford, A., Nicholls, A.P.
QinetiQ
Farnborough, Hampshire, UK
Shawl, C
Dstl
Portsdown West, Hampshire, UK
The future of training involves simulation, which is often delivered by Commercial Off-The-Shelf (COTS)
products. This principle was embraced by the United Kingdom (UK) Ministry of Defence (MOD) in 2010,
under its Strategic Defence and Security Review (SDSR), and heralded the beginning of a paradigm shift
in the way the MOD customer viewed simulation and COTS solutions. Given budgetary constraints and
the need for a flexible, adaptive training capability in support of future military operations, the
commitment to embrace COTS is a bold and sound step. However, whilst COTS offers obvious
advantages, there is a perception that COTS provides the entire answer, and that procuring a training
capability is as simple as walking into a store, picking a training solution from the shelf, taking it home
and plugging it in.
Unfortunately, although this approach can work for commodity items, it’s not always suitable as the basis
for developing mission-critical training capability. In particular, the technical specification or level of
innovation in an COTS product are not the only things that customers should consider when contemplating
their options. They need to ask if they have considered the implications for meeting the training objectives,
technical integration, safety, business or procurement or commercial processes, and legislative compliance.
Does the training task really need a 6 Degree of Freedom motion platform? Does a foreign product meet
your country’s safety legislation? Can the product be easily integrated with existing solutions?
Asking the right questions early can save time and money and avoid disappointment. This paper reports
the lessons-learnt during a series of training-related Technology Demonstrators undertaken under a MODfunded research programme investigating COTS. The lessons are presented in a simple check-list to help
providers and customers manage and mitigate risks early in planning and delivery phases, helping to
maximize the benefits gained from exploiting COTS.
59
PERSPECTIVES ON EXPORTABILITY AND PROGRAM PROTECTION IN VIRTUAL
TRAINING SYSTEMS
Michael Coleman & Ricky Denny
Naval Air Warfare Center, Training Systems Division
Orlando, Florida, USA
Department of Defense (DoD) and industry acquisition integrated product teams delivering virtual training
systems to international customers must consider exportability and program protection issues common to,
and often beyond, those of the corresponding live platforms. DoD Instruction 5000.02 requires DoD
program managers to consider exportability and program protection throughout the acquisition lifecycle,
ensuring the ability for international partners to procure defense articles while mitigating risks of potential
loss of critical program information or technology to potential adversaries. Virtual training systems may
contain classified military information, controlled unclassified information, or proprietary information
required to replicate or simulate the live platform and its behavior in a synthetic environment. DoD’s
ability to provide Government-furnished information for International Armament Cooperative Programs
and Foreign Military Sales programs is constrained by numerous DoD policies and issuances as well as
federal law. Incorrect assumptions by industry, DoD, and international customers regarding DoD’s ability
to provide classified military information, controlled unclassified information, or proprietary information
may lead to cost and schedule overruns and inability to provide capabilities previously advertised to the
customer. This paper defines perspectives on exportability and program protection in the DoD acquisition
lifecycle and discusses the relevance of these perspectives to acquisition of virtual training systems. After
defining methods of international acquisition of defense articles, the paper aggregates numerous DoD
issuances regarding exportability and program protection into perspectives that DoD acquisition personnel
may reference in drafting documents and conducting other program activities relating to virtual training
system acquisition. The paper concludes with recommendations for DoD, industry, and international
customers to consider with the mindset of delivering a valid training system within customer cost and
schedule constraints.
TUESDAY, 2 DECEMBER, 2014 ROOM S320A
P-2 Commonality and Data Sharing in the LVC Environment
1600
LVC, Translating DoD Policy into
Action (14059)
1630
Data Sharing: The Standard
Specification is Just the Start
(14130)
1700
Establishing Sharing for Geospatial
Environment Data (14255)
Notes
60
LVC, TRANSLATING DOD POLICY INTO ACTION
2014 IITSEC Paper No.14059
LCDR Daniel Cain, USN; CAPT Robert Snyder, USN (Ret.)
OPNAV N980T
Arlington, VA
“I don't know what the hell this "logistics" is that Marshall is always talking about, but I want some of it.”
- Admiral E. J. King (1942). As with WWII logistics, today’s Services’ leadership also “…want some of
it…” when referring to Live, Virtual and Constructive (LVC) capabilities. They use the term ‘LVC’ to
conceptualize a futuristic rendition of affordable, net-centric warfare training. However, current DOD
policy on LVC minimizes actionable direction and definition that is needed to ensure a Joint as well as
Service approach to LVC training. For example, DOD Directive 1322.18 Military Training policy does not
identify an LVC Service lead, nor does it provide any guidance or standardization to ensure inter-Service
LVC interoperability. Also, key technical elements such as a 5th generation waveform and security
encryption have been left to each Service to solution/innovate independently. As a result, each Service has
separately invested in LVC Modeling and Simulation (M&S). These individual, costly approaches are a
barrier to interservice LVC compatibility.
Perhaps DOD, as well as individual Service LVC policy should include direction to exploit an integrated
LVC approach to both Training and Test and Evaluation (T&E) requirements. In the past, the Services
addressed separately LVC M&S investments to support either Training or T&E requirements. A common
investment strategy to both Training and T&E requirements coupled with a cooperative LVC strategy
would assure affordable inter-Service as well as Training and T&E LVC development.
This paper will first provide some background into the LVC journey from a Naval Aviation perspective.
Next, the paper will identify key barriers to interservice LVC implementation. Finally, the paper will
propose a clarified DOD-wide LVC definition along with policy direction to overcome LVC barriers and
facilitate the translation of LVC policy into actionable, coherent, funded Programs of Record.
DATA SHARING: THE STANDARD SPECIFICATION IS JUST THE START
2014 IITSEC Paper No.14130
Robert F. Richbourg & George E. Lukes
Alexandria, Virginia
Data sharing across multiple lines of effort is an often-cited component of reducing costs, improving
efficiency, supporting interoperability, and providing other potential benefits. However, achieving a state
where data can be readily shared is far from trivial and, as a first step, requires standards to be universally
accepted among the data users. To achieve real success, many other steps must follow.
The Multinational Geospatial Co-production Program (MGCP) is an international cooperative effort where
32 nations together are coordinating the production and sharing of digital geospatial data that will
eventually provide high-resolution vector data at a scale equivalent of 1:50,000 or 1:100,000 for much of
the world’s landmass. The MGCP is a successful data sharing program that continues to provide benefits
for all member nations. As an example, much of the 1:50,000 data that was used in Afghanistan was
produced by 7 different MGCP nations. All of the multi-purpose Atlas data the United States used to
provide humanitarian relief in Haiti following the 2010 earthquake was produced from MGCP data. While
there are many other success stories, the enduring value of the MGCP extends beyond its ability to provide
timely, accurate geospatial data. The MGCP is a role model exemplifying the potential benefits of
standards that are fully supported throughout the enterprise.
This paper describes key components of the MGCP effort, starting with the MGCP standard development
processes and the importance of the supporting technologies that the MGCP has put in place to complete
the standards. These include mechanisms for standards evolution, adjudication, compliance assessment,
and enforcement. After developing these elements, the paper describes how they could be extended to
provide similar benefits to other problem areas and thus form a domain-independent model for successful
data sharing.
61
ESTABLISHING SHARING FOR GEOSPATIAL ENVIRONMENT DATA
Mark Faulk
Cornerstone Software Solutions
Oviedo, Florida
Robert Cox & Bill Reese
U.S. Army PEO STRI
Orlando, Florida
Live, virtual, constructive, and gaming (LVC-G) integrated training environments bring challenges to data
providers with increased target formats for geospatial environment data and visual models. New sharing points
along the processing pipeline support consumer expectations of both managed correlation for interoperability
and fair fight and optimized runtime content. Sharing points now include cleaned source, intensified source,
confederate differentiated source, and runtime formats providing benefit of increased interoperability and fair
fight through managed, well-defined levels of correlation. While datasets available from a single provider have
increased, so has the number of providers bringing differing standards and conventions. With this increased
sharing comes the complexity of managing the number and variety of datasets and providing efficient search and
retrieval by data consumers. Oftentimes consumers ask for data from one sharing point without fully realizing
the intended use for that share, resulting in poor reuse performance and consumer frustration. Maximum reuse
requires incorporating externally developed, value-added data submitted with a variety of formats, data models,
dictionaries, fidelity, and specialization levels. Provider reuse policies must balance between accepting unvalidated data, risking contaminating their repository and full data validation which may be as costly as using
raw source data. Effective data sharing across this vast set of available data possesses potential for improved
approaches to managing the acquisition of geospatial environment data for the M&S community. Multiple
initiatives have been established or proposed to address the standardization of metadata, exchange protocols, and
data product formats toward improved interoperability both between sharing sites and with consumers. This
paper describes how some of those efforts are converging to support improved human and machine discovery
and selection and interoperability between providers. We describe real world experiences solving these
problems from the perspective of a large data provider and propose future direction for effective data sharing.
P-3 Wave of the Future
1600
Agile Program Management on
Software Intensive Training
Systems (14311)
1630
Simulations in the Cloud – A
Manager’s Challenge (14104)
1700
Continuous Monitoring of
Cybersecurity in a Training System
Environment (14121)
Notes
.
62
AGILE PROGRAM MANAGEMENT ON SOFTWARE INTENSIVE TRAINING SYSTEMS
Gregory Owens, Petra L. Robinson, Barry Minchey
Naval Air Warfare Center, Training Systems Division
Orlando, Florida
For years, training systems acquisitions supporting Department of Defense (DoD) programs have used the
Systems Engineering Technical Review (SETR) process to guide system development. Programs using
SETR are driven toward traditional waterfall methodologies, leading to a development process that can be
rigid, expensive and time consuming. The traditional SETR process may not be the best approach for
software intensive training system development. The Information Technology (IT) industry makes
widespread use of Agile practices in the management and development of software. This paper is based on
the results from a literature review and interviews with program teams who recently used or are actively
using Agile methodologies. The DoD programs can take advantage of these practices to reduce cost,
deliver training capability to the warfighter sooner and with fewer overall technical defects. The intent of
this paper is to inform the Defense Modeling and Simulation Training community on the possibility of
shifting to Agile project management on DoD software intensive training systems. The paper will discuss
benefits and possible drawbacks of shifting to Agile methodologies, applicability to software intensive
training system projects, implementation concerns, contract development issues, and future activities
which may lead to greater success. This paper may be of interest to a wider government audience that may
be struggling with similar challenges and have a need to maximize budget effectiveness while delivering
capabilities on a tight timeline.
SIMULATIONS IN THE CLOUD – A MANAGER’S CHALLENGE
Lawrence A. Rieger, CMSP
U.S. Army Capabilities Integration Center
Fort Eustis, VA
In August 2013, The Army Capabilities Integration Center (ARCIC) Director challenged that ARCIC
could save big dollars if we could put OneSAF “in the cloud” in the Army Battle Lab Collaborative
Simulation Environment (BLCSE). What seemed like a simple technical migration was actually a
significant change in the way simulations and simulation federations would be structured and operated
within a major distributed simulation environment (BLCSE). Software (including M&S) as a service is the
push of the DoD Cloud Computing strategy, and the necessary result of the decreasing resources available
to military M&S for large distributed simulation federations and events. DoD must maintain a trained and
ready force, relying on simulations to both define and design the future force as well as ensuring it is
properly trained. Given Federal Government and DoD cloud policy, it’s not a matter of if, but when and
how, our simulations will move to the cloud. This paper provides a detailed introduction to what it means
to have your simulations “in the cloud” together with practical planning steps and lessons learned for the
migration of a distributed simulation network into a community cloud environment, with particular
attention to the Mission Analysis planning process and the federation management processes which require
change. The author addresses the technical architecture problems associated with cloud computing,
community issues of network redesign and the DoD Information Assurance Program (DIACAP)/Risk
Mitigation Framework as well as the resource investment and cost benefit analysis for distributed
workstations vice central blade servers or rack servers. The more demanding configuration management
and configuration control issues of simulation federations in the cloud, providing Modeling and Simulation
as a service, are also addressed. Virtualization is a major component of the ARCIC BLCSE Modernization
initiative, with summer 2014 initial technical evaluations and detailed modernization technical review
being addressed.
63
CONTINUOUS MONITORING OF CYBERSECURITY IN A TRAINING SYSTEM
ENVIRONMENT
Graham Fleener & Marco Mayor
Andrew Maxon
U.S. Army PEO STRI
Cybernet Systems Corporation
Orlando, FL
Orlando, FL
There are a number of upcoming paradigm shifts within Information Assurance (IA), to include policy and
technical mandates, affecting IA in today’s training and simulation systems. Maintaining situational
awareness of a system’s IA posture has been a challenge DoD wide. Specifically, in the training and
simulation community it has been especially difficult given the closed, restricted networks the systems
create or may intermittently traverse. A number of DoD wide policies and technical solutions have been
developed and procured to ensure a system owner has continuous oversight of their system’s IA posture.
Over the years the Defense Information System Agency (DISA) has provided tools and solutions to Project
Managers (PMs) to easily assess a given systems IA posture at a given time. The most popular example of
these tools was the Gold Disk. However, the Gold Disk program was discontinued in 2012. Next came a
suite of products much more scalable and robust in capabilities, but also with significant complexity.
Assured Compliance Assessment Solution (ACAS), Host Based Security System (HBSS), and Continuous
Monitoring and Risk Scoring (CMRS) are a few of the latest DISA licensed Commercial Off The Shelf
(COTS) and Government Off The Shelf (GOTS) solutions available to PMs for integration into their
systems at no cost. These solutions were designed for an enterprise Information Technology (IT)
environment, but must be scaled to integrate with training and simulation systems. This paper will discuss
the continuous monitoring requirements, benefits, emerging security practices, implementation concepts,
and a training system example. This paper will document how the U.S. Army Program Executive Office
for Simulation, Training, and Instrumentation (PEO STRI) is addressing the growing cybersecurity threats
through continuous monitoring and improved situational awareness by leveraging DISA licensed COTS
and GOTS solutions to secure training and simulation systems. All DISA licensed COTS and GOTS
described in this paper are available at no cost to the Government to implement.
TUESDAY, 2 DECEMBER, 2014 ROOM S320B
S-1 Leveraging Cloud and High Performance Computing Environments
1400
Embracing The Cloud – Providing
Simulation as a Service (14018)
1430
Cloud Terrain Generation and
Visualization Using Open
Geospatial Standards (14308)
1500
Enabling External Player
Connections To Kerberos-secured
Systems (14202)
Notes
64
EMBRACING THE CLOUD – PROVIDING SIMULATION AS A SERVICE
Dr. Daniel Lacks
Cole Engineering Services, Inc. (CESI)
Orlando, FL
Lawrence A. Rieger, CMSP
U.S. Army Capabilities Integration Center
Fort Eustis, VA
In August 2013, The Army Capabilities Integration Center (ARCIC) Director challenged that ARCIC
could save big dollars if we could put OneSAF “in the cloud” in the Army Battle Lab Collaborative
Simulation Environment (BLCSE). What seemed like a simple technical migration was actually a
significant change in the way simulations would be structured and operated within a major distributed
simulation environment (Battle Lab Collaborative Simulation Environment – BLCSE). While typical
distributed simulation environments use either the High Level Architecture or Distributed Interactive
Simulation protocols to exchange data between federates, a cloud environment seeks to remove as much of
these data exchanges, and the resulting network infrastructure and latency, as possible. Software as a
Service (SaaS) is the push of the DoD Cloud Computing strategy, and Simulation becomes the Software
being provided as a service within cloud simulation. This paper details a simulation cloud testbed and
several technical evaluations conducted to determine Simulation as a Service within the DoD Cloud
Computing Strategy. It provides lessons learned and practical planning steps for the migration of a
distributed simulation network into a community cloud environment with particular attention to the
intricacies of establishing a robust Virtual Machine simulation environment. The authors also address the
technical architecture problems associated with cloud computing, community issues of network redesign
and the DoD Information Assurance Program (DIACAP) as well as the resource investment and cost
benefit analysis for distributed workstations vice central blade servers. The more demanding configuration
management and configuration control issues of simulations in the cloud, providing Modeling and
Simulation as a service, are also addressed. The paper is based on an IRAD simulation cloud testbed and a
series of distributed technical tests demonstrating SaaS over an Army secure simulation network.
CLOUD TERRAIN GENERATION AND VISUALIZATION USING OPEN GEOSPATIAL
STANDARDS
Samuel Chambers
Joint Staff J7 Environmental Development Division
Suffolk, VA
Jay Freeman
CAE USA
Orlando, FL
The Joint Training Data Services (JTDS) is a web-based set of services that provide Modeling and
Simulation (M&S) ready data and scenario development tools to the DoD Enterprise to support Joint and
Service theater level constructive & virtual training. JTDS provides persistent web access to an Order of
Battle Service and a Terrain Generation Service (TGS) that leverage unique data repositories and tools to
generate training scenario initialization files. Historically, the terrain generation service proved difficult to
maintain and extend given its closed architecture, stove pipe terrain generation capabilities and stagnant
source data collection. Given the lessons learned from the legacy terrain generation service, a technology
update and refresh was undertaken to create an updated terrain generation service that supports open
source formats, accessibility through an easy to use web interface, and dynamic terrain during runtime.
The new terrain generation service heavily utilizes open simulation data standards and geospatial web
mapping interfaces to share and distribute simulation products and geospatial data. The open source
Common Database (CDB) structure is used as the underlying source data format based on its ability to
promote sharing, reuse and utility by storing geospatial and simulation data sets in non-proprietary formats
structured to facilitate rapid access, rendering and visualization. Open Geospatial Consortium (OGC) web
standards are used to maximize connectivity to the CDB by enabling most geospatial tools to natively
visualize and navigate the Terrain Repository. This paper will share the lessons learned and architectural
updates of the new terrain generation service.
65
ENABLING EXTERNAL PLAYER CONNECTIONS TO KERBEROS-SECURED SYSTEMS
Peter G. Raeth
Chinhoyi University of Technology
Zimbabwe, Africa
Sean B. Ziegeler
Stennis, Mississippi
Rhonda Vickery
Dayton, Ohio
For applications in real-time distributed simulation, it had previously been difficult for users of DoD's
high-performance computer (HPC) shared resources to connect internal players with external players.
Kerberos authentication and the resources' batch orientation were the main barriers. This paper describes
the authors' method of overcoming such barriers. We show the approach we took to linking internal and
external players via the public Internet. This approach is totally user driven, follows a standard process,
and requires no system administrator interaction or special permissions. All Kerberos authentication
requirements are met. Batch submittals are not required. Utility of the technology was demonstrated by
linking two widely-used customer players, one on an external Windows PC with a player running on a
remote Linux HPC compute node. The result of this effort makes previously inaccessible equipment
available to an entirely new customer base. This result is important to the simulation community because it
facilitates real-time access to a large already-funded collection of remote Kerberos-secured HPC resources.
These resources enable higher-fidelity modeling and expanded throughput for complex players, processes,
and automated interactions. Included are specialized hardware under test or evaluation, or performing
some function within a real-time scenario. No additional capital expenditure was made, yet new
computational and storage resources are now available to a much wider user community.
TUESDAY, 2 DECEMBER, 2014 ROOM S320B
S-2 LVC Interoperability
1600
Sensor Placement Optimization in
LVC Environments for
Training, Analysis, and Operational
Applications (14314)
1630
Towards Interoperability of
Simulations Systems of Ground
Force: Progress and Challenges
(14082)
1700
Integrating Distributed Virtual
Command and Control
Platforms into Live Training
(14318)
Notes
66
SENSOR PLACEMENT OPTIMIZATION IN LVC ENVIRONMENTS FOR TRAINING,
ANALYSIS, AND OPERATIONAL APPLICATIONS
Jennifer Lewis and Joyner Livingston
Science Applications International Corporation
Huntsville, AL
Advances in specialized processor capabilities, such as Graphics Processing Units (GPUs), have contributed to
the ability to efficiently process high density terrain. Using these technologies, the Aviation and Missile
Research Development and Engineering Center (AMRDEC), System Simulation and Development Directorate
(SSDD), Soldier Protection Laboratory (SPL) developed a physics-based sensor-terrain interaction model that
accurately predicted and synthesized radio frequency (RF) coverage in dense foliage for the Army
Expeditionary Warrior Experiment (AEWE) conducted at Ft. Benning in August 2013. Since that time, the
development team has built upon the initial capability to include unique user features such as intuitive
comparisons of the mathematically optimal placement for baseline and experimental sensor sets and the ability
to respond on-the-fly to changes in the underlying terrain. This paper describes the capabilities of the Automated
Sensor Placement Engine (ASPE) and potential operational applications, such as Intelligence Preparation of the
Battlefield (IPB) and mission planning and rehearsal. It also describes the technical design of the tool and
underlying models as well as its transparent middleware approach to integrating with existing toolsets and
visualization options. During the past year, the SPL team has successfully integrated ASPE into a range of Live,
Virtual and Constructive (LVC) environments, including networked tactical sensors, command and control (C2)
nodes, constructive simulations such as OneSAF, and web-based interfaces such as Ozone Widget Framework
(OWF). Integration leveraged the use of both tactical and simulation interoperability standards, including the
Security Equipment Integration Working Group (SEIWG) Interface Control Document (ICD) series and
Distributed Interactive Simulation (DIS). The paper will discuss lessons learned and document repeatable
processes developed while integrating these multi-architecture environments.
TOWARD INTEROPERABILITY OF SIMULATION SYSTEMS FOR GROUND FORCES:
PROGRESS AND CHALLENGES
Sérgio Simas Lopes Peres, Jonathan Rosa Moreira
Brazilian Army
Brasília, Brazil
This paper presents the Brazilian Army approach to simulation system based training and how it evolved
in the course of time, from the development of GOTS, to the use of COTS and their integration. A
particular focus is dedicated to the technical challenges faced during the integration of virtual and
constructive simulators.
67
INTEGRATING DISTRIBUTED VIRTUAL COMMAND AND CONTROL PLATFORMS INTO
LIVE TRAINING
Ryan McLaughlin, Orlando Torres, Mike Aldinger
Northrop Grumman Corporation
Orlando, FL
As the United States Air Force’s budget continues to shrink, so does the budget for live fly training. This
drives aircrews to increase their usage of virtual trainers. These reduced budget constraints will directly
impact live training ranges such as the Pacific Air Force’s (PACAF) Joint Pacific Alaska Range Complex
(JPARC) ability to fund live aircraft participation in joint exercises such as Red Flag-Alaska and Northern
Edge. One approach to reduce live exercise costs is to provide critical Command and Control (C2)
platforms such as E-3, E-8, and RC-135V/W as virtual assets.
The JPARC has taken this approach and has integrated the Combat Air Force Distributed Mission
Operations (CAF DMO) E-3 and E-8C mission training centers to support live training exercises. The
integration of these virtual C2 platforms required the merging of two disparate training architectures, the
CAF DMO Virtual-Constructive solution and the JPARC’s live range solution. A primary difference
among these architectures is the standards each are based on. CAF DMO implements the Distributed
Interactive Simulation (DIS) standard while the JPARC implements the Test and Training Enabling
Architecture (TENA). The merging of TENA and DIS architectures has resulted in significant challenges.
(1) How to accurately model live airspace in the DIS domain with multiple live aircraft data sources, (2)
live-to-virtual and virtual-to-live radio communications, (3) tactical data links, and (4) inconsistencies of
live data. These challenges proved vital to accurately representing the live battlespace for C2 assets.
To overcome the difficulties of merging two disparate architectures with individually unique standards, the
JPARC utilized a progressive solution that enabled bi-directional information to be accurately exchanged,
formatted, and processed between TENA and DIS architectures. Our paper will discuss this solution and
lessons learned during the integration of virtual C2 platforms into live fly exercises.
WEDNESDAY, 3 DECEMBER, 2014 ROOM S320B
S-3 Simulated Movement
0830
Mission Integrated Simulation – A
Case Study (14085)
0900
An Instructor Operating System
(OIS) Framework for
Interactive Instructor-Station
(14112)
0930
Development of a Microscopic
Artificially Intelligent Traffic
Model for Simulation (14003)
Notes
68
MISSION INTEGRATED SIMULATION – A CASE STUDY
Per Wikberg, Mirko Thorstensson, Peter Hammar, Gustav Tolt
Swedish Defence Research Agency (FOI)
Sweden
Currently available modeling and simulation technology has the potential to increase capability of military units.
The purpose of this study was to explore the potential benefit of 3D modeling and simulation as mission
integrated tools for preparing, executing and evaluating a ranger mission. The study was undertaken during an
eight day exercise in which two ranger squads were tasked to ambush a communication hub. Access to an
interactive 3D model of the mission area was expected to enhance planning and task force performance and also
provide means for better debriefing. During the planning, virtual mission rehearsals and reconnaissance were
undertaken in a virtual 3D model of the target area in the simulation system Virtual Battlespace 2. The execution
in the real target area was documented by observers, questionnaires, GPS, voice recording, helmet mounted
video cameras and interviews. Results indicated that “virtual reconnaissance” was a more appreciated function
compared to “virtual mission rehearsal”. Results also indicated that the 3D model had given the rangers a spatial
mental model which enhanced their execution. Finally, replaying the mission in the model enhanced the
possibility to draw conclusions. One conclusion is that mission integrated simulation does not replace, but rather
complements conventional tools or procedures. Possibly an urban terrain would render the “virtual mission
rehearsal” more valuable compared to this case of forest with a limited number of significant artefacts. Still, a
virtual 3D model which is “good enough” in terms of adequate level of detail in the mission area gives a
supplementary perspective which increases the understanding of the limitations of 2D maps. Consequently, the
concept of mission integrated simulation will be explored further. By utilizing already available tools and
platforms and focusing on solutions that might be realized within 5-10 years it should be possible to enhance
mission performance with limited investments.
AN INSTRUCTOR OPERATING SYSTEM FRAMEWORK FOR INTERACTIVE
INSTRUCTOR-STATION
Kim Leng, Koh & Shih Yeong, Wah
ST Electronics (Training & Simulation Systems) Pte Ltd
Singapore
Most training simulators engage a pool of experienced instructors to train and impart their invaluable skill
sets to trainees. Simulator training hours have proven to help trainees to acquire the necessary skill sets
more effectively. In most simulators, the instructor(s) preside over the instructor station while the
trainee(s) is (are) housed in the respective simulated cockpit or cabin. Conventional instructor stations
revolve around scenario planning, exercise execution control and communication to the trainees in the
simulated cockpit or cabin. In any simulation training session, the instructor relies on his communication
with the trainee(s) and the necessary skill sets are picked up over several (repeated) sessions. This paper
explores an Instructor Guidance-Assistance Role (IG-AR) enabled instructor operating station (IOS)
framework in building an instructor station. An instructor station built using this framework enables the
instructor(s) to engage trainee(s) interactively and aims to bring forth an instructor’s demonstrative
approach in imparting skill sets, thus enhancing the value of simulation training. This paper also presents
two case studies where the IG-AR enabled IOS framework has been adopted to build the instructor station.
The first case study illustrates how the instructor in a driving simulator guides the driver trainees to acquire
skill sets in open terrain driving. The IG-AR enabled IOS shortened the number of training sessions
required before driver trainees become proficient. A further focus study shows that the IOS helped to
qualify drivers effectively. The next case study illustrates how the IG-AR enabled IOS facilitates an
instructor playing an adversary role trains and assists maritime force crew in the Maritime Warfare
Tactical simulator to be exposed to more exhaustive adversary tactics. The IOS enables the instructor to
achieve greater learning synergy for the trainees. The paper concludes with a discussion on future
expansion of this framework to enhance building future simulator trainers.
69
DEVELOPMENT OF A MICROSCOPIC ARTIFICIALLY INTELLIGENT TRAFFIC MODEL
FOR SIMULATION
Viral Raghuwanshi, Sarthak Salunke & Kevin F. Hulme
Yunfei Hou
NYSCEDII
Department of Computer Science and Engineering
Roadway safety continues to be a major public health concern. Recent statistics show that more than
30,000 fatalities occur due to motor vehicle accidents, and in the year 2012, motor vehicle crashes resulted
in more than 2 million injuries. As a result of these ongoing trends, simulators continue to become more
abundant in applications ranging from Intelligent Transportation Systems (ITS) research, autonomous
driving, human factors studies, rehabilitation, and driver training and workload applications. However,
many current simulators lack realism with regards to accompanying traffic, which often does not
satisfactorily respond to the real-time actions of the human subject who is operating the simulation.
Artificial traffic simulation models found within many modern-day driving simulators are often
“macroscopic” in nature – they aggregate the description of overall traffic flow, which is based on
“idealistic” driver behavior. This lack of network realism (particularly in the vicinity of the human subject
operating the simulator) limits the application scope.
In this paper, we evaluate traffic simulation models for supporting next-generation ITS research
applications. This survey justified the need for the design and development of a microscopic Artificially
Intelligent Traffic Model (AITM) intended for civilian ground vehicle research applications. The AITM
generates a fleet of semi-intelligent vehicles with which a human driver interacts within a virtual driving
simulation environment. The behavior of the vehicles is based upon the basic principles of rigid body
physics and real-time collision detection, and includes a rule-base for: road-appropriate travel speed
behavior, behavior at intersections (e.g., stop signs, street lights), and interactions with other AI and
human-driven vehicles on the virtual roads (i.e., lane changing, headway distance). In this paper, the
design and development of the baseline AITM is described, and a use-case application is presented, along
with recommendations for improvements required subsequent to the deployment of the preliminary model.
S-4 Culture, Reaction and Movement: Simulating Human Behavior
1030
Teaching Cross Cultural Social
Competence in a Dynamic,
Synthetic Environment (14289)
1100
Advanced Animation Techniques in
a Dismounted Solider
System (14136)
1130
Game-based Simulation for
Philippine Post-typhoon Stability
Operations Training (14329)
Notes
70
TEACHING CROSS CULTURAL SOCIAL COMPETENCE IN A DYNAMIC, SYNTHETIC
ENVIRONMENT
William Ferguson, Bruce Roberts, David Diller
Raytheon BBN Technologies
Cambridge, MA
Dan Shapiro, Michael Mateas
University of California Santa Cruz
Santa Cruz, CA
To prevail in modern, asymmetric conflicts, most warfighters must socially engage with people of diverse
cultures to accomplish a variety of military missions. In spite of this pervasive need, no scalable solution
exists for training social skills. Live role-playing is prohibitively expensive and dynamic social skills
cannot be learned using traditional, virtual environment based training architectures that rely on carefully
scripted scenarios and statically animated, synthetic characters. This paper offers a possible solution for
social training by describing an exploitation of Expressive Artificial Intelligence (AI) and an adaptation of
cognitive apprenticeship to create a synthetic, mentored, social practice environment. Expressive AI views
AI as an expressive medium, and aims at the algorithmic and architectural research necessary to create
highly interactive and generative experiences. To allow for true social interaction, our team focused on
creating combinable chunks of behavior that enable synthetic characters to participate in a wide variety of
jointly meaningful social activities with each other and with a human learner. To meet the challenge of
mentoring in this environment, our team borrowed from the deep teaching method of cognitive
apprenticeship, exploiting techniques such modeling and scaffolding. To this mix was added real time
coaching using the same social simulation mechanisms that create the synthetic characters in the simulated
world. A demonstration version of the system was developed under the Defense Advanced Research
Projects Agency (DARPA) Strategic Social Interaction Modules (SSIM) program, which in part is
designed to illustrate training of general social competence in unfamiliar contexts rather than culturespecific knowledge and skills, using computer controlled characters and instruction in
synthetienvironments. This work on the design and engineering trade-offs and innovations in simulation
control structure should spark interesting debate in the education and simulation communities as well as
serving as the basis for others heading in this same direction.
ADVANCED ANIMATION TECHNIQUES IN A DISMOUNTED SOLDIER SYSTEM
Scott M. Johnson, John Carswell
Intelligent Decisions, Inc.
Orlando, FL
Pat Garrity
U.S. Army Simulation and Training Technology Center
Orlando, FL
The Dismounted Soldier Training System (DSTS) is a program of record with systems fielded by PEO
STRI throughout the US Army. The system provides a hardware platform that instruments each Soldier
trainee with eight worn Inertial Measurement Unit (IMU) based motion tracking sensors and a motion
tracked, instrumented weapon. The U.S. Army Research Laboratory, Human Research and Engineering
Directorate, Simulation and Training Technology Center (ARL-HRED-STTC) is performing research and
development to leverage the motion tracking capabilities of the DSTS system as well as emerging motion
tracking technologies to develop a more seamless and natural fusion of soldiers’ physical movements with
their body movement within the virtual environment and interactions with objects in it. Achieving this
objective requires the injection of real-time data from the motion tracking system into the animation
system of the underlying game engine in order to control the virtual avatar. Game engine frameworks
provide mechanisms that support injection through features such as forward and inverse kinematic solvers
and animation blending. Individually, these features are adequate to support simple representations of the
soldiers’ actions, but more complex actions require a fusion of techniques. This paper describes our
approach to solving the challenges in fusing many animation techniques together towards the goal of
suspension of disbelief that the virtual avatar’s motion is entirely the motion of a single Soldier.
71
GAME-BASED SIMULATION FOR PHILIPPINE POST-TYPHOON STABILITY OPERATIONS
TRAINING
Marjorie Zielke, Ph.D., Djakhangir Zakhidov, MFA, Gary Hardee, MA, & Michael Kaiser, MA
UT Dallas
Richardson, Texas
This paper discusses the use of The First Person Cultural Trainer (FPCT) platform to develop predeployment stability operations training scenarios for typhoons in the Philippines and other natural
disasters which require Army humanitarian missions. The FPCT platform, sponsored by TRADOC G2
Intelligence Support Activity, is a composable game-based simulation system capable of representing the
cognitive complexity of non-kinetic population engagement in zones affected by natural disasters. The
platform utilizes a PMESII (Political, Military, Economic, Social, Infrastructure, and Information Systems)
model as the design framework for modular and interconnected training scenarios -- engaging users in
cultural and communications decision-making for specific geographic regions and cultures. Within FPCT
game simulations, players must communicate with game characters in a culturally appropriate manner to
achieve stability post natural disaster, create alliances and ensure balance and stability between conflicting
cultural and political groups. Using the FPCT platform, FPCT Philippines was created -- inspired by
events that followed Typhoon Bopha which occurred in December 2012. At the time of the storm, Bopha
was the costliest and most severe typhoon to ever hit the Philippines. However, in November 2013, less
than a year after the creation of FPCT Philippines, Typhoon Haiyan also hit the region, and was
magnitudes greater in severity and aftermath. This paper explores the construct of the original FPCT
Philippines model and potential enhancements necessary to accommodate scenarios for Typhoon Haiyan –
thereby analyzing the overall use and flexibility of the FPCT platform for pre-deployment training of
stability operations in an environment of uncertainty and the usefulness of game-based simulations to train
for humanitarian missions using the PMESII model.
S-5 Decision Support Systems & Methods
1030
A Decision Aid for Optimizing
Experimental Design Involving
LVC Environments (14139)
1100
A Practitioner’s Approach using
MBSE in Systems of Systems
(14383)
1130
Robotic Simulators: A Case for
Return on Investment (14129)
Notes
72
A DECISION AID FOR OPTIMIZING EXPERIMENTAL DESIGN INVOLVING LVC
ENVIRONMENTS
Sylvain Bruni, Kenyon Riddle,
Andres Ortiz, Danielle Dumond,
Spencer Lynch
Aptima, Inc.
Woburn, MA
Henry Marshall, Chris Gaughan
U.S. Army Research Laboratory Human Research
and Engineering Directorate Simulation and
Training Technology Center
Orlando, FL
Jay Saffold
Research Network,
Inc.
Kennesaw, GA
Increasingly, Modeling and Simulation (M&S) is playing a key part in the decisional process Program Managers
(PM) make in the development of new systems, testing, doctrine, and other processes. Unfortunately, the PM
must navigate their decisions about leveraging M&S without any supporting aids, making entry and efficient
utilization difficult. There is currently no systematic method for assembling environments and designing
experiments from multiple M&S perspectives like Live vs. Virtual vs. Constructive simulations to provide
decisional data. This process typically requires multiple stakeholders to meet many times in an effort to
assemble modeling and simulation-based experiments “that work.” As more models, simulators, and scenarios
become networked and available to experimenters, a solution is needed to facilitate and accelerate the setup of
complex experiments that involve these assets. To meet this need, research was conducted to develop the Live
Virtual Constructive & Game - Assisted Experimental Designer tool (LVC&G-AED), an interface and software
solution that guides individuals through a ten-step research process, from defining research questions and
choosing variables of interest, to developing relevant measures and specifying the environment’s software and
hardware apparatus. This process is designed to be high-level, capturing the questions of the various
professionals involved in simulation development, while being sufficiently rigorous to ensure that specific
research questions are addressed. Partially Observable Markov Decision Process algorithms, coupled with an
intuitive user interface, allow for interactive exploration of the state space of experimental configurations of
simulators, equipment, and other resources available to the user. Through the LVC&G-AED decision-aid,
experimenters are provided with recommendations for optimal experimental design configurations. Ultimately,
LVC&G-AED translates experimental and simulation requirements into machine-actionable constraints, to
facilitate the complex setup of experiments that involve combinations of Live, Virtual, Constructive, and Game
M&S environments. This paper focuses on the development lessons learned during this research and the way
forward.
A PRACTITIONER’S APPROACH USING MBSE IN SYSTEMS OF SYSTEMS
Richard Deakins & Doug Parsons
US Army Aviation and Missile Research, Development and Engineering Center
Colorado Springs, CO
Recognizing the value of systems engineering (SE) as a key enabler of successful systems acquisition, and the
growing importance of systems interdependencies affecting the ability for mission success of highly complex
development systems, the Deputy Under Secretary of Defense for Acquisition and Technology developed the
[2008] “Systems Engineering Guide for Systems of Systems.” While this guide provides excellent insight into
the Systems of Systems (SoS) environment, as well as core SoS SE elements, the process to apply them in a
DoD acquisition environment is not included. The purpose of this paper is to extend those concepts by defining
a system of system acquisition process from receipt of modeling and simulation (M&S) needs through to
development of individual requirements for the constituent systems by leveraging the power of Model Based
Systems Engineering (MBSE) practices. Because it’s not atypical for constituent developers to read the same
need statement and interpret what occurs and what is needed on opposite sides of an interface differently, severe
issues can result of which are not discovered until the resolution is extremely costly. The use of MBSE and the
Systems Modeling Language (SysML) provides formal methods and notations that can remedy SoS
misunderstandings prior to development. Further, our proposed process will facilitate collaboration amongst the
system constituents and other stakeholders using MBSE throughout the acquisition process will create a shared
common understanding and agreement for the efforts required for success of the SoS mission. Included in the
discussion of the proposed acquisition process will be conceptual modeling, architecture and design reviews.
The nature of DoD missions and the simulations that describe them are becoming more complex with increasing
interdependence among the systems involved. This paper intends to provide the practitioner with systems
engineer processes that will result in avoidance of the unintended consequences impacting mission success.
73
ROBOTIC SIMULATORS: A CASE FOR RETURN ON INVESTMENT
Roger D. Smith PhD
Celebration, FL
Khara M. Simpson MD
Columbia University Medical Center
New York, NY
Simulation has been integrated into the education and certification process in aviation and military arenas
with significant success in providing cost effective training. The transition from the apprenticeship model
to simulation has been slower in the field of medicine with cost, lack of curricula and high fidelity
exercises and equipment being the main reasons. With recent improvements in all areas, cost remains a
significant challenge.
This report describes our novel analysis of the return on investment (ROI) that can be achieved through the
inclusion of simulator use within a robotic surgery business practice and as an alternative source of
training revenue. Information was gathered through an extensive literature review and expert interviews
for the development of an interactive calculator for institutions to utilize when considering an investment
in robotic surgery simulators.
This ROI model presents the core improvements to existing operations which may be realized through the
use of simulators of robotic surgery. Category headings include simulator investment costs, surgeon
productivity, surgeon health, hospital costs, and other training costs. The user of the model is able to enter
their own numbers for their unique facilities. The spreadsheet model will calculate the costs and benefits
associated with each area, create category subtotals, and then an overall total for all areas. Using these
numbers, it can then calculate an ROI percentage for the simulators. This model represents one tool to
assist organizations in making the investment in these devices and training programs.
S-6 Innovative Approaches to Environment and Behavior Simulation
1400
Creating a Re-useable Knowledge
Repository for UK MoD CGF
Behaviours (14080)
1430
Improving Air-to-Air Combat
Behavior Through Transparent
Machine Learning (14298)
1500
Improving Material Classification
Quality with Elevationderived
Metrics (14380)
NOTES
74
CREATING A RE-USEABLE KNOWLEDGE REPOSITORY FOR UK MOD CGF
BEHAVIOURS
Mark Lewis
Centre for Simulation and Analytics, Cranfield University Defence
Academy of the United Kingdom
Dan Allison
Discovery Machine Inc.
Williamsport, PA
The United Kingdom's Ministry of Defence (MOD) has a large investment in both Computer Generated
Forces (CGF) systems and in the supporting data and models that run within them. During the last audit
over 20 different CGF were identified in use across the three service domains. A major concern is that the
investment in behaviour modelling and data in one area is not easily re-useable due to the fundamental
differences in implementation of that data in those systems.
To address this issue, the UK Defence Science and Technology Laboratory (dstl) under the Centre for
Defence Enterprise (CDE) set a challenge to investigate the feasibility of achieving methods for capturing
models and data within common knowledge repositories for re-use across CGF and common modelling
services.
The purpose of this paper is to describe the work of Cranfield University at the Defence Academy to
evaluate a new approach for knowledge capture and simulation agnostic execution of CGF Behaviours. It
discusses the elements required to define a framework for the UK to develop a Behaviour Repository in
the context of informing a possible future MoD CGF Service. This includes a discussion on the software
architecture and processes; the lessons learned in the development of a multi-simulation behaviour
authoring console; the skills required to populate such a repository and the governance required to provide
a MoD CGF Service (including Non-run-time services). It describes the challenges both technical and nontechnical in developing such a framework that enables greater re-use of verified and validated data models
and behaviours across different simulation systems without being tied to a single CGF supplier.
IMPROVING AIR-TO-AIR COMBAT BEHAVIOR THROUGH TRANSPARENT MACHINE
LEARNING
Armon Toubman, Jan Joris Roessingh
National Aerospace Laboratory NLR
Amsterdam, the Netherlands
Pieter Spronck
Tilburg University
Tilburg, the Netherlands
Aske Plaat, Jaap van den Herik
Leiden University
Leiden, the Netherlands
Training simulations, especially those for tactical training, require properly behaving computer generated forces
(CGFs) in the opponent role for an effective training experience. Traditionally, the behavior of such CGFs is
controlled through scripts. There are two main problems with the use of scripts for controlling the behavior of
CGFs: (1) building an effective script requires expert knowledge, which is costly; and (2) costs further increase
with the number of ‘learning events’ in a scenario (e.g., a new opponent tactic). Machine learning techniques
may offer a solution to these two problems, by automatically generating, evaluating and improving CGF
behavior. In this paper the application of the dynamic scripting technique to the generation of CGF behavior for
training simulations is described. Dynamic scripting is a machine learning technique that searches for effective
scripts by combining rules from a rule base with predefined behavior rules. Although dynamic scripting was
initially developed for artificial intelligence (AI) in commercial video games, its computational and functional
qualities are also desirable in military training simulations. Among other qualities, dynamic scripting generates
behavior in a transparent manner. Also, dynamic scripting’s learning method is robust: a minimum level of
effectiveness is guaranteed through the use of domain knowledge in the initial rule base. In this research, the
application of dynamic scripting for generating behaviors of multiple cooperating aircraft in air-to-air combat is
investigated. Coordination in multi-agent systems remains a non-trivial problem. Explicit team coordination is
enabled through communication between team members. This coordination method was tested in an air combat
simulation experiment, and compared against a baseline that consisted of a similar dynamic scripting setup,
without explicit coordination. In terms of combat performance, the team using the explicit team coordination
was 20% more effective than the baseline. Finally, the paper will discuss the application of dynamic scripting in
a practical setting.
75
IMPROVING MATERIAL CLASSIFICATION QUALITY WITH ELEVATION-DERIVED
METRICS
Christopher Fink, Ph.D.
JRM Techn ologies, Inc.
Fredericks burg, VA
The goal of material classification is to identify the type of surface material present at each pixel of a satellite image or
aerial photograph, given metadata specific to the image and imaging sensor (e.g., geodetic area, day-of-year, time-ofday, sensor channel wavelength ranges, etc.). Typically, automated spectral algorithms are employed which attempt to
separate this large cloud of pixels into specific clusters representing different material classes (e.g., soil, vegetation,
road, rooftop, water, etc.), but often there are too few available channels to guarantee unambiguous identification. It is
well-known that RGB space data alone is typically insufficient to provide such distinction, because most of the
spectral reflectivity variation among these material types exists at higher wavelengths (near infrared [NIR] channels).
The problem is compounded when the classifier is asked to also find “artificial” (i.e., contextual) distinctions, e.g.,
between asphalt roofs and asphalt roads. As much of the available satellite image data is captured during daytime,
additional ambiguities arise as the result of strong shadows over much of the terrain.
Fortunately, spatially-correlated Digital Elevation Map (DEM) data are also often available with the RGB imagery.
Such data can provide not only a means by which image shadows can be identified and corrected, but also additional
contextual axes along which distinctions can be sought. In this paper, the author describes the algorithms behind a
novel tool capable of preprocessing such elevation data for two purposes: (1) to identify and remove shadows prior to
material classification, and (2) to define and compute a pair of normalized statistical metrics which can be put into the
same form as the color-channel data, such that the downstream clustering algorithms can use them as additional axes
of distinction. Finally, the author will present results showing the increased level of classification certainty achieved
by this method.
S-7 Improving Healthcare with Simulation
1600
Improving and Proving Healthcare
Quality and Value through
Physical Simulation (14144)
1630
Employing Modeling and
Simulation to Improve Patient Care
(14034)
1700
The Effect of Difficulty Levels
within a Virtual Medical
Simulation (14228)
Notes
76
IMPROVING AND PROVING HEALTHCARE QUALITY AND VALUE THROUGH PHYSICAL
SIMULATION
Timothy R. Brock, PhD, CPT
The Institute 4 Worthy Performance
Winter Park, FL
Mary Holtschneider, RN-BC, BSN, MPA, NREMT-P, CPLP
US Department of Veterans Affairs
Durham, NC
Healthcare providers world-wide are discovering how simulation modalities in clinical care settings
change practice behaviors. Not only must new behaviors result in improved and safer patient care, but the
investment of limited resources must prove worth the tangible and intangible outcomes. This paper
presents an Organizational Change Management (OCM) framework used by two healthcare provider
systems (one government and the other private) to satisfy this dual imperative—to improve healthcare
quality through physical simulation; then prove its value. The first example involves the US Department of
Veterans Affairs (DVA) hospital in Durham, NC. To meet current American Heart Association (AHA)
guidelines for resuscitation, the DVA used an OCM framework to guide physical simulation efforts during
the rollout and implementation of three, necessary changes: 1) a stroke code emergency policy; 2) use of
new emergency code carts; and 3) intraosseous needle (IO) use for cardiac arrest patients. The OCM
framework positioned grassroots stakeholders to increase adoption and sustained commitment by roleplaying emergency response situations requiring decision-making. The second example involves a
healthcare system in Birmingham, AL with an objective to reduce central line blood infections in intensive
care units at local hospitals. This healthcare system also implemented a comprehensive value stream
measurement methodology which generated six types of quantitative and qualitative metrics to prove the
value (including Return on Investment (ROI)) of quality care and patient safety culture change initiative.
Both the OCM framework and the ROI value stream evaluation methodology assisted with planning and
proving the value of simulation to support organizational and behavioral change initiatives as well as
address organizational education and training requirements.
EMPLOYING MODELING AND SIMULATION TO IMPROVE PATIENT CARE
James Thomas & Allen J. Giannakopoulos, Ph.D.
Baptist Health South Florida
Miami, FL
In a Labor & Delivery (L&D) environment, new mothers that experience hemorrhages are a medical
emergency. How clinicians deal with that emergency is critical for the mother’s health. Computer
modeling software provides an effective tool to simulate and understand how different treatment processes
affects patient care. In order to improve patient safety and quality of care, the department developed a new
protocol to request blood and medication supplies for a hemorrhagic mother. This protocol required
requesting an additional nurse for the patient and one phone call to the lab for supplies. However,
clinicians did not follow the protocol uniformly. This paper describes the data and the model developed at
the request of the Medical Director of the Patient Simulation Lab. The model uses data collected from an
Electronic Health Record (EHR) system that describes a patient’s encounter from admission to discharge.
The model compares instances of two patient care scenarios for coping with a hemorrhagic emergency.
The first scenario represented the original state, which included multiple phone calls to the lab. The second
scenario employed the communication protocol developed to improve the speed of delivering lab and
pharmacy supplies. The pharmacy and lab are included as Labor & Delivery informs them of the patient’s
clinical information collected when the mother-to-be arrives in the Labor & Delivery unit alerting both
departments of needed supplies quickly. The simulation clearly demonstrated that following the new
protocol decreased time from ordering to administering blood supplies and medication. The model
provided Labor & Delivery opportunities to experiment with changing variables within the simulation to
deliver better care to new mothers and hemorrhagic emergencies.
77
WEDNESDAY, 5 THE EFFECT OF DIFFICULTY LEVELS WITHIN A VIRTUAL MEDICAL
SIMULATION
Matthew Hackett, Kevin Fefferman
ARL HRED-STTC
Orlando, FL
Steve McIlwain, Bradley Willson
Applied Research Associates, Inc.
Raleigh, NC
Virtual environments provide medical professionals a risk-free setting to practice their skills. Within these
environments, medical professionals receive training to reinforce triage, communication, and treatment
protocols. Recently, researchers created a medical virtual environment geared towards Combat Medics,
focusing on step-by-step training for individual medical procedures. The software requires trainees to
manage their aid bag and utilize the appropriate equipment for each procedure. To introduce a ‘crawlwalk-run’ training modality, researchers implemented a difficulty system into the simulation. In the novice
level, simulated patients present readily apparent symptoms with no complications. In the intermediate
level, simulated patients have multiple injuries with complications occurring throughout the scenario. The
advanced level includes multiple casualties, requiring triage skills, in addition to the skills required at the
intermediate level. By altering the difficulty level, researchers studied the impact on trainees in terms of
cognitive load and performance. Researchers then conducted a usability study to further evaluate the
performance of the system. The results of these studies are reported, including conclusions and discussion
regarding successful implementation of difficulty systems within virtual training applications.
THURSDAY, 4 DECEMBER, 2014 ROOM S320B
S-8 Synthetic Environment
0830
A Paradigm Shift in the Test and
Evaluation of Terrain
Databases (14200)
0900
Measuring the Impact of Natural
Environment Representation
on Combat Simulation Outcomes
(14305)
0930
Implementation of Real-time Snow
Layers in Game-based
Simulation (14361)
Notes
78
A PARADIGM SHIFT IN THE TEST AND EVALUATION OF TERRAIN DATABASES
Thomas Kehr, Trey Godwin
US Army – PEO STRI
Orlando, FL
Ryan McIntire
Leidos, Inc.
Orlando, FL
The Synthetic Environment Core (SE Core) program is a primary provider of Terrain Databases (TDB) for
the US Army’s training and simulation systems. Through an open format, non-proprietary, image
generator independent TDB generation process, SE Core produces terrain and models to link Live, Virtual,
and Constructive domains into a common operating environment. The test and evaluation processes of
these Terrain Databases for US Army virtual simulators have taken many forms over time. Until recently,
this process has been disjointed and often far removed from the system level testing of the major database
consumers. As a primary TDB provider for the US Army simulation and training, the SE Core program
has taken steps to improve the testing process by developing the Major Evaluation of Geospatial Areas
(MEGA) Review. By incorporating the test procedures of SE Core’s primary virtual customers, the Close
Combat Tactical Trainer (CCTT) and the Aviation Combined Arms Tactical Trainer (AVCATT), SE Core
has developed an efficient test process that incorporates incremental system-level testing early on in the
TDB test schedule. The new process also involves a series of set milestone review events designed to
gather input from Subject Matter Experts (SMEs) and site representatives. To better understand the new
process, this paper will first provide a brief historical background on test and evaluation of Program
Executive Office for Simulation, Training, and Instrumentation (PEO STRI) virtual TDBs followed by its
evolution into the current MEGA Review process. Additionally, the paper will present lessons learned
throughout the development of this process along with the cost, schedule and performance benefits
realized.
MEASURING THE IMPACT OF NATURAL ENVIRONMENT REPRESENTATION ON
COMBAT SIMULATION OUTCOMES
Karl D. Pfeiffer
Atmospheric and Environmental Research
Hampton, VA
Theresa Tamash
Dignitas Tec hnologies
Orlando, FL
Weather affects military operations, and simulated military operations should be similarly affected if these
simulations are to deliver value to training, mission rehearsal, acquisition and other simulation-enabled
communities across the DoD. Immersive simulations must derive human visibility with explicit or implicit
assumptions about temperature, dew point and aerosol content in the space between player and target.
Simulations of land, sea and air vehicles must make some set of assumptions about trafficability, wave
heights, and turbulence or wind shear, even if the assumption is that these conditions are benign. The space
environment, ionosphere and sensible weather (e.g., rain showers or thunderstorms) dramatically impact
real command, control, communications, and computers (C4), and simulated C4 systems should be
similarly, realistically affected. Achieving this level of fidelity in constructive simulations requires an
authoritative representation of the natural environment driving a set of validated, calibrated behaviors
within these simulations.
In this study, the Army OneSAF simulation system is used to revisit combat operations in the early days of
Operation Iraqi Freedom during a severe and extended dust storm event (March 25-27, 2003). With the
passage of a long, dry cold front through the region, sand and dust obscured visibility in the lower
atmosphere in a wide swath down to the Arabian Gulf. This event limited ground, air and maritime
operations and drove commanders in the field to operational and tactical improvisation. Using an
authoritative representation of weather in southern Iraq, OneSAF simulation outcomes are examined and
these results compared to known combat outcomes and mission limitations. These results provide insight
and a starting point for improving model behaviors in OneSAF and other simulation systems.
79
IMPLEMENTATION OF REAL-TIME SNOW LAYERS IN GAME-BASED SIMULATION
Dr. Michael D. Woodman & Peter Morrison
Bohemia Interactive Simulations
Orlando, Florida
While many games have “snow” environments, they are only artists’ representations. There is a
requirement for game-based simulations to provide a realistic, real-time virtual snow environment for
militaries that operate in snow. Among the many considerations for simulated snow are changing snow
depth over time; depth of snow based on slope angle and direction relative to sun and wind; and varying
snow depths on and around buildings, under trees, and on roads (which may be plowed). Because of these
considerations, the snow layer is not uniform; it will require a tremendous amount of data, especially for
large maps. Therefore, we decided to generate the snow procedurally, using optimized rendering. A
simulation is not static; we have to consider the interactions: vehicles must have particle effects from
driving through snow, they must leave tracks, they may sink into the snow, and they may plow through the
snow. We need to calculate the force acting on each wheel for PhysX vehicles with a defined “floating
zone” from the top of the snow layer which will depend on snow density. Of course, this also affects the
snow height where the vehicle has driven, so we must update the height of the snow each time we simulate
an object. The equation will take into account the mass of the object and the snow density. Similarly,
soldiers are affected by the snow simulation; we must consider the increased difficulty of foot movement
as well as the tracks left behind. This paper will discuss the tradeoff decisions, engineering solutions to
creating snow layers, and lessons learned in developing snow layers for simulation. It will be of great
interest to attendees who are considering implementing snow in their simulations, as it is important to
understand the complexity of such a task.
S-9 Emerging Visual Technologies
0830
Integration of Low-cost HMD
Devices in Existing Simulation
Infrastructure (14190)
0900
Simulating Realistic Light Levels in
Next Generation Image
Generators (14233)
0930
Vergence and Accommodation in
Simulation and Training with
3D Displays (14147)
Notes
80
INTEGRATION OF LOW-COST HMD DEVICES IN EXISTING SIMULATION
INFRASTRUCTURE
Tomer J. Michael, Yaniv Minkov & Rami Rockah
IDF Ground Forces Command Battle-Lab
Tel-Aviv, Israel
Recent years have seen a sharp increase in the availability of low cost Head Mounted Display Devices or,
as they are colloquially referred to, "VR Goggles". These devices pair the live tracking of the orientation
of their user's head, with a full stereoscopic view of a 3d environment. Thus, providing users with the
illusion that they been transported to a virtual world where they are free look around in a realistic manner.
It is this functionality that brought the HMD to the attention of the research simulation world, in particular
because of the field's vested interest in providing its test subjects with the most realistic experience
possible within a virtual environment.
However, the task of integrating the HMD presented a set of unique challenges. From the logistical, such
as the lack of visibility between the human subject and input devices, to the physiological, such as the
potential and prominent increase in so called "simulation sickness" (a subset of motion sickness),
sometimes associated with even short encounters with the device. These phenomena raise questions in
regards to the HMD's usefulness in research environments, where unintended side effects directly clash
with the realism of the virtual environment and, by extension, with the validity of a given experiment's
results.
This paper describes an attempt made between 2013 and 2014 to integrate Oculus VR's "Rift" HMD with
the IDF Ground Forces Command Battle-Lab's existing simulation infrastructure. It discusses solutions
and lessons learned for the integration of the device, technical hurdles encountered in making an HMD
work with simulators built on existing frameworks like Vega Prime and Virtual Battlespace, and the
application of different methodologies - explored for setting up or converting different simulators for use
with an HMD - and their respective effectiveness with human participants.
SIMULATING REALISTIC LIGHT LEVELS IN NEXT GENERATION IMAGE GENERATORS
Brett Chladny
Renaissance Sciences Corporation
Richardson, TX
Kenny Hebert
Birmingham, AL
Brad Colbert
San Diego, CA
The dynamic range of modern day display systems have greatly improved over the past few years.
However, they are still not capable of displaying the full intensity range that the human eye is capable of
perceiving. The computational power of Graphical Processing Units (GPUs) that are incorporated into
modern day Image Generators (IGs) have greatly improved over the past few years, particularly when
operating on floating point values. Rendering the entire scene as accurate in-band radiance values enables
sophisticated processing to be applied that can help compensate for the limited dynamic range of modern
day display systems. The resulting realism can significantly improve training when high contrast scene
content is present. Examples include landing on an aircraft carrier that is steaming into the sunset and
spotting an entity that is in the direction of the sun. Furthermore, rendering light points using in-band
radiance values improves training by providing pilots with realistic visual representations that can take the
display’s limited dynamic range into account. This can be critical to training when displaying navigational,
runway, and anti-collision lights is required. This paper shows results from a new experimental IG that
incorporates both accurately rendering the scene using floating point radiance values as-well-as post
processing the resulting values to compensate for the dynamic range of the display system. This processing
is accomplished by mimicking various aspects of the Human Vision System while still maintaining the
commonly required 60 Hz update rate.
81
VERGENCE AND ACCOMMODATION IN SIMULATION AND TRAINING WITH 3D
DISPLAYS
David L. Page, PhD; C. E. (Tommy) Thomas, PhD; Steve L. Kelley; Paul G. Jones
Third Dimension Technologies
Knoxville, TN
David A. Miller
AYA Associates, Inc.
Savannah, GA
Mismatches in vergence and accommodation cues can lead to visual discomfort and possibly display sickness.
Conventional 3D glasses-based—and more recently lenticular-based—displays create unnatural conflicts
between vergence and accommodation. Accommodation and vergence cues differ for a far object compared to a
near object. The cues are based on object depth and thus are an important consideration that is often ignored in
3D displays. When vergence and accommodation cues are not correctly reproduced, the information conflict
inside the human brain can lead to asthenopia, i.e., visual fatigue, headaches and sickness. Accommodation and
vergence responses are normally coupled, which is to say their cues are not independent of one another.
Specifically, accommodative changes evoke vergence changes (accommodative vergence), and vergence
changes evoke accommodative changes (vergence accommodation). Thus, having correct or nearly correct
vergence-accommodation cues is important to comfortable, long-term viewing of 3D displays. Holographic 3D
displays, as a step beyond glasses-based and lenticular-based approaches, offer the promise of reproducing all
human visual cues, including matched vergence accommodation cues, to enable 3D displays to provide truly
immersive environments for simulation and training. This paper presents the vergence-accommodation problem
in the context of conventional 3D displays as well as with emerging holographic 3D displays, specifically
holographic stereogram-based displays. The paper defines holographic stereography and discusses the principles
using an electronic version developed by the authors, known as the Holographic Angular Slice 3D Display
(HAS3D). The paper concludes with experimental results based on an operational prototype of the HAS3D
display relative to game-based simulation and training environments.
S-10 IG, Synthetic Environment and Scalable Simulation
1030
A Distributed Scene Graph
Approach to Scaled Simulation
Based Training Applications
(14033)
1100
Pseudo-Specific High-resolution
Data Boundary Techniques
(14317)
1130
Solving the Innovator’s Dilemma for
Simulation and Training
Image Generator Architectures
(14373)
Notes
82
A DISTRIBUTED SCENE GRAPH APPROACH TO SCALED SIMULATION-BASED
TRAINING APPLICATIONS
Douglas B. Maxwell
Joe Geil, William A. Rivera
U.S. Army Research Laboratory, Human Research and
Engineering Directorate
Orlando, FL
Orlando, FL
Dr. Huaiyu Liu
Intel Research
Hillsboro, OR
Current infantry training simulators are based on first person shooter gaming products and have been used for
many years for individual and small unit training. There is a need for a broader application of simulation-based
training systems to train multiple small teams in concert or larger unit operations. Additionally, the systems will
need to accurately present the operational area with larger numbers of civilian and opposing forces. This
requires a simulation-based trainer to scale from currently tens of users to hundreds of users and entities in the
same virtual space at the same time. The biggest limiting factor for this activity has been the inability for the
backend simulation architectures of the first person shooters to simultaneously broker the large numbers of
entities needed to support the scaled simulation. The U.S. Army Research Laboratory’s Simulation and Training
Technology Center (ARL STTC) and the Intel Corporation entered into a Cooperative Research and
Development Agreement (CRADA) in February of 2013 to address core simulation scaling issues. The
ARL/STTC and Intel Corp. performed a series of five joint scalability experiments over the summer of 2013 to
test new prototype architectures that support scaled operations. These scalability experiments were open to the
public and included volunteers from industry and academia. The experiments were able to show significant
increases in the number of humans who could log into a coherent training simulation and interact with each
other while performing a mission. This paper will present the results of one of the events, including the data
collected from the distributed simulators which were located at various locations across the continental United
States. We will discuss the architecture of the prototype simulator, provide performance findings, the statistical
approaches used to analyze this data and provide an interpretation of findings. Finally, we discuss a model
developed from the autonomous agent simulator loads and compare it to the performance of the simulators when
loaded with large numbers of human users.
PSUEDO-SPECIFIC HIGH-RESOLUTION DATA BOUNDARY TECHNIQUES
Daniel J. Lowe & Michael A. Cosman
Rockwell Collins
Salt Lake City, UT
Training requirements for a variety of platforms are quickly expanding to include larger and larger gaming
areas in response to customer demand and the availability of data. However, there still remain several
drawbacks to using worldwide high-resolution photo-specific data: size of the data, the ability to correlate
data with sensor and SAF versions, the time required to validate and correct data. Instead using autogenerated simulation models coupled with real-world data to quickly and economically create training
environments remains an attractive option.
This paper describes two techniques recently developed to build realistic terrain texture that is pseudospecific data (from low resolution data, i.e., Feature Identification Codes, or FICs). When using low
resolution theme data resulting textures can appear "blocky" and unnatural. One way to improve this is to
super-sample the boundaries between themes to a higher resolution in such a way that they appear more
natural and less blocky when viewed up close. Stencils are defined for blending two or more theme types
to create natural looking edges. Multiple stencils applied in specific ways are used to vary edges thereby
avoiding repeating image patterns. Next, the super-sampled theme data is used with correlated templates of
three-dimensional features to generate 3D content on-the-fly without the need of "pre-compiling" or
"publishing" the database. The end result is the appearance of higher resolution terrain texture with
accurately correlated 3D features.
83
SOLVING THE INNOVATOR’S DILEMMA FOR SIMULATION AND TRAINING IMAGE
GENERATOR ARCHITECTURES
Bob Grange, Michael Cosman, Nephi Lewis, & Brad Southwick
Rockwell Collins
Salt Lake City, UT
Today, high performance image generators can be built utilizing Commercial, Off-The-Shelf (COTS) PC hardware,
graphics cards and operating systems, leveraging custom software at several system levels. Image generators (IGs)
based solely on COTS PC technology and custom software produce impressively powerful simulations within the
COTS constraints on memory size, processor speed, processor algorithms, multi-threading, and PC graphics video
outputs. This technology is being employed for fast-jet training for the F-35 “Lightning II” Joint Strike Fighter (JSF),
FAA/EASA level D, ground warfare, part-task trainer, unmanned aerial vehicle (UAV) and dismounted infantry
applications.
Purpose-built rendering hardware also delivers impressive and powerful simulations by employing COTS FieldProgrammable Gate Array (FPGA) technology to create targeted rendering solutions that exactly meet specific
simulation and training requirements. Considering baseline hardware costs, these systems are expensive (today), but
deliver higher quality imagery and more effective training scenarios because they are uninhibited by third party PC
graphics card constraints. Today, this technology is being delivered on various devices, including those requiring
FAA/EASA level D fidelity, weapons and targeting simulations in various sensor domains, and for multi-crew tactical
helicopter training devices like the Apache Longbow Crew Trainer for the pilot and copilot gunner stations.
PC graphics technology, largely driven by the video game industry and its variants, is here referred to as gameCOTS.
FPGA technology, when delivering purpose-built image generation systems, is here referred to as simCOTS because it
specifically emphasizes simulation training requirements. This paper compares and contrasts these two innovative
rendering approaches to highlight the need for the simulation industry to employ a broad variety of solutions in
effecting world-class training solutions, across the training spectrum, that remain squarely positioned on the cost-value
curve.
S-11 Automation and Autonomy
1030
A Framework for Enabling Virtual
Observer Controllers in
Synthetic Training (14268)
1100
UAV Flight Control Software
Development based on COTS
Product (14206)
1130
Lessons Learned in Creating an
Autonomous Driver for OneSAF
(14106)
Notes
84
A FRAMEWORK FOR ENABLING VIRTUAL OBSERVER CONTROLLERS IN SYNTHETIC
TRAINING
Mandira Hegde, Dan Allison, Todd W. Griffith, Ph.D.
Discovery Machine, Inc.
Williamsport, PA
This paper presents a modeling framework to enable the creation of custom virtual observer controllers
(VO/Cs) to help naval students meet training objectives. First, we describe an approach to create behavior
models to drive the behavior of automated entities used in synthetic training. Next, we describe an
approach to create behavior models supporting the functions of intelligent VO/Cs. Specifically, we
describe the development of an authoring console and training task blocks used to create custom VO/C
architectures as well as the creation of student cognitive process models representative of varying training
proficiency levels. We also describe a method to integrate these models into simulations used in synthetic
training and a communication architecture supporting communication between VO/Cs and our behaviordriven automated entities. We end with a use case of a custom VO/C running end-to-end in the Joint SemiAutomated Forces™ (JSAF) simulation in a Navy Anti-Submarine Warfare (ASW) training scenario.
Although the Navy ASW training domain is the focus of the examples described in this paper, the
modeling framework described is not domain specific.
UAV FLIGHT CONTROL SOFTWARE DEVELOPMENT BASED ON COTS PRODUCT
Jung-ho Moon & Da-hyoung Jeon
Yeong-cheol Kim
Korean Air, R&D Center
Agency for Defense Development
Republic of Korea
Republic of Korea
Flight management and control software is the most safety-critical software of Unmanned Aerial Vehicle (UAV)
and it has to be verified by several development steps such as simulation, unit test, formal test, stand-alone test
and hardware integrated test based on hardware-in-the-loop simulation environment. To support these activities,
a variety of software packages are required such as flight control software, flight dynamic software, avionics
model, image generator, software test tool and operation training simulator. To increase development efficiency,
these tools have to be integrated and share a core data and models with various embedded hardware
components. Model-based development (MBD) technology and commercial off-the-shelf (COTS) products
could be helpful for engineers to cover various areas of software development and test where there are limits to
costs and time. This paper describes MBD application on the flight control software, hardware-in-the-loop
simulator, software test, and operation training simulator (OTS). The flight control software was developed
using Matlab/Simulink®, and engineers performed unit testing and system testing using a Hardware-In-TheLoop Simulator (HILS), and an operation training simulator. The HILS was embedded on dSPACE® integrated
with image generator based on PrePar3D®. HILS real time data is distributed using equipment based on AGI
STK® to visualize communication links and flight data. Most of the dynamic and subsystem models were
developed in Simulink and then C-code generated to reduce development efforts dramatically. This includes the
development of gimbal dynamics, aircraft dynamics, avionics model, data link model, target calculation, and
tracking models. This paper presents an integrated development environment for UAV flight control software
that uses model based development technology and COTS software. It further details the software development
environment, testing, hardware integration and verification capabilities. This environment was linked with an
operational training simulator to evaluate the camera guided modes and radar based automatic landing system
verification and the results are presented as part of this paper.
85
LESSONS LEARNED IN CREATING AN AUTONOMOUS DRIVER FOR ONESAF
Dr. Jonathan Stevens, Latika Eifert
Army Research Laboratory (ARL)
Orlando, FL
Dean Reed, Eugenio Diaz
Orlando, FL
Oleg Umanskiy
STILMAN Advanced Strategies
Denver, CO
The high cost of live training has always been a major challenge for the military. This challenge will only
grow as current fiscal uncertainty leads to declining training budgets. Constructive simulations, such as
One Semi-Automated Forces (OneSAF), have shown to partially reduce some costs associated with
warfighter training. However, further cost reductions in simulation are always sought to ensure that
simulation remains an attractive training option for the Commander. The Army Research LaboratoryHuman Research and Engineering Directorate, Simulation and Training Technology Center focused on an
effort to lessen costs by creating an automatic 'driver' for OneSAF with less need for human intervention.
For this initial effort, we describe how the Linguistic Geometry Real-Time Adversarial Intelligence and
Decision-making (LG-RAID) lightweight simulation generated and sent to OneSAF tactically valid
cooperative entity behaviors for an entire company-size-force of friendly and enemy combatants. We
discuss how this was accomplished, for both scenario creation as well as scenario execution. For this initial
paper, our results primarily focused on scenario creation, with follow-on studies concentrating on scenario
execution. This paper describes key principles developed behind the 'driver' and offers potential areas for
future research based upon our lessons learned in this study.
S-12 Engineering Based Modeling
1330
How the U.S. Navy is Migrating
from Legacy/Large Footprint to
Low Cost/Small Footprint Sonar
Simulation Systems (14090)
1400
Use of Automated Intelligent
Entities in ASW Simulation
(14109)
1430
A Physics-Based Approach to
Simulate Jet Engines (14030)
Notes
86
HOW THE U.S NAVY IS MIGRATING FROM LEGACY/LARGE FOOTPRINT TO LOW
COST/SMALL FOOTPRINT SONAR SIMULATION SYSTEMS
Mr. Sean M. Reilly
The AEgis Technologies Group, Inc.
North Kingstown, RI
Mr. Jonathan Glass
Naval Air Warfare Center Training Systems Division
Orlando, FL
This paper describes the migration process undertaken by the US Navy to migrate from legacy, large
footprint mainframe computer-based sonar simulation systems to next-generation sonar simulation systems
with a smaller footprint, lower costs and better accuracy than the legacy models. The paper will describe
the development efforts to create a faster and more accurate acoustic transmission loss (TL) and
reverberation model for sonar simulation/stimulation systems in littoral environments based on ray theory
for active sonar frequencies (above 1000 Hz). The paper also describes how the next-generation model
augments ray theory with Gaussian beam techniques (based on the Gaussian Ray Bundling or GRAB),
which enables simulation of frequencies as low as 150 Hz. The paper will detail the integration challenges
faced by the US Navy to migrate from the legacy models to the next-generation sonar simulation model
into the Navy’s Live Virtual Constructive Modeling and Simulation (LVCMS) product line that includes
PACT3, BATTT, and EFAAS simulators/simulations. The paper will also describe the results of these
integration efforts, including the ability to provide trainees with improved training via more complex
scenarios in the LVCMS training suite without increasing their hardware costs or footprint.
USE OF AUTOMATED INTELLIGENT ENTITIES IN ASW SIMULATION
Geoff Tompson
Jared Snyder
Decisive Encounters Limited
Discovery Machine Inc.
London, UK
Kongsberg, Norway
Williamsport, Pa, USA
As defense budgets are cut, assets and personnel are being increasingly stretched to meet operational tasks,
making it ever more difficult to allocate platforms and subject matter experts (SMEs) to train the next
generation of operators. One key area where there is a shortage of platforms and experienced SMEs is in
anti-submarine warfare (ASW). At the very time when many countries are purchasing sophisticated
submarines and the potential submarine threat is increasing, fewer operational submarines and SMEs are
available for training tasks. An innovative solution to these shortages is to use an ‘expert system’ ASW
simulator employing automated intelligent entities to generate realistic threat actions. This innovative
solution provides added benefits as it improves the quality of simulator training whilst reducing the
workload on the available ASW simulator instructors. This Paper describes the process used for collecting
expert knowledge and then using that knowledge to create the automated intelligent behaviors employed
by automated intelligent entities in simulators. The collection process enables SMEs to ensure the
behaviors represent tactically realistic actions and, for the highest quality simulation, that they do not
become predictable. Therefore, for any tactical situation, the system must select the most appropriate
behavior and the entity should react realistically to the tactics employed against it by the student. Such
autonomous entities allow instructors to perform complex maneuvers and actions with a low level of
interaction with the simulation. An additional benefit of the low level of interaction with the simulation is
the reduction in the instructor's workload, giving them more time to focus on the overall simulator exercise
objectives. As an illustrative example, we present a case study of a system created for the Royal
Norwegian Navy (RNoN), which now uses such automated intelligent behaviors in its ASW simulator.
Morten Kolve
Kongsberg Defence Systems
87
A PHYSICS-BASED APPROACH TO SIMULATE JET ENGINES
Sami S. Mina
Rockwell Collins Simulation and Training Solutions
Sterling, VA
Creating jet engine simulations that replicate the behavior of actual engine parameters at finite flight conditions is only
one step toward meeting the requirements for pilot training. Reproducing realistic performance trends throughout the
flight envelope and generating proper responses to malfunctions and pilot-initiated events, including secondary and
cascading effects, is critical to achieving positive pilot training.
Traditionally, jet engine simulation for pilot training purposes is based on table-lookup of steady-state engine
parameters, such as rotor rotational speed, fuel flow, exhaust gas temperature, engine pressure ratio and net thrust.
This approach does not inherently meet all the aforementioned requirements and exhibits the following shortcomings:
The dynamic engine performance has to be approximated as a lagged transition between steady-state points. It is
unreliable to predict the behavior of the engine parameters when excursions outside the bounds of the tables take
place. Malfunction effects have to be programmed individually for each engine parameter and for different flight and
operational conditions. Additionally, the interdependencies between the different engine parameters can be violated
during the model tuning process. Accordingly, a new approach to model jet engines is needed.
The objective of this paper is to present a physics-based jet engine simulation approach which addresses the
shortcomings of table-lookup solutions, is data-driven and generic, while also distinguishing itself from other physicsbased simulations (Claus, Townsend, 2010) by being computationally efficient. This approach can be used to simulate
any turbojet or turbofan engine by accounting for the physical processes and the geometric and mechanical
characteristics that govern the performance and behavior of the engine. These include the fan, compressors and
turbines maps, the rotors inertia, and the thermodynamics of the flow entering the engine from its free-stream state
ahead of the engine intake, through the intake duct, the fan, the compressors, the combustion chamber, the turbines and
the nozzles.
The paper discusses the methodology used in applying the physics-based approach to simulate a two-spool turbofan
engine, the technical challenges involved and demonstrates how this new approach advantageously compares with a
table-lookup model in matching actual flight test data and in providing realistic performance trends.
The paper also assesses the physics-based approach’s ability to meet the requirements of the different levels of flight
simulators and flight training devices, as defined in FAR 14 CFR Part 60.
TUESDAY, 2 DECEMBER, 2014 ROOM S320D
T-1 Keepin’ It Real with VR
1400
Declarative Knowledge Acquisition
in Virtual Learning
1430
Virtual World Room Clearing: A
Study in Training Effectiveness
(14045)
1500
Simulating Participant Training Data
to Test Mixed-reality
Training Systems (14252)
Notes
88
DECLARATIVE KNOWLEDGE ACQUISITION IN IMMERSIVE VIRTUAL LEARNING
ENVIRONMENTS
Rustin Webster, PhD
Intuitive Research and Technology Corporation
Huntsville, AL
Motivated by a learners’ general lack of engagement and passive receiving of information from lectures,
and the increased use of interactive media from the millennial generation, the author investigated the
interaction effect of immersive virtual reality (VR) in the classroom. The objective of the project was to
develop and provide a low-cost, scalable, and portable VR system containing purposely designed and
developed immersive virtual learning environments for the U.S. Army. The purpose of the mixed design
experiment was to compare lecture-based and immersive VR-based multimedia instruction, in terms of
declarative knowledge acquisition (i.e. learning) of basic corrosion prevention and control with military
personnel. Participants were randomly assigned to the control group (N = 115) or investigational group (N
= 25) and tested immediately before and after training. The author accessed learning outcomes from the
pre- and post-exam scores and VR system usability from exit questionnaires. Results indicate that both
forms of instruction will increase learning. VR-based did produce higher gain scores and there was a
statistically significant interaction between instruction type and time. Lecture-based instruction continues
to be a cheaper and more efficient method for large group settings while VR-based instruction advocates
individual training, active learning, and condensed training time.
VIRTUAL WORLD ROOM CLEARING: A STUDY IN TRAINING EFFECTIVENESS
Stephanie J. Lackey, Julie N. Salcedo, & Gerald
Matthews
Institute for Simulation & Training
Orlando, FL
Douglas B. Maxwell
U.S. Army Research Laboratory
Simulation and Training Technology Center
Orlando, FL
Large-scale live training exercises require significant funding investments in personnel, equipment, and
other resources. Given the current state of budget constraints, Simulation-Based Training (SBT),
specifically, Game-Based Virtual Environments (GBVE), represents an opportunity to alleviate such
challenges. However, from the SBT perspective, scalability (e.g., the number of simultaneous trainees
supported) and flexibility (e.g., resources required to build and render new scenarios, real-time scenario
adaptation) hinder the ability of GBVEs to address large-scale training. Virtual Worlds (VW) offer a viable
solution to resolve the challenges facing the U.S. Army’s existing training paradigm. Emerging research in
the field of VW training for operational tasks seeks to understand where in the training cycle such
technology is most beneficial, how to implement such capabilities, in addition to the return on investment.
The U.S. Army Research Laboratory Simulation and Training Technology Center has entered into a
cooperative agreement with the University of Central Florida to conduct research that includes a series of
empirical evaluations of VW training through 2017. Ultimately, this body of research will result in
empirically-driven recommendations for designers, developers, and decision-makers within the training
systems acquisition community. This paper presents the findings from the initial field study focused on the
effectiveness the VW training provided during a refresher room clearing battle drill event involving 64
reserve unit Soldiers. This Training Effectiveness Evaluation (TEE) compared the impact of traditional
classroom training to VW training on performance outcomes, stress, and workload. The results reported
from this inaugural TEE provide the foundation for future research.
89
SIMULATING PARTICIPANT TRAINING DATA TO TEST MIXED-REALITY TRAINING
SYSTEMS
Ken Kopecky, Eliot Winer
Ames, Iowa
Julio de la Cruz
Army Research Laboratory HRED-STTC
Orlando, FL
As simulation-based, mixed-reality, and virtual reality training systems are more widely adopted in the military, the
process of verification and validation (V&V) for these systems becomes similarly more complicated and
timeconsuming.
It is critical to verify and validate these simulation-based training systems so they operate properly and as expected.
Often, live trainees are brought in, hardware set up, and different configurations of a system tested as part of the V&V
process. No actual training has occurred. In much the way that these simulation systems act as a substitute for live
action, the subsystems and trainees that use training systems can also be replaced with simulations to dramatically
speed-up V&V. This paper examines the potential for replacing live trainees and hardware with virtual simulations in
a mixed-reality training environment for the purposes of V&V of a simulation training system. ! A case study is
presented, composed of a method that allows multiple tracking systems, from different vendors, to be combined into a
single system. The system, used in a large mixed-reality training environment, allows different aspects of the physical
layout to be tracked depending on the training being performed. In order to test the system’s robustness, virtual
tracking data was generated, having been calibrated from actual tracked entities, to test metrics including positional
error correction and data throughput capability. Comparing this data with results obtained using real tracking hardware
allowed the development of models to predict the system’s behavior in new situations, such as the introduction of a
new tracking system, or introducing a second tracked space to the training simulation. Using simulated tracking data,
errors were identified in the system without the need for testing with humans or additional equipment. Finally, the
virtual data was used to test the simulation itself, to ensure it would handle the data requirements encountered during
actual training. Results indicated that simulated data can be used to test the various factors necessary for V&V of a
simulation-based training system. The training system behavior was the same whether actual or simulated data was
used. The use of simulated data allowed scenarios to be tested without the need to bring in additional human and
equipment resources.
TUESDAY, 2 DECEMBER, 2014 ROOM S320D
T-2 Nothing Here To See
1600
Enhancing the Utility and
Effectiveness of Combat Medic
Simulation (14075)
1630
Training Effects for First-responder
Competency in Cholinergic
Crisis (14241)
1700
Simulation Environments for
Offshore Oil and Gas Emergency
Training (14344)
Notes
90
ENHANCING THE UTILITY AND EFFECTIVENESS OF COMBAT MEDIC SIMULATION
Danielle Julian, John Killilea, Patricia Bockelman, Margaret Nolan
MESH
Orlando, FL
Teresita Sotomayor
ARL-HRED STTC
Orlando, FL
The U.S. Army Research Laboratory (ARL), Human Research and Engineering Directorate (HRED), Simulation and
Training Technology Center (STTC) enhances warfighter readiness through research and development of engineering
solutions by placing the right technology in the hands of soldiers in the shortest time. To accomplish that goal, the
STTC supports training transformation and the promotion of learning to reach diverse Army specialties, such as
medical training. Contributing to the efforts of specialized Army medical training, the present work expands on
research the team conducted under a 2013 Front-End Analysis (FEA) examining Army nursing training gaps and best
practices. Participants reported the largest barrier to using the available simulators is in creating and implementing
appropriate scenarios.
To determine whether other military medical training personnel report the same barrier, a follow-on FEA was
conducted targeting combat medics (also known as 68W), who serve as specialized warfighters tasked with providing
pre-hospital care under the complex and stressful conditions of conflict. Scenario-based training is an integral part of
their required course for the 68W designation. Effective scenarios that incorporate best practices in the use of available
simulators create the conditions to maximize Return on Investment (ROI).
The data collected from the combat medic FEA will be leveraged to provide recommendations for best practices in
scenario design and Simulation-Based Training (SBT). These recommendations are intended as practical, jargon-free
considerations that training developers and decision makers can apply to combat medic training. The best practices
include, but are not limited to, the following areas: integrating simulation into training curriculum, methods for
debriefing and achieving skill acquisition, methods for achieving critical thinking, confidence and perceived
competency, and evaluation/assessment. The paper will close by mapping out the relationship between successful
scenario development and return on investment for simulation technologies to support the larger STTC mission.
TRAINING EFFECTS FOR FIRST-RESPONDER COMPETENCY IN CHOLINERGIC CRISIS
MANAGEMENT
Madsen JM, Hurst CG
Talbot TB
University of Minnesota Medical
School
U.S. Army Medical Research
Institute for Chemical Defense
(USAMRICD)
Telemedicine & Advanced
Technology Research Center
(TATRC)
Aberdeen, MD
Fort Detrick, MD
Minneapolis, MN
Background: Military and civilian first-responders must be able to recognize and effectively manage mass disaster
casualties. Clinical management of injuries resulting from nerve agents provides different challenges for first
responders than those of conventional weapons. We evaluated the impact of a mixed-methods training program on
competency acquisition in cholinergic crisis clinical management. Methods: We developed a multimedia and
simulation-based training program based on the more comprehensive USAMRICD courses. The training program was
designed to provide first-responders with the necessary abilities to recognize and manage a mass casualty cholinergic
crisis event. Training included a learner controlled multimedia iPad app and hands-on instruction using SimMan3G™
mannequin simulators. We evaluated the impact of the training through a purposively selected sample of 204 civilian
and military first responders who had not previously completed either of the referenced USAMRICD courses. We
assessed knowledge, performance, affect, and self-efficacy measures pre- and post-training using previously validated
assessment instruments. We calculated results using analysis of variance with repeated measures, and with statistical
significance set at p < .05. Results: Analyses demonstrated a significant improvement (p = .000) across all domains
(knowledge, performance, self-efficacy, and affect). Knowledge scores increased from 60% to 81% correct.
Performance scores increased from 16% to 68% correct. Self-efficacy scores increased from 51% to 87% confidence
in ability to effectively manage a cholinergic crisis event. Affect scores increased from 75% to 81% personal comfort
during procedures. Conclusions: These findings could aid in the selection of instructional methodologies available to a
broad community of first-responder personnel in military and civilian service. Although less comprehensive than the
USAMRICD courses, training outcomes associated with this easily distributed instruction set demonstrated its value in
increasing the competency of first responders in recognizing an managing a mass casualty cholinergic event. Retention
outcomes are in process.
91
SIMULATION ENVIRONMENTS FOR OFFSHORE OIL AND GAS EMERGENCY TRAINING
Mr. Randy Billard & Captain Anthony Patterson
Virtual Marine Technology Inc.
St. John’s, NL, Canada
Emergency Response and evacuation training for offshore oil and gas workers has traditionally been
performed using land-based test facilities and live exercises. Due to risks associated with practicing drills
using live exercises, this training has been limited to controlled environments and benign weather
conditions. As offshore oil and gas activities move into harsher environments including deeper waters and
ice covered waters, operators are required to demonstrate to regulators that personnel are prepared for
emergencies in these conditions. Simulation technologies have been specifically created for offshore
personnel to practice emergency scenarios in harsh environments using representative equipment and
immersive virtual environments. A specific example includes the use of a simulator to practice launching
lifeboats in severe sea states and ice. As simulation is adopted by the oil and gas industry as a supplement
to existing training or as an alternative to replace specific drills, operators will be required to demonstrate
that simulators are effective training tools. Human factors studies have been performed to assess how
simulation technologies improve the performance of offshore personnel. These studies assess the value of
simulation as a means to reduce time-to-competence. Validation studies are carried out to measure how
students become immersed in the simulation environment and how accurately the virtual environment
matches the real world. This paper discusses a case study of how a lifeboat simulator was created for the
oil and gas industry and how the technology is used to allow oil and gas personnel to practice safely and
effectively to increase the competence of offshore personnel. The paper also addresses the design
philosophy for developing training technologies to maximize value to the operators, which includes
designing simulators to achieve learning objectives which are derived from recognized training standards
and regulated courses.
WEDNESDAY, 3 DECEMBER, 2014 ROOM S320D
T-3 Comparative Analysis for Clinical Training
0830
Comparison of the Usability of
Robotic Surgery Simulators
(14168)
0900
Forces Applied on Laryngoscope
during Intubation: A Study on
Airway Simulators (14203)
0930
Outcomes from Two Forms of
Pediatric and Neonatal
Intubation Training (14240)
Notes
92
COMPARISON OF THE USABILITY OF ROBOTIC SURGERY SIMULATORS
Alyssa Tanaka, M.S., Courtney Graddy, M.S., Roger Smith, Ph.D.
Celebration, FL
Haider M. Abdul-Muhsin, M.D.
Mayo Clinic
Scottsdale, AZ
The introduction of simulation into minimally invasive robotic surgery is relatively recent and has seen
rapid advancement; therefore, a need exists to develop training curriculums and to identify systems that
will be most effective at improving surgical skills. Several robotic simulators have been introduced to
support these aims, but their effectiveness has yet to be fully evaluated.
Currently, there are three simulators -- the daVinci Skills Simulator, Mimic dV-Trainer, and Surgical
Simulated Systems’ RoSS. While multiple studies have been conducted to demonstrate the validity of each
system, no studies have been conducted which compare the value of these devices as tools for education
and skills improvement. This paper presents the results of an experiment comparing value, usability, and
validity of all three systems. Subjects who were qualified as medical students or physicians (n=105)
performed one exercise on each of the three simulators and completed two questionnaires, one regarding
their experience with each device and a second regarding the comparative effects of the simulators. This
data confirmed the face, content, and construct validity for the dV-Trainer and Skills Simulator. Similar
validities could not be confirmed for the RoSS. Greater than 80% of the time, participants chose the Skills
Simulator in terms of physical comfort, ergonomics, and overall choice. However, only 55% thought the
skills simulator was worth the cost of the equipment. The dV-Trainer had the highest cost preference
scores with 71% percent of respondents feeling it was worth the investment. This work is the second
component of a three-part analysis. In the previous study, the simulators were objectively reviewed and
compared in terms of their system capabilities. The third part will evaluate the transfer of training effect of
each simulator. Collectively, this work will offer end users and potential buyers a comparison of the value
and preferences of robotic simulators.
FORCES APPLIED ON LARYNGOSCOPE DURING INTUBATION: A STUDY ON AIRWAY
SIMULATORS
Matthew Mui, M.S.
University of
Central Florida
College of
Medicine
Christine Allen, Ph.D.
Mojca R. Konia, M.D., Ph.D.
Army Research
University of Minnesota Department of
Laboratory (ARL)
Anesthesiology
Human Research
Directorate (HRED) Simulation and
Engineering
Training Technology Center (STTC)
Jack Stubbs
David Hananel
SimPORTAL &
CREST University
of Minnesota
Excessive forces applied during endotracheal intubation may cause damage to laryngeal structures leading
to patient morbidity and mortality. Simulators are widely used for intubation training, but recent studies
have shown significant differences in airway anatomy as well as forces applied during intubation when
compared to humans. This study assesses the differences in intubation training on three partial-task
trainers, TruCorp AirSim Standard, Laerdal Airway Management Trainer, and VBM Air Management
Simulator Bill I, against cadavers. Objective force measurements and subjective ratings of difficulty and
force used were measured. Using ANOVA and paired t-tests, endotracheal intubation on simulators was
found to have significantly different force profiles (i.e., locations and magnitudes of the applied forces in
comparison to cadavers. In particular, the Laerdal Airway Management Simulator differed in all three
measurement variables, namely torque applied on the laryngoscope, force applied at the laryngoscope tip,
and force exerted on the simulator’s teeth. These findings are further supported by the surveys of the
participants in the Laerdal group. For the TruCorp and VBM simulators, significant differences are found
only in torque and tip forces, respectively. These results suggest that a simulator that offers more realistic
endotracheal intubations may be necessary for airway management training. In addition, this study sets a
foundation for future studies to further elucidate the effects of various airway simulators on intubation
training.
93
OUTCOMES FROM TWO FORMS OF PEDIATRIC AND NEONATAL INTUBATION
TRAINING
Dooley-Hash S, House J
University of Minnesota Medical School
University of Michigan Medical School
Minneapolis, MN
Ann Arbor, MI
Background: Contextually relevant factors within a training environment facilitate performance transfer to applied
clinical settings. Live animals are often used for pediatric and neonatal intubation training because the scale of their
airways and the living condition of the animal are assumed to facilitate transfer of aquired skills to intubating infants
and children, despite species-specific anatomical variation. Mannequin simulators provide an alternative for training,
but have anatomical, physiological, and tissue limitations. We evaluated the impact of two clinical training methods
(live animal v. simulation) on the acquisition of performance abilities in pediatric and neonatal intubation. Methods:
We implemented a quasi-experimental design with purposive sampling to assess performance differences between 294
subjects after completing a training intervention that included either direct interaction with a (1) live animal or (2)
mannequin simulator. All other training elements were identical and followed American Heart Association
(PALS/NRP) clinical protocols. We used validated instruments to assess knowledge, performance, and self-efficacy
outcomes before and after training, and at three retention intervals (6, 18, and 52 weeks). Results: Post-training
outcomes were significantly better that pre-training outcomes for both groups (p < .001), and there were no significant
differences between the outcomes for the two types of training. There were significant differences between the
performance outcomes after 18 and 52 weeks (p < .01), and for cognitive outcomes after 52 weeks (p < .01), all
favoring the simulator training. Conclusions: These findings could aid in the selection of instructional methodologies
that minimize the uses of live animals for instruction in pediatric and neonatal intubation without sacrificing the
quality of training. Variation in retention outcomes are likely due to opportunity for repeated deliberate practice using
a simulator rather than associated with fidelity issues. Improvements in simulator technology would likely improve
training outcomes.
T-4 “Assess”-orizing Your Training and Performance Outcomes
1030
Data & Analytics Tools for Agile
Training & Readiness
Assessment (14064)
1100
Lessons Learned Integrating Mobile
Technology into Two
Army Courses (14128)
1130
Experience API and Team
Evaluation: Evolving Interoperable
Performance Assessment (14157)
NOTES
94
DATA & ANALYTICS TOOLS FOR AGILE TRAINING & READINESS ASSESSMENT
Jared Freeman
Aptima, Inc.
Washington, DC
Denise Nicholson
Orlando, FL
Peter Squire & Amy Bolton
Arlington, VA
The return of American warfighters to their bases and their garrisons presents an opportunity to bolster
scarce training resources and expertise with new assessment technologies. America made a similar
investment in the 20th century as it shifted its intelligence budget to supplement human intelligence
gathering with technologies that unobtrusively captured data concerning the activities of foreign powers.
Here, we present a unifying vision of several emerging technologies that can improve military training.
Following a human systems engineering approach, we first define the functional requirements of future
training and readiness assessment systems, describe the architectural requirements for providing those
functions, and then describe systems for the Marine Corps and Air Force that instantiate this architecture.
Next we focus on two fundamental and new components of this emerging architecture: sensors that capture
human performance data unobtrusively, and big data analytics that make sensor data meaningful and
actionable. Finally, we identify several scientific and technical challenges encountered during the initial
implementation and planned testing of these architectures.
LESSONS LEARNED INTEGRATING MOBILE TECHNOLOGY INTO TWO ARMY COURSES
Gregory A. Goodwin
Simulation and Training
Technology Center,
Orlando, FL
Michael Prevou
Holly C. Baxter
Strategic Knowledge
Solutions
Leavenworth, KS
Heather Wolters
U.S. Army
Research Institute
Fort Belvoir, VA
Mike Hower
Strategic Knowledge
Solutions
Colorado Springs,
CO
Linda McGurn
Engility
Corporation
Leavenworth,
KS
As the Army considers using mobile computing to improve training and assessment, it must be confident
in the benefits of that technology and more importantly, it must be able to articulate the requirements
needed to achieve those benefits. Although mobile devices and software have proven to be extremely
popular in the commercial market, research is needed to identify both the benefits and requirements of this
technology before the Army considers its wholesale adoption for training and education. This paper reports
on the results of using mobile devices in two Army courses: the Signal Captains Career Course (SCCC)
and the School for Command Preparation (SCP). The software developed for the SCCC was an interactive
performance assessment tool for the topic of power distribution while the software developed for the SCP
was a practice tool for media engagement. In the first experiment, 182 SCCC students either took the
traditional paper and pencil practical exercise or the interactive tablet-based version. The tablet-based
version significantly reduced the time needed to complete the exercise (1h vs. 3h) without affecting student
understanding of the topic. In the second experiment 161 SCP students practiced for the final exercise
(mock media engagement) with and without the aid of a tablet-based practice tool. Although the group
using the app reported practicing more, their performance on the final exercise was the same as those who
practiced without the app. These findings indicate that although mobile technologies have the potential to
benefit students and instructors, neither the magnitude nor the type of benefit is easy to predict at this
point. These findings and other lessons learned are used as the basis for a proposed strategy for developing
mobile applications for use in Dept. of Defense training.
95
EXPERIENCE API AND TEAM EVALUATION: EVOLVING INTEROPERABLE
PERFORMANCE ASSESSMENT
Michael Hruska
Problem Solutions
Johnstown, PA
Charles Amburn, Rodney Long
United States Army Research Laboratory
Human Research and Engineering Directorate
Simulation and Training Technology Center
Orlando, FL
Tara Kilcullen
Raydon
Daytona
Beach, FL
Tiffany R.
Poeppelman
Google (Contractor)
London, England
Simulation and training technologies continue to advance the ways we assess individual and team
performance on a range of skills. Given that training is costly, military organizations are developing
solutions for tailored learning since they represent a path to larger efficiencies. While many training
systems can assess and report a trainee’s performance, most have no way to share collected learner data
with other training systems. Collecting data in a complete profile of performance could lead to the ability
to leverage performance data to save time and money training personnel or increase training effectiveness.
Limited interoperability of performance assessment and tracking across training systems continues to
constrain the ability of these solutions to adapt, or personalize, across a lifeline continuum of the learning
experiences. The Advanced Distributed Learning Initiative is supporting community developed
specifications and tools, such as the Experience Application Programming Interface (xAPI). The Army
Research Laboratory (ARL) is exploring the use of the xAPI for Interoperable Performance Assessment
(IPA) to support the assessment of individuals and teams across multiple training systems. These efforts
are beginning to establish best practices to create a “universal language” for Live, Virtual, Constructive,
and Gaming systems to share performance data and provide adaptive learning regardless of the
technologies or platforms used. Previous work established example methods, an architecture, and tools to
capture interoperable data to support individual adaptations. In this paper, we will describe and provide
best practices for this evolving approach of tracking and using team performance data. Tracking this data
in an interoperable way can provide the basis to support both macro and micro adaptations at the
individual level. Practical examples using a single gunner simulator along with team-based data from a
crew trainer will be provided. Lessons learned will also be outlined to inform considerations for approach
and usage.
T-5 Critical Flight Decisions
1400
Developing and Evaluating
Performance Measures for MannedUnmanned Teaming (14024)
1430
Using Temporal Occlusion to Assess
Carrier Landing Skills
(14171)
1500
Distributed Live/Virtual
Environments to Improve Joint Fires
Performance (14041)
NOTES
96
DEVELOPING AND EVALUATING PERFORMANCE MEASURES FOR MANNEDUNMANNED TEAMING
John E. Stewart & Scott E. Graham
Army Research Institute
Fort Benning, GA
Courtney R. Dean
Aptima, Inc.
Woburn, MA
Troy Zeidman
Imprimis, Inc.
Huntsville, AL
The role of U.S. Army unmanned aircraft systems (UAS) is becoming increasingly important in tactical
combat missions. Consequently, training critical skills required for manned-unmanned teaming (MUM-T)
becomes more important, especially for UAS operators. In order to effectively train MUM-T skills,
reliable and valid performance measures are required. Scaled observer-based performance measures can
add objectivity to the process of assessing training outcomes, providing formative feedback, and tracking
team progress. To this end, 36 performance measures were developed and evaluated to assess trainingcritical MUM-T skills. Draft performance measures were developed and refined with input from senior
UAS operators and scout- attack pilots with MUM-T experience. For each performance measure, fivepoint behaviorally-anchored rating scales were produced representing “good,” “average,” and “poor”
performance of the skill. The content validity of the measures and the usability of the rating scales were
determined by a second group of senior UAS operators and scout-attack pilots. Most MUM-T measures
were deemed relevant to the mission and observable. Six measures with low consensus by participants on
relevance and/or observability were determined not to be practically usable. Some of these unusable
measures did not reflect the role of UAS aircrews in current MUM-T operations. The measures were
designed to be collected as “over the shoulder” observations. As such, a trainer, in the live or the virtual
environment, could easily apply the measures. Because the resulting measures use quantitative scales that
include exemplars of good-to-poor performance, they can be easily applied to unit performance assessment
sessions, such as training ” hot wash” and after action reviews.
USING TEMPORAL OCCLUSION TO ASSESS CARRIER LANDING SKILLS
Webb Stacy, Jeff Beaubien, Sterling
Melissa Walwanis
Amy Bolton
Wiggins
Naval Air Warfare Center Training
Office of Naval
Aptima, Inc.
Systems Division
Research
Woburn, MA & Dayton, OH
Orlando, FL
Arlington, VA
Military skills often have perceptual and motor components that need to be trained and measured. An emerging
approach for doing so is temporal occlusion. In this approach, videos are played and stopped just before a key
event, and the participant is asked to make a judgment about subsequent events. The term temporal occlusion
refers to the blacking out of the time period following the key event, forcing the participant to use available
perceptual cues to correctly anticipate what will happen next. The paradigm has been used successfully in
research on sports such as tennis (Ward, Williams, & Bennett, 2002) and baseball (Fadde, 2006) but has not yet
found widespread use in military training (Williams, Ericsson, Ward, & Eccles2008).
In this paper, we will discuss the use of a temporal occlusion paradigm to assess the perceptual skills of expert
and novice pilots as they land on an aircraft carrier. Videos were created from expert landings in a simulator,
and subject matter experts (SMEs) identified the situations that would most clearly require either standard or
aggressive corrections in order to stay on track for a skilled and safe landing. The resulting stimuli were used in
tests administered on an ordinary laptop computer before and after training sessions. The temporal occlusion test
was embedded in an overarching experiment concerning the relation of simulator fidelity to training
effectiveness, and it was used to assess pilots’ implicit perceptual learning during the experiment.
Test results were used to measure the degree to which expert-novice differences on the pretest were reduced in
the posttest; that is, the degree to which novice perceptual performance moved in the direction of expert
perceptual performance. We will discuss the temporal occlusion results from the experiment and will conclude
by discussing several other promising uses of the approach.
97
DISTRIBUTED LIVE/VIRTUAL ENVIRONMENTS TO IMPROVE JOINT FIRES
PERFORMANCE
Emilie A. Reitz, Kevin Seavey
JS J6, Joint Fires Division, CTR support
Alion Science and Technology, Suffolk, VA
An infantry platoon maneuvers toward a village; above them, an unmanned aerial system (UAS) provides
a video feed to the operations center. The platoon reacts to nearby mortar strikes, while a Joint Terminal
Attack Controller (JTAC) assigned to the company Tactical Operations Center (TOC) confirms the target
with radio communication and the UAS’s video. The platoon leader is relieved when an AC-130 takes out
the mortar pit, but has to handle additional hostile contact from the village itself as the insurgents inside
are emboldened by the mortar fire. It takes only a few moments for the situation to spiral out of the platoon
leader’s control; the company commander calls mission end to the virtual scenario. The pilot in the AC130 simulator continues his own training mission. After the after action review (AAR) with the ground
forces, the JTAC turns his attention from the virtual mission and resumes controlling live aircraft from his
observation post (OP) in the desert, 1,500 miles away from the ground forces.
Building on three years of live and virtual environment development during Bold Quest (Reitz & Richards,
2013), BQ 14.2 will assess methods to improve joint fires performance using a mix of distributed live and
virtual training systems. As a first for the Bold Quest live-virtual event, seasoned JTACs will be inserted
into the virtual environment while they are still at their live OP, allowing them to control virtual aircraft in
their natural environment with their real equipment.
This paper discusses the planning, execution and initial results of using a mixed live/virtual environment to
improve individual and team performance in joint fires.
T-6 Combating Stress: Performance Under Fire
1600
Accelerating Unit Adaptability: A
Principle-based Approach
to Unit Communication (14038)
1630
Utilizing Simulation and Gamebased Learning to Enhance
Incident Commander Training
(14148)
1700
Inducing Stress in Warfighters
during Simulation-based
Training (14201)
NOTES
98
ACCELERATING UNIT ADAPTABILITY: A PRINCIPLE-BASED APPROACH TO UNIT
COMMUNICATION
Tara Rench, Zachary Horn, Alexander Walker
Steve Zaccaro
Aptima, Inc.
George Mason University
Woburn, MA
Fairfax, VA
Mission success in today's decentralized military relies increasingly upon highly adaptive decision-making by
small units. Successful adaptation requires units to communicate in ways that facilitate coordination and shared
understanding within and outside of the unit (Marks, Zaccaro, & Mathieu, 2000). However, communication
breakdowns are prevalent within small units, especially when operating in highly stressful environments. While
unit leaders and members are formally trained in communication basics (e.g., how to operate communication
devices, preparing and delivering orders, etc.), it is possible that additional training on the deeper principles of
team communication can promote improved unit awareness, decision-making, and adaptation. In response, a
framework was developed to support training, monitoring, and assessment of Coordinated Tactical
Communications in Teams (CONTACT), particularly in the face of situational stressors that create a need for
unit adaptation. Leveraging existing Navy-funded team communication research (e.g., Bowers, Jentsch, Salas, &
Braun, 1998; Entin & Serfaty, 1999; Smith-Jentsch, Zeisig, Acton, & McPherson, 1998; Waller, 1999) and
operational expertise from active duty Marines, six distinct communication principles were identified:
Relevance; Quality; Timeliness; Frequency; Information Flow; and Confirmation and Response. These
principles provide a common language that help leaders and units align pre-mission communication
expectations, assess and adjust within-mission communication, and conduct post-mission reviews of
communication strategy. Additionally, six situational stressors are described that significantly affect the
application of these communication principles: Uncertainty, Risk, Time Demand, Mental/Physical Demand,
Lack of Unit Familiarity, and Broken Communications. Present to varying degrees in most situations, units must
recognize these stressors and adapt communications appropriately. Feedback from Marine Corps instructors
highlights the promise and utility of the CONTACT framework to help leaders set communication expectations,
assess communication during missions, and hold more efficient after-action reviews (AARs). The CONTACT
framework and its intended training and assessment applications will be discussed in depth in the current paper.
UTILIZING SIMULATION AND GAME-BASED LEARNING TO ENHANCE INCIDENT
COMMANDER TRAINING
Ronald W. Tarr, Eileen Smith
Institute for Simulation and Training/UCF
Orlando, FL.
Eric Totten, Michael Carney
IST/UCF
Chief Michael Wajda
OCFRD
Across the country, Incident Commander training is viewed as a critical part of Fire Officer readiness and safety.
However, this training can often vary from department to department and be rather passive and static. This is the exact
opposite of most fireground scenes, which require Fire Officers to make split second life and death judgments in
rapidly changing environments.
Unsatisfied with traditional training techniques, the Orange County (FL) Fire Rescue Department (OCFRD) partnered
with the University of Central Florida’s Institute for Simulation and Training (UCF-IST) to create a modernized
training program for Lieutenant, Captain and Battalion Chief Incident Commanders (IC). For this project, UCF-IST
conducted a rigorous performance and needs analysis on OCFRD’s Incident Command Academy. This analysis
identified key environmental cues and critical skills that led to specific decision points and tactical direction for ICs.
With this in hand, UCF-IST created an engaging, performance oriented, multimedia training program and unique
immersive simulator that allowed ICs to learn and practice critical skills through scenario-based learning. In addition,
the open-ended nature of the simulation allows for multiple personnel to participate, increasing crew coordination
through collective team training.
Since completion OCFRD trainers have run over 400 Fire Officers through the program which employs pre-training
web modules, step by step instruction and e-learning activities designed to gradually ramp the Incident Commander
into the instructional material and interactive technology, thereby improving their comfort level and acceptance of the
program. The training academy classes were organized by Battalion, which improved unit cohesion while raising the
tactical knowledge level of participating Fire Officers. OCFRD reports near unanimous improvements in the tactical
IC performance and morale of Battalions and their acceptance of this modernized training academy.
99
INDUCING STRESS IN WARFIGHTERS DURING SIMULATION-BASED TRAINING
Meredith Carroll, Brent Winslow,
Christina Padron, Glenn Surpris
Design Interactive, Inc.
Orlando, FL
Jason Wong, Peter Squire
Jennifer Murphy
Quantum Improvements Consulting, LLC
Arlington, VA
Orlando, FL
Over the past decade, the U.S. Marine Corps has shifted its training focus towards enabling effective and
efficient decision making in its small unit leaders. Small unit leaders with relatively little experience are
increasingly required to make tactical decisions with critical second and third order effects. These near
strategic level decisions are not being made in a Command Operations Center (COC), but in the heat of the
battle, where the decision maker is surrounded by high levels of physical and emotional stress. Studies
have shown significant adverse effects of combat stressors on cognitive performance (Lieberman et al.,
2005) as well as persistent changes in brain functional connectivity (Van Wingen et al., 2012). To ensure
military success, and the health and wellness of our veterans, it is critical that these small unit leaders
receive training necessary to develop strategies which enable them to make effective decisions under stress
and mitigate long term physiological and psychological impacts of stress. However, a challenge with
implementing such training in the military is the ability to induce high enough levels of stress to elicit
physiological and psychological responses similar (maybe not in magnitude, but in nature) to those
experienced in combat. Simulation-based training provides a less resource-intensive alternative to live
exercises and greater opportunity for variation in decision dilemmas, situations, and stressors.
Unfortunately, there is little empirically-validated guidance on how to utilize simulation to train decision
making under stress. An approach for integrating cognitive, emotional, and socio-evaluative stressors into
simulation-based training was developed and evaluated in a study conducted with experienced Marines.
The results found significant increases in both physiological stress response (i.e., increased electrodermal
activity), and perceived stress (i.e., State Trait Anxiety Index responses) during this simulation based
training approach, suggesting the method may be an effective means of inducing stress in experienced
Warfighters.
THURSDAY, 4 DECEMBER, 2014 ROOM S320D
T-7 Intelligent Tutors: Just How Smart Are They?
0830
Developing and Evaluating an
Intelligent Tutoring System for
Advanced Shiphandling (14014)
0900
Developing Models of Expert
Performance for Support in an
Adaptive Marksmanship Trainer
(14214)
0930
A Digital Tutor for Accelerating
Technical Expertise (14272)
NOTES
100
DEVELOPING AND EVALUATING AN INTELLIGENT TUTORING SYSTEM FOR
ADVANCED SHIPHANDLING
Jason H. Wong, Lauren Ogren
Naval Undersea Warfare Center
Newport, RI
Stanley Peters, Elizabeth O. Bratt
Stanford University
Stanford, CA
The goal of an Intelligent Tutoring System (ITS) is to improve training efficiency by monitoring student
performance and providing automated tutoring advice with the goal of increasing student learning and
throughput. Traditional ITS development has focused on static problems, such as math and physics (Koedinger,
Anderson, Hadley, & Mark, 1997; VanLehn, et al., 2005). Recent systems have targeted dynamic environments,
such as Navy shipboard damage control and basic shiphandling maneuvers (Iseli, Koenig, Lee & Wainess, 2010;
Peters, Bratt & Kirschenbaum, 2011). The research described here examines the advanced shiphandling task of
mooring to a pier, which is the graduation exercise at the Surface Warfare Officers School (SWOS). To develop
an ITS for mooring, many variables were considered, including ownship parameters (e.g., engine and rudder
status), predicted future paths, and student behavior (e.g., number of orders, gaze direction). This development
process involved creating and vetting a task analysis with SWOS subject matter experts (SMEs) and several
iterations of system prototype testing. An effectiveness evaluation of the prototype was conducted with twenty
novice shiphandling students at SWOS, split into groups that received either human or ITS tutoring only for a
mooring to a pier scenario. Afterward, all students completed another mooring scenario without any tutoring.
Across both runs, performance was evaluated using ship parameters, student behavior, and instructor scoring
metrics. Analyzing a wide variety of performance measures showed no differences between the two groups,
suggesting that the ITS was able to tutor as effectively as human instructors. Future work will involve
developing additional advanced shiphandling scenarios and examining how the student-to-teacher ratio can be
increased using a combination of ITS tutoring and instructor supervision.
DEVELOPING MODELS OF EXPERT PERFORMANCE FOR SUPPORT IN AN ADAPTIVE
MARKSMANSHIP TRAINER
Benjamin Goldberg, Charles Amburn, Keith Brawner
U.S. Army Research Laboratory—Human Research
and Engineering Directorate
Orlando, FL
Marko Westphal
German Federal Office of Bundeswehr
Equipment—Information Technology and
In-Service Support
Koblenz, Germany
The U.S. Army’s Engagement Skills Trainer (EST) uses sensors on simulated weapons to collect valuable
data about a soldier’s performance during marksmanship exercises. That data is available to an instructor
for coaching and remediation purposes. However, experience shows that accessing the data, reviewing the
data, and providing feedback to a trainee can be a time consuming process. This environment presents
challenges when considering the number of trainees who must complete this training and the limited
number of instructors available. This also assumes that instructors are capable of accurately interpreting
the data and applying effective remediation. Simulators like the EST are prime candidates for the
incorporation of an Intelligent Tutoring System’s (ITS) capabilities. The goals of an ITS are to collect data
from a system, make inference on that data as it relates to defined metrics, and to provide formative
feedback when data is found to deviate from a specified standard. For this purpose, a system requires
models to compare data against. In this paper, we will present the results of the first phase of a study to
apply ITS technology to the fundamentals of marksmanship. Models created in this phase will be
integrated into an adaptive training system prototype built within the Generalized Intelligent Framework
for Tutoring (GIFT) for future experimentation. Data was collected across eight experts from the U.S.
Army Marksmanship Unit’s service rifle team as they conducted marksmanship tasks. These models are
built around sensor data collected during execution, with each sensor being selected based on their link to
the fundamentals of marksmanship. We will review the techniques applied to the data for model
construction, trends found in the data that are generalized across each expert, and how the models will be
used to diagnose error and trigger remediation.
101
A DIGITAL TUTOR FOR ACCELERATING TECHNICAL EXPERTISE
J. D. Fletcher
Alexandria, VA
William D. Casebeer
Lockheed Martin Advanced Technology Laboratories
Cherry Hill, NJ
Information Technology is as vital and ubiquitous in the national economy as it is in military operations. A
16-week Digital Tutor for the Navy Information Systems Technicians (IT) rating was produced for
DARPA as a way to accelerate the development of expertise in IT and related areas. Graduates from the
DARPA Tutor program were found by independent, third-party assessment to exceed significantly (p <
0.01) and practically (effect sizes ranging as high as 3.00 standard deviations) the knowledge and practical
troubleshooting skills of IT graduates with 35 weeks of classroom training and sailors averaging over 9
years of IT experience in the Fleet. The Tutor was revised and extended to 18 weeks in order to include
preparation for higher-level IT certifications and topics relevant to civilian employment. Ninety-seven
veterans have completed the newer 18-week version. Eighty-three of these veterans were unemployed after
an average 5 years of separation from active duty. There were no academic dropouts from the course.
Nearly all who sought employment after finishing the course have found jobs averaging $65,000 per year.
This paper briefly reviews learning and economic findings from the Navy assessments, and provides up-todate information on learning, quality of life, and economic findings from the veterans’ project, including
returns to government investment at various scales of implementation. It also provides an overview of
strategies underlying intelligent tutoring systems, those used to develop the Digital Tutor, and practical
tactics the Tutor applies to accelerate acquisition of technical expertise. This paper is important to the
community because it addresses an important topic (Information Technology); the promise of the Tutor’s
technology; the perennial military need to accelerate development of technical expertise; the magnitude of
the assessment findings; and the responsibility to prepare people likely to be separated from the military in
the near future.
T-8 Aviation Training: The Ups and Downs!
1030
Evolving Aviation Live Training in
the Future (14078)
1100
Training Fidelity of an Unmanned
Aerial Systems
Complementary Family of Trainers
(14135)
1130
ASOC Training Research: Joint
Theater Air Ground Simulation
System (14166)
NOTES
102
EVOLVING AVIATION LIVE TRAINING IN THE FUTURE
Wanda Fuentes, Anne Dunlap, Jim Grosse, Tien Pham, Patrick Sincebaugh
U.S. Army PEO STRI
Orlando, FL
Army Aviation is evolving from over a decade of training aircrews with aircraft using analog systems to an
era of digitization, simulation, and embedded processing. Training Centers are adapting to this changing
environment characterized by a common operating environment by reusing hardware and software across
platforms. This evolving environment not only enhances individual aircrew skill levels, but collective air
to ground integrated (AGI) operations training.
The Army’s Aviation Tactical Engagement Simulation System (Aviation TESS) is capable of conducting
individual, crew, and collective training to facilitate unified land and air operations training at the Combat
Training Centers (CTCs) and Home Stations. Aviation TESS material developers must plan for life-cycle
upgrades to adapt and integrate new technology, new tactics, techniques, and procedures (TTPs), and new
training venues. Aviation TESS is currently used to collect aviation data across existing Army Aviation
platforms (Apache, Black Hawk, Kiowa, and Chinook) and, in the future, from the ground component of
the Unmanned Aircraft Systems (UAS).
This paper provides information on current aviation training capabilities, challenges faced by live aviation
training stakeholders, and how PM TRADE will use the principles of Better Buying Power to evolve
training systems to meet long-term Army aviation training requirements.
TRAINING FIDELITY OF A UNMANNED AERIAL SYSTEMS COMPLEMENTARY FAMILY
OF TRAINERS
Sharon L. Conwell, Christine M.
Covas-Smith, Leah J. Rowe
Wright-Patterson AFB, OH
Andrew Shepherd
Sinclair Community College
Dayton, OH
John B. Bridewell
University of North Dakota
Grand Forks, ND
The use of Unmanned Aerial Systems (UAS) in the National Airspace System (NAS) is of concern by a
number of entities. The public is concerned about safety and privacy issues. The private sector is interested
in how to exploit the technology to drive down operational costs and create profit. Finally, the public
sector is concerned about air safety, privacy, and policy issues, as well as, making use of the technology to
drive down operational costs for numerous oversight activities from traffic monitoring to fighting forest
fires. Private and public sector implementation pioneers will be faced with unique challenges. Despite the
name unmanned, there is a plethora of people in the UAS operational loop from air support crews to pilots
to air traffic controllers which creates an extraordinarily complex training requirement. Adding to
complexity is the size and mission of the UAS as each size dictates airspace considerations, location of
Ground Control Stations (GCS) and operational environments. While the US military has invested in UAS
training and research, it is a new area constrained by limited resources. Thus it is incumbent on early UAS
adopters to address their complex training challenges and leverage the training resources and research done
by the US military. One critical NAS integration issue is training pilots to safely operate UAS, particularly
medium altitude, long range UAS which will share airspace with private aircraft and amateur pilots. This
paper discusses the need for and creation of a complementary family of UAS trainers. The authors draw on
the training research, and training simulators and technologies developed and used at the Air Force
Research Laboratory, and the commercial simulators used at University of North Dakota, and Sinclair
Community College. The findings from the training fidelity assessment are presented and conclusions are
drawn.
103
ASOC TRAINING RESEARCH: JOINT THEATER AIR GROUND SIMULATION SYSTEM
Leah J. Rowe & Sharon L. Conwell
Wright-Patterson AFB, OH
Across the armed forces, warfighters are required to perform in complex, dynamic, networked
environments. Adequate preparation requires training in robust simulations that replicate these situations.
Designing a training environment for Air Support Operation Center (ASOC) personnel comes with a
unique set of challenges. The ASOC warfighters are required to work within their team of nine as well as,
with an undefined number of personnel external to their location. The problem in creating a simulated
training environment for local training at the operational unit is that it has been proven to be difficult to
access, on a regular basis, thirty role-playing warfighters across different locations in order to staff an
exercise. The purpose of this paper is to introduce an ongoing effort to create a robust ASOC training
environment – the Joint Theater Air Ground Simulation System (JTAGSS) – which overcomes this
limitation. JTAGSS incorporates the nine personnel who work within the ASOC, as well as, reflex agents
playing the role of team members external to the ASOC (e.g., Joint Terminal Attack Controllers, F16, Joint
Fires, etc.). Reflex agents are software-generated role players which are smart, synthetic, team members.
Using Mission Essential CompetenciesSM (MEC) the research team identified the primary competencies,
supporting competencies, knowledge, skills, and developmental experiences that an ASOC warfighter
requires for successful mission completion. The MECs were then leveraged to design and create JTAGSS
to provide a training platform to address current ASOC training gaps. This paper presents the successes
and lessons learned through creating this unique training environment.
T-9 A Variety of Training Approaches
1030
1100
1130
Fusing Self-Reported and Sensor
Data from Mixed-reality
Training (14158)
A Competency Based Approach to
Marine and Weapons
Engineering Training (14224)
Scenario-based Training for
Development of LeaderSubordinate Mental Models and
Cohesion (14333)
NOTES
104
FUSING SELF-REPORTED AND SENSOR DATA FROM MIXED-REALITY TRAINING
Trevor Richardson, Stephen Gilbert, Joseph Holub, Frederick
Thompson, Anastacia MacAllister, Rafael Radkowski, Eliot Winer
Ames, Iowa
Paul Davies, Scott Terry
The Boeing Company
St. Louis, MO
Military and industrial use of smaller, more accurate sensors are allowing increasing amounts of data to be acquired at
diminishing costs during training. Traditional human subject testing often collects qualitative data from participants
through self-reported questionnaires. This qualitative information is valuable but often incomplete to assess training
outcomes. Quantitative information such as motion tracking data, communication frequency, and heart rate can offer
the missing pieces in training outcome assessment. The successful fusion and analysis of qualitative and quantitative
information sources is necessary for collaborative, mixed-reality, and augmented-reality training to reach its full
potential. The challenge is determining a reliable framework combining these multiple types of data.
Methods were developed to analyze data acquired during a formal user study assessing the use of augmented reality as
a delivery mechanism for digital work instructions. A between-subjects experiment was conducted to analyze the use
of a desktop computer, mobile tablet, or mobile tablet with augmented reality as a delivery method of these
instructions. Study participants were asked to complete a multi-step technical assembly. Participants’ head position
and orientation were tracked using an infrared tracking system. User interaction in the form of interface button presses
was recorded and time stamped on each step of the assembly. A trained observer took notes on task performance
during the study through a set of camera views that recorded the work area. Finally, participants each completed pre
and post-surveys involving self-reported evaluation.
The combination of quantitative and qualitative data revealed trends in the data such as the most difficult tasks across
each device, which would have been impossible to determine from self-reporting alone. This paper describes the
methods developed to fuse the qualitative data with quantified measurements recorded during the study.
A COMPETENCY BASED APPROACH TO MARINE AND WEAPONS ENGINEERING
TRAINING
Commander Richard Clarke Royal Navy
Ministry of Defence, Royal Navy
Portsmouth, Hampshire, UK
Within the United Kingdom Ministry of Defence a number of competency based models for training have
been introduced. Unlike traditional systems based approaches the use of competency is less mature –
producing a number of significant challenges. The increasing popularity of Competency Frameworks
stems from their stated ability to deliver a breadth of outcomes relating not only to training but also to
issues such as professional development, talent management, appraisal and organisational development.
This paper reports on the progress of a pathfinder project to redefine all Marine and Weapons Engineering
training conducted for the UK Royal Navy within a competency based approach to both career
management and training. The paper describes the evolution of that approach – focusing upon why the
previous Systems Approach and its job focused Operational Performance Statement (OPS) behaviourist
methodology was judged too limited to underpin future training requirements. It will describe what new
outputs and deliverables a competency based approach will offer and how, in a training environment
utilizing significant contractor led training delivery, the project has evolved in conjunction with the key
prime contractor. The approach being taken is that of a hybrid – with significant linkage to Job Analysis
based thinking. Support for the traditional behaviourist based systems approach remains strong in many
areas of maritime training in the UK. As competency based approaches gain in popularity the result is
increasingly the use of hybrid approaches. Some fuse elements of competence/competency models with
elements of systems based models – others seek simply to align the two processes. The paper will describe
the evolution of current policy and thinking within the Royal Navy. Finally the paper will present a
sequence of “Lessons Learned” and outline a generic model for the application of competency to the
training and HR environment of the Royal Navy over the next 5 years.
105
SCENARIO-BASED TRAINING FOR DEVELOPMENT OF LEADER-SUBORDINATE
MENTAL MODELS AND COHESION
Kara L. Orvis
Aptima, Inc.
San Diego, CA
Gregory A. Ruark
Ft. Leavenworth, KS
Krista L. Ratwani
Aptima, Inc.
Washington DC
Mission Command doctrine emphasizes that command is a human endeavor in which leaders help develop
their supporting teams through instilling cohesion and shared understanding (ADRP 6-0). One of the six
principles of mission command is to provide a clear commander’s intent. Whereas the communication and
interpretation of tactical orders can be relatively straightforward, the same cannot be said of commander’s
intent. Misinterpretations of commander’s intent can lead subordinate leaders to take inappropriate or
ineffective actions that compromise mission success – outcomes that have been demonstrated in the
laboratory, field training exercises, and the operational environment. Pigeau and McCann (2000)
differentiate between two aspects of command intent: explicit and implicit. Explicit intent is the actual
content of the order expressed by the words that are used to convey what the leader wants a subordinate to
do (e.g. command intent statement). Implicit intent, on the other hand, constitutes the underlying meaning
of the command as it is conceptualized by the leader. Implicit intent communicates the expectations and
idealized solution that the leader envisions. The purpose of the research was to investigate how shared
interpretation of command intent is developed within Army company leadership teams. Research findings
led to the development and validation of a straightforward, scenario-based, leader led, hip pocket training
tool to enhance shared understanding of implicit command intent between company commanders and their
subordinate leadership team. This paper describes the development of the training tool, the benefits of
scenario-based training for developing shared mental models of command intent, and findings from a field
validation study conducted with 166 Soldiers (49 teams). Validation results showed an increase in team
cohesion and shared interpretation of commander’s intent after two hours of use. The implications of this
research for military readiness and suggested future use will be discussed.
T-10 Evaluating Training Effectiveness
1330
1400
1430
Post-fielding Training Assessment
of Dismounted Infantry
Simulation (14022)
Evaluating the Impact of Individual
Training on Units’
Operational Performance (14123)
Using LMS Technology for
Kirkpatrick Level 3 Evaluation
on Human Trafficking Training
(14162)
NOTES
106
POST-FIELDING TRAINING ASSESSMENT OF DISMOUNTED INFANTRY SIMULATION
Martin L. Bink & Victor J. Ingurgio
Fort Benning, GA
David James
Northrop-Grumman
Columbus, GA
The assessment of training effectiveness for virtual-training systems is best accomplished with two
complimentary approaches: experimentation and post-fielding assessment. Obviously, experimentation involves
the controlled assessment of system capabilities that should have an impact on training. By contrast, a postfielding assessment determines the practical advantages and limitations of the system in the context of mission
training and can be used to discover ways to increase system utilization and effectiveness. This paper reports the
results of a post-fielding assessment of the U.S. Army’s recently fielded dismounted infantry simulator:
Dismounted Soldier Training System (DSTS). The goal of the assessment was to collect input on DSTS training
effectiveness and training issues from Leaders and Soldiers who have used DSTS at home station. A total of 58
surveys and 122 interviews from Leaders (82), Soldiers (90), and DSTS Operators (8) were collected over six
months from five U.S. Army installations. On the surveys, respondents indicated that DSTS is capable of
providing a training environment for collective task training and assessment. The after action review (AAR)
system, as a training tool, received the most positive responses. Respondents were equally vocal in identifying
training distracters. Technical issues and simulator sickness were identified as causes of suspended training,
while over half of the Soldiers reported less than a complete feeling of immersion. The interview responses
echoed these themes and provided insights on effective planning, preparation, and execution of DSTS training.
Overall, the results showed that dismounted infantry simulation showed promise as an effective training device.
However, improvements to the technology were still needed to provide a reliable training environment. The
results also highlighted the positive impact of effective AAR capabilities on training. Future development of
dismounted infantry simulation training should formalize the unique AAR capabilities of the technology to
provide formative feedback to Soldiers.
EVALUATING THE IMPACT OF INDIVIDUAL TRAINING ON UNIT'S OPERATIONAL
PERFORMANCE
Dr. Jay Brimstin & Toumnakone Annie Hester
U.S. Army, Maneuver Center of Excellence
Ft Benning, GA
The evaluation of training programs has long been considered one of the critical components of the
analysis, design, develop, implementation, and evaluation (ADDIE) process. Typically though, training
evaluations at best measure learning, with no measure of the learner’s ability to apply what they learned on
the job and no measure of the impact on organizational performance. This is especially true for military
training programs designed to prepare soldiers for combat operations, as it is difficult to trace the results of
the training program to organizational performance. This paper describes a project that implemented a
robust training evaluation of an Army institutional training program that was designed to better prepare
soldiers in units to deal with the improvised explosive devices (IED) threat. The Dismounted Counter IED
Tactics Master Trainer (DCT-MT) course provided a unique opportunity to pilot an evaluation method
using Kirkpatrick’s four-level model for an institutional training course. The course’s mobile training team
conducted training for soldiers from the same brigade combat team prior to their deployments to
Afghanistan. The evaluation employed a mixed method design to evaluate Levels I and II during the
training and Levels III and IV during and after the unit’s deployment. The findings from this evaluation
that spanned a two-year period showed positive soldier reaction to the learning experience and significant
improvement in their knowledge. Soldiers applied their knowledge in their units under certain conditions.
A comparison of specific variables between units that did and did not receive the DCT-MT training
showed a positive impact on the units’ operational performance. This evaluation project demonstrated the
ability to quantify the impact that training had on units’ operational performance in a combat environment
by using Kirkpatrick’s model, and provides an evidence based approach for conducting future training
program evaluations.
107
USING LMS TECHNOLOGY FOR KIRKPATRICK LEVEL 3 EVALUATION OF HUMAN
TRAFFICKING TRAINING
Jill Shepherd, Lewis Harris, Casey O’Conor
Booz Allen Hamilton
McLean, VA
Traditional methods of gathering Kirkpatrick Level 3 evaluation data, such as observation, interview, work
reviews, and focus groups, are resource and time-intensive; consequently, this level of data is often not
gathered. To assess its training for law enforcement professionals using the Kirkpatrick Four Levels of
Evaluation model at Levels 1, 2, and 3, the Department of Defense (DoD) Combating Trafficking in
Persons (CTIP) Program Management Office (PMO) created an evaluation plan to make non-traditional
use of existing Learning Management System (LMS) technology. Course effectiveness was assessed using
an immediate post-course evaluation (Level 1), a post-test to assess mastery of the learning objectives and
perceived learning (Level 2), and a six-month follow-up evaluation (Level 3). This paper describes how
the CTIP PMO designed a new course for DoD law enforcement professionals, a one-hour, scenario-based
web-based training intervention that models a realistic and complicated crime investigation, and an
evaluation plan to use existing functions available on multiple LMSs to gather Kirkpatrick Level 3
evaluation data. The paper discusses the background of the CTIP PMO and its efforts to train law
enforcement professionals and the background of the use of LMSs to gather Level 3 Kirkpatrick evaluation
data. The paper describes the CTIP PMO’s design of a new course and evaluation plan, the evaluation plan
itself, the advantages and disadvantages of non-traditional uses of LMS technology to gather Kirkpatrick
Level 3 data, the preliminary Level 3 data gathered, lessons learned, tips for implementing a similar plan,
and next steps.
TRAINING PUBLISH ONLY: RESEARCH DIRECTIONS FOR FUTURE SIMULATION-BASED
TRAINING DESIGN IN DEFENCE
Luke G. Thiele, Ph.D.
Rheinmetall Simulation Australia Pty Ltd
Adelaide, South Australia
New technologies provide potential increases in operational capability through improved survivability,
mobility, and lethality. For technologies with a human in-the-loop, however, this potential is only
translated to actual combat power through the expertise of the human controllers. This places training on
the critical path for maximising the return on investment for technologies of this type. Modern systems
also support distributed command and control through increased connectivity. To exploit this technology
and seize the operational initiative, commanders must have the mental agility to respond as the situation
changes and new information becomes available. This includes creating novel solutions when faced with
unexpected situations. This places an emphasis on training to support greater personal adaptability. This
paper discusses collaborative research by Rheinmetall Simulation Australia and the Australian Defence
Science and Technology Organisation Land Division to investigate the impact of these issues on future
training needs analysis and simulation-based training design in Defence. It discusses training as the
selected sampling of the environment to provide the experience and feedback opportunities required for the
accelerated development of expertise. Using this model, it evaluates the requirements for future Defence
training, compares this with current approaches, and identifies a series of future research
questions/“roadblocks” for supporting the goal of more efficient, effective, and responsive simulationbased training. The discussion is intended to share Australian research with the international audience.
108
PDF FILES OF THE 2014 TUTORIAL PRESENTATIONS ARE INCLUDED ON THE
PROCEEDINGS CD. PLEASE SEE THE TUTORIALS SECTION OF THIS BOOK FOR
SCHEDULE AND SYNOPSES DETAILS.
NOTES
109
TABLE OF AUTHORS
Abbott, Robert, 51
Abdul-Muhsin, Haider, 93
Acharya, Girish, 43
Akbari, Morris, 34
Alban, Angela, 41
Aldinger, Mike, 68
Alexander, Thomas, 51
Allen, Christine, 57, 93
Allison, Dan, 75, 85
Amburn, Charles, 56, 96, 101
Anderson, Amanda, 48
Anderson, Benjamin, 51
Andreatta, P, 57, 91, 94
Armstrong, Stuart, 25
Astwood, Randy, 38
Avenoso, Amanda, 37
Baker, Erin, 21
Balaban, Mariusz, 12
Barrera Kristen, 50
Barton, Meg, 10
Baxter, Holly, 95
Beaubien, Jeff, 54, 97
Beidel, Deborah, 30
Bickley, WIlliam, 24
Billard, Randy, 92
Bink, Martin, 107
Bland, Kevin, 25
Blankenbeckler, Paul, 23
Bockelman, Patricia, 11, 91
Boland, Irene, 56
Bolton, Amy, 54, 95, 97
Bowers, Clint, 30
Branzoi, Vlad, 43
Brathen, Karsten, 36
Bratt, Elizabeth, 101
Brawner, Keith, 101
Bridewell, John, 103
Brimstin, Jay, 107
Brock, Timothy, 77
Bruni, Sylvain, 73
Bruzzone, Agostino, 51
Cain, Daniel, 61
Carlton, Bruce, 33
Carney, Michael, 99
Carpenter, Jim, 43
Carroll, Meredith, 100
Carswell, John, 71
Casebeer, William, 102
Catrambone, Richard, 52
Ceranowicz, Andy, 8
Chadderdon, George, 21, 47
Chambers, Samuel, 65
Champney, Roberto, 21
Chan, Jit, 21
Chartrand, Donald, 29
Chavez-Knott, Camilla, 24
Cheng, Huaining, 26
Cheng, Zhiqing, 26
110
Chladny, Brett, 81
Clarke, Richard, 105
Clegg, Benjamin, 39
Cohn, Joseph, 29, 40
Coiro, Cheryl, 41
Colbert, Brad, 81
Coleman, Michael, 60
Connell, Michael, 31
Conwell, Sharon, 103, 104
Cooley, Tim, 9
Cosman, Michael, 83, 84
Covas-Smith, Christine, 37, 103
Cox, Rob, 33, 62
Cummings, Paul, 46
Dargue, Brandt, 13, 55
Darken, Rudolph, 35
Davies, Paul, 105
Davis, Fleet, 32
De Jong, Steven, 51
De la Cruz, Julio, 90
Deakins, Richard, 73
Dean, Courtney, 24, 97
Dean, Frank, 43
Dechmerowski, Sara, 47
DeJong, Rendel, 22
Denny, Ricky, 60
DeVisser, Ewart, 29
Diaz, Eugenio, 86
Dierich, Fred, 24
Diller, David, 71
Dinger, Jason, 44
Djahandari, Kelly, 45
Donsbach, Jamie, 37
Dooley-Hash, S, 94
Dumond, Danielle, 73
Dunlap, Anne, 103
Dunne, Robb, 9
Eddy, Jason, 45
Eifert, Latika, 86
Ender, Tommer, 35
Ezell, Barry, 12
Faben, Carol, 20
Fabiano, Gregory, 53
Faulk, Mark, 62
Fautua, David, 11
Fefferman, Kevin, 42, 78
Ferguson, William, 71
Fink, Christopher, 76
Flanagan, Scott, 24
Fleener, Graham, 45, 64
Fletcher, JD, 102
Folkerth, Leese, 15
Folkestad, James, 39
Fonkalsrud, Dave, 27
Forsythe, Chris, 51
Franke, Warren, 48
Freed, Michael, 44
Freedy, Elan, 29
Freeman, Jared, 95
Freeman, Jay, 65
Frueh, Christopher, 30
Fuentes, Wanda, 103
Gallagher, Patrick, 39
Gallant, Scott, 35
Garrity, Pat, 43, 71
Gaughan, Chris, 35, 51, 73
Geil, Joe, 83
Georgiadis, Konstantinos, 29
Gervasio, Melinda, 44
Geyer, Alexandra, 40
Giannakopoulos, Allen, 77
Gilbert, Stephen, 105
Ginader, Kenneth, 28
Glass, Jonathan, 87
Godby, Kevin, 48
Godwin, Trey, 79
Goerger, Simon, 35
Goldberg, Benjamin, 101
Goldiez, Brian, 57
Goodwin, Gregory, 95
Gordon, Steven, 9
Graddy, Courtney, 93
Graffeo, Clarissa, 31
Graham, Scott, 97
Grange, Bob, 84
Graves, Thomas, 23
Griffith, Todd, 85
Grosse, Jim, 103
Guarino, Sean, 31
Gunselmann, Glenn, 51
Gupton, Kevin, 33
Hackett, Matthew, 42, 78
Hale, Kelly, 21
Hammar, Peter, 69
Hananel, David, 93
Hardee, Gary, 72
Harris, Lewis, 108
Hearn, Gareth, 11
Hebert, Kenny, 81
Hegde, Mandira, 85
Hester, Annie, 107
Hester, Robert, 58
Hodges, Glenn, 49
Holt, Jerred, 15, 47
Holtschneider, Mary, 77
Holub, Joseph, 105
Horn, Zachary, 99
Hou, Yunfei, 53, 70
House, J, 94
Hower, Mike, 95
Hruska, Michael, 96
Huey, Christopher, 45
Huffam, Christopher, 19
Huiskamp, Wim, 51
Hulme, Kevin, 53, 70
Humphrey, Elliot, 47
Hurst, CG, 57, 91
Husband, Leland, 58
Hyndoy, Jens Inge, 36
Ingurgio, Victor, 52, 107
Isaksen, Geir, 18
Jacko, Julie, 21
Jacobson, Dov, 13, 55
James, David, 107
Jarvis, Ryan, 31
Jeon, Da-hyoung, 85
Johnson, Scott, 71
Jones, David, 47
Jones, Pau, 82
Julian, Danielle, 91
Kaiser, Michael, 72
Kassam, Karim, 10
Keeney, Michael, 54
Kehr, Thomas, 79
Kelley, Steve, 82
Kenski, Kate, 39
Kent, JR, 59
Kenyon, Peggy, 19
Keren, Nir, 48
Kilcullen, Tara, 96
Killilea, John, 91
Kim Leng, Koh, 69
Kim, Yeong-cheol, 85
Kincaid, Peter, 9
Klotz, J, 57, 91, 94
Knaplund, Justin, 27
Kolve, Morten, 87
Konia, Mojac, 93
Kopecky, Ken, 90
Korris, James, 10
Kratzig, Gregory, 49
Kristofek, Charles, 45
Kumar, Rakesh, 43
Kunkle, Christina, 37
Lackey, Stephanie, 89
Lacks, Daniel, 65
Leland, Richard, 28
Lewis, Jennifer, 67
Lewis, Joanna, 39
Lewis, Mark, 14, 75
Lewis, Nephi, 84
Ligouri, Anthony, 26
Lilja, Andrew, 48
Linn, Terry, 27
Liu, Huaiyu, 83
Livingston, Joyner, 67
Long, Richard, 21
Long, Rodney, 56, 96
Lopez-Couto, Stephen, 25
Lopez-Rodriguez, Jose-Maria, 33
Lowe, Daniel, 83
Lukes, George, 61
Lynch, Spencer, 73
MacAllister, Anastacia, 105
Madsen, JM, 57, 91
Mahoney, Samuel, 31
Mall, Howard, 41
Malone, Amelia, 15
Marshall, Henry, 73
Martey, Rosa, 39
Martinez-Salio, Jose-Ramon, 33
Mastaglio, Thomas, 12
Mateas, Michael, 71
Matthews, Gerald, 89
Maxon, Andrew, 64
Maxwell, Douglas, 83, 89
Mayor, Marco, 64
Mazzeo, Mark, 57
Mazzone, Leslie, 17
McClain, Jonathan, 51
McCraty, Rollin, 29
McGonagle, Mike, 34
McGurn, Linda, 95
McIlwain, Steve, 78
McIntire, Ryan, 79
McLaren, Elizabeth, 39
McLaughlin, Ryan, 68
Mellone, Ingrid, 20
Metevier, Christopher, 35
Mevassvik, Martin, 36
Michael, Tomer, 81
Miller, David, 82
Mina, Sami, 88
Minchey, Barry, 63
Minkov, Yaniv, 81
Moody, Reed, 27
Moon, Jung-ho, 85
Moore, Jimmy, 15
Morewedge, Carey, 10
Morris, Karen, 53
Morris, Sean, 37
Morrison, Peter, 80
Mui, Matthew, 93
Murphy, Jennifer, 100
Murray, Glenn, 43
Murray, Kate, 35
Napoletano, Nat, 55
Nauer, Kevin, 51
Neer, Sandra, 30
Nelson, Andrew, 21
Nelson, Robert, 47
Newman, Michael, 28
Nguyen, Anh Bao, 17
Nicholls, AP, 59
Nicholson, Denise, 40, 95
Niehaus, James, 40
Nolan, Margaret, 40, 91
O’Connell, Ralph, 33
O’Conor, Casey, 108
O’Neill, Elizabeth, 29, 40
Ogren, Lauren, 101
Oppelaar, Richard, 22
Ortiz, Andres, 73
Orvis, Kara, 106
Oudiette, Delphine, 40
Overholtzer, Adam, 44
Owens, Gregory, 63
Padron, Christina, 100
Pagan, Jennifer, 38, 54
Page, David, 82
Parsons, Doug, 73
Patterson, Anthony, 92
Patterson, John, 39
Perez, Manuela, 11
Perez-Sanagustin, Mar, 44
Peters, Stanley, 101
Pfeiffer, Karl, 79
Pham, Tien, 103
Phillips, Henry, 38
Pihera, Drew, 35
Pingel, Thomas, 57
Plaat, Aske, 75
Poeppelman, Tiffany, 96
Prestwich, Shenan, 39
Prevou, Michael, 17, 95
Priest-Walker, Heather, 13, 32
Pruett, Andrew, 58
Puckett, Kimberly, 15
Qiao, Chunming, 53
Quinn, Mary, 10
Radkowski, Rafael, 105
Raeth, Peter, 66
Raghuwanshi, Viral, 70
Rahmes, Mark, 34
Ratwani, Krista, 24, 106
Reber, Paul, 40
Reed, Dean, 86
Reed, Theodore, 51
Reese, Bill, 62
Reilly, Sean, 87
Reitz, Emilie, 11, 98
Remily, Helen, 19
Rench, Tara, 99
Richardson, Trevor, 105
Richbourg, Robert, 61
Riddle, Kenyon, 73
Rieger, Lawrence, 63, 65
Riley, Jennifer, 32
Rivers, William, 83
Rizzo, Albert, 30
Roberts, Bruce, 71
Robinson, Petra, 63
Robson, Elliot, 23
Robson, Robby, 23
Rockah, Rami, 81
Rodabaugh, Timothy, 50
Roessingh, Jan Joris, 75
Romeu, Amanda, 57
Rosa Moreira, Jonathan, 67
Roschelle, Jeremy, 44
Rowe, Leah, 37, 103, 104
Ruark, Gregory, 106
Sadek, Adel, 53
Saffold, Jay, 73
Sainfort, Francois, 21
Salcedo, Julie, 89
Salunke, Sarthak, 70
Samarasekera, Supun, 43
Sanders, John, 13, 55
111
Saulnier, Tobi, 39
Schatz, Sae, 11
Schmidt-Daly, Tarah, 21, 47
Scielzo, Sandro, 32
Scrudder, Roy, 33
Seavey, Kevin, 98
Shapiro, Dan, 71
Shaw, Adrienne, 39
Shawl, C, 59
Shepherd, Jill, 108
Shepherd, Andrew, 103
Shih Yeong, Wah, 69
Silva, Austin, 51
Simas Lopes Peres, Sergio, 67
Simpson, Khara, 11, 74
Sims, Ed, 56
Sincebaugh, Patrick, 103
Sjoestedt, Peter, 16
Skinner, Anna, 21
Smith, Eileen, 99
Smith, Peter, 25
Smith, Roger, 11, 74, 93
Snyder, Jared, 87
Snyder, Robert, 61
Sokolowski, John, 12
Sotomayor, Teresita, 41, 91
Southwick, Brad, 84
Spaulding, Aaron, 44
Speed, Mark, 50
Spronck, Pieter, 75
Squire, Peter, 29, 40, 95, 100
St. Benoit, Tracy, 31
Stacy, Webb, 54, 97
Stafford, A, 59
Stanney, Kay, 21
Stevens, Jonathan, 9, 86
Stewart, John, 97
112
Stone, Robert, 14
Stromer-Galley, Jennifer, 39
Strzalkowski, Tomek, 39
Stubbs, Jack, 93
Surpris, Glenn, 100
Symborski, Carl, 10
Talbot, TB, 57, 91
Tamash, Theresa, 79
Tanaka, Alyssa, 11, 93
Tarr, Ronald, 99
Terry, Scott, 105
Thiele, Luke, 108
Thomas, CE, 82
Thomas, James, 77
Thompson, Fredrick, 105
Thorstensson, Mirko, 69
Toli, Gustav, 69
Tompson, Geoff, 87
Torres, Orlando, 68
Tossell, Chad, 47
Totten, Eric, 99
Toubman, Armon, 75
Tremori, Alberto, 51
Tripp, Lisa, 47
Troung, Mireille, 11
Tucker, Christina, 15
Umanskiy, Oleg, 86
Van Den Bosch, Karel, 51
Van Den Herik, Jaap, 75
Van Der Pal, Jelke, 29
VanBuren, John, 22
VanDeVelde, Mandy, 22
VanGinkel, Hilde, 22
Vassall, Jelani, 25
Vickery, Rhonda, 66
Villamil, Ryan, 43
Vitovich, Nicholas, 43
Vogt, Brian, 35
Waisel, Laurie, 17
Wajda, Michael, 99
Walker, Alexander, 99
Walwanis, Melissa, 97
Wampler, Richard, 23, 52
Wan, Jingyan, 53
Wary, Robert, 13
Webb, Timothy, 26
Webster, Rustin, 89
Weerasinghe, Chalinda, 46
Weltman, Gershon, 29
Westheimer, Keith, 37
Westphal, Marko, 101
Wiggins, Sterling, 54, 97
Wikberg, Per, 69
Wikoff, Dennis, 19
Wilson, Bradley, 78
Winer, Eliot, 90, 105
Winner, Jennifer, 15, 47
Winslow, Brent, 47, 100
Wolters, Heather, 95
Wolverton, Michael, 43
Wong, Jason, 100, 101
Woodman, Michael, 80
Wray, Curtis, 50
Wu, Changxu, 53
Yanoschik, Thomas, 59
Yarnall, Louise, 43, 44
Zaccaro, Steve, 99
Zakhidov, Djakhangir, 72
Zeidman, Troy, 97
Zhao, Yunjie, 53
Zhu, Zhiwei, 43
Ziegeler, Sean, 66
Zielke, Marjorie, 72
Zou, Cliff, 45
Papers are available on the 2014 I/ITSEC CD ROM included in the Conference Attendee meeting bag, or visit the I/ITSEC
Website (www.iitsec.org) for ordering information. (Limited numbers of CDs from 1998-2014 are also available.) All
papers from 1966 through 2000 are available in the I/ITSEC Compendium. Individual papers from 1966 through 2014 may
also be ordered through the www.iitsec.org portal.
http://www.iitsec.org • (703) 247-2569
Interservice/Industry Training, Simulation and Education Conference
The National Training and Simulation Association (NTSA)
An Affiliate of the National Defense Industrial Association (NDIA)
Suite 400, 2111 Wilson Boulevard
Arlington, VA 22201

2014IITSECProgram_Full_LowRes

Transcription

Similar documents

sendai mediatheque - toyo ito

Army Looks to Training Center Upgrades

MDA and the Design of Design

Mary Ellen Smith Glasgow, PhD, RN, ACNS-BC

Inside IUSM-FW - IU School of Medicine - Fort Wayne

UNMC Global Center for Advanced Interprofessional Learning

the analisys of naval electromagnetic systems using software

NeuroLab - Sites at Lafayette

Manuscript