2015IITSECProgramGuideLoRes

Transcription

NATIONAL TRAINING AND SIMULATION ASSOCIATION
T H E W O R L D ’ S L A R G E S T M O D E L I N G & S I M U L AT I O N E V E N T
I/ITSEC
INTERSERVICE/INDUSTRY TRAINING,
SIMULATION & EDUCATION CONFERENCE
F O R G I N G T H E F U T U R E T H R O U G H I N N O V AT I O N
ch eck
o
t
e
r
Be su
fron t
e
h
t
e
in sid
rs fo r
e
v
o
c
k
c
a n d ba
u id e
G
t
e
k
c
o
you r P
.
a n d CD
W W W. I I T S E C . O R G
u
NOVEMBER 30 - DECEMBER 4, 2015
PROGRAM
GUIDE
u
O R L A N D O, F L O R I D A
n AGENDA
Pre-Conference Agenda
Dress Code
Conference Agenda
Continuing Education Units/Continuous Learning Points
Orange County Convention Center Diagram
Hyatt Regency Diagram
In Memoriam
7
7
8
12
13
14
15
n TUTORIALS
Tutorial Grid
Tutorial Synopses & Schedule
17
18
n SPECIAL EVENTS
Signature Events
Floor Events
Focus Event
Community of Interest
International Programs
Special Guests
29
39
41
47
50
52
n PAPER SESSIONS
Paper Session Grid
Papers/Authors Presentation Schedule
53
58
n STEM
STEM Workforce Initiative
Future Leaders • Students at I/ITSEC
America’s Teachers at I/ITSEC • Educators (techPATH)
Serious Games Showcase & Challenge
STEM Pavilion: Project Based Learning
I/ITSEC Scholarships
Post I/ITSEC Professional Development Workshops
69
70
71
72
73
74
75
n EXHIBITS
Hall Happenings
2015 Exhibitors
77
82
n COMMITTEES
Conference Committee • Council of Chairs
Program Subcommittees
Special Teams
85
86
88
TA B L E O F C O N T E N T S
n WELCOME & INTRODUCTIONS
Conference Welcome
2
Keynote Speakers
3
Conference Leadership
4
Interservice Executives
5
Principals & Advisor6
n CONFERENCE INFORMATION
Registration Information • Parking • Dress Code
89
Lodging90
Getting Around
91
Publications & Media
92
Association Sponsors
93
Safety & Security
94
Golf Tournament
95
5K Run
96
n I/ITSEC 2016
I/ITSEC 2016 Save the Date
Call for Papers and Tutorials
Serious Games Showcase & Challenge
97
98
99
n ABSTRACTS101
1
ELCOME ATTENDEES OF I/ITSEC 2015:
WELCOME
On behalf of the United States Navy and the United States Marine Corps, this year’s Lead Services, our sponsoring
organization, the National Training and Simulation Association; the Service Executives and their Principals; and
the 250-plus volunteers from the Army, Navy, Marine Corps, Air Force, Coast Guard, Office of the Secretary of Defense, Industry, and Academia, it is my distinct honor and great pleasure to welcome you to the 2015 Interservice/
Industry Training, Simulation and Education Conference.
With continuing and emerging threats around the world, our military services and government organizations—
both here in the U.S. and abroad—are being asked to do more with less. Now more than ever before, innovative
modeling and simulation strategies and technologies can help organizations train in Live, Virtual, and/or Constructive (LVC) environments reducing the time to achieve proficiency and saving significant dollars.
This year’s Conference theme, “Forging the Future through Innovation,” underscores this crucial need for new
and innovative approaches to how we train. Building on the Conference’s military-focused history, I/ITSEC has
emerged as the predominant cross-industry forum, drawing an increasing number of attendees from industries
including healthcare, energy, transportation and manufacturing, who are in search of new innovations to change
the way people learn and perform, to drive down costs, and increase their ability to compete.
The volunteer members of our six subcommittees, the Tutorial Board and the Conference Committee have spent
this year finding the best technical papers (150), tutorials (21), educationally-focused special events (12), and
a range of special programs including the inaugural Operation Blended Warrior multi-year Special Event which
will highlight readiness through LVC activities on the exhibit floor. Other special programs include a Black Swan
panel, the Future Leaders Pavilion, Serious Games Showcase & Challenge, and Warfighters Corner wrapping up
Friday with our Professional Development Workshops. As a result, I am certain you will find the 2015 Program to
be rich, diverse, and highly relevant. All tutorials, papers and professional development workshops, as well as the
Fellow Special Event are available for Continuing Education Units (CEU) and Continuous Learning Points (CLP).
Be sure to visit the I/ITSEC exhibit hall, which hosts the largest display of training systems capabilities in the
world. This year, we also welcome continued growth in the number of international attendees. With nearly 2,000
attendees from over 60 countries expected, we have created even more opportunities for networking to promote
dialogue and idea exchange.
Finally, I want to express my sincere thanks and appreciation to our dedicated volunteers and their sponsors.
Their commitment and support has made I/ITSEC 2015 a reality and ensured this Conference remains the premier
professional development event across the globe for the training and simulation professional. As you attend the
events and walk the exhibit floor over the next few days, please take a moment to thank the authors and other
volunteers who have helped make this program a success.
Sincerely,
D. David Hutchings
2015 Program Chair
2
2 0 1 5 I N T E R S E R V I C E / I N D U S T RY T R A I N I N G, S I M U L AT I O N & E D U C AT I O N C O N F E R E N C E
Service Keynote
KEYNOTE SPEAKERS
Admiral John M.
Richardson, USN
Chief of Naval Operations
Admiral John Richardson graduated from the U.S. Naval Academy in 1982 with a Bachelor of
Science in Physics. He holds master’s degrees in electrical engineering from the Massachusetts
Institute of Technology and Woods Hole Oceanographic Institution, and National Security
Strategy from the National War College.
At sea, Richardson served on USS Parche (SSN 683), USS George C. Marshall (SSBN 654)
and USS Salt Lake City (SSN 716). He commanded USS Honolulu (SSN 718) in Pearl Harbor,
Hawaii.
Richardson also served as commodore of Submarine Development Squadron (DEVRON)
12; commander, Submarine Group 8; commander, Submarine Allied Naval Forces South; deputy commander, U.S. 6th Fleet; chief of staff, U.S. Naval Forces Europe and U.S. Naval Forces
Africa; commander, Naval Submarine Forces, and director of Naval Reactors.
His staff assignments include duty in the attack submarine division on the Chief of Naval
Operations staff; naval aide to the President; prospective commanding officer instructor for
Commander, Submarine Forces, U.S. Pacific Fleet; assistant deputy director for Regional Operations on the Joint Staff; and director of Strategy and Policy at U.S. Joint Forces Command.
Richardson served on teams that have been awarded the Presidential Unit Citation, the
Joint Meritorious Unit Award, the Navy Unit Commendation, and the Navy “E” Ribbon. He
was awarded the Vice Admiral Stockdale Award for his time in command of USS Honolulu.
Industry Keynote
Waymon Armstrong
Founder and Chief
Executive Officer
Engineering & Computer
Simulations
Waymon Armstrong is founder and CEO of Engineering & Computer Simulations, Inc. (ECS).
As CEO, Armstrong provides strategic leadership to a growing technology company that employs over 110 simulation professionals, computer programmers, artists, and designers who
develop, produce, and operate interactive and immersive advanced learning technologies.
Armstrong is a visionary and an implementer of novel learning technologies that change
the way we implement the art and science of learning – and the way we conduct training.
ECS, headquartered in Orlando, Florida, is a leader in modeling simulation and training
industry. The company delivers innovative software and simulations technology and services
to both government and commercial markets, and has changed the way our military, first responders, emergency management personnel, and professionals in all business sectors learn
and practice their professions.
Armstrong led the company as a first adopter of computer game technology for learning.
His philosophy of early adoption, and implementation of evolving technology solutions continued as the company invested in mobile and virtual world technologies.
From its position as a leading U.S. military solution-provider, to its bold early expansion
into the commercial sector, Armstrong’s devotion to excellence in learning technologies sets
the company apart as a leader in the simulation industry. ECS is a four time I/ITSEC Serious
Games Showcase and Challenge winner. The company has also won National awards including Inc. 500/5000, Deloitte Technology Fast 500, and other pre-eminent accolades in the United States and Canada. In 2010 Armstrong was selected by the Small Business Administration
as the National Small Business Person of the Year.
Armstrong has called Orlando home for over 40 years. He attended Valencia College and
the University of Central Florida. He remains a is strong supporter in the Orlando community,
currently serving as the chair of Orange County Mayor Teresa Jacobs’ Modeling and Simulation Blue Ribbon Commission and is the immediate past chair of Orlando Inc. He is a passionate business leader, and can be frequently heard speaking at CEO events and forums such
as Rollins College Center for Advanced Entrepreneurship Center, GrowFL, and other events
encouraging individual business owners and company growth.
3
Conference Chairs
WELCOME
Brent Smith
I/ITSEC 2015
Conference Chair
David Hutchings
I/ITSEC 2015
Program Chair
BRENT SMITH is the Vice President and Chief Technology Officer for Engineering & Computer
Simulations in Orlando, Florida. As the CTO, Mr. Smith is responsible for the overall design of the
company’s next generation of learning technologies which include intelligent systems, virtual environments, modeling & simulation applications, and serious games that improve training efficiency.
In this role, he is responsible for the strategy and vision of the company’s portfolio of training and
education products. He also coordinates research and development activities across the organization
with a focus on modeling & simulation, blended learning, performance assessment, and managing
big data. Mr. Smith brings over 20 years’ experience in developing enterprise learning architectures
and is a nationally recognized expert in the modeling and simulation (M&S) industry. Mr. Smith
has written numerous papers, managed large multi-disciplinary programs, and developed award
winning simulation content that has been recognized in numerous publications. Mr. Smith has a
long standing association with I/ITSEC. Prior to serving as the 2013 Program Chair, he has been
an author, a presenter, and exhibitor, and a technology consultant working to help modernize the
conference infrastructure.
DAVID HUTCHINGS is the Senior Vice President, Strategic Business Development for Raydon Corporation headquartered in Port Orange, Florida. In this role, his primary responsibility is the planning
and execution of the strategic growth plan with primary focus on strategic pursuits, partnerships,
and M&A. Mr. Hutchings has over 32 years of comprehensive business development and senior management operations experience in DOD, commercial and other government agencies with a strong
focus on custom training and simulation solutions. His experience has included playing a significant
role in corporate start-ups, turnarounds, acquisitions, business strategy, marketing initiatives, and
establishing strategic business partnerships. He has held senior level management positions in regional and national business development, operations, and sales for Raydon Corporation, Camber
Corporation, Teledyne Brown Engineering, Interactive Media Corporation, APEX Technology, Simms
Industries, and General Physics Corporations. Mr. Hutchings has served on the NDIA Central Florida
Board of Directors since 2003 with his current role as the Vice President of Recognition and Awards.
Prior to serving as the I/ITSEC 2015 Program Chair, he served as the I/ITSEC STEM Student Tour
Coordinator from 2009 - 2014 as well as other I/ITSEC subcommittees in prior years including the
Training, Education, Best Paper and PSMA Subcommittees. He is a member of AUSA, NTSA, and
NDIA. He holds a B.S. degree in Business Management/Marketing and has served on the Huntsville,
AL BizTech Small Business Incubator Board of Advisors. Mr. Hutchings also served six years in the
U.S. Navy’s Submarine Force.
Conference Sponsor
Following graduation from Rensselaer
Polytechnic Institute, designation as a
Naval Aviator and training in the F-14
Tomcat, Admiral Robb deployed nine
times across the globe accumulating over
5000 hours and 1000 carrier landings.
Following a tour flying Russian fighters
in the Nevada desert, he commanded
Fighter Squadron Fifty One, Carrier Air
Wing Nine, the Navy Fighter Weapons
RADM James Robb, School (TOPGUN), and Carrier Strike
Group Seven. As a Flag Officer he manUSN (Ret.)
President National aged all Naval Aviation Programs (N980)
and was the Director of Navy Readiness
Training and
(N43). Following 9/11, he joined USSimulation
CENTCOM as the Director of Plans (J5)
Association
deploying to the Middle East in support
of combat operations. Retiring in 2006,
he built a successful small consulting
business before joining the National
Training and Simulation Association as
President in June 2012.
4
Craig R. McKinley, Gen (ret), USAF, became President and CEO, National Defense Industrial Association (NDIA) in
January 2015. He had previously worked
at the Air Force Association (AFA). His final assignment in uniform was Chief, National Guard Bureau, where he also served
as a member of the Joint Chiefs of Staff.
Craig received his commission in 1974
as a distinguished graduate of the ROTC
program at Southern Methodist UniverGen Craig R.
McKinley, USAF (Ret.) sity (SMU). He has served in numerous
assignments in flying and operations;
President and
Chief Executive command positions at group, wing, sector
and field operating agency levels. He was
Officer
National Defense a command pilot with more than 4,000
hours. He graduated with a bachelor’s deIndustrial
gree in business administration from SMU
Association
in 1974 and his Master’s in management
and economics from Webster College in
1979. He completed studies at the National War College, National Defense University, in 1995.
Col Walter Yates, USMC
Program Manager, Marine
Corps Systems Command
PM Training Systems
Navy Service Executive (Lead Service)
CAPT WES NAYLOR, USN: The Naval
Air Warfare Center Training Systems Division (NAWCTSD) is the Navy’s principal center for modeling, simulation, and
training systems technologies. The command provides training solutions and
research for a wide spectrum of military
programs, including aviation, surface
& undersea warfare, and other specialized requirements. CAPT Naylor leads
a workforce of more than 1,000 scientists, evaluators, engineers, technicians,
logisticians, contracting specialists, and
support personnel. A native of McLean,
Virginia, CAPT Naylor is a graduate
of George Mason University, where he
earned his Bachelors of Science degree
in Government, and a 2007 graduate of
the National War College, where he was
awarded a Master’s Degree in National
Security Strategies. As a Naval Aviator,
CAPT Naylor flew the Navy’s P-3 Orion
maritime patrol aircraft. He also has experience serving in various acquisition
and staff positions. CAPT Naylor served
as the Executive Officer of NAWCTSD
for two years prior to assuming command in June 2014. His awards include
the Meritorious Service Medal, Air Medal, Navy Commendation Medal, Navy
and the Achievement Medal.
Marine Corps Service Executive (Lead Service)
Col WALTER YATES, USMC, Program
Manager, Training Systems: As the Marine Corps Systems Command Program
Manager for Training Systems Col Yates
is responsible for managing a workforce
of over 150 personnel in the acquisition
and sustainment of training systems used
throughout the Marine Corps. Col Yates
graduated from Texas A&M University in
1990 with a B.S. in Mechanical Engineering Technology. He graduated from the
Naval Postgraduate School in 2004 with
a M.S. in the field of Modeling, Virtual
Environments and Simulation. As a Field
Artillery officer he served ashore and deployed afloat in billets including forward
observer, platoon commander, fire direction officer, and battery commander. Col
Yates served as the officer in charge of
the Battle Simulation Center for MAGTF
Training Command from 2004 to 2007.
Col Dan Marticello, USAF
Chief, Simulators Division,
Air Force Materiel
Command
MG Jon Maddux, USA
U.S. Army Program
Executive Officer for
Simulation, Training and
Instrumentation
From 2007 to 2011, he served on the Program Manager for Training Systems staff,
including a year as the Marine Corps
Systems Command Liaison Officer to
the Command Element of Multinational
Force-West, Iraq. In 2011, Col Yates returned to Marine Corps Systems Command, as the Deputy for Modeling and
Simulation. He completed his assignment
as a Secretary of Defense Corporate Fellow at Norfolk Southern Corporation prior to reporting for duty as PM TRASYS.
Air Force Service Executive
Col DAN MARTICELLO, USAF, is Chief
of the Simulators Division, AF Life Cycle
Management Center, Wright-Patterson
AFB, Ohio. He directs 400 employees in
the acquisition and sustainment of more
than 40 USAF and 10 foreign AF training
systems. Col Marticello graduated from the
USAF Academy and earned an M.S. degree
at the University of Colorado. He graduated
from USAF Test Pilot School, and Air Command and Staff College. His assignments
have included serving as flight test program
manager for the F-15 Joint Helmet-mounted Cueing System and AIM-9X missile;
overseeing F-117A developmental flight
test; working on the HQ AFMC Commander’s Action Group; managing upgrades to
the F-15C/D air superiority fleet; and commanding a squadron executing classified
acquisition programs. He attended MIT as
an AF Fellow, and then managed a C-17
capability upgrade program. He deployed
with NATO Training Mission-Afghanistan,
where as Chief of Plans and Operations
(DCGS), he created the Afghan Defense
Acquisition & Resource Management Institute. He was then assigned as Chief of the
HQ AFMC Capabilities and Requirements
Division, managing the AF Agile Combat
Support Core Function portfolio.
Army Service Executive
MG JON MADDUX, USA, is responsible
for simulation, training and test/instrumentation to support the U.S. Army. PEO
STRI annually executes a multi-billion
dollar program with a workforce of 1200
employees. He oversees approximately
1600 contracts valued at over $28 billion.
MG Maddux has held key positions in the
Army, including Assistant to the Princi-
EXECUTIVES
CAPT Wes Naylor, USN
Commanding Officer,
Naval Air Warfare Center,
Training Systems Division
Frank C. DiGiovanni, SES
Director, Force Readiness
and Training, Office of
the Assistant Secretary of
Defense (Readiness)
pal Military Deputy, Assistant Secretary
of the Army Acquisition, Logistics and
Technology (ASA(ALT)); Deputy Commanding General, Support, Combined
Security Transition Command-Afghanistan; PEO Ammunition/Commanding
General, Picatinny Arsenal; Chief of
Staff, Office of the ASA(ALT); Director
for Army Evaluation Task Force Integration, Directorate for Program Manager for
Future Combat Systems (Brigade Combat
Team); and Project Manager for Future
Combat Systems Network Systems Integration. MG Maddux entered the Army
as an enlisted Soldier in 1976 and he has
earned several awards, including four
Legion of Merit awards and the Bronze
Star Medal. MG Maddux graduated from
the U.S. Army War College and has three
Masters degrees.
Senior Advisor for Readiness and Training
FRANK C. DIGIOVANNI, SES, serves as
the Director, Force Readiness and Training, Office of the Assistant Secretary of
Defense (Readiness). His responsibilities
include policy and oversight of military
training readiness and capability modernization. He leads the Department’s
$4.3B Combatant Commander Exercise
and Engagement and Training Transformation, the sustainment of military training ranges, the development of Live, Virtual and Constructive Training Standards
and Architectures, the Advanced Distributed Learning Initiative, the creation of
a “virtual world” training capability, and
ensures training is properly incorporated
into major acquisition programs. He also
serves as a senior DoD training member
on the Modeling and Simulation Steering
Committee and collaborates with interagency partners to develop training strategy and policy to ensure Government
civilians and Service members are better
prepared to conduct reconstruction and
stabilization operations. He oversees efforts and policies associated with sustaining access to DoD’s land, air and sea
training space and for developing policy, strategic communication and the research agenda associated with energy infrastructure and its impact on the ability
of the Department to conduct readiness
training activities.
5
Service Principals
PRINCIPALS & ADVISOR
Diana Teel
Navy
Industry Outreach, Naval Air
Warfare Center Training Systems
Division (NAWCTSD)
Martin Bushika
Marine Corps
Director, Strategic Business
Operations, PM TRASYS
MARCORSYSCOM
Tony DalSasso
Air Force
Chief Engineer, Simulators
Division, Air Force Materiel
Command (AFMC)
Traci Jones
Army
Deputy Chief of Staff,
U.S. Army Program Executive
Office, Simulation, Training and
Instrumentation
(PEO STRI)
OSD Principal
Education and Training Advisor
Sae Schatz, Ph.D.
Director, Advanced Distributed
Learning Initiative, Office of the
Assistant Secretary of Defense
(Readiness)
6
VADM Al Harms, USN (Ret.)
Vice President (Emeritus)
for Strategy, Marketing,
Communications and Admissions,
University of Central Florida
Agenda
WEDNESDAY, 25 NOVEMBER 2015
TIME
LOCATION
Exhibitor Registration Open
1700
Exhibitor Registration Close
S220 of the South Concourse
PRE-CONFERENCE AGENDA
0730
THURSDAY, 26 NOVEMBER 2015
CLOSED FOR THANKSGIVING
FRIDAY, 27 NOVEMBER AND SATURDAY, 28 NOVEMBER 2015
TIME
LOCATION
0730
1800
Exhibitor Registration Close
SUNDAY, 29 NOVEMBER 2015
TIME
LOCATION
0730
1200
Conference Registration Open
1200
All Satellite Registrations Open
Multiple Hotels
1800
All Registrations Close
Dress Code
BRANCH
CONFERENCE AND GENERAL SESSIONS
BANQUET
Army
ACUs or Duty Uniform
Army Blue (Army Evening Mess optional)
Marine Corps
Service “C”
Evening Dress (Dress Blue “B” or Service “A” optional)
Navy
Service Khaki, Navy Service Uniform
Dinner Dress White (Service Dress White optional)
Air Force
Short or Long Service Blues
Service Dress Blue with tie and jacket (Mess Dress optional)
Coast Guard
Tropical Blue Long
Civilian
Business attire
Black tie (optional)
7
MONDAY, 30 NOVEMBER 2015
TIME
LOCATION
AGENDA
0730
Conference and Exhibit Registration Open
0730
Multiple Hotels
0830 - 1000 TUTORIALS (Synopses begin on page 18) 
Transmedia Learning in the Wild: Supporting Military Training Through Story-driven
Engagement (1504) %
Room S320A
Introduction to HLA (1514) ñ
Room S320C
Fundamentals of Modeling and Simulation (1507)
Room S320GH
Enterprise Solutions for M&S Asset Discovery and Reuse (1518)
Room S320E
Natural Language Interaction in Simulations: Moving Beyond Pre-Coded Scripts (1536) %
Room S320B
Breathing New Life Into Old Assets (1539)
Room S320F
Building Intelligent Tutoring Systems for Teams: What Matters (1531)
Room S320D
1030 - 1200 SIGNATURE EVENT: Congressional Modeling and Simulation Event (page 29)
Room S310BCD
1400
Elevate Your Instruction: Practical Tactics to Maximize Military Learning (1516) 
Room S320A
TENA/JMETC: An Innovative Architecture for Testing and Training (1509)
Room S320C
Planning and Execution of a Large Multi-Architecture Distributed LVC Event (1508)
Room S320GH
Simulation Conceptual Modeling Theory and Application (1534)
Room S320E
Speech-based Interaction: Myths, Challenges and Opportunities (1515) ñ
Room S320B
Three-Dimensional Display Technologies for Simulation and Training (1529)
Room S320F
Simulator Fidelity in Training: Informed Decision-Making to Maximize Effectiveness (1505)
Room S320D
Exhibits Open
Exhibit Hall
1430 - 1600 FLOOR EVENT: O
peration Blended Warrior (OBW) (page 39)
Disaster Strikes
Booth 339 and other floor
locations
1430 - 1600 COMMUNITY OF INTEREST: International Outreach (page 47)
Room S310A
An Introduction to Cognitive Systems for Modeling & Simulation (1511) 
Room S320A
Distributed Interactive Simulation 101: The Basics (1538)
Room S320C
Model Verification and Validation Methods (1506)
Room S320GH
U.S. Export Controls 2015: Next Phase of Export Control Reform for the International
Simulation Industry (1503)
Room S320E
Using HTML5 to Develop IMI, Simulations, and Games (1521) %
Room S320B
When Simple Becomes Complicated: Using Machinima for Training and Education (1533) %
Room S320F
Team and Collective Training Needs Analysis (1537) ñ
Room S320D
1530 - 1700 SIGNATURE EVENT: DoD Small Business Programs (page 30)
1800
Exhibits Close
1800
All Registration Stations Close
Warfighters Corner
LEGEND (one or more of the following may appear on this page). The number in parentheses following Paper Title is the ID tracking number.
 Nominated for Best Paper Award  Honorable Mention for Best Paper Award
 Continuing Education Units (see pg. 12) % Game-related Subject Matter
ñ International Author h Medical-related Subject Matter
8
Ï Cyber Security
Mobile
TUESDAY, 1 DECEMBER 2015
TIME
LOCATION
0700
Multiple Hotels
0745
Pre-Ceremony Music
Central Florida VFW Musical Ensemble
Hyatt Regency,
Windermere Ballroom
0830 - 1000 OPENING CEREMONIES
AGENDA
0700
Hyatt Regency/Windermere
Call to Order
Presentation of Colors
National Anthem
Invocation
OPENING REMARKS
Brent Smith, 2015 Conference Chair
KEYNOTE ADDRESSES
Admiral John M. Richardson, USN
Waymon Armstrong
Chief of Naval OperationsFounder and CEO of Engineering
& Computer Simulations
1030 - 1200 SIGNATURE EVENT: General/Flag Officer Panel (page 31)
1200
Exhibits Open
Hyatt Regency/Windermere
Exhibit Hall
1200 - 1330 Lunch (Opening of Exhibits and Lunch will occur at 1200 or upon adjournment of the
General/Flag Officer Panel)
South B
1300 - 1330 COMMUNITY OF INTEREST: PEO STRI Updates to TSIS Briefs: Field OPS (page 47)
Booth 2280
1330 - 1400 COMMUNITY OF INTEREST: PEO STRI Updates to TSIS Briefs: PM TRADE (page 47)
Booth 2280
1400 - 1430 COMMUNITY OF INTEREST: PEO STRI Updates to TSIS Briefs: Army Contracting
Command (page 47)
Booth 2280
1400 - 1530 PAPER SESSIONS
Rooms S320A-F; S330C
(Title/Author List begins on page 58. Session schedules for this timeframe are on page 53.)
1400 - 1530 SIGNATURE EVENT: Forging the Future of Navy Training (page 32)
Room S330AB
1400 - 1530 FOCUS EVENT: Live-Synthetic Blended Training: From Home Station Environment to
Battlespace Execution (page 41)
Room S320GH
Immediate Response
locations
1530 - 1700 FLOOR EVENT: Warfighters Corner, From the Tip of the Spear (page 40)
Joint Warfighter Panelists: recently deployed Service men and women.
Booth 2280
1530 - 1700 COMMUNITY OF INTEREST: Air Force Acquisition Update (page 47)
Room S320GH
Rooms S320A-F
1700 - 1830 Exhibitor Networking Event
Exhibit Hall
1800
Senior Leaders Networking Hour and M&S Awards Dinner, by invitation from NTSA only Hyatt Regency
1800
1830
Exhibits Close
9
WEDNESDAY, 2 DECEMBER 2015
TIME
0700
LOCATION
AGENDA
Rooms S320A-F
0830 - 1000 SPECIAL PAPER SESSION: Best Papers from Around the Globe (page 54)
Room S210D
0830 - 1000 SIGNATURE EVENT: Forging the Future of Marine Corps Training (page 33)
Room S310BCD
0830 - 1000 FOCUS EVENT: Medical Simulation and Training – A Joint Endeavor (page 42)
Room S320GH
0830 - 1000 COMMUNITY OF INTEREST: International Outreach (page 47)
Room S310A
0830 - 1000 COMMUNITY OF INTEREST: Asia Pacific Simulation Alliance Workshop (page 47)
Room S330C
0900 - 1730 COMMUNITY OF INTEREST: ADL Virtual World Sandbox PlugFest: Hands-on Tutorial
(page 48)
Room S330D
0930
Exhibits Open
Exhibit Hall
Rooms S320A-F
1030 - 1200 SPECIAL PAPER SESSION: Presentations from Future Leaders Pavilion (page 70)
Room S210D
1030 - 1200 SIGNATURE EVENT: Training the Cyber Warrior: A Learner-Centric Model (page 34)
Room S330AB
Countering Exploitation
locations
Booth 2280
1030 - 1200 COMMUNITY OF INTEREST: ETSA/SISO Briefing: International M&S Standards (page 48) Room S330C
1200 - 1330 Lunch
South B
1200 - 1730 COMMUNITY OF INTEREST: I/ITSEC Energy Forum: Demonstration and discussion of
the latest training and simulation requirements in the energy sector (page 48)
Room S210D
1300 - 1330 COMMUNITY OF INTEREST: PEO STRI Updates to TSIS Briefs: PM ITTS (page 47)
Booth 2280
1330 - 1400 COMMUNITY OF INTEREST: PEO STRI Updates to TSIS Briefs: PM ITE (page 47)
Booth 2280
1400 - 1430 COMMUNITY OF INTEREST: PEO STRI Updates to TSIS Briefs: FMS (page 47)
Booth 2280
Rooms S320A-F
1400 - 1530 SIGNATURE EVENT: F-35 Lightning II – Live, Virtual, Constructive (LVC) (page 35)
Room S320GH
1400 - 1530 FOCUS EVENT: Industry Involvement Initiative for NATO Exercises (I3X) – Lessons
Learned (page 43)
Room S330C
Rooms S320A-F
10
1600 - 1730 SIGNATURE EVENT: I/ITSEC Fellows (page 36)
Room S330AB
Securing the Skies
locations
1600 - 1730 FOCUS EVENT: The Serious Games Showcase & Challenge: Celebrating 10 Years of
Forging the Future Through Innovation (page 44)
Room S320GH
1600 - 1730 COMMUNITY OF INTEREST: Hiring America’s Service Members - DoD SkillBridge
Forum (page 49)
Warfighters Corner
1800
1800
Exhibits Close
THURSDAY, 3 DECEMBER 2015
TIME
0700
LOCATION
AGENDA
Rooms S320A-F
0830 - 1000 SIGNATURE EVENT: Black Swan – Setting the Stage (page 37)
Room S330AB
0830 - 1000 FOCUS EVENT: IGNITE! (page 45)
Room S330C
0830 - 1000 COMMUNITY OF INTEREST: Geospatial Environmental Database Standards Forum
(page 49)
Room S330D
0930
Exhibits Open
Exhibit Hall
Rooms S320A-F
1030 - 1200 SIGNATURE EVENT: Training Adaptive Leaders for a Black Swan World (page 38)
Room S330AB
Overmatch: Forward from the Sea
locations
Booth 2280
1030 - 1200 FOCUS EVENT: Visual Computing Event – Special Event (page 46)
Room S320GH
1030 - 1200 COMMUNITY OF INTEREST: Navy Vision: From the Training Systems Program
Manager’s Perspective (page 49)
Room S330C
1030 - 1200 COMMUNITY OF INTEREST: Design of Learning Games (page 49)
Room S330D
1200 - 1330 Lunch
South B
1300
AWARDS CEREMONY: Serious Games Showcase & Challenge (page 72)
Warfighters Corner Stage
1345
AWARDS CEREMONY: Future Leaders (page 70)
Warfighters Corner Stage
Rooms S320A-F
1500
Exhibit Hall and Registration Close
Exhibit Hall
1800
Hosted Reception sponsored by Lockheed Martin
Hyatt Regency, Windermere Foyer
1900
Reception
Awards
&
Conference Awards Banquet
Hyatt Regency
Dinner Music provided by Bob Dehne on the Vibraphones
Windermere Ballroom
Best Paper Award Presentation
RADM Fred Lewis Postgraduate I/ITSEC Scholarship
After Dinner Entertainment provided by The Flashback Four
b a n q u e t Passing of the Flag for I/ITSEC 2016
Dance Music provided by Captain Harry and the Surfriders
FRIDAY, 4 DECEMBER 2015
TIME
PROFESSIONAL DEVELOPMENT WORKSHOPS (Synopses can be found on pages 75-76)
0800 - 1700 Serious Game Design Tutorial
Room S330A
0800 - 1200 Certified Modeling & Simulation Professional (CMSP) Exam Preparation
Room S330B
0800 - 1200 Live-Virtual-Constructive (LVC) Interoperability Techniques
Room S330C
0800 - 1200 Modeling & Simulation for Acquisition
Room S330D
0800 - 1200 Seamless Mobile Learning and Simulations
Room S330E
0800 - 1200 Measuring the Impact and ROI of Training, Simulation, and Education Programs
Room S330F
11
Continuing Education Units: An I/ITSEC Opportunity
What sessions are CEU Eligible? (Updated for 2015.)
C O N T I N U I N G E D U C AT I O N U N I T S / C O N T I N U O U S L E A R N I N G P O I N T S
“CEUs are a convenient and efficient
way to keep track of my participation in
professional development activities.”
Continuing Education Units (CEU) were established in 1970 to
create a unit of measurement to quantify continuing education
and training activities. CEUs apply to technical and educational
settings such as I/ITSEC. The primary focus of I/ITSEC is to
highlight innovative implementation of simulation and education
technologies as tools to achieve cost efficient training and increased
military readiness. Therefore CEUs are offered for all Tutorials, Paper
Sessions, and the Professional Development Workshops. CEUs are being
sponsored and maintained by the University of Central Florida,
Division of Continuing Education.
Why should I earn CEUs at I/ITSEC?
• Participation in the tutorials, papers and/or Post-conference
Workshops for CEU credit reinforces your commitment to remain current in the evolving technologies relating to training
and simulation.
• The CEU transcript indicates your active participation in the
technical program of the conference to your employer.
• Previous attendees have indicated that CEUs have assisted them
in securing approval to attend the conference.
• All Tutorials, Papers, and Professional Development Workshops are CEU eligible. Additionally, the I/ITSEC Fellows
Signature Event is also Eligible.
• Who may attend these events? Tutorials and Professional
Development Workshops and the Fellows Event are open
to everyone. The Paper Sessions are limited to registered
conference attendees.
• Does attending mean I automatically receive CEU credits? No.
You have to let us know, via your registration, that you are interested in the credits. There is no charge for Paid Conference
Attendees. However, if you are in an unpaid category (i.e.
Exhibitor Personnel) there is a $45 charge, payable during
registration. You may also register separately for the CEUs if
you missed this step in your conference registration process.
• We also ask that you be sure to mark your registration if you
wish to attend the Professional Development Workshops held
on Friday so that appropriate rooms/seating are in place.
How do I receive CEUs at I/ITSEC?
1. Be sure you are appropriately registered (you can confirm
when you check in onsite) for CEU credits.
2. Follow directions provided at each session to answer a few
basic questions about the presentation.
3. Your CEU transcript will come to you via the University
of Central Florida, Division of Continuing Education. Ten
contact hours equate to one CEU credit.
Contact Maria Cherjovsky at (407) 882-0247 or
mariac@mail.ucf.edu for additional information
Continuous Learning Points (CLPs)
The U.S. Department of Defense (DoD) acquisition workforce members are expected to earn Continuous Learning Points (CLPs) to stay current in leadership
and functional acquisition skills that augment the minimum education, training and experience standards established for certification purposes within their
acquisition career fields. It is each acquisition member’s responsibility to meet the goal of 40 CLPs each year and to meet the mandatory requirement of
80 CLPs every two years. Acquisition Professional Activities are allowed to count toward CLPs. CLPs are awarded in accordance with DoD-wide guidelines
as augmented by Service-specific policies. I/ITSEC provides an excellent opportunity for the DoD acquisition workforce members to earn mandatory CLPs.
EARNING THE CMSP DESIGNATION WILL:
• Demonstrate expertise in the field of M&S to your
employer and the larger M&S community
• Provide opportunities for professional advancement
Requirements include 3-8 years of work experience
(depending on level of highest collegiate degree), 3
professional letters of reference, and successful
completion of an online examination.
The completion of CEUs has long been used to demonstrate
dedication and career interest to supervisors, employers
CMSP Applicants now have a choice between
CMSP-Technical and CMSP-Management exams.
or rating officers. CEUs earned at I/ITSEC can be applied TO LEARN MORE ABOUT THE REQUIREMENTS AND TO APPLY,
PLEASE VISIT WWW.SIMPROFESSIONAL.ORG OR CONTACT
towards CMSP Recertification.
PATRICK ROWE AT PROWE@NDIA.ORG.
12
South Concourse
Orange County Convention Center, Orlando, Florida
CONVENTION CENTER
Although the buildings are large, a five minute walk will take you...
• from the South Concourse to the North Concourse
• from the South Concourse to the Hilton
• from the South Concourse to the Hyatt Regency, formerly Peabody Orlando
• from the South Concourse to the Rosen Center
(In comparison, a walk from Hall A to Hall F in the West Concourse takes about ten minutes.)
LEVEL 3 (Presentations/Events/Practice Rooms)
LEVEL 2 (Entry Level)
To Exhibit Floor
To Exhibit Floor
First Aid
Self-Registration
Onsite Registration
Property Check
Lost & Found
Hyatt Regency
Hilton
Rosen Centre
13
RECREATION LEVEL
H YAT T R E G E N C Y
MEZZANINE LEVEL
B-Line Diner
ENTRY LEVEL
Opening Ceremonies
Keynote Speakers
General/Flag Officer Panel
Closing Banquet
CONVENTION LEVEL
Bus Drop Off
Hyatt Regency Satellite
Registration Desk
14
IN MEMORIAM
LIEUTENANT COLONEL EARLE L. DENTON, U.S. Army (Ret.)
The I/ITSEC family, along with the
overall modeling and simulation
technology community, mourns
the loss of a significant participant
in the history of I/ITSEC as well as
other NTSA events. Earle Denton’s
involvement in training issues is
well known, to government and
industry alike, for he was a professional of the first magnitude.
As for training, Second Lieutenant
Denton, had full appreciation in the
value of being prepared and for his
actions in the Battle for Pork Chop Hill. In the Korean War
he was decorated with the Silver Star medal for his combat
leadership in reclaiming a US Army strategic location that had
been lost in an earlier battle. This experience, along with his
leadership in the Vietnam War, where he commanded the 1st
Battalion, 18th Infantry, 1st Infantry Division, provided impetus for his personal commitment to ensure that the U.S. Army
is the best trained in the world.
During his military career, Earle received a number of
awards and decorations including the Silver Star to Legion of
Merit with OLC, Purple Heart, and the Combat Infantryman
Badge. In 2001 he was presented the Korean Ambassador of
Peace and Appreciation Medallions by the Republic of Korea War Veterans. Subsequent to his military career Earle was
recognized by various organizations for his lifelong contributions to their efforts in the advancement of technology development and employment of Modeling and Simulation for
IN MEMORIAM
March 16, 1930 – March 1, 2015
training. In 2001 Earle was presented the Interservice/Industry Training, Simulation and Education Conference (I/ITSEC)
Lifetime Achievement Award.
In 2012 Orange County Mayor Theresa Jacobs selected
Earle to receive the Col W. Kittinger Medal of Achievement
for his selfless act of giving to his country, his county and
his community. Also in 2012, Earle was selected as the
Distinguished Citizen of the year for Orange County, Florida
District III.
Unabashed and colorful, Earle often appeared wearing one
of a collection of patriotic neckties; the accompanying photograph depicts friends at his funeral service wearing some
of those ties, so symbolic of Earle’s patriotism and love of
country.
Receiving the Lifetime Achievement Award in 2001
Friends share and model Earle’s tie collection at his Memorial Service in Orlando in March 2015
15
IN MEMORIAM
WILLIAM F. (BILL) WAITE
IN MEMORIAM
October 17, 1946 - July 25, 2015
Bill and AEgis co-founder Steven Hill.
16
The I/ITSEC Community mourns the
loss of Bill Waite, Co-founder and Chairman of The AEgis Technologies Group,
Inc. and a leading advocate and promoter of modeling and simulation.
Bill was impassioned about M&S
education and worked diligently in advocacy and support of establishment of
Master’s Degree and Ph.D. programs in
M&S at UAH, UCF, Old Dominion Uni- Attendees at the recent SISO Simulation Inversity, the Naval Post Graduate School, teroperability Workshop remembered Bill by
and others. He was an influential, active wearing ribbons denoting his familiar blue
member of the Modeling and Simulation
shirt/red striped tie “uniform.”
Professional Certification Commission,
which established the concept, protocols, testing, and training for the Certified
Modeling & Simulation Professional (CMSP) program.
He took his passion and energy to a national and international scale, creating
and leading the international SimSummit Round Table from its inception in 2002,
and providing critical and influential support to Congressmen Randy Forbes and
Bobby Scott and the Congressional M&S Caucus in passing House Resolution 487
in 2007, which established M&S as a National Critical Technology. Bill was also a
leading proponent of the sustained effort that resulted in standup of the National
Modeling and Simulation Coalition in 2012. In addition, during the past two decades, he twice led the Society for Modeling and Simulation International (SCS) as
Chairman of the Board of Directors, and served twice on the Executive Committee
of the Simulation Interoperability Standards Organization (SISO). Bill was also active in the National Training and Simulation Association (NTSA), served the National Defense Industrial Association (NDIA) as a member of its Board of Trustees,
and was a member of the Board of Directors of Economic Development Partnership
of Alabama Foundation.
Beyond his leadership of AEgis, Bill spent a significant amount of his time working to raise national and international awareness and use of Modeling & Simulation
technology. He worked diligently to transform the industry’s ability to collaborate
and communicate through the establishment of industry forums such as the Alabama Modeling and Simulation Council (AMSC) and the Virginia Modeling and
Simulation Partnership (VMSP). He additionally was a member of various NATO
technical forums and groups.
Bill’s energy, passion, expertise, and advocacy for M&S will long be remembered
as singularly instrumental in the development of M&S as a Technology, Profession,
Industry, and Marketplace.
Tutorials
M on d ay, 30 N ov e m b e r 2015
 All Tutorials are eligible for CEU/CLP credits. (See page 12) 
I/ITSEC Tutorials are designed to serve three purposes:
• Provide foundational educational material, including material essential to prepare for Certification as a Modeling and Simulation Professional (CMSP).
• Serve as a refresher and more advanced learning opportunity for those seeking to maintain
their certification.
• Bring topics of special interest in Training, Simulation and Education to I/ITSEC attendees.
TUTORIALS
OPEN TO ALL ATTENDEES
FOR YOUR CONVENIENCE,
FOOD STATIONS ARE OPEN
ON LEVEL TWO DURING
MONDAY SESSIONS.
TUTORIALS SCHEDULE
ROOM
S320A
S320C
TRACK/CHAIR
Track 1:
Simply The Best
Michael Freeman,
Ed.D.
Track 2:
Architectures
Robert Lutz
Track 3:
S320GH Simple to Complex
James Coolahan, Ph.D.
S320E
S320B
Track 4:
Policy & Processes
Denise Nicholson,
Ph.D.
Track 5:
Interacting with Users
Charles Cohen, Ph.D.
S320F
S320D
Track 6:
Visual Arts
Thomas Mastaglio,
Ph.D.
Track 7:
New Dimensions in
Training
Luis Miguel
Encarnação, Ph.D.
0830 – 1000
1245 – 1415
1430 – 1600
Transmedia Learning in the
Wild: Supporting Military
Training Through Story-driven
Engagement (1504) %
Elevate Your Instruction:
Practical Tactics to Maximize
Military Learning (1516) 
An Introduction to Cognitive
Systems for Modeling &
Simulation (1511) 
Introduction to HLA
(1514) ñ
TENA/JMETC: An Innovative
Architecture for Testing and
Training (1509)
DIS 101: The Basics
(1538)
Fundamentals of Modeling
and Simulation (1507)
Planning and Execution of
a Large Multi-Architecture
Distributed LVC Event (1508)
Model Verification and
Validation Methods (1506)
Enterprise Solutions for M&S
Asset Discovery and Reuse
(1518)
Simulation Conceptual Modeling
Theory and Application (1534)
U.S. Export Controls 2015:
Next Phase of Export Control
Reform for the International
Natural Language Interaction
in Simulations: Moving Beyond
Pre-Coded Scripts (1536) %
Speech-based Interaction:
Myths, Challenges and
Opportunities (1515) ñ
Using HTML5 to Develop
IMI, Simulations, and Games
(1521) %
Breathing New Life Into
Old Assets (1539)
Three-Dimensional Display
Technologies for Simulation and
Training (1529)
When Simple Becomes
Complicated: Using Machinima
for Training and Education
(1533) %
Building Intelligent Tutoring
Systems for Teams: What
Matters (1531)
Simulator Fidelity in Training:
Informed Decision-Making to
Maximize Effectiveness (1505)
Team and Collective Training
Needs Analysis (1537) ñ
LEGEND (one or more of the following may appear on this page). The number in parentheses following Tutorial Title is the ID tracking number.
 Nominated for Best Tutorial Award ñ International Author % Game-related Subject Matter
17
Mo n d a y, 30 N ov e m b e r 2015
TUTORIALS
TRACK 1: M&S: SIMPLY THE BEST
0830 – 1000 • ROOM S320A
TRACK 2: ARCHITECTURES
0830 – 1000 • ROOM S320C
Transmedia Learning in the Wild:
Supporting Military Training Through
Story-driven Engagement (1504) %
Introduction to HLA
Technology-mediated solutions for learning, instruction, and
assessment are often intended for use by schoolhouses or
formal training programs. Few address informal learning or
self-paced “learning in the wild.” This tutorial addresses this
gap by discussing transmedia learning design specifically for
informal learning and by offering open source software approaches to unobtrusively track learner progress. Transmedia
learning is social, story-driven, unfolds across multiple media, and designed to promote self-directed engagement. The
first section of the tutorial defines and describes how and
why transmedia learning can be a game-changer for training and education. The second section explores the design
and development of a “Warrior-Athlete” transmedia learning
ecosystem, a use case shared by the Services, VA, NATO, industry, and Federal Government alike. Strategies for the use
of massively open online course (MOOC) platforms, mobile
apps, social media, videos, machinima, virtual environments,
and games are provided. The third section demonstrates open
source software for tracking engagement in transmedia learning ecosystems such as the experience API, learning record
store, analytics dashboard, competency frameworks, and a
game/collaborative 3D virtual environment. No prerequisite
knowledge is required. Program managers, researchers, designers, and developers who are interested in 1) transmedia
storytelling to engage learners, and 2) the design and implementation of transmedia learning ecosystems will take away
practical strategies, resources, tools, and software for their
own use.
The High-Level Architecture (HLA) is the leading international standard for simulation interoperability. It originated
in the defense communities but is increasingly used in other
domains. This tutorial gives an introduction to the HLA standard. It describes the requirements for interoperability, flexibility, composability and reuse and how HLA meets them. It
also describes the new features of the most recent version:
HLA Evolved (IEEE-1516-2010) and the road ahead. Finally it
provides some recent experiences of the use of HLA in NATO
M&S groups as well as an overview of recent evolution of
Federation Object Models for military platform simulation.
This tutorial is intended for all audiences; however, some
familiarity with basic principles of distributed computing is
recommended.
Presenter
ELAINE M. RAYBOURN, Ph.D., is a Principal Member of the Technical Staff in Cognitive Science & Systems at Sandia National Laboratories and a European Research Consortium for Informatics and
Mathematics (ERCIM) Fellow who has worked in research laboratories in Germany, England, and France. Elaine is on assignment
to the Advanced Distributed Learning Initiative, Office of the Assistant Secretary of Defense (Readiness). As a social scientist, her
greatest passion involves designing immersive experiences that
present opportunities to hone intercultural communication competence, self-awareness, and adaptability. To that end she led the development of an award-winning Government game. She serves on
several editorial boards including Interactive Technology and Smart
Education, Journal of Game-based Learning, and Simulation & Gaming. She conducts research on transmedia learning, next generation
learning ecosystems, and the anthropology of next generation learners’ interactions with future technology. Elaine is a recipient of the
Department of the Army Award for Patriotic Civilian Service.
(1514) ñ
Presenters
BJÖRN MÖLLER is the vice president and co-founder of Pitch Technologies, the leading supplier of tools for the High-Level Architecture
(HLA). He leads the strategic development of Pitch HLA products.
He serves on several HLA standards and working groups and has a
wide international contact network in simulation interoperability. He
has twenty years of experience in high-tech R&D companies, with an
international profile in areas such as modeling and simulation, artificial intelligence and Web-based collaboration. He is currently serving as the chair of the SISO RPR FOM Product Development Group
and the vice chair of the SISO HLA Evolved Product Support Group.
ROBERT LUTZ is a principal staff scientist at The Johns Hopkins
University Applied Physics Laboratory in Laurel, MD. His background includes 35 years of practical experience in the development, use, and management of models and simulations across all
phases of the DoD systems acquisition process. He currently serves
as the Airspace Integration Modeling and Simulation (M&S) lead for
the Navy’s Triton Program and has led the development of several
M&S standards (IEEE 1516.2, IEEE 1516.3, IEEE 1730). Mr. Lutz also
serves as the Chair of the Simulation Interoperability Standards Organization (SISO) Board of Directors, serves on the Tutorial Board
and Fellows Committee at the Interservice/Industry Training, Simulation and Education Conference (I/ITSEC), and is a guest lecturer on M&S-related topics in The Johns Hopkins University Whiting
School of Engineering.
18
TRACK 4: POLICY AND PROCESSES
0830 – 1000 • ROOM S320E
Fundamentals of Modeling
and Simulation (1507)
Enterprise Solutions for M&S
Asset Discovery and Reuse (1518)
TUTORIALS
TRACK 3: M&S: SIMPLE TO COMPLEX
0830 – 1000 • ROOM S320GH
This tutorial has been designed by a team of subject matter
experts to prepare attendees to understand the scope of I/ITSEC presentations and demonstrations. It provides definitions
of widely-used technical terms, while explaining the range
and types of models and simulations that are commonly applied in the M&S domain. The tutorial reviews major simulation architectures (HLA, TENA, DIS), the basics of instructional design, a description of the major standards and best
practices available for use across the M&S problem space, and
a brief presentation of resources that can provide further information. The tutorial introduces topics that are examined
more extensively in other tutorials.
The tutorial is designed to be technically focused and is
not intended to overview management or governance of M&S
within the US DoD.
Reuse of M&S assets including models, simulations, data, architectures, designs, and requirements (collectively referred
to as M&S assets hereafter) is critical to providing cost-effective M&S solutions. The first step in reuse is to understand
what assets exist, and then to have mechanisms to access
them. However, most M&S practitioners and managers are
largely unaware of the wealth of existing M&S assets or where
to find them. Significant advances have been made recently to
more effectively support discovery and reuse of assets across
the M&S enterprise, both within the U.S. DoD and internationally. Following sound industry practices and DoD guidance, we have migrated to a set of federated discovery and
access mechanisms. Tutorial attendees will learn what types
of M&S assets can be discovered, how and where to search
for them, and how to access them. Attendees will also learn
how they can make their own assets available to others for
reuse. The tutorial will begin with the motivation and busiPresenters
JAMES E. COOLAHAN, Ph.D., is the Chief Technology Officer of ness case for M&S asset reuse, and then through descriptions
Coolahan Associates, LLC, having retired from full-time employment and demonstrations, show how M&S assets can be discovered
at the Johns Hopkins University Applied Physics Laboratory (JHU/ and accessed.
APL) in December 2012 after 40 years of service. He currently chairs
the M&S Committee of the Systems Engineering Division of the National Defense Industrial Association, and teaches courses in M&S
for Systems Engineering in the JHU Engineering for Professionals
M.S. program. He holds B.S. and M.S. degrees in aerospace engineering from the University of Notre Dame and the Catholic University
of America, respectively, and M.S. and Ph.D. degrees in computer
science from JHU and the University of Maryland, respectively.
Presenters
ROY SCRUDDER is the Program Manager for the M&S Information
Management Group at the Applied Research Laboratories, The University of Texas at Austin (ARL:UT). Mr. Scrudder served as the Associate Director for Data at the DoD M&S Coordination Office (DMSCO) from 2005-2009. He has over 30 years’ experience in information
systems analysis and development, recently concentrating in M&S
information management. His experiences are data management
S. K. NUMRICH, Ph.D., CMSP, holds an AB, MA and Ph.D. in physand data engineering, specializing in metadata. He has contributed
ics and worked as a research physicist at the Naval Research Labto the Enterprise Data Services, Joint Strike Fighter Product Developoratory plying her trade in a variety of fields including underwater
ment Metadata Specification, M&S Community of Interest Discovery
sound in the Arctic (yes, aboard ship), fluid-structure interactions,
Metadata Specification, and DoD M&S Resource Repository Board of
parallel processing, modeling and simulation and virtual reality.
Directors. Mr. Scrudder holds a B.S. in Applied Mathematics from
Upon leaving government service, Dr. Numrich has joined IDA.
the University of Tennessee.
ROBERT RICHBOURG, Ph.D., is a member of the Research Staff
at the Institute for Defense Analyses. He is a retired Army officer
who holds a B.S. in Mathematics, and M.S. and Ph.D. in Computer
Science. In his last active duty assignment, he was an Academy Professor and Director of the Artificial Intelligence Center at the United
States Military Academy, West Point. He is the 2015 Chair of the I/
ITSEC Tutorial Board.
FRANK MULLEN is a senior scientist at SimVentions and was previously an associate director at the DoD M&S Coordination Office
(DMSCO), helping develop the Defense M&S Catalog. He gained over
twenty‐five years of defense‐related engineering and management
experience at the Charles Stark Draper Laboratory, specializing in
integrated circuits and microwave devices, and contributed to development of tactical and strategic guidance systems. On active and
reserve duty, he served afloat and ashore, deployed to the Mideast,
and spent four years on the Pentagon’s Joint Staff. He retired as a
captain after thirty years of service. Mr. Mullen is a graduate of the
US Coast Guard Academy, the U.S. Naval War College, Defense Acquisition University, and the California Institute of Technology. He
holds degrees in electrical engineering and physics.
19
TUTORIALS
TRACK 5: INTERACTING WITH USERS
0830 – 1000 • ROOM S320B
TRACK 6: VISUAL ARTS
0830 – 1000 • ROOM S320F
Natural Language Interaction in Simulations:
Moving Beyond Pre-Coded Scripts (1536) %
Breathing New Life Into Old Assets (1539)
Natural language interactions can increase engagement in
online training, simulation, and performance support environments. Avatars that give prompts, hints, and feedback can
be used with great effect as non-player characters (NPC’s)
in games and simulations or as tutors in intelligent tutoring
systems. In these applications, the avatars have traditionally
interacted with and responded to learners using pre-encoded
dialogue trees or templates. This results in unnatural sounding dialogue and limits the ability of the avatar to respond
when the learner wanders off topic or changes the subject.
Today, however, it is possible to build dialogue engines that
process spoken input and respond in surprisingly natural and
intelligent ways. This tutorial is an introduction to the current
capabilities of dialogue engines, their implementation, and
their use in training environments. Topics include the AI behind chat bots, selecting a dialogue engine, building a library
of domain knowledge, training-the-trainers for virtual tutors,
matching technology to training goals, and incorporating chat
into training and simulation. This tutorial provides insights
into how these technologies work and demonstrates ways to
leverage open source products and native browser functionality to start making your training talk with learners.
Computer graphics hardware has come a long way. Modern
games provide unprecedented levels of visual quality that often surpasses the quality of existing visual simulation systems. Trying to bring modern effects into older systems faces
several challenges: a need for new hardware, new software
and new assets. The last one is especially problematic, as
in many cases it represents the bulk of the required effort.
For modernizing an existing visual environment in the face
of budget constraints it can be impossible to argue for a full
set of new assets. The goal of this tutorial is to introduce
methods for developers and maintainers of existing visual
environments to add modern graphics effects without needing to fully rebuild existing assets. The core element is the
ability to freely program the current graphics hardware using
so-called shaders. Effects like per-pixel lighting, bump and
normal maps, forward and deferred shading as well as shadows and image effects like motion blur, screen-space ambient
occlusion for soft shadows and others can be added with no
or limited modeling effort. The Unity development framework
will be used to demonstrate fast experimentation and present
outlines on how to integrate the described effects into custom
software using libraries like OpenGL.
Presenters
ROBBY ROBSON, Ph.D., began developing web-based learning content and learning management systems in 1995 and has led multiple
Department of Defense and National Science Foundation projects
that have applied emerging technologies to learning, education,
and training. He has published extensively in areas ranging from
mathematics to web-based learning and standards development. Dr.
Robson chaired the IEEE Learning Technology Standards Committee
from 2000 to 2008 and co-founded Eduworks Corporation in 2001
where he has guided research, services, and product development
and serves as CEO. He holds a doctorate in mathematics from Stanford University and has held leadership posts in both academia and
industry.
Presenters
DIRK REINERS, Ph.D., is an Associate Professor in the Department
of Information Science at the University of Arkansas at Little Rock.
His research focuses on interactive 3D graphics and Virtual Reality applications development and making high-quality, high-speed
graphics usable. He has an MS and a Ph.D. in Computer Science
from the Technical University Darmstadt and has worked for more
than 10 years in applied research at Fraunhofer IGD before joining
academia at Iowa State University, The University of Louisiana at
Lafayette and now at UALR. He is the project lead for the OpenSG
Open Source scenegraph project.
CARSTEN NEUMANN is a Senior Research Scientist in computer
graphics and visualization at the Emerging Analytics Center at the
ELAINE KELSEY is a software engineer at Eduworks focusing on de- University of Arkansas at Little Rock. He has been working on a varivelopment of conversational dialogue agents for intelligent tutoring ety of applications of Virtual Reality for the last 10 years and is a lead
systems, natural language processing and semantic analysis. She has developer of the OpenSG Open Source scenegraph project. He has an
developed multi-cultural interfaces for global software deployments; M.S. in Mathematics from the Technical University Darmstadt.
designed algorithms for improving cost pool allocation in financial
planning and analysis software for global nonprofit organizations;
and worked on the integration of idiomatic and dialectal forms in
natural language translation. Elaine speaks eight languages and has
a B.S. in Computer Science from Oregon State University, a B.A. in
Scandinavian Languages and B.S. in Cell and Molecular Biology from
the University of Washington, and a MPH from Tulane University.
20
1245 – 1415 • ROOM S320A
Building Intelligent Tutoring Systems
for Teams: What Matters (1531)
Elevate your Instruction: Practical Tactics
to Maximize Military Learning (1516) 
The military has long recognized having a successful team
requires more than assembling individuals who are highly
competent in their jobs. Successful teams require both taskwork and teamwork skills in order to effectively coordinate in
an adaptive manner (Salas et al., 1992). This awareness has
caused the military to invest heavily in team training. Over
the last thirty years much has been learned. Recently, there
has been a renewed interest in the use/creation of intelligent
tutoring systems (ITS) for teams which can be adapted to
multiple training situations. From a technological standpoint
much can be leveraged from the work on intelligent tutors
for individuals; however, teams are more than the individual
sum of their parts (i.e., members). The purpose of this tutorial will be to highlight the process of working through the
complexity involved and offering scientifically-grounded best
practices which illustrate a way ahead in building an ITS for
teams. Best practices and principles will be extracted from
the scientific literature and the authors’ combined experience
(over 25 years) in conducting research on military teams. Discussions will cover the lifecycle of development – e.g., learning goals/content, scenario development, and measurement/
evaluation.
The impact of quality instructors (teachers, trainers, coaches,
and educators) can’t be understated. Research findings reveal
that “…the most important factor affecting student learning
is the teacher” (Sanders, Wright, & Horn, 1997, p. 61), and
typical results show that a one standard deviation increase
in teacher quality raises student outcomes by approximately
.20-.24 standard deviations (Rockoff, 2004). When discussing instructional technologies, the message is much the same:
Their delivery and interaction methods have a profound effect
on learner outcomes.
This tutorial summarizes the existing research on instructor effectiveness, and it translates those concepts into a military setting. Attendees will leave with a clear argument for
defending the value and return on investment of instructor
development, and they’ll gain a practical, actionable set of
instructional strategies, tactics, and assessment methods to
begin using immediately with human teachers and trainers
as well as instructional technologies and training simulations.
Presenters
C. SHAWN BURKE, Ph.D., is an Associate Professor (Research) at
the Institute for Simulation and Training of the University of Central Florida. Her expertise includes teams and their leadership, team
adaptability, team training, measurement, evaluation, and team effectiveness.
ROBERT A. SOTTILARE, Ph.D., leads adaptive training research
within the US Army Research Laboratory focusing on automated authoring, automated instructional management, and evaluation tools
and methods for intelligent tutoring systems.
TUTORIALS
TRACK 7: NEW DIMENSIONS IN TRAINING
0830 – 1000 • ROOM S320D
Presenter
SAE SCHATZ, Ph.D., currently serves as the Director of the Advanced Distributed Learning (ADL) Initiative, a research and development unit overseen by the Office of the Assistant Secretary of
Defense (Readiness). Before joining ADL, she worked in industry, as
the chief scientist for MESH Solutions LLC, and before that in academia, as an Assistant Professor at the University of Central Florida.
During her time as a technical performer, Sae contributed to a variety of science and technology efforts for agencies such as the Office
of Naval Research, Army Research Lab, Marine Corps Training and
Education Command, and Special Operations Command, and she’s
earned accolades for this work. In 2010, for instance, Sae led the
team who received an NTSA Modeling & Simulation Award for Training, and she has received the I/ITSEC best paper award twice: first
in 2012 for her work on the Marine Corps’ Making Good Instructors
Great and again in 2014 for work on Joint Blended Learning.
EDUARDO SALAS, Ph.D., is a Professor of Psychology at Rice University. His expertise includes helping organizations on fostering
teamwork, the design and implementation of team straining strategies, facilitating training effectiveness, and the design of learning
and simulation-based environments.
JOAN JOHNSTON, Ph.D., has been a U.S. Military research psychologist for 25 years. Her current research focus is on training effectiveness
with an emphasis on training transfer. Dr. Johnston’s areas of expertise include training and decision support systems for tactical decision
making under stress, team performance and team training technologies, and embedded and distributed simulation-based training.
21
TUTORIALS
1245 – 1415 • ROOM S320C
1245 – 1415 • ROOM S320GH
TENA/JMETC:
An Innovative Architecture
for Testing and Training (1509)
Planning and Execution of a Large
Multi-Architecture Distributed
LVC Event (1508)
The Test and Training Enabling Architecture (TENA) and
the Joint Mission Environment Test Capability (JMETC) program provide an advanced set of interoperability software,
interfaces, and connectivity for use in joint distributed testing and training. This tutorial will provide information about
how TENA works and why it is important to the test and
training communities, with some comparison to other interoperability architectures. TENA provides testers and trainers software such as the TENA Middleware—a high-performance, real-time, low-latency communication infrastructure
that is used by training range instrumentation software and
tools during execution of a range training event. The standard TENA Object Models provide data definitions for common range entities and thus enables semantic interoperability
among training range applications. The TENA tools, utilities,
and gateways assist in creating and managing an integration
of range resources. The current version of the TENA Middleware, Release 6.0.4, is being used by the range community for
testing, training, evaluation, and feedback and is being used
in major exercises in the present.
JMETC has created a persistent test and evaluation capability throughout the US DoD, connecting many test ranges
together, and including a bridge to the Joint Training and
Experimentation Network (JTEN); a set of TENA-compliant
software middleware, interfaces, tools, and databases; and a
process for creating large distributed test events. The combination of TENA and JMETC gives testers and trainers unprecedented power to craft a joint distributed mission environment that forges the future for innovative testing and training.
Distributed simulation technologies have changed the way
the DoD does RD&E, training, analysis, and testing. These
technologies and associated standards have been in use for 20
years and have been documented in many forums. However
a critical element that has not been widely documented is
the processes and tools required to execute a large multi-architecture distributed event. There are organizations that do
these types of events very well, but the only way for the new
practitioner to lean these skills is to be a member of one of
these teams.
This tutorial provides a guide to the planning and execution
of a large multi-architecture distributed event. This guide will
include the steps to planning and executing an event including design of the simulation architecture, planning integration
spirals, scenario development and rehearsal, conduct of the
event, data collection and analysis. While the steps described
in this tutorial are applicable to all large distributed events,
special emphasis will be placed on multi-architecture based
events. An example will be provided showing how to select
architectures and object models. This tutorial is applicable to
anyone involved in the development a large test event. The
material will be applicable to simulation architects, analysts,
scenario developers, simulation users, and managers.
Presenter
EDWARD T. POWELL, Ph.D., is a lead architect for the Test and
Training Enabling Architecture. After receiving his Ph.D. in Astrophysics from Princeton University, he worked for the Lawrence Livermore National Laboratory performing simulation-based analysis.
He moved to SAIC (now Leidos) in 1994, and participated as lead architect in some of the most complex distributed simulation programs
in DoD, including the Joint Precision Strike Demonstration (JPSD),
the Synthetic Theater of War (STOW)and the Joint Simulation System (JSIMS). He then worked in the intelligence community for two
years on architectures for integrating large-scale diverse ISR systems.
He has been the lead architect for TENA for over twelve years, and
is currently working on expanding the applicability of TENA, and integrating TENA with broader DoD-wide Data Management systems.
Presenter
MICHAEL J. O’CONNOR, CMSP, is a Senior Program Manager at
Trideum Corporation. Mr. O’Connor has more than 25 years’ experience in Modeling and Simulation (M&S). He has been a key participant in the development of distributed modeling and simulation
standards, including IEEE 1278 and IEEE 1516. He has held many positions in the community, including Chairman of the SISO Standards
Activities Committee and Chairman of the SISO Executive Committee. He served as the chair of the I/ITSEC Simulation Subcommittee. Mr. O’Connor currently supports the technical integration of the
“Always On – On Demand” program. He has led the development of
multiple simulations using DIS, HLA, and TENA. Mr. O’Connor has
led the technical integration of several large multi-architecture distributed events including the Multi-Served Distributed Event (MSDE)
in 2005 that used DIS, HLA, and TENA. MSDE included over 20 sites
and included organizations from the Army, Navy, and Air Force. He
holds a bachelor’s degree in Computer Engineering from Auburn
University, and a master of science in Computer Science from the
University of Alabama in Huntsville.
22
1245 – 1415 • ROOM S320B
Simulation Conceptual Modeling
Theory and Application (1534)
Speech-based Interaction:
Myths, Challenges and Opportunities (1515) ñ
Simulation conceptual modeling is a critical step in simulation development frequently overlooked in the rush to
demonstrate program progress. A simulation conceptual
model is an abstraction from either the existing or a notional
physical world that serves as a frame of reference for further
simulation development by documenting simulation-independent views of important entities and their key actions and
interactions. A simulation conceptual model describes what
the simulation will represent, the assumptions limiting those
representations, and other capabilities needed to satisfy the
stakeholder’s requirements. It bridges between these requirements, and simulation design. This tutorial will present the
theory and application of simulation conceptual modeling
as documented during the research done by the NATO MSG
058 and SISO SCM SG/SSG/PDG. In addition, Use Cases that
have been drawn from previous conference presentations will
be presented to illustrate how conceptual modeling has been
performed. Additional work is necessary to mature the stateof-the-art of simulation conceptual modeling before a recommended practices guide could be standardized. This tutorial
has been created to continue the maturation of the simulation
conceptual modeling best practices.
Recent developments have enabled advanced interaction so
that users can more realistically interact with serious games
in virtual environments. Unfortunately, it is complex to allow
users to fully interact through speech, particularly in areas
where the task is unconstrained and performed under adverse
conditions. As such, speech has been often neglected as a
modality that can enhance the naturalness of interacting with
virtual training systems. Furthermore, user-based evaluations
of speech interfaces are intrinsically difficult. Recent research
indicates there are several interesting areas and approaches
that could improve the design and implementation of training
systems.
This tutorial will explain how Automatic Speech Recognition and Speech Synthesis work; the challenges in enabling
speech as a modality for hands-free interaction; some usability issues in speech-based interaction systems; opportunities
for researchers and developers to enhance system interactivity by enabling speech, and how to enable speech-based interaction within immersive, mixed-reality environments. The
tutorial is intended for developers interested in implementing
speech recognition in interactive applications, as well as for
researchers dedicated to developing methods and systems
that allow humans to naturally interact with technology.
Presenter
JAKE BORAH, CMSP, was a Senior Member of Technical Staff for
AEgis Technologies Group, Inc. He has been assigned as Project
Manager or Technical Lead on several projects that require a high
degree of modeling and simulation expertise and a capability to integrate leading edge technology into ongoing processes. His most
recent conceptual modeling has been for the Air Force Modeling and
Simulation Training Toolkit (AFMSTT). He has frequently supported
US and Canadian government sponsored military simulation projects
because of his mastery of the M&S technology, and expertise in High
Level Architecture federation development. He is a Charter Certified
Modeling and Simulation Professional (CMSP). He is a recognized
expert and a prominent member of the worldwide M&S community
as reflected by his contributions to the Simulation Interoperability
and Standards Organization (SISO) workshops and products. He has
taught M&S classes and given tutorials in academic, government and
industrial forums throughout the world from Asia to Europe during
the last 18 years. He graduated from the United States Air Force
Academy in 1974 and possesses a Master of Aeronautical Science
degree from Embry-Riddle Aeronautical University.
TUTORIALS
1245 – 1415 • ROOM S320E
Presenters
COSMIN MUNTEANU, Ph.D., is an Assistant Professor at the Institute for Communication, Culture, Information, and Technology
(University of Toronto at Mississauga). Until 2014 he was a Research
Officer with the National Research Council of Canada. His area of expertise is at the intersection of Automatic Speech Recognition (ASR)
and Human-Computer Interaction (HCI), having extensively studied
imperfect speech recognition systems, and designed and evaluated
systems that improve humans’ interaction with immersive technologies through speech and natural language (such as advanced
learning systems and mixed reality training simulators). His interests
include speech and natural language interaction for mobile devices,
mixed reality systems, learning technologies for marginalized users,
and assistive technologies. He has authored numerous publications
in HCI, ASR, and Computational Linguistics.
GERALD PENN, Ph.D., is a Professor of Computer Science at the
University of Toronto. His area of expertise is in the study of human
languages, both from a mathematical and computational perspective. Dr. Penn is one of the leading scholars in Computational Linguistics, with significant contributions to the formal study of natural
languages. His publications cover many areas, from Theoretical Linguistics, to Mathematics, and to Automatic Speech Recognition, as
well as Human-Computer Interaction.
23
TUTORIALS
1245 – 1415 • ROOM S320F
1245 – 1415 • ROOM S320D
Three-Dimensional Display Technologies
for Simulation and Training
Simulator Fidelity in Training:
Informed Decision-Making to Maximize
Effectiveness (1505)
(1529)
The human visual system naturally processes information in
three dimensions, yet the majority of simulation and training applications utilize two-dimensional display technology.
Three-dimensional displays introduce significant benefits to
human performance, including reduction in cognitive load,
improved spatial understanding, improved processing time,
greater knowledge gains, and improved student perceptions
on learning. These improvements exist in diverse subject areas, including medical diagnosis, navigation, imagery analysis, and geospatial visualization. Based on these proven
benefits, the use of 3D display technologies for training and
simulation has the potential for significant beneficial results.
Within this tutorial, the basics of 3D perception, including the
human visual system and depth cues, are explained. The presentation then covers current 3D display technologies, including the functionality of different displays and the depth cues
they present. The presenter will discuss the beneficial impacts
3D displays have on human performance. The tutorial will
cover areas of simulation and training suitable for 3D display
technology, and will discuss ongoing research in those areas.
This tutorial is designed for those interested in understanding more about 3D displays, their potential benefits, and application areas in training and simulation. Engineers, project
managers, researchers, and scientists should attend. No prior
exposure to 3D display technology is required.
Presenter
MATTHEW HACKETT is a research engineer for the Medical Simulation Research Branch of the Army Research Laboratory, Simulation
and Training Technology Center. He manages a variety of projects
including medical hologram, virtual patients, and medical serious
games. Mr. Hackett received his Bachelor of Science in Computer
Engineering from the University of Central Florida and his Masters
of Science in Biomedical Engineering from the University of Florida.
He is currently pursuing his Ph.D. in Modeling and Simulation at the
University of Central Florida.
The term Fidelity is formally defined as “the accuracy of the
representation when compared to the real world”. This notion typically demands that the customer and simulator (or
system) designer iterate to specify the required hardware and
software characteristics to meet the stated training objectives.
Fidelity, as it relates to cost, certainly speaks to why simulators have been vastly underutilized in civilian training. Often
times, the simulation Fidelity present is governed not by what
is required, but by the financial limitations of the training
organization in question. This necessitates that the customer
prioritize their requirements, which ultimately results in compromise and trade-offs.
Accordingly, the primary goal of this Tutorial is to introduce the notion of Fidelity in simulation-based training, and
explore how its determination requires a systematic process
of informed decision-making. We will summarize techniques
for establishing preferences and priorities based on Fidelity
needs, and offer guidelines for optimizing related trade-off decisions for training system acquisition. As the centerpiece of
this Tutorial, we will highlight three Case Studies (of advancing complexity), both to demonstrate the specific techniques
presented in this Tutorial, and to justify the critical need for
appropriately specified simulation technology in both civilian
and military training applications.
Presenter
KEVIN HULME, Ph.D., received his Ph.D. from the Department of
Mechanical and Aerospace Engineering at the University at Buffalo
in 2000, where his area of expertise was multidisciplinary analysis and optimization of complex systems. For the past decade, Dr.
Hulme has served as the technical lead of the Motion Simulation
Laboratory at the Center for Engineering Design and Applied Simulation at the University at Buffalo. He and his research team focus on
the custom design and development of ground vehicle simulations
for applications in: clinical research, education and training, and
next-generation transportation and safety studies.
24
1430 – 1600 • ROOM S320C
An Introduction to Cognitive Systems
for Modeling & Simulation (1511) 
DIS 101: The Basics
There are increasing requirements for automated reasoning
abilities across the broad spectrum of modeling and simulation, as well as in battlefield information and control systems.
Additionally, the cognitive capabilities that have been developed and tested in simulation are migrating to real-world systems. Cognitive systems represent a maturing computational
approach to intelligence that can provide robust, scalable,
and adaptive decision making. This tutorial provides an introduction to cognitive systems, concentrating on production
system computation and high-level design of human-like reasoning systems. We draw examples and comparisons from
existing cognitive systems, focusing on the tradeoffs between
cognitive and non-cognitive modeling approaches. The tutorial content does not require any specialized knowledge,
but some experience with software engineering or behavior
modeling can be helpful. Attendees will learn to recognize
problems that suggest cognitively based solutions, and they
will be better able to assess risks, costs, and benefits of different approaches. This tutorial is targeted toward developers
interested in cognitive approaches to software engineering, as
well as customers who have problems that may be amenable
to a cognitive approach.
Distributed Interactive Simulation (DIS) is one of the three
major modeling and simulation standards used in military
M&S. This tutorial will provide an overview of the problem
domain for modeling and simulation in virtual environments,
briefly discuss the standards used by modeling and simulation, and then provide an overview of the DIS protocol. The
messages that constitute DIS and the techniques used by DIS
to implement a virtual environment are discussed, as are simple examples for sending and receiving DIS messages.
Presenters
RANDOLPH M. JONES, Ph.D., Senior Artificial Intelligence Engineer at SoarTech, is a leading developer of knowledge-rich intelligent agent software. He has been principal investigator for a variety
of advanced R&D projects for ONR, ARI, DMSO, DARPA and other
agencies. He has previously held positions at Colby College, the University of Michigan, the University of Pittsburgh, and Carnegie Mellon University. His areas of research include computational models
of human learning and problem solving, executable psychological
models, and full-spectrum intelligent behavior models. He earned a
B.S. in Mathematics and Computer Science at UCLA, and M.S. and
Ph.D. degrees from the University of California, Irvine.
TUTORIALS
1430 – 1600 • ROOM S320A
(1538)
Presenters
DON MCGREGOR is a research associate at the Naval Postgraduate
School. He is the primary author of Open-DIS, an implementation of
the Distributed Interactive Simulation protocol in Java, C++, C#,
and Javascript. His research interests include web-based simulation
and scalable server side architectures.
DON BRUTZMAN, Ph.D., is Technical Director for 3D Visual Simulation and Networked Virtual Environments in the MOVES Institute.
As an Associate Professor at the Naval Postgraduate School in Monterey California, he is a member of two Academic Groups: Undersea
Warfare (UW) and Modeling, Virtual Environments and Simulation
(MOVES). He is an investigator in the NPS Center for Autonomous
Underwater Vehicle (AUV) Research. His research interests include
underwater robotics, real-time 3D computer graphics, artificial intelligence and high performance networking. He is a member of the
Institute of Electrical and Electronic Engineers (IEEE), the Association for Computing Machinery (ACM) Special Interest Group on
Graphics (SIGGRAPH) and the American Association for Artificial
Intelligence (AAAI).
DYLAN SCHMORROW, Ph.D., Chief Scientist at SoarTech, leads the
effort to build intelligent systems for defense, government, and commercial applications that emulate human decision making. He also
serves as a Potomac Institute for Policy Studies Senior Fellow, Editor
of the Theoretical Issues in Ergonomics Journal, and the Technical
Advisor for the Applied Human Factors and Ergonomics Conference
Series. He is a leading expert on national security research, technology, and policy related to information technology, medical research and human performance applications. Past service includes
OSD, DARPA, NAWC, NRL, ONR, NPS, and Executive Assistant to
the Chief of Naval Research. Dr. Schmorrow holds a Ph.D. in Experimental Psychology from Western Michigan University, as well as
M.S. degrees in Psychology and Philosophy. He retired from the U.S.
Navy as a Captain in 2013.
25
TUTORIALS
1430 – 1600 • ROOM S320GH
1430 – 1600 • ROOM S320E
Model Verification and
Validation Methods
(1506)
U.S. Export Controls 2015: Next Phase of
Export Control Reform for the International
Verification and Validation (V&V) are essential prerequisites
to the credible and reliable use of a model. But what are V&V,
and what is their purpose within a modeling and simulation
project? What types of potential errors can occur during V&V
and how can they be avoided? What methods are available to
perform verification and validation in a rigorous and effective
manner?
The tutorial is intended to answer these questions. It has
three distinct parts. The first part motivates the need for V&V,
provides definitions necessary to their understanding, and explains why all V&V methods can be understood as comparisons and how this informs their application. The second part
provides guidelines for selecting V&V methods for a particular
model, introduces a widely used categorization of V&V methods, defines four categories of V&V methods, and describes
two or more methods from each category. Example applications of the described methods are presented. In the third
part, longer case studies of V&V in practice are presented,
showing how V&V methods have been applied in actual modeling and simulation projects.
This tutorial will focus on the continuing changes in 2015
as a result of the ongoing Export Control Reform initiative
that has transformed the controls applicable to simulation
hardware, software and services. The tutorial will include a
focus on examples and practical application of the revised
regulations to simulation products and services. Revisions to
the International Traffic in Arms Regulations (U.S. Munitions
List Category IX) and the Export Administration Regulations
(ECCN 0A614 et al.) have dramatically altered the controls
on simulation products from software to services. Definitional changes are expected for key terms such as “defense services”, “technical data”, “public domain” and “export” and
will continue to change the rules on exporting and importing
simulation technology. Participants will understand the scope
of the U.S. export laws, continuing changes and reform, how
the U.S. Government applies them to the simulation industry, including controls on software, hardware, services and
activities at trade shows such as I/ITSEC, as well as strategies
for ensuring compliance in commercial, U.S. government and
foreign contracts.
Presenter
MIKEL D. PETTY, Ph.D., CMSP, is Director of the University of Alabama in Huntsville’s (UAH) Center for Modeling, Simulation, and
Analysis and an Associate Professor of Computer Science. Prior to
joining UAH, he was Chief Scientist at Old Dominion University’s
Virginia Modeling, Analysis, and Simulation Center and Assistant
Director at the University of Central Florida’s Institute for Simulation and Training. He received a Ph.D. in Computer Science from
the University of Central Florida in 1997. Dr. Petty has worked in
modeling and simulation research and education since 1990 in areas that include verification and validation methods, simulation interoperability and composability, simulation software frameworks,
and human behavior modeling. He has published over 190 research
papers and has been awarded over $16 million in research funding.
He served on a National Research Council committee on modeling
and simulation, is a Certified Modeling and Simulation Professional, and is an editor of the journal SIMULATION. He has served as
dissertation advisor to five graduated Ph.D. students, including the
first two students to complete a Ph.D. in Modeling and Simulation at
Old Dominion University and the first student to complete a Ph.D. in
Modeling and Simulation at UAH.
Presenters
JEREMY HUFFMAN is a founding member and partner of Huffman
Riley PLLC. Mr. Huffman concentrates his practice advising U.S. and
foreign clients concerning the U.S. export control laws, including the
International Traffic in Arms Regulations (“ITAR”); Export Administration Regulations (“EAR”); and the Office of Foreign Assets Control
(“OFAC”) regulations.
DARREN RILEY is a founding member and partner of Huffman Riley
PLLC. Mr. Riley has extensive experience advising clients on matters
involving U.S. export controls and government contracts issues. He
counsels clients on issues related to the International Traffic in Arms
Regulations, Export Administration Regulations, the regulations of
the Office of Foreign Assets Control and the Foreign Corrupt Practices Act.
26
1430 – 1600 • ROOM S320F
Using HTML5 to Develop IMI,
Simulations, and Games (1521) %
When Simple Becomes Complicated: Using
Machinima for Training and Education (1533) %
TUTORIALS
1430 – 1600 • ROOM S320B
HTML5 turns the web into an interactive multimedia platform
that does not require plug-ins and can run on any device.
Because of this, it has become the standard for developing
web sites, web applications, mobile applications, eBooks
and, most importantly for the I/ITSEC community, interactive
multimedia instruction (IMI), simulations, and games. This
tutorial covers the basics of HTML5, presenting the “why,”
the “how,” the “so what?” and the “wow!”. This tutorial addresses the big picture issues that are important to managers
and procurement personnel as well as the details of how to
code in HTML5. Attendees will learn how (and why) to use
HTML5 to render cross-platform multimedia, implement interactivity, optimize the user experience, add semantic tags,
and create cloud-based simulations and games. All of these
topics will be illustrated with live demonstrations, and the
tutorial will provide pointers to HTML5 authoring tools and
other resources. The tutorial will also cover recent developments in HTML5.
The use of virtual technologies to visualize specific events
on battlefields, both real and imagined, has become incredibly important in training the leaders of today and tomorrow.
Reviewing an event visualization for specific lessons learned,
both positive and negative, is far more effective than reading
a 30 page after action review. With the advent of new abilities
of video game engines, it is now possible to quickly create
these event visualizations from a desktop computer with minimal training. However, as the events to be replicated deviate further from squad-based, traditional combat, it becomes
increasingly difficult to utilize a game engine designed with
artificial intelligence programmed to avoid certain conflicts,
collisions, and movements.
This tutorial will provide an overview of the video creation
process utilized by the Training Brain Operation Center’s
(TBOC) SIMS organization, with a focus on specific problems
encountered by scenario developers utilizing a video game
engine as the visualization tool for event recreation.
This tutorial is intended for anyone interested in how a
visualization request is broken down by the TBOC SIMS team
Presenters
ROBBY ROBSON, Ph.D., began developing web-based learning con- into component parts, evaluated for recreation, and then
tent and learning management systems in 1995 and has led multiple filmed for use as a military training tool.
Department of Defense and National Science Foundation projects
that have applied emerging technologies to learning, education,
and training. He has published extensively in areas ranging from
mathematics to web-based learning and standards development. Dr.
Robson chaired the IEEE Learning Technology Standards Committee
from 2000 to 2008 and co-founded Eduworks Corporation in 2001
where he has guided research, services, and product development
and serves as CEO. He holds a doctorate in mathematics from Stanford University and has held leadership posts in both academia and
industry.
SHERRIE VIEIRA is a media specialist who served honorably in the
U.S. Army for four years, receiving a Joint Services Achievement
medal or her work at the Medina Regional SIGINT Operations Center
in San Antonio, Texas. She has a BS in digital arts from the University of Oregon and has worked at Hewlett-Packard and with the State
of Oregon as well as at Eduworks. Sherrie has develop a variety of
HTML5 web applications and web sites and has worked on projects
converting older interactive web formats to HTML5.
Presenter
BRIAN HALL is a former Artilleryman who spent a year in downtown Baghdad dodging indirect fire and IEDs while collecting intelligence and escorting industry officials to various Ministries. After his
return and subsequent departure from the US Army, his recent experience in Iraq and computer science degree was leveraged to complement the scenario development team at TBOC SIMS. In four years
there, he has developed over 60 military training videos, consisting
of more than five hours of content. He also conceived, designed, and
developed a virtual land navigation trainer, of which there are now
12 different terrain variants.
27
TUTORIALS
1430 – 1600 • ROOM S320D
Team and Collective Training
Needs Analysis (1537) ñ
An organisation’s capability is delivered, almost without exception, by the team or set of teams that make up its structure. Effective team training is a significant precursor to the
delivery of team performance at the level required for organisational success. The complexity of team training goes far
beyond that of individual training as a consequence of both
the complexity of the overall team task and its associated task
environment, and the training delivery capability required to
implement such training. Consequently, the front end analysis
techniques required to identify team training requirements,
specify training solutions and evaluate training options have
to address these complexities. Team and Collective Training
Needs Analysis (TCTNA) is a methodology that has been developed for the UK MOD specifically to address this front end
analysis challenge.
The purpose of this tutorial is to provide an understanding
of how analysis and design concepts familiar from Instructional Systems Design have been extended within TCTNA to
address the complexities of team and collective training, and
to demonstrate its application by means of a Maritime Force
Protection Case Study. The iterative application of TCTNA to
support key stages in the acquisition process will also be discussed.
This tutorial is aimed at anyone with an interest in specifying team and collective training requirements and identifying
viable training solutions.
Presenters
JOHN HUDDLESTONE, Ph.D., is a Senior Research Fellow in the
Human Systems Integration Group within the Engineering and Computing Faculty at Coventry University in England. A co-author of
the Team and Collective Training Needs Analysis Methodology, his
research interests include team training, human factors methods and
aviation human factors. Current research projects include the human factors of future flight deck technologies and single pilot operations, and the team and collective training implications of future
maritime unmanned systems concepts. He holds a Ph.D. in applied
psychology from Cranfield University, a Master’s degree in Computing Science from Imperial College, London and BS in Education from
Notting Trent University.
JONATHAN PIKE, is a freelance learning and development consultant currently living in Perth, Western Australia. Since 2005, while
working at Human Factors departments of Cranfield University and
Coventry University, he has conducted research for the UK MOD under the auspices of the Human Factors Integration Defence Technology Centre and Defence Human Capability Science and Technology
Centre. A visiting researcher at Coventry University, he holds a B.S.
in Biology from University College London and an M.S. in Applied
Computing Technology from Middlesex University.
28
Special Events
MONDAY, 30 NOVEMBER
1030 - 1200 • ROOM S310BCD
SE1
Select Members of
Congress will participate
in this I/ITSEC Event
S I G N AT U R E E V E N T
Congressional
Modeling and
Simulation Event
This special event invites everyone attending the conference or exposition to hear from the
training and simulation leaders in congress. It is also a great opportunity for you to interact
with Congressional Members on issues of importance to you or your company and to impress
upon them your priorities. With defense budgets constrained, this forum provides you an
opportunity to advocate for value of training and simulation in support of national security.
Attendees will hear how Congressional Members see the situation in Washington and have
the opportunity to make their case for timely investments in Modeling and Simulation. With
every budget dollar being scrutinized, strong advocacy for training and readiness has never
been more important. This event is always standing room only, so get there early.
The following members of Congress have been invited to address the M&S Community at I/ITSEC:
Robert Aderholt*
Steve Cohen*
Virginia Foxx*
Jeff Miller*
Jeff Sessions
Alabama 4th District
Tennessee 9th District
Florida 1st District
Alabama
Gus Bilirakis*
Mike Conaway*
North Carolina 5th
District
Bill Nelson
Richard Shelby
Florida 12th District
Texas 11th District
Phil Gingrey*
Florida
Alabama
Diane Black*
John Cornyn
Georgia 11th District
Richard Nugent*
Niki Tsongas*
Tennessee 6th District
Texas
Richard Hanna*
Barbara Boxer
Ander Crenshaw*
Scott Peters*
Massachusetts 3rd
District
California
New York 22nd
District
Jim Bridenstine*
Ted Cruz
California 52nd
District
Tim Walz*
Duncan Hunter
Minnesota 1st District
Oklahoma 1st District
Texas
California 50th District
Bill Posey*
Mark Warner
Mo Brooks*
Susan Davis*
Tim Kaine
Virginia
Alabama 5th District
California 53rd District
Virginia
Tom Rooney*
Elizabeth Warren
Corrine Brown
Blake Farenthold
Doug Lamborn*
Massachusetts
Texas, 7th District
Colorado 5th District
Marco Rubio
Joe Wilson*
Vern Buchanan*
Dianne Feinstein
Ed Markey
Florida
California
Massachusetts
Ken Calvert*
J. Randy Forbes*
John Mica
C.A. Dutch
Ruppersberger*
South Carolina 2nd
District
California 42nd
District
Virginia 4th District
Chair, M&S Caucus
Maryland 2nd District
Virginia 1st District
Candice Miller*
Bobby Scott*
Michigan 10th District
Virginia 3rd District
*denotes members of the
Congressional M&S Caucus
John Carter*
Robert Wittman*
Texas 31st District
Congressmen Scott and Mica address the audience and visit the Exhibit Floor during I/ITSEC 2014.
29
Small Business and the Government
MONDAY, 30 NOVEMBER
1530 - 1700
WARFIGHTERS CORNER
(BOOTH 2280)
SE2
DoD Small
Business Programs
Moderator
Kenyata L. Wesley, SES
Acting Director, DOD Office
of Small Business Programs
Office of Under Secretary
of Defense, Acquisition,
Technology and Logistics
30
Mr. Wesley, SES
M
r. Wesley’s duties encompass multiple functional areas within the Office Under Secretary
of Defense (Acquisition, Technology and Logistics), including Policy, Goaling and Portfolio Management; Comprehensive Subcontracting; SBIR/STTR; Commercialization; Technology and Innovation; and Acquisition Strategies.
This Special Event will enlighten listeners and provide a distinct perspective on the importance of a robust and technologically agile Small Business industrial base, the Department of
Defense Mentor Protégé Program, Small Business Innovation Research (SBIR) / Small Business Technology Transfer (STTR), Commercialization, streamlining contracting processes,
and Better Buying Power 3.0 initiatives as they relate to how the Department will continue
“Forging the Future through Innovation”.
Following Mr. Wesley’s presentation, the audience will be able to engage in a question and
answer segment covering all small business topics of concern. The segment will also focus
on the significance of developing highly skilled small business professionals empowered to
contract with exceedingly capable small businesses that provide a good value to the taxpayer
while also meeting warfighter requirements.
TUESDAY, 1 DECEMBER
1030 - 1200
HYATT REGENCY
WINDERMERE BALLROOM
SE3
General/Flag
Officer Panel
RADM Robb, USN (Ret.)
Mr. DiGiovanni, SES
VADM Gonzalez-Huix, ESP N
VADM Moran, USN
LTG Williamson, USA
Maj Gen Post, USAF
MajGen Lukeman, USMC
Moderator
Rear Admiral James A. Robb,
USN (Ret.)
President, National Training
and Simulation Association
Panelists
Defense (Readiness)
Vice Admiral Javier
Gonzalez-Huix, ESP N
Deputy Chief of Staff Joint
Force Trainer, NATO HQ
SACT
Vice Admiral Bill Moran, USN
Chief of Naval Personnel
Lieutenant General Michael E.
Williamson, USA
G
lobal forces are squeezed between declining budgets and expanding threats. The U.S.
Army and Marine Corps have posted plans to significantly reorganize and downsize.
Meanwhile, terrorist threats continue to spread across the globe. Nations are experiencing
increasing pressure from immigration issues on their borders and a wave of cyber attacks
from the net. Our Senior Officer panel will be challenged to address current and future environments and give perspective on how we “forge the future through innovation”. This year’s
panel will include senior representatives from all the U.S. Military Services and NATO. Following opening remarks, the audience will be able to interact with the panel through written
questions. Don’t miss the opportunity to hear from national leaders on the way ahead.
Military Deputy/Director,
Army Acquisition Corps,
Office of the Assistant
Secretary of the Army
(Acquisition, Logistics and
Technology)
Major General James N. Post
III, USAF
Director, Current Operations,
Headquarters Air Force
Major General James W.
Lukeman, USMC
Commanding General,
Training and Education
Command
Panelists address Global Defense issues during the 2014 General/Flag Officer Panel.
31
Training Innovation:
Key to Maritime Strategy
TUESDAY, 1 DECEMBER
1400 - 1530 • ROOM S330AB
SE4
Forging the Future
of Navy Training
Moderator
Vice Admiral Bill Moran, USN
Chief of Naval Personnel
Panelists
VADM Moran, USN
Vice Admiral Paul A. Grosklags,
USN
Commander, Naval Air
Systems Command
Acquisition for Training
Rear Admiral Michael E. White,
USN
Naval Education and
Training Command
Future of Naval Training
Rear Admiral Mathias W. Winter,
USN
Chief of Naval Research/
Director, Innovation
Technology Requirements,
and Test & Evaluation
The Role of Research and
Technology in Training and
Simulation
VADM Grosklags, USN
RADM White, USN
RADM Winter, USN
T
his year’s conference theme, “Forging the Future through Innovation” focuses on the ever
changing landscape, and how innovation continues to be one of the key factors in our
future.
The United States Maritime Strategy calls for a force that is forward, engaged and ready.
In times of limited budgets and force size, it’s the readiness of our personnel that will provide
our continued warfighting advantage. Training sits at the very core of that readiness, and
remains one of the greatest tools available to ensure our Sailors remain our greatest asymmetrical advantage.
In ensuring our future force remains flexible, agile and ready, our Navy will use innovative
training methods to train them more efficiently and to ever-higher levels of proficiency. The
maritime strategy calls for the sea services to “create a true learning competency that unites
our acquisition, requirements, and programming efforts to deliver the latest in technology and
design, resulting in realistic simulation and live, virtual, and constructive scenarios before our
people deploy…” It is understood that high-quality training is an investment in our people.
We capitalize on this investment in training by making affordability a priority throughout the
training system cycle of research, development, acquisition and sustainment.
The sea services of the United States will remain critical to securing the nation’s national
security objectives, and well-trained forces are critical to our Navy’s ability to meet its mission. This panel of senior Navy leaders will provide insight into how training our Sailors fits
into our national maritime strategy, now and in the future, from an acquisition, research and
technology and mission readiness perspective.
Session Chair:
Rob Matthews
NAWCTSD
32
WEDNESDAY, 2 DECEMBER
0830 - 1000 • ROOM S310BCD
SE5
Forging the Future
of Marine Corps
Training
Training Innovation:
Preparing for the Future Naval Campaign
Session Keynote
General Robert B. Neller, USMC
37th Commandant of the
Marine Corps
Gen Neller, USMC
Moderator
Rear Admiral James A. Robb,
USN (Ret.)
President, National Training
and Simulation Association
Panelists
Major General James W.
Lukeman, USMC
Commanding General,
Training and Education
Command
Brigadier General Julian D.
Alford, USMC
Commanding General,
Marine Corps Warfighting
Laboratory
Brigadier General Joseph F.
Shrader, USMC
Commander, Marine Corps
Systems Command
Brigadier General Raymond R.
Descheneaux, USMC
Assistant Deputy
Commandant for Aviation
(Mobilization)
RADM Robb, USN (Ret.)
MajGen Lukeman, USMC
BGen Alford, USMC
BGen Shrader, USMC
BGen Descheneaux, USMC
T
his year’s conference theme, “Forging the Future through Innovation” focuses on the ever
changing landscape, and how innovation continues to be one of the key factors in our
future.
‘Expeditionary Force XXI’ and ‘A Cooperative Strategy for 21st Century Seapower’ calls for
a force that is forward, engaged and ready. In times of limited budgets and force size, it’s the
readiness of our personnel that will provide our continued force dominance. Training sits at
the very core of that readiness, and remains one of the greatest tools available to ensure our
Marines remain our greatest asymmetrical advantage.
In ensuring our future force remains expeditionary, agile and ready, our Marine Corps will
use innovative training methods to train them more efficiently and to ever-higher levels of
proficiency. The Cooperative Strategy for 21st Century Seapower calls for the sea services to
“create a true learning competency that unites our acquisition, requirements, and programming efforts to deliver the latest in technology and design, resulting in realistic simulation
and live, virtual, and constructive scenarios before our people deploy…” It is understood
that high-quality training is an investment in our people. We capitalize on this investment in
training by making affordability a priority throughout the training system cycle of research,
development, acquisition and sustainment.
The sea services of the United States will remain critical to securing the nation’s national
security objectives, and well-trained forces are critical to our Marine Corps ability to meet its
mission.
This panel of senior Marine Corps leaders will provide insight into how training our Marines fits into our national security strategy, now and in the future, from an acquisition, research and technology and mission readiness perspective.
Session Chair:
Martin Bushika, Director, Strategic
Business Operations, PM TRASYS
MARCORSYSCOM
33
Fostering the Cyber Mind
1030 - 1200 • ROOM S330AB
SE6
Training the Cyber
Warrior: A LearnerCentric Model
Moderator
Frank C. DiGiovanni
Defense (Readiness)
Panelists
John Rigney
Co-Founder, Chief
Technology Officer, Point3
Security, Inc.
Martin C. Carlisle, Ph.D.
Professor of Computer
Science, Director of the
Academy Center for
Cyberspace Research,
and Coach of the Cyber
Competition Team,
U.S. Air Force Academy
Jeff Moss
Founder of Black Hat and
DEF CON Conferences
Brian Markus
Chief Executive Officer and
co‐Founder of Aries Security
T
he timing for this special event could be no more opportune. Although cyber remains a
high priority for the DoD, our training capability for cyber is nascent and our methods
and tools to train this force are still emerging. This world-class panel of cyber learning experts
will share lessons from their cyber training experiences and assess the current cyber training
landscape. Together, they will address critical cyber readiness issues: How can we train cyber
in the right way, and what role can DoD play in taking cyber training to the next level, in
partnership with academia and industry?
Session Chair:
Damon Regan, Ph.D.
The Tolliver Group
34
Lightning II, plugged in to LVC
1400 - 1530 • ROOM S320GH
SE7
F-35 Lightning II –
Live, Virtual,
Constructive (LVC)
Moderator
Lieutenant General Christopher
C. Bogdan, USAF
Program Executive Officer,
Lightning II Joint Program
Office
Panelists
Rear Admiral Michael C.
Manazir, USN
Director, Air Warfare
(OPNAV N98)
Brigadier General Raymond R.
Descheneaux, USMC
Assistant Deputy
Commandant for Aviation
(Mobilization)
Major General James. N. Post
III, USAF
Director, Current Operations,
Headquarters Air Force
S
ervice leaders will discuss the future of training for the F-35 including how LVC will be
leveraged for increased proficiency of the aircrew. The Marine Corps, Navy and Air Force
share a common vision and goal for LVC with respect to training on the F-35, where in the
end state there will be no discernable differences between live and constructive adversaries
beyond visual range for the F-35 pilots. Panel members will discuss the future of joint virtual
and constructive distributed training from their Service perspectives.
Key near-term LVC objectives to be discussed include:
• F-35 integration with distributed training networks
• Future LVC systems integration of USMC-USN-USAF fourth and fifth generation aircraft
• Challenges and benefits of incorporating live training into joint, virtual and constructive
training networks.
While the path to achieve the goal is not pre-determined, integration of efforts and collaboration on initiatives are paramount to achieving LVC success for not only the F-35 but also
for legacy and future advanced weapons platforms.
Session Chair:
Lewis Harris
Booz|Allen|Hamilton
35
1600 - 1730 • ROOM S330AB
SE8
Attendees at this event may earn CEU credits.
I/ITSEC Fellows
I/ITSEC 2015 Fellow
Duncan (Duke) Miller, Ph.D.
I/ITSEC Fellows is an
annual series of presentations by technical
leaders responsible for
the seminal contributions
that have fundamentally
shaped the simulation
and training capabilities
being delivered today.
The paper authored by
Duncan (Duke) Miller,
Ph.D. may be found on
the 2015 Proceedings CD
provided to conference
attendees. The abstract of
his paper is also included
in this Program Guide as
part of the 2015 I/ITSEC
Abstract Section.
D
r. Duncan (Duke) Miller has been a key figure in the development of distributed simulation for 30 years. In 1983, he formed and led the development of the SIMNET system and
protocols. He chaired the Distributed Interactive Simulation (DIS) Technical Committee that
developed the DIS Standards, and served on the government/FFRDC team that developed the
High Level Architecture (HLA). He was a founding member of the Simulation Interoperability
Standards Organization (SISO), where he served as Chair of SISO’s Board of Directors, Chair
of SISO’s Conference Committee, and as a member of SISO’s Executive Committee. From
2001-2012, he was SISO’s Executive Director.
In this presentation, Dr. Miller provides a unique perspective on how distributed simulation was conceived and developed, including major milestones, tests, and demonstrations. He
offers anecdotes and insights regarding key events and individuals, as well as comments on
subsequent developments.
Key quotes from “SIMNET and Beyond:
A History of the Development of Distributed Simulation”:
“The core concept of SIMNET was the networking of multiple simulators, with each simulator
providing its own controls, displays, and computational resources. No central control system
scheduled events or resolved interactions among the simulation nodes. Instead, each node was
autonomous, maintaining authoritative status for one simulated entity (e.g., a tank, helicopter, or missile system) and transmitting messages about the state and actions of its simulated
entity to other nodes on a peer-to-peer basis. Each node was also responsible for receiving, interpreting, and responding to messages regarding events that might affect its own entity (e.g.,
a missile impact, an exploding mine, a collision, etc.) and for reporting any resulting changes
in its entity’s state (e.g., damaged, destroyed, or unaffected.)”
“In 1985, the Undersecretary of the Army agreed to redirect funding to DARPA to support SIMNET development, “Because if you can do what you’re telling me, it will change the way the
Army manages its weapon systems procurement.” And in many respects, it has. In 1991, a
study of various DARPA initiatives by the Potomac Institute for Policy Studies listed SIMNET as
one of six programs that have had the most profound effects on the DoD.”
“The SIMNET protocols were the foundation for the Distributed Interactive Simulation (DIS)
protocols, which were used for the Army’s Close Combat Tactical Trainer (CCTT), Aviation
Combined Arms Tactical Trainer (AVCATT), and subsequent procurements. DIS, in turn, was
a primary source for the High Level Architecture (HLA).”
Session Chair:
Robert Lutz
The Johns Hopkins University/
Applied Physics Laboratory
36
THURSDAY, 3 DECEMBER
0830 - 1000 • ROOM S330AB
SE9
Moderator
Garth Jensen
Director of Innovation, Naval
Surface Warfare Center
Panelists
David Earnest, Ph.D.
Professor of Political Science
and International Studies,
Old Dominion University
Kira Hutchinson
TRADOC Intelligence
Support Activity (TRISA)
Rebecca Law
Faculty Research Associate,
Naval Postgraduate School
WHAT IS A BLACK SWAN?
A Black Swan event refers to a high impact/low probability event. It is an occurrence that is
very difficult to predict, and carries a massive impact. Throughout history, these events, both
good and bad, have had a substantial impact on individual organizations, entire industries,
nations, or populations on a global scale. Black Swan is typically associated with the phrases
“That will never happen”, “That can never be done”, “No one would ever do that” or “What
if?” Because they will never happen we assume away their risk or potential value. Are they
truly unpredictable?
Black Swan –
Setting the Stage
Join Us for the I/ITSEC Black Swan KICKOFF!
WHY BLACK SWAN AT I/ITSEC?
We rarely take time to try to imagine the unimaginable. Why spend time studying something
that will never happen? Why should we spend time and resources attempting to conceptualize
and plan for something that will never happen? The answer is obvious — They do happen!
The significance of Black Swan to I/ITSEC is that you can’t study the effects of a Black Swan
in reality — you have to do it in Simulation and Simulation is what we do best.
WHAT WILL WE TALK ABOUT?
This inaugural Black Swan event is the start of a multi-year effort to highlight the value of
Modeling, Simulation and Analysis in the study of Black Swan scenarios. This event will bring
together a panel of globally recognized experts that will focus on the following topics:
• What is a Black Swan?
• What entities and organizations are thinking in the Black Swan space?
• How do these organizations frame their thinking and analysis of Black Swans?
• How do we prepare, organize, train and equip for Black Swan resiliency?
• How can Modeling and Simulation be used to analyze and prepare or create a Black
Swan?
• Can we develop complex adaptive models and simulations tools that will enable the
analysis?
RELATED EVENTS THIS WEEK
• Operation Blended Warrior, the LVC demo on the show floor will be set off by a Black Swan
scenario. Events all week — see OBW flyer.
• Technology Innovation Poster Session and Demonstration — highlighting M&S capabilities
and techniques that foster future research and development in the Black Swan domain Wednesday
• Training Adaptive Leaders for a Black Swan World — Thursday at 1030
Session Chair:
Anne Little, Ph.D.
Addx Corporation
37
Thriving in Uncertainty
1030 - 1200 • ROOM S330AB
SE10
Training Adaptive
Leaders for a Black
Swan World
Moderator
Defense (Readiness)
Panelists
LTG Robert B. Brown, USA
Commanding General, U.S.
Army Combined Arms
Center at Fort Leavenworth
Meir Finkel, Ph.D.
Commander of the Israeli
Defense Force (IDF) Dado
Center on Interdisciplinary
Military Studies, retired IDF
Brigadier General
MG Karl L. E. Engelbrektson
Head of Training and
Education, Swedish Armed
Forces
I
s the world actually getting more complex, or are we just getting bombarded with more
information? There is no simple answer. However, with the world’s increasing interconnectedness, we do know one thing. There is a growing awareness (in the military and elsewhere)
of a meteoric rise in potential “Black Swan” challenges and events that are difficult to predict.
How to prepare for this?
This Signature Event will focus on the critical human dimension — the importance of
training and developing truly adaptive leaders, able to challenge the status quo and embrace
innovation.
The event features a diverse panel of top experts. Expect a wide-open discussion that will
apply to the future of military training, business industry practices, academic research, and
learning how to thrive in ambiguous environments.
This panel builds upon the earlier Black Swan Signature Event as well as I/ITSEC’s new
Operation Blended Warrior floor activity.
Session Chair:
Randy Crowe, Ph.D.
Lockheed Martin Mission Systems
and Training
38
MONDAY, 30 NOVEMBER
1430 - 1600
DISASTER STRIKES
Addressing 23 Years of LVC Status Quo
FLOOR EVENT
TUESDAY, 1 DECEMBER
1530 - 1700
IMMEDIATE RESPONSE
1030 - 1200
COUNTERING EXPLOITATION
1600 - 1730
SECURING THE SKIES
1030 - 1200
OVERMATCH: FORWARD FROM
THE SEA
BOOTH 339 AND OTHER
FLOOR LOCATIONS
FL1
Operation Blended
Warrior (OBW)
I
n this Live-Virtual-Constructive (LVC) Special Event, industry members and government organizations are joining forces
to create an LVC environment for the purpose of: 1) showcasing their capabilities, and 2) collecting data on challenges
encountered during their efforts. The relative effort to establish an LVC event versus the benefit of, or the effort to conduct
the LVC event is out of kilter; the former takes considerably more time and effort, and too little is being done to address this
discrepancy.
This year, Operation Blended Warrior will provide attendees with glimpses of the state of LVC capabilities constrained to
I/ITSEC confines, as well as insights into the number, degree and priority of challenges encountered. This year’s concentration
is on standards, after action review, and traditional LVC and cyber-contested environments using a Black Swan (humanitarian
assistance) event as a backdrop. In future years, additional complexities and concentrations will be added to the event construct — both to showcase additional capabilities, and to tease out additional challenges that need to be addressed.
BY WORKING TOGETHER, NTSA, INDUSTRY AND GOVERNMENT PERSONNEL BELIEVE
SOLUTIONS CAN BE FOUND TO IMPROVE THE CREATION AND EXECUTION OF LVC EVENTS.
Check your Meeting Bags or stop by Booth 339 for a full listing of the participants and the programs to be demonstrated.
39
From the Tip of the Spear:
Joint Warfighter Panels
TUESDAY, 1 DECEMBER
1530 - 1700 • BOOTH 2280
FLOOR EVENT
1030 - 1200 • BOOTH 2280
1030 - 1200 • BOOTH 2280
FL2
Warfighters Corner:
From the Tip of the
Spear
A
s combat operations in the Middle East diminish, our warfighters are expected to perform
around the globe without hesitation. In recognition of their service, Warfighters Corner:
From the Tip of the Spear provides an opportunity to meet and thank the warfighters and to
hear about their personal experiences on recent deployments.
Warfighters Corner: From the Tip of the Spear presents multiple events during the conference featuring service men and women. These Soldiers, Sailors, Airmen, Marines, and Coast
Guardsmen derive great benefit from I/ITSEC and the organizations and industries that support the conference. Many of the speakers have served multiple tours and will be sharing their
stories; their personal experiences; and their views of what was, or was not effective in terms
of the training they received prior to deployment.
All From the Tip of the Spear sessions will include representatives from each of the Services. The presenters will discuss operations and also provide insights into the role of Allies,
international organizations and private organizations in theater. You’ll hear first hand how
your products and services have enhanced warfighter combat readiness.
The Wednesday morning session will be attended by veterans groups from the local Central
Florida area.
The audience listens intently to the presenters
from Warfighters Corner 2014.
40
Blending Live and Synthetic is Reality
TUESDAY, 1 DECEMBER
1400 - 1530 • ROOM S320GH
FE1
FOCUS EVENT
Live/Synthetic
Blended Training:
From Home Station
Environment
to Battlespace
Execution
Moderator
Defense (Readiness)
Panelists
Col Patrick Kline, USMC
MARFORCOM
LtCol Troy Havener, USAF
LVC LNO, U.S. Air National
Guard
COL Craig S. Unrath, USA
Director, National Simulation
Center, U.S. Army Combined
Arms Center
Stuart Bullard
Aviator, Producer, and
Founder, Unmanned
Autonomous Vehicle System
Association (UAVSA)
Chip Carpenter
Simulation Engineer, U.S.
Navy Fleet Forces Command
O
ur force of the future will expect a realistic and blended environment. Their training must
be demanding and challenging enough to give them success in complex and ambiguous
21st century environments.
Technological advantage is giving way to the need for training superiority in order to win
the wars of the future. Future training capabilities must provide that realistic Live/Synthetic
Blended Training environment. Live/Synthetic Blended Training is important for three reasons:
• With deployments coming to an end, budgets decreasing, and home station training
increasing, the training the warfighter receives must be realistic and relevant to prepare
for combat operations.
• Demand on, and expectations of, training technologies will only increase. Future training
must seamlessly blend live and synthetic environments in order to maximize the training
outcomes.
• There is an implied shift away from winning with better technology to winning through
better training and preparation.
This panel will discuss the ways and means of how existing and emerging technologies can
provide a blend of live and synthetic capability. The panel session will close with a question
and answer period that will give the audience an opportunity to provide their thoughts about
the state of Live/Synthetic Blended Training and possible avenues for advancement.
Session Chair:
Brian Holmes
The AEgis Technologies Group, Inc.
41
0830 - 1000 • ROOM S320GH
FE2
Medical Simulation Across the Services
FOCUS EVENT
Medical Simulation
and Training:
A Joint Endeavor
Moderator
M. Beth H. Pettitt
Chief, Medical Simulation
Research Branch, ARL HRED
STTC
Panelists
Col Meletios Fotinos, USAF,
M.D.
Chief, Medical
Modernization, Office of the
Command Surgeon, AETC
COL Daniel Irizarry, USA, M.D.
U.S. Army PEO STRI Medical
Advisor, JPO Medical
Simulation
Kevin Kunkler, M.D.
Chair Medical Simulation
and Training, Joint Program
Committee-1
Jack Norfleet
Chief Engineer, Medical
Simulation Research Branch,
ARL HRED STTC
Ray Perez, Ph.D.
Program Officer, Human
& Bioengineered Systems
Division, Warfighter
Performance Department,
Office of Naval Research
M
edical simulation and training is a growing field in both the military and civilian sectors.
Improvements in medical training technology enable more effective caregivers, and ultimately results in improved patient outcomes. The adoption of medical simulation has rapidly
grown over the past decade, in medical schools, residency programs, nursing, EMS training,
Combat Medics/Corpsman Training, and first responder training. Over the past 5-10 years,
there has been a huge endeavor to create a Joint Medical Simulation and Training community
within the Department of Defense. The commonalities in medical care across the Services
allow for significant collaboration and cost-savings by combining resources and leveraging
research. The Defense Health Agency has provided funding to support the joint medical infrastructure as well as research and development across this emerging community.
This panel will showcase the successes of this Joint endeavor and highlight its future direction. The discussions will cover current research objectives across the Services, and future
technological challenges facing the medical simulation community. The research discussion
will include both near term and long term research goals. The acquisition discussion will address current programs, highlight transition of R&D into acquisition, and lifecycle sustainment
related to medical modeling and simulation. The joint community discussion will speak to the
creation of requirements, the progress made towards creating a cohesive medical simulation
community, and the challenges encountered along the way.
The panel represents a broad segment of the Services, and has representatives involved in
diverse phases of the acquisition lifecycle — requirements generation, technology development, and fielding. The event provides excellent background to the joint nature of medical
simulation and training, and can help facilitate coordination among academia, industry, and
the appropriate Service(s). Additionally, the program provides the near and far term vision of
joint medical training within the DoD.
The audience does not need to have any medical simulation background to benefit from
this panel session. The intended audience is anyone with an interest in the progress and future vision for joint medical training, including engineers, scientists, business development,
corporate executives, and Service members.
Session Chair:
Mike Flanagan
CACI-Federal
42
INDUSTRY AND EXERCISES
How can business learn from NATO exercises?
1400 - 1530 • ROOM S330C
FE3
Moderator
Wayne Buck
Allied Command
Transformation, I3X
Panelists
Wayne Fujito
Past Chairman NATO
Industrial Advisory Group
(Invited)
Jean-Pierre Faye
Thales Raytheon Systems
FOCUS EVENT
Industry
Involvement
Initiative for NATO
Exercises ñ
N
ATO completed Exercise Trident Juncture (TRJE 15), the largest NATO exercise since the
Cold War in November of this year. TRJE 15 brought together 36,000 troops, 200 aircraft,
and 75 ships performing maneuver warfare, amphibious assault and other large-scale combat
training events. The NATO exercise program is the centerpiece of the NATO training strategy
but also provides an opportunity for Commanders to be imaginative and innovative and further develop requirements to meet future challenges.
This year, the Supreme Allied Commander Transformation (SACT) started a program for
industry called Industry Involvement Initiative for NATO Exercises (I3X). The aim of this new
initiative is to welcome and encourage innovation by allowing industry to have a much better
foundational understanding of how NATO exercises are initiated, planned, executed, evaluated and used to prepare for operations. Industry participation further contributes to aligning
industry research and development efforts with emerging NATO requirements.
For TRJE 1, industry visitors from 51 companies were embedded in the Computer Assisted Exercises (CAX) and Live Exercises where they received special briefings, participated in
facilitated sessions, received training, got qualitative comments from field users and bettered
understand operational requirements.
This special event panel is made up of NATO, national government, and industry participants who will discuss the lessons learned, benefits and challenges of NATO efforts to engage
industry inside the exercise planning and execution cycle. Come hear what industry learned
from this rare and valuable window into the war fighting experience.
Steve Joyce
Pulau Corporation
Gonzalo Arechaga
Thales Programas
de Electrónica y
Communicaciones (Invited)
Session Chair:
K. Denise Threlfall, Ph.D.
Kratos Defense & Security
Solutions
43
1600 - 1730 • ROOM S320GH
FE4
FOCUS EVENT
The Serious Games
Showcase &
Challenge (SGS&C)
The Next 10 Years of
Forging the Future
Through Innovation
Moderator
Lisa Scott Holt, Ph.D.
Senior Research Scientist,
Intelligent Automation, Inc.
& SGS&C 2015 Event Lead
Panelists
Joe Kreiner
Head of Engine Licensing
in the Western Hemisphere,
Epic Games
Mike Macedonia, Ph.D.
Assistant Vice President for
Research and Innovation,
Russell Shilling, Ph.D.
Executive Director of STEM,
US Department of Education
David Martz
President, BroadReach
Education
Michael D. Armbruster, Ed.D.
Senior Executive Director,
Career and Technical
Education, Orange County
Public Schools
A
s the SGS&C celebrates its 10-year anniversary, we look back to understand the impact
that serious games have had thus far, and more importantly, we look forward at the huge
potential that lies ahead. This SGS&C Special Event reflects on the current and future impact
of serious games, and provides the opportunity to learn from, and interact with experts in the
field. This special event begins when exhibits open with a Serious Game Hunt on the exhibit
floor for all I/ITSEC registrants and culminates with this lively, expert panel discussion.
SGS&C is focused on showcasing games that are used for learning or training, and the annual Challenge has played an important role in advancing the genre. Thought leaders from the
serious game industry and government organizations will share their perspectives on the last
10 years and the future of serious games. The panel will explore the impact of serious games
on workforce education, STEM education, and the simulation community. This event will be
accented with gameplay video segments from past SGS&C finalists. Don’t miss this once in a
lifetime event!
The Serious Games Hunt kicks off the 10th Anniversary on the exhibit floor with an engaging and fun introduction to the many different uses of serious games and gaming technology
in the training, simulation and education community. The I/ITSEC floor is open season where
participants are invited to “shoot out” targets on a Hunt card by engaging with serious game
developers and consumers on the exhibit floor to collect special stamps from designated exhibitors. The Hunt culminates at this expert panel discussion where a prize will be awarded
to one lucky Game Hunter (with a complete target card). The Game Hunter must be present
to win. Look for details in your I/ITSEC bag.
Session Chair:
Randy Jensen
Stottler Henke Associates, Inc.
44
Enlighten Us, But Make It Quick!
0830 - 1000 • ROOM S330C
FE5
FOCUS EVENT
IGNITE!
Moderator
John H. Aughey
The Boeing Company
C
ome and hear industry experts speak on topics such as non-traditional agile, serious
games, cognitive biases, and more.
Have you ever sat through a long presentation and lamented that there were only five minutes of content? Imagine if you could hear only that five minutes of targeted, compelling, and
maybe even provocative content... that’s Ignite!
Ignite is a presentation format that allows dynamic, high octane speakers a platform to
share their passion and ideas. I/ITSEC’s version of Ignite focuses on topics that are relevant
and thought-provoking, and which embody this year’s theme of “Forging the Future Through
Innovation.”
So bring your short attention span and prepare to be inspired, entertained, educated and
amazed by an array of talented speakers. Seven presenters have been selected from over 30
nominations, and each talk is jam-packed with inspiration and information using 20 slides
that auto-advance every 15 seconds, creating a fun and dynamic event.
Speakers
Pete Morrison
Brian Simpson
Brandt Dargue
Bohemia Interactive
Simulation, Inc.
Subject: Gaming Supercharges
Military Training
Q4 Services, LLC
Subject: High Performance
Collimated Optical Displays for
Simulation
The Boeing Company
Subject: Fun, Fast, and
Unforgettable Learning Experiences
Bill Reuter
Al Brannan
R-Squared Solutions, LLC
Subject: Leadership from the Flight
Deck
CAE USA, Inc.
Subject: Cybersecurity
Anne Little, Ph.D.
Addx Corporation
Subject: Unconventional Agile
Lisa Scott Holt, Ph.D.
Stu Armstrong
QinetiQ Training and
Simulation, Inc.
Subject: 2025 – A look BACK over
the last 10 years
Subject: Serious Games
Session Chair:
Denny Shockley
Motion Analysis Corporation
45
1030 - 1200 • ROOM S320GH
FE-6
Visual Computing – Cloud to Virtual Reality
FOCUS EVENT
Visual Computing
Moderator
Doug Traill
Principal Engineer, NVIDIA
Panelists
Douglas Maxwell
Science and Technology
Manager, Virtual World
Strategic Applications, U.S.
Army Research Lab
Tim Woodard
CTO, Diamond Visionics
Matt Tovey
Training and Simulation
Systems Programme
UK Ministry of Defence
One of these images is a photograph, the other
is a computer rendering using physically-based
rendering techniques. (Image courtesy of NVIDIA)
Real-time image generation from different
data sources. (Image courtesy of Diamonod Visionics)
V
isual computing refers to the rising importance of GPUs (Graphical Processor Units) to
the generation of visual simulation — not only in creating the visuals but increasingly
influencing how databases are created and training is deployed. The event will include a panel
of experts from the computer graphics and visualization industry as well as the training community related to these challenges.
This event focuses on three challenges to the visual simulation community:
Realistic Visuals — How does new technology enable more photo-realistic visualization to
support immersive and VR training? The first challenge, photo-realistic visualization, is about
suspending disbelief and getting the warfighter to believe they are immersed within a training
exercise. However, photo-realism is not only computationally extensive, it takes a long time to
render a realistic image and it is reliant on the skill of the programmer or artist who generates
the database. This event will explore the concept of physically-based rendering which uses
scanned materials to create photo-realistic images.
Data on-the-fly — What are the opportunities for automatically generating visual databases
“on the fly” to provide timely training material? The second challenge being discussed is the
time it takes to generate a 3D database. Typically, this development targets specific runtimes
or Image Generators that require the database to be processed in intermediate steps. Even
with open database standards this approach leads to proprietary datasets. One method to
overcome this is to compute the database in real-time utilizing the power of today’s modern
computing platforms. Some Image Generation platforms already do this and will be discussed
in this event.
osted in the cloud? — Does this provide for more robust deployment of training in the cloud
H
so visual training can be utilized anywhere? The last challenge to be discussed includes the
numerous advances in the technologies relating to cloud gaming, virtualized desktop infrastructures and training as a service that have occurred over the past 3 years. These advances
have the potential to revolutionize how training is delivered to the warfighter. This event will
discuss the current and future state of cloud technology, lessons learned from current deployments, lessons from other industries and future research.
Session Chair:
Klainie Nedoroscik
Camber Corporation
46
DATES & TIMES LISTED BELOW
BOOTH 2280 (WARFIGHTERS
CORNER)
COI2
PEO STRI to
Provide TSIS
Updates at I/ITSEC
Session Chair: Traci Jones,
U.S. Army PEO STRI
The U.S. Army Program Executive Office for Simulation,
Training and Instrumentation
(PEO STRI) will offer updates
to the business opportunities
portion of the annual Training
and Simulation Industry Symposium (TSIS). The briefings
will occur in the Warfighters
Corner as follows:
TUESDAY, 1 DECEMBER • 1300 - 1330
COL Bill Canaley, USA, Director for Field Operations
COL Vince Malone, USA, Project Manager for Training
Devices (PM TRADE)
Joe Giunta, Army Contracting Command-Orlando
WEDNESDAY, 2 DECEMBER • 1300 - 1330
COL Rich Haggerty, USA, Project Manager for
Instrumentation, Targets and Threat Simulators (PM ITTS)
WEDNESDAY, 2 DECEMBER • 1330 - 1400
COL Ron Gaddy, USA, Project Manager for Integrated
Training Environment (PM ITE)
WEDNESDAY, 2 DECEMBER • 1400-1430
Dale Whittaker, Director of International Programs Office
(IPO)
TUESDAY, 1 DECEMBER
1530 - 1700 • ROOM S320GH
COI3
Air Force
Acquisition Update
Session Chair: Anthony
DalSasso, Air Force Materiel
Command
Presenters
COMMUNITY OF INTEREST
Discover Teaming Opportunities for International Industry
and International GovernWEDNESDAY, 2 DECEMBER
ment/Military organizations
0830 - 1000 • ROOM S310A
to solve needs common with
COI1
U.S. DoD. Sign up for one-onInternational
one meetings with program
principals to discuss your
Outreach ñ
technology interests in or poSession Chair: Cathy
tential cooperative programs
Matthews, Matthews
at http://www.iitsec.org/attendees/
Systems Engineering, Inc.
Pages/InternationalAttendees.aspx
Both the Monday and
Wednesday sessions will discuss the following topics:
International Industry Opportunities with the Foreign
Comparative Testing (FCT) Program will be presented by Col
Scott Wallace, USAF, OSD Director for the Comparative Technology Office.
• Electro-magnetic Spectrum (EMS) Agility
• Autonomous Systems
• Space Capability Resilience
• Asymmetric Force Application
Government-to-Government opportunities in cooperative
research and development programs will be presented by Col
Mike Malley, USAF, OSD Director, Coalition Warfare Program.
• Address strategic technology gaps for current and future
missions
•
Develop interoperability solutions for coalition operations
• Strengthen current defense partnerships and developing
new relationships through research and development
MONDAY, 30 NOVEMBER
1430 - 1600 • ROOM S310A
Lynda Rutledge, SES, Program
Executive Officer and Director
for the Agile Combat Support,
AF Materiel Command; Col
Daniel Marticello, USAF,
Chief, Simulators Division,
AF Materiel Command
This Special Event will provide the latest information
from the Air Force regarding acquisition policy and upcoming training system acquisition actions. It will feature remarks
by Ms. Rutledge, the Air Force Program Executive Officer for
Agile Combat Support, who will share her perspective on the
current state of the Air Force acquisition process and ongoing
initiatives, as they apply to the I/ITSEC community. This will
be followed by a presentation by Col Marticello, the Chief of
the Simulators Division. He will provide an update on Air
Force simulator business opportunities, as a follow-on to the
Simulation and Training Community Forum (STCF) held earlier this year.
0830 - 1000 • ROOM S330C
COI4
Asia Pacific
Simulation
Alliance: Advances
in Healthcare
and Emergency
Management
Simulation ñ
Session Chair: Jim Godwin,
The Tolliver Group, Inc.
Presenter
Alisha Fisher, Founder and Director, Asia Pacific Simulation
Alliance
This event will focus on the
unique possibilities of simulation to increase productivity
and improve health outcomes
across the region. It will showcase key industry developments and thought leaders to
discuss the impact these advances will have on our jobs,
our economy, our education
and our well-being.
Who is the Asia Pacific Simulation Alliance?
The Asia Pacific Simulation Alliance provides an international platform to bring together simulation users, developers, researchers and government to share, learn and collaborate. Our
purpose is to provide a forum to fulfill three main objectives:
1. For users of simulation to share research, simulation technologies and experience
2. For simulation developers to promote their latest technologies and also share latest research and development
3. For users and developers to engage together in a rich dialogue to further advance simulation as an enabler across a
wide range of industry sectors
47
Why attend?
The purpose of this event is
to highlight the importance
Healthcare systems across the Asia Pacific region, along with
1030 - 1200 • ROOM S330C
of standards in modeling and
the rest of the world, are experiencing a period of significant
COI6
simulation across Europe.
change as the pressure grows to train more healthcare proETSA/SISO Briefing: Of particular focus will be
fessionals, at a lower cost, whilst at the same time providing
defense budget constraints
state of the art training programs and facilities.
International M&S
which are making reusability
Simulation technologies are seen as key enablers in the
Standards ñ
and interoperability in Simudrive to increase productivity and achieve improved health
Session Chair: Robert
lation and Training essential.
outcomes across the region.
Hester, Ph.D., University of
European Training and SimWe invite you to join us to hear about developments across
Mississippi Medical Center
ulation Association (ETSA),
the Healthcare and Emergency Management sector where simulation technologies are being embraced to improve the train- which has an agreement with the Simulation Interoperability
ing of the healthcare workforce and address issues around Standards Organization (SISO) to support each other across
Europe, can be seen as leading in an important, money savpatient safety, within a fiscally responsible environment.
ing, technology area. The presentations will not be detailed
explanations of any particular standard, but will illustrate
Who should attend?
This meeting is aimed at those working in the following or- what standards are doing for NATO and in individual nations
and how SISO standards are supporting the way forward unganisations:
der tight defense budgets.
• Government Departments
• Health Services
Moderator
• Public Hospitals
• Private Hospitals
RADM James Robb, USN
1200 - 1730 • ROOM S230D
• Universities and other healthcare training providers
(Ret.), National Training and
COI7
• Medical Device Suppliers and Management Companies
Simulation Association
Whether you be a healthcare provider or educator; an academic or technician; a CEO, Chief Medical Director or Director of Nursing; this meeting will have something for you.
0900 - 1730 • ROOM S330D
COI5
ADL Virtual World
Sandbox PlugFest:
Hands-on Activity
Moderator
Robert Chadwick, ADL
Initiative
Attention game and simulation developers, decision makers, and instructional technology designers—this activity is
Session Chair: Rolando
for you!
Paredes, ADL Initiative
The Virtual World Sandbox
lets you collaboratively author and deploy 2D/3D simulations
and games via any internet connection—with absolutely no
software installation required. The Sandbox comes with a
large library of 3D models, embedded physics, and extendable behavior scripts. Plus, simulations made with it support
robust human performance assessment and integrate seamlessly with xAPI.
At this activity, participants will explore the Sandbox from
a multi-user perspective, learn to author a 3D game environment from the Sandbox’s open-source library, and leave with
a new tool to enhance your organization’s training and education.
This activity is intended for anyone from semi-technical
instructional designers to seasoned software developers. The
first 45 minutes will describe the Sandbox’s purpose, value,
and use (and decision makers are encouraged to attend this
portion). The rest of the activity will involve hands-on work
in the Sandbox. If you know the basics of JavaScript, you can
become an expert game developer with this open-license tool.
So, bring your laptop and start building!
48
I/ITSEC Energy
Forum
Join the President of NTSA at
roundtable discussions where
leaders from the energy sector
will discuss power plant operations, safety and training
requirements. Industry and Academia will showcase and discuss the latest solutions.
1200-1730: Demonstrations
1330-1500: I/ITSEC Energy Roundtable (Energy Companies’
requirements)
1530-1700: Simulation Solutions (Simulation industry’s solutions)
Session Chair: Catherine
Emerick, QinetiQ Training
and Simulation, Inc.
Invited participants include representatives from Duke Power and Dominion Power, with industry being represented by
TRAX Corporation, L-3 MAPPS, Areva North America, ABB,
Central Virginia Community College, Lockheed Martin and
SAAB Security. Companies will also have demonstrations running in the room from 1200 to 1730.
Hiring America’s
Service Members:
DoD SkillBridge
Forum
Moderator
Frank C. DiGiovanni, SES,
and Training, Office of the
Assistant Secretary of Defense
(Readiness)
Presenters
Bruce Sobczak, Director,
Workforce Development for
Session Chair: Elaine
the Commonwealth Center
Raybourn, Ph.D., Sandia
for Advanced Manufacturing
National Laboratories, ADL
Initiative
(CCAM); Mick Yauger,
Chicago Teamster 786,
Teamsters Military Assistance Program, Vietnam Veteran;
Elizabeth Murray-Belcaster, International Brotherhood of
Teamsters Consultant, Building Materials & Construction
Trade Division; Amy J. Moorash, Division Chief, U.S. Army
Continuing Education System; Tim Thorne, CEO, ABF
Freight
Each year, over 250,000 Service members transition out of the
military. Many seek employment in the civilian workforce.
The new DoD SkillBridge initiative allows eligible transitioning Service members to participate in job skills training, including apprenticeships and internships, starting up to the
last six months of their active duty. Join us for this I/ITSEC
forum featuring a town hall with DoD SkillBridge program
managers, training providers, Service members, and Veterans
who will discuss how DoD SkillBridge offers training opportunities in a range of professions and industries. Whether you
are a Service member or a potential training provider, you will
learn firsthand how to get involved.
0830 - 1000 • ROOM S330D
COI9
Presenters
Dan Visone, Army Geospatial
Center (AGC); Shun Steward,
USAF AFLCMC Air Force
Geospatial
Common Dataset (AFCD);
Environmental
Frank Rhinesmith, PEO STRI
Synthetic Environment (SE)
Database
CORE; Earl Miller, SOCOM
Standards Forum
Common DataBase (CDB);
Session Chair: Tom
Rob Cox, PEO STRI
Yanoschik, SAIC
Enterprise Data Services
(EDS); Bruce Riner,
NAWCTSD NAVAIR Portable Source Initiative (NPSI)
1030 - 1200 • ROOM S330C
COI10
Navy Vision:
From the Training
Systems Program
Manager’s
Perspective
Moderator
Rob Matthews, NAWCTSD
Deputy Technical Director
1600 - 1730 • BOOTH 2280
(WARFIGHTERS CORNER)
COI8
Presenters
CAPT Wes Naylor,
Commanding Officer,
Naval Air Warfare Center
(NAWCTSD); CAPT Craig
Dorrans, Program Manager,
Session Chair: Diana Teel,
Naval Aviation Training
NAWCTSD
Systems (PMA-205); CAPT
Jeffrey Sinclair, Program
Manager, Naval Surface Training Systems (PMS-339); Jimmy
Lee, Program Manager, Undersea Training Systems (07TR)
Each year at I/ITSEC, a panel of Navy Captains and senior
civilian leaders representing the Navy’s training acquisition
organizations convenes to discuss the year’s highlights and
share their strategic vision. I/ITSEC participants are welcome
and encouraged to attend to hear about the state of the Navy’s
training systems.
1030 - 1200 • ROOM S330D
COI11
Presenters
Curtiss Murphy, Technical
Director, Alion Science; Talib
Hussain, Senior Scientist,
Design of Learning BBN Raytheon Technologies;
Michael Pillitiere, Lead Artist/
Games
Designer, ICF International;
Clint Little, Director of
Interactive Development, Security Mentor; Vance Souders,
Producer, JANUS Research
This is the sixth workshop held by the multi-disciplinary Design of Learning Games Community of Practice. This 90-minute workshop provides the opportunity for serious games
practitioners to meet and discuss key research and development issues and guidelines. We encourage instructional system designers, game designers, and simulation developers,
program managers to attend and contribute your thoughts,
methods and concerns to our discussions. The workshop will
cover several topics of current research and development interest. For each topic, a short introduction of current issues
and approaches will be presented by a facilitator from the
community, followed by a longer group discussion among all
workshop participants.
This forum provides a unique opportunity for the Components’ geospatial database producers to meet with leaders in
geospatial standardization, and geospatial dataset consumers and suppliers to discuss goals and challenges. Emerging
technology, standards and services to reduce data production
times to empower the agile force will be explored.
49
I N T E R N AT I O N A L P R O G R A M S
INTERNATIONAL ATTENDEES - INTERNATIONALE TEILNEHMER - LES PARTICIPANTS INTERNATIONAL
INTERNATIONAL DELTAKERE - INTERNATIONELL DELTAGARE - INTERNATIONAL DEELNEMERS
International Pavilion Room S310E-H
International attendees can meet and connect with counterparts from around the world.
Limited private meeting space is available on a first-come, first-served basis to our
international participants and may be scheduled at the International Pavilion’s Welcome Desk. Additional information about the many international activities throughout
I/ITSEC are readily available in the International Pavilion. Sponsored by AVT Simulation.
International Registrants should register at the dedicated International Check-in station positioned
near the Main Registration Desk in S220 of the South Concourse. International Conference Attendees’ Meeting Bags will be available for pick-up at the Welcome Desk in the International
Pavilion. More information specific to international attendees will be available at that location.
International Pavilion Hours of Operation
Sunday, 29 November
1400-1800
Monday, 30 November
0800-1800
Tuesday, 1 December
1030-1800
Wednesday, 2 December
0800-1500
Thursday, 3 December
0800-1500
Program Notes of Special Interest for International Attendees
Papers
Explore your Program for the ñ indicating Papers from International Authors.
Tutorials
Monday, 1 December • Room S320C • 0830 – 1000
Introduction to HLA
Monday 1 December • Room S320B • 1245 – 1415
Speech-based Interaction: Myths, Challenges and Opportunities
Monday 1 December • Room S320D • 1430 – 1600
Team and Collective Training Needs Analysis
Events
Wednesday, 2 December • Room S210D • 0830 – 1000
Best Papers from Around The Globe
SimTecT Best Paper
Col Lorna Swinyard, British Army, Director of Training Development, Directorate of Training
Procurement (Strategy to Deliver) Critique & Lessons
ITEC Best Paper
Amanda Davies, Ph.D., School of Policing, Charles Stuart University
Reflecting and Gaining Wisdom: Self-Assessment Rubric Model for Optimising Simulation Based Learning
50
I N T E R N AT I O N A L P R O G R A M S
INTERNATIONAL ATTENDEES - INTERNATIONALE TEILNEHMER - LES PARTICIPANTS INTERNATIONAL
INTERNATIONAL DELTAKERE - INTERNATIONELL DELTAGARE - INTERNATIONAL DEELNEMERS
SIGNATURE EVENT
Thursday, 3 December • Room S330AB • 1030 – 1200
Training Adaptive Leaders for a Black Swan World
COMMUNITY OF INTEREST EVENTS
Monday, 30 November • Room S310A • 1430 – 1600
Wednesday, 2 December • Room S310A • 0830 – 1000
(See page 37 for more information)
International Outreach
Is the world actually getting more complex, or are we just bombarded with information? There is no simple answer. However,
with the world’s increasing interconnectedness, we do know one
thing. There is a growing awareness (in the military and elsewhere) of a meteoric rise in potential “Black Swan” challenges
and events that are difficult to predict. How to prepare for this?
The event features a diverse panel of top experts. Expect a
wide-open discussion that will apply to the future of military
training, business industry practices, academic research, and be
of particular value to those with an interest in preparing for unknown threats. This panel builds upon the earlier Black Swan
Signature Event as well as I/ITSEC’s new Operation Blended
Warrior floor activity.
FOCUS EVENT
Wednesday, 2 December • Room S330C • 1400 – 1530
Industry Involvement Initiative for NATO Exercises
Asia Pacific Simulation Alliance: Advances in
Healthcare and Emergency Management Simulation
NATO completed Exercise Trident Juncture (TRJE 15), the largest
NATO exercise since the Cold War in November of this year. TRJE
15 brought together 36,000 troops, 200 aircraft, and 75 ships performing maneuver warfare, amphibious assault and other largescale combat training events. The NATO exercise program is the
centerpiece of the NATO training strategy but also provides an
opportunity for Commanders to be imaginative and innovative
and further develop requirements to meet future challenges.
This year, the Supreme Allied Commander Transformation
(SACT) started a program for industry called Industry Involvement Initiative for NATO Exercises (I3X). The aim of this new
initiative is to welcome and encourage innovation by allowing
industry to have a much better foundational understanding of
how NATO exercises are initiated, planned, executed, evaluated
and used to prepare for operations. Industry participation further
contributes to aligning industry research and development efforts with emerging NATO requirements.
This special event is panel made up of NATO, national government, and industry participants who will discuss the lessons
learned, benefits and challenges of NATO efforts to engage industry inside the exercise planning and execution cycle. Come hear
what industry learned from this rare and valuable window into
the war fighting experience.
This event will focus on the unique possibilities of simulation to
increase productivity and improve health outcomes across the
region. It will showcase key industry developments and thought
leaders to discuss the impact these advances will have on our
jobs, our economy, our education and our well-being.
Healthcare systems across the Asia Pacific region, along with the
rest of the world, are experiencing a period of significant change
as the pressure grows to train more healthcare professionals, at
a lower cost, whilst at the same time providing state of the art
training programs and facilities.
Simulation technologies are seen as key enablers in the drive
to increase productivity and achieve improved health outcomes
across the region.
Discover Teaming Opportunities for International Industry and
International Government/Military organizations to solve needs
common with U.S. DoD. Sign up for one-on-one meetings with
program principals to discuss your technology interests in or
potential cooperative programs at http://www.iitsec.org/attendees/Pages/InternationalAttendees.aspx
International Industry Opportunities with the Foreign Comparative Testing (FCT) Program will be presented by Col Scott Wallace, USAF, OSD Director for the Comparative Technology Office.
Government-to-Government opportunities in cooperative research and development programs will be presented by Col Mike
Malley, USAF, OSD Director, Coalition Warfare Program.
ETSA/SISO Brieifing: International M&S Standards
The purpose of this event is to highlight the importance of standards in modeling and simulation across Europe. Of particular
focus will be defense budget constraints which are making reusability and interoperability in Simulation and Training essential.
European Training and Simulation Association (ETSA), which
has an agreement with the Simulation Interoperability Standards
Organization (SISO) to support each other across Europe, can
be seen as leading in an important, money saving, technology
area. The presentations will not be detailed explanations of any
particular standard, but will illustrate what standards are doing
for NATO and in individual nations and how SISO standards are
supporting the way forward under tight defense budgets.
51
SPECIAL GUESTS
Central Florida Veterans Organizations
The citizens of the United States are more supportive of today’s men and women in
Military Service than they have been since WWII. The Overseas Contingency Operations
A veteran…is
someone who, at one
continue to be of great concern to all, whether in uniform or not. This I/ITSEC effort is
point in his or her
to especially inform U.S. Veterans about the dramatic changes in training methodologies
life, wrote a blank
and systems since their time in Service. Since 2006, I/ITSEC has had the pleasure and
check made payable
honor to have select members of Central Florida Veterans Organizations visit the exhibit
floor on Wednesday. The successful experiences in broadening the Veterans’
to The United States
understanding of today’s training transformation as well as the appreciation of the other
of America for an
I/ITSEC attendees in seeing the Veterans, some in their uniforms and with their
amount of “up to and
decorations, have made this an annual event for I/ITSEC. When you see these Veterans,
including my life.”
thank them for their service to the Nation.
52
Papers
TUESDAY, 1 DECEMBER •  All Papers are eligible for CEU/CLP credits. (See page 12) 
ROOM
1430
1500
T-1 Designing for Training
Effectiveness
Susan Coleman, Ph.D.
Measuring Training Effectiveness
of Lightweight Game-based
Constructive Simulation (15007)

Validating Scenario-based
Training Sequencing: The
Scenario Complexity Tool (15102)

Adaptive Instructor Operating
Stations: Design to Decrease
Instructor Workload and Increase
Effectiveness (15336) 
S320B
S-1 Cutting Edge Training
Angela Alban
Mobile Augmented Reality for
Force-on-Force Training (15223)
h
Emergency Medical Card
Augmented Reality: Training
Evaluation (15266)  h
Real-Time Cutting of Organs with
Scissors (15333)  h
S320C
EC-1 Easy Button
John Dzenutis
Automated Simulation Creation
from Military Operations
Documents (15227) 
Scheduling Training to Manage
Acquisition & Decay (15340) 
M&S as a Service: Paradigm for
Future Simulation Environments
(15324)  ñ
ED-1 Assessment and
Evaluation: Isolating Effects
Robert “Buddha” Snyder
Student Retention in STEM
Career Paths: Primary Influences
on the Decision to Stay or Leave
(15018) 
Overcoming the Challenge of
Evaluating Skills Transfer from
Training to Job (15151) 
Measuring a Moving Target:
Validating Deployed Training
Courses (15189) 
H-1 This Won’t Hurt a Bit
Robert Hester, Ph.D.
Piloting a Groundbreaking
Virtual Continuing Competency
Platform: Results and
Recommendations (15325) 
%h
Mobile App Design for Veterans
with Physical and Cognitive
Limitations (15182)   h
Structured Development of
Interventions to Improve
Physician Knowledge Retention
(15095)  h
P-1 Enterprise, Architecture,
and Standards
Jan Drabczuk
The Live-Synthetic Training and
Test & Evaluation Enterprise
Architecture (15076)  
Early Adoption of Common
Operating Environment (COE)
Standards and Guidelines (15098)

EC-2 Are We There Yet?
Scott Ariotti
Modelling a Helicopter Training
Continuum to Support System
Transformation (15165)  ñ
Military Vehicle Training with
Augmented Reality (15180) 
S320D
S320E
S320F
S330C
ROOM
SESSION/CHAIR
1600
1630
1700
S320A
T-2 VR, AR, and AV for
Training “Reality”
Paul Lyon
Training Effectiveness Evaluation
of Augmented Virtuality for Call
for Fire Training: Insights from a
Novice Population (15014) 
Empirically Derived
Recommendations for Training
Novices Using Virtual Worlds
(15038) 
Using Augmented Reality to
Tutor Military Tasks in the Wild
(15050) 
S320B
S-2 Getting to the Right
Scenarios & Data
Roy Scrudder
Virtual Battlespace Scenario
Encoding for Reuse (15027)
%
Multi-Federate Scenario
Development and Testing: “A
Good Plan, Violently Executed”
(15249) 
The Expected Results Method for
Data Verification (15020) 
EC-3 Pinging and the Brain
Luis Velazquez
Cognitive Two-Way Interactions
In an Immersive Virtual Reality
Environment (15332)  
Modeling and Integrating
Cognitive Agents Within the
Emerging Cyber Domain (15232)

Tablet Computer Call for Fire
Simulation Proof of Concept
Study Results (15008) 
ED-2 Navigating the Social
and Educational Terrain
Martin Bink, Ph.D.
Enhancing Good Stranger Skills:
A Method and Study (15071)
Curriculum GPS: An Adaptive
Curriculum Generation and
Planning System (15369) 
H-2 Analyze This
Bill Gerber, Ph.D.
Work Domain Analysis for
Ecological Interface Design of
Tangible Interfaces (15130) 
Adaptive Testing: Adapt and
Overcome the Shortfalls
of Traditional Proficiency
Assessments (15196) 
P-2 Design, Build, Track and
Train – Here and Abroad
Mary Trier
Sejong the Great Class DDGs:
How ROK Navy Trained and
Embraced Them (15032)  ñ
A System-Model-Centric
Collaborative Environment for
the Acquisition Lifecycle (15093)

EC-4 Avatars Crossing
James (Josh) Jackson
Turn-Based Gaming for Convoy
Commander Training (15036)
ñ%
An Immersive Live / Virtual
Bridge Approach with Ultra
Wideband Tracking Technology:
Phase II (15024) 
S320C
S320D
S320E
S320F
S330C

Cognitive Load Assessment for
Intelligence Analysts through
Full Motion Video Analytics
(15142) 
Virtualizing Humans for Game
Ready Avatars (15023) 
PAPER SESSIONS
1400
S320A
SESSION/CHAIR
Ï Cyber Security
Mobile
53
WEDNESDAY, 2 DECEMBER •  All Papers are eligible for CEU/CLP credits. (See page 12) 
PAPER SESSIONS
ROOM
SESSION/CHAIR
Developing Game-Based
Leadership Training for Robotic
Surgeons (15198)  h
The Use of Hyper-Realistic
Surgical Simulation (15244) 
S320A
T-3 Physician Know Thyself!
Enhancing Surgical Team
Skills through Gaming and
Simulation
Robert Heinlein
S-3 Convergence of Testing
& Training
Karen Williams
Polygone LVC: The New
Paradigm for EW Training
(15213) 
Live Synthetic Training and
Test & Evaluation Infrastructure
Architecture (LS TTE IA)
Prototype (15099) 
Measuring Realism in
Simulations for Training and
Testing (15206)  
EC-5 Fidelity Matters
Eric Weisel, Ph.D.
Reliably Assessing the
Effectiveness of a Plan with
Models of Verying Fidelity and
Under Time Constraints (15060)
ñ
Requirements for Future SAFs:
Beyond Tactical Realism (15193)

Required Fidelity of Simulated
Wound at the Point of Injury
(15351)  h
S320D
ED-3 Full STEM Ahead
Nina Deibler
Mars Game: Creating and
Evaluating an Engaging
Educational Game (15105) 
The Secret for STEM Success:
Employing Technology for Math
Proficiency (15145)  
Inquiry and Design Approach to
STEM Education Using Projectbased Learning (15238) 
S320E
H-3 Do You Understand?
John Schlott
Multi-measure Assessment of
Internal Distractions on Driver
Performance(15017) 
Measuring Trust of Autonomous
Vehicles: A Development and
Validation Study (15049) 
Building Trust in a HumanRobot Team with Automatically
Generated Explanations (15315)

Virtual Interview Training
Increases Job Offers for Veterans
and Others (15013) 
Stepping Stones – An Augmented
Reality Rehabilitation Game
(15181) 
The VA Virtual Medical Center:
Implementing a Vision for a
Virtual Healthcare Campus for
our Veterans (15358)  h
S320B
S320C
S320F
EC-6 It’s All About Veterans
Brian Holmes
0830
S210D
ROOM
0900
0930
Special Paper Session: Best Papers from Around the Globe
SESSION/CHAIR
1030
1100
1130
Human Performance Analysis and
Engineering: You Cannot Hit
What You Do Not Shoot (15209)

BP-1 Best Papers from
Education, Training, Human
Performance Analysis and
Engineering
Anne Little, Ph.D.
Education: Stress Exposure
Training for the Dismounted
Squad: The Human Dimension
(15150)  
Training: Differentiating Measures
of Learning (MOL) from
Measures of Performance (MOP)
During Aircraft Carrier Landing
Practice (15210)  
S-4 From Reality to
Simulation
Mark Soodeen
Network Bandwidth’s Effect
on Virtual World Simulator
Performance Optimization
(15360) 
3D Immersive Environment Using Battle Damage Computation
X-Plane for Depth Perception
Server (15051) 
Research (15261) 
S320C
EC-7 Let’s Get Physiological
Jennifer Murphy
Visualizing fMRI Data Using
Volume Rendering in Virtual
Reality (15253) 
Professional Soldier Assessment
of a Rifle-Mounted Target HandOff System (15039)  ñ
Empirical Support for BrainBased Assessment in SimulationBased Training (15300) 
S320D
ED-4 Getting In Front of It
Jay White
Antecedents of Adaptive
Collaborative Learning
Environments (15211) 
Adaptable Resilience Training
for Transitioning Veterans Using
Existing Technologies (15285) 
The Changing Face of Military
Learning (15327) 
H-4 It’s a Wild, Wild Cyber
World
Jennifer Arnold
Automated Performance
Assessment in Cyber Training
Exercises (15044)  Ï
Command Shift: Exploring Modern
Gaming Technologies to Create
Next-Generation OCO Interfaces
(15091)   % Ï
Embedded Cyber-Physical
Systems for Assessing
Performance in Training
Simulations (15263)  Ï
P-3 Developing Competency
– The Future for M&S
Jeffrey A. Raver
Improving Education, Training
STEMulating: An Integrated
and Career Advancement through Approach to Cultivating Our
Competency Portability (15117)
Future (15270) 
S320A
S320B
S320E
S320F
S210D
Modeling and Simulation
Professionals – Meeting the
Demand (15342) 

Special Paper Session: Presentations from the Future Leaders Pavilion
54
Ï Cyber Security
Mobile
WEDNESDAY, 2 DECEMBER •  All Papers are eligible for CEU/CLP credits. (See page 12) 
SESSION/CHAIR
S320A
BP-2 Best Papers from
Emerging Concepts and
Innovative Technologies;
Simulation; Policy Standards
Management and Acquisition
Tara Kilcullen
Emerging Concepts and Innovative
Technologies: Safe Testing
of Autonomous Systems
Performance (15348)  
Simulation: Cyber Operational
Architecture Training System –
Cyber for All (15108)   Ï
Policy Standards Management and
Acquisition: Measuring Virtual
Simulation’s Value in Training
Exercises – USMC Use Case
(15114)  
S320B
S-5 Innovation in
Environmental Modeling
Roy Scrudder
High Fidelity Wind Model
Software for Real-Time
Simulation Platforms (15362)
ñ
Automated Runtime Terrain
Database Correlation Assessment
(15218) 
Automated Modelization in
Terrain Database Production
(15290) 
S320C
S-6 Human Interface to
Artificial Intelligence
Gerald Dreggors
Modeling CGF Behavior with
Machine Learning Techniques:
Requirements and Future
Directions (15128)  ñ
Automatic Speech Recognition
in Training Systems:
Misconceptions, Challenges and
Paths Forward (15205) 
On the Peripheral Application of
Head Mounted Display (HMD)
Devices in Infantry Simulation
(15186)  ñ
ED-5 Challenging the Game
Adelle Lynch
Game-Based Training for HumanIntelligence Skills (15067)
%
A Conceptual Model of Pedagogic
Design to Support Critical
Thinking in Commanders (15075)
%ñ
H-5 Maintaining Your Cool
Elaine Raybourn, Ph.D.
Human Performance in Content
Using Micro and Macro Studies of
Design for Interactive Augmented Tablets to Improve Maintenance
Reality Systems (15156)
(15279)
P-4 Getting Your Money’s
Worth
Larry Rieger
Calculating Simulation-Based
Training Value: Cost Avoidance
and Proficiency (15199) 
S320D
S320E
S320F
ROOM
S320A
S320B
S320C
S320D
S320E
S320F
SESSION/CHAIR
1400
1430
1500
Large Scale Adoption of Training
Simulations: Are We There Yet?
(15256) 
Automated Surveys: Lowering
the Respondent’s Burden (15080)
ñ
1630
1700
1600
T-4 Team Building: All for
One and One for All
Brian Cairns
Assessing the Effects of Virtual
Emergency Training on Mine
Rescue Team Efficacy (15119)

Alternative Front End Analysis
for Automated Complex
Systems (15121) 
Evaluating Distributed Teams
with the Team Multiple Errands
Test (15264) 
S-7 Next Gen LVC
Robert Kleinhample
M&S Training Transformation:
Bridging the Next Generation
Joint LVC (15167) 
Leveraging Cloud Computing
Technology for LVC Training
(15101) 
Osseus, An Experiment in What’s
Next in LVC M&S Architecture
(15085) 
EC-8 Experience with the
Experience API
Nick Giannias
Putting Live Firing Range Data to
Work Using the xAPI (15019) 
Adapting Gunnery Training Using
the Experience API (15179) 
Opening Legacy Data Silos: Using
Experience Data for Educational
Impact (15043) 
ED-6 Instructors for the
Force of the Future
Jim Threlfall
Designing Instructor Support
Tools for Virtual Shiphandling
Training (15133) 
Hey, Remember to Add
Motivational Design to Your
E-learning (15030)  ñ
Achieving Educational
Excellence: What do Effective
Instructors do? (15226) 
H-6 Flight Life
Dennis Vincenzi, Ph.D.
Helicopter Pilot’s Modeling
Including the Stress Factor
(15168)  ñ
Human-in-the-Loop Flight
Simulation Study of Virtual
Constructive Representation on
Live Avionics Displays (15197) 
Practical Recommendations for
Validating Survey Apparatus in
Coalition Training Environments
(15299) 
P-5 Cybersecurity: New
Threats, New Policies, New
Solutions
Robby Robson, Ph.D.
Risk Management Framework
(RMF) Transition Impacts in
Training Simulation Systems
(15009)  Ï
Cybersecurity Controls: Then and Cybersecurity Challenges and
Resolutions for Simulator &
Now (15010)  Ï
Training Systems (15063)  Ï
PAPER SESSIONS
ROOM
Ï Cyber Security
Mobile
55
THURSDAY, 3 DECEMBER •  All Papers are eligible for CEU/CLP credits. (See page 12) 
ROOM
SESSION/CHAIR
0830
0900
0930
PAPER SESSIONS
T-5 Where’d Who Go? –
Operational Considerations
for LVC Training
William Reuter
Exercise Management
Considerations for Live, Virtual,
and Constructive (LVC) Training
(15281) 
Integrating Warship Bridge,
Combat, and Deck Teams in LVC
Environment (15191) 
Capability Assessment of Test
and Live Training Systems for
Real Time Casualty Assessment
(15364) 
S-8 Simulation Supported
Training
Scott Hooper
Embarking on a Home Station
Training Revolution (15176) 
Implementation of Role-Based
Command Hierarchy Model for
Actor Cooperation (15166)  ñ
Innovative Division/Brigade
Level CO Training Solution for
Influence Operations (15107)
ñ
EC-9 3D Psycho
John Dzenutis
Rapid 3D Geospatially Oriented
Structure Extraction from
Minimal Image Sets (15323) 
Extending Intelligent Tutoring
Beyond the Desktop to the
Psychomotor Domain (15029) 
Delivering 3D Virtual
Maintenance Training Content:
Examining the Deployment
Options (15239) 
S320D
ED-7 It’s All About the
Learning
Mark Friedman
Using a Skill Acquisition Theory
as a Framework for the Army
Learning (15125) 
Transmedia (Social) Learning
in the Wild: DoD SkillBridge for
Transitioning Service Members
(15162) 
S320E
H-7 Is That Your Final
Decision?
Eric Jarabak
Stealth Assessment of ProblemSolving Skills from Gameplay
(15212)  %
“Fixing” the Military DecisionMaking Process (15220) 
The Small Unit Decision
Making Assessment Battery:
Development and Psychometric
Analysis (15143) 
S320F
S-9 UAS Engineering Design
Simulation
Ron Dionne
Reducing Operational Risk
through Better Performance
Testing (15138) 
Design of an Educational Tool for
Unmanned Air Vehicle Design
and Analysis (15086) 
Using Simulation to Test MannedUnmanned Teaming (15112) 
S320A
S320B
S320C
ROOM
SESSION/CHAIR
S320A
T-6 ISR, CRM, TDM, Oh My! –
Different Domains,
Universal Strategies
Anne Little, Ph.D.
Improving Military Crew
Resource Management Using a
Commercial Strategy Game
(15097) ñ
Simulation and Training
Challenges for Intelligence,
Surveillance and Reconnaissance
Analysts (15175) 
Effectiveness of Process Level
Feedback at Training Tactical
Decision Making (15201)  
S320B
S-10 Preparation Through
Virtualization
Brent Terwilliger, Ph.D.
Design and Development of a
General Virtual Maintenance
Training Platform (15312)  ñ
Virtual Environment ComputerBased Training for Bridge and
Tunnel Inspections (15276) 
Developing Authoring Tools
for Skills Models that Enable
Adaptive Game-Based
Maintenance Training (15129) 
S320C
EC-10 To Game or Not To
Game
Stu Armstrong
Gamers Today, Surgeons
Tomorrow? (15235)  ñ h
Relationship Between Learner
and Environment: Learner Traits
in Serious Games (15092) 
Learning Stories: Design
Considerations for Narrative
Elements in Serious Games
(15303)  %
ED-8 Make a Mobile
Ellen Menaker, Ph.D.,
Innovative Mobile Technologies
for Assessing and Enhancing
Soldier Performance (15082) 

A Reference Model for
Designing Mobile Learning and
Performance Support (15225)

Development and Evaluation
of Mobile Adaptive Training
Technologies (15231) 
H-8 From the Halls of
Montezuma
Jerry Stahl
MarineNet User Engagement
Exercise (15011) 
Supporting Unit Training
Management Through Mobile
Performance Assessment Tools
(15034) 
Marine Corps Instructor Master
Model: A Foundation for Marine
Faculty Professional Development
(15146) 
S-11 Soldier Representation
Lisa Jean Bair
Toward Acquiring a Human
Behavior Model of Competition
vs. Cooperation (15316) 
Embedded Simulation to Prevent
Tactical Surprise and Improve
Soldier Performance (15054) 
Distributed Soldier
Representation: Improving M&S
Representation of the Soldier
(15123) 
S320D
S320E
S320F
1030
1100
1130
56
Ï Cyber Security
Mobile
THURSDAY, 3 DECEMBER •  All Papers are eligible for CEU/CLP credits. (See page 12) 
ROOM
SESSION/CHAIR
1330
1400
1430
Procedural Reconstruction of
Simulation Terrain Using Drones
(15041) 
Assessment of Unmanned
Aircraft Platform Performance
Using Modeling and Simulation
(15006) 
S320B
S-12 The Business of
Simulations
Randy Crowe, Ph.D.
A MBSE Approach in Modeling
Systems Using Hybrid Simulation
Techniques (15368) 
Simulator Architecture Upgrade
Utilizing Virtual Machines (VMs)
(15219) 
Improved Process for Bridging
the Technology Transition Valley
of Death (15103) 
S320C
EC-12 A Little Design Will
Do Ya
Susan Harkrider
Early Synthetic Prototyping:
When We Build It, Will They
Come? (15187) 
Implementation of Agile Methods
within Instructional Systems
Design: A Case Study (15094) 
Development and Evaluation of
a Venipuncture and Phlebotomy
Training Ssystem (15084)  h
H-9 Experience Counts
Randy Jensen
Realism and Effectiveness of
Robotic Moving Targets (15118)

Soldier Physiological Changes
of Shooting Performance in the
Tank Simulator (15192)  ñ
Novice and Experience Police
Officer Simulation Experience:
Guiding the Future (15370)  ñ
S320A
S320E
PAPER SESSIONS
EC-11 Droning On
John Aughey
Reception
Awards
banquet
&
T H U R S D AY E V E N I N G
HYATT REGENCY
WINDERMERE BALLROOM
WE WILL NOT PROVIDE SHUTTLE SERVICE TO/FROM THE CLOSING BANQUET.
IF YOU HAVE A SPECIAL NEED PLEASE CONTACT BMCDANIEL@NDIA.ORG.
BANQUET TICKET REQUIRED
Ï Cyber Security
Mobile
57
 All Papers are eligible for CEU/CLP credits. (See page 12) 
B E S T PA P E R S
E DU CATIO N
B P - 1 WED NES DAY, 2 DEC EMB ER • 1030 -1200 • S320A
The Best Paper for this category will be presented
on Wednesday in Room S320A at 1030
PAPERS AND AUTHORS
Best Papers from Education, Training, Human
Performance Analysis and Engineering
Session Chair: Anne Little, Ph.D., ADDX Corporation
Session Deputy:Sowmya Ramachandran, Ph.D., Stottler Henke
Associates, Inc.
Education: Stress Exposure Training for the Dismounted
Squad: The Human Dimension (15150)   Jay Brimstin,
Ph.D., U.S. Army Maneuver Center of Excellence; SGM
Higgs, Senior Enlisted Officer at PEO STRI; Rob Wolf,
Brian Kemper PM TRADE; Rob Parrish, PM CATT; Dr.
Joan Johnston, Pat Garrity, ARL HRED STTC; Jim Moon,
Chris Jacques, Milton Fields, Sam Rhodes, Fort Benning
MCoE; Anita Zabek, Paul Butler, Rick Osborne, Patrick
Ogden, MITRE; Bill Ross, Brandon Woodhouse, Cognitive
Performance Group
Training: Differentiating Measures of Learning (MOL) from
Measures of Performance (MOP) During Aircraft Carrier
Landing Practice (15210)   Jeffrey M. Beaubien, Ph.D.,
E. Webb Stacy, Ph.D., Sterling M. Wiggins, Michael J.
Keeney, Ph.D., Aptima, Inc.; Amy Bolton, Ph.D., Office
of Naval Research; LCDR Jefferson D. Grubb, Ph.D.,
USN, Naval Aviation System Program Office; Melissa M.
Walwanis, Heather A. Priest, Ph.D., NAWCTSD; Christian S.
Riddle, NAVAIRSYSCOM Manned Flight Simulator
Human Performance Analysis and Engineering: You Cannot Hit
What You Do Not Shoot (15209)   Martin Bink, U.S.
Army Research Institute; Elizabeth Uhl, Ph.D., ARL HRED
STTC; David James, Northrop Grumman Corporation
B P - 2 W ED NES DAY, 2 DEC EMB ER • 1400 -1530 • S320A
Best Papers from Emerging Concepts and Innovative
Technologies; Simulation; Policy Standards
Management and Acquisition
Session Chair: Tara Kilcullen, Raydon Corporation
Session Deputy:Karen Williams, U.S. Army PEO STRI
Emerging Concepts and Innovative Technologies: Safe Testing of
Autonomous Systems Performance (15348)   David
Scheidt, Robert Lutz, William D’Amico, Ph.D., Dean
Kleissas, Robert Chalmers, Robert Bamberger, Johns Hopkins
University Applied Physics Laboratory
Simulation: Cyber Operational Architecture Training System –
Cyber for All (15108)   Ï David “Fuzzy” Wells, Ph.D.,
Derek Bryan, USPACOM J81
Policy Standards Management and Acquisition: Measuring
Virtual Simulation’s Value in Training Exercises – USMC
Use Case (15114)   Nathan Jones, Greg Seavers,
MARCORSYSCOM PM TRASYS; Christin Capriglione,
NAWCTSD
STRESS EXPOSURE TRAINING FOR THE DISMOUNTED SQUAD:
THE HUMAN DIMENSION (15150)
ED -1
TUESD AY, 1 D ECEM BER • 140 0 -1 5 3 0 • S3 2 0 D
Assessment and Evaluation: Isolating Effects
Session Chair: Robert “Buddha” Snyder, WBB, Inc.
Session Deputy:Sally Carter, AETC
Student Retention in STEM Career Paths: Primary Influences
on the Decision to Stay or Leave (15018)  Jennifer Winner,
LT Christopher Faxon, USAF, AFRL; Michael D. Coovert
Ph.D., University of South Florida Tampa
Overcoming the Challenge of Evaluating Skills Transfer from
Training to Job (15151)  Toumnakone (Annie) Hester, Jay
Brimstin, Ph.D., Maneuver Center of Excellence Fort Benning
Measuring a Moving Target: Validating Deployed Training
Courses (15189)  Timothy R. Brock, Ph.D.; Denise R.
Stevens, Ed.D., General Dynamics Information Technology
ED -2
TUESD AY, 1 D ECEM BER • 160 0 -1 7 3 0 • S3 2 0 D
Navigating the Social and Educational Terrain
Session Chair:Martin Bink, Ph.D., U.S. Army Research
Institute
Session Deputy:Anthony Carboniari, MARCORSYSCOM PM
TRASYS
Enhancing Good Stranger Skills: A Method and Study (15071)
 Robert Hubal, Mike van Lent, Ph.D., Bob Marinier, Ph.D,
Chris Kawatsu, Bob Bechtel, Ph.D., Soar Technology, Inc.
Curriculum GPS: An Adaptive Curriculum Generation
and Planning System (15369)  Mustafa Ilhan Akbas,
Prateek Basavaraj, Özlem Garibay, Ivan Garibay, Michael
Georgiopoulos, University of Central Florida
ED -3
W ED N ESD AY, 2 D ECEM BER • 083 0 -1 0 0 0 • S3 2 0 D
Full STEM Ahead
Session Chair: Nina Deibler, Serco Inc.
Session Deputy:Tiffany Parrish, NAWCTSD
Mars Game: Creating and Evaluating an Engaging
Educational Game (15105)  Kevin Dill, Spencer Frazier,
Lockheed Martin; Barbara Freeman Ed.D., UC Berkeley; Juan
Benito, Cooperative Entertainment Inc.
The Secret for STEM Success: Employing Technology for
Math Proficiency (15145)   Edward P. Harvey, Jr.,
Advanced Training & Learning Technology, LLC; Marvin G.
Fuller, Ph.D., Oglethorpe Charter School; Edward P. Harvey,
III, Harvard University
58
Ï Cyber Security
Mobile
ED -7
TH URSD AY, 3 D ECEM BER • 0830-1000 • S3 2 0 D
It’s All About the Learning
Session Chair: Mark Friedman, Adayana Government Group
Session Deputy:Claudia Clark, Ph.D., NETC
PAPERS AND AUTHORS
Inquiry and Design Approach to STEM Education Using
Project-based Learning (15238)  Danielle McNeely, Robert
Seltzer, NAWCTSD; Stephen Priselac, Ed.D., Nancy Priselac,
Ed.D., nCASE; Heather Norton, Alicia Frascati, Orlando
Science Center; Susan Nelson, AUVSI Foundation; Eileen
Smith, University of Central Florida; Abdul Siddiqui, U.S.
Army PEO STRI
Using a Skill Acquisition Theory as a Framework for the
Army Learning (15125)  LTC Glenn A. Hodges, Ph.D.,
U.S. Army, TRADOC
E D-4 WED N ES D AY, 2 DEC EMB ER • 1030- 1200 • S 3 20D Transmedia (Social) Learning in the Wild: DoD SkillBridge
for Transitioning Service Members (15162)  Elaine
Getting In Front of It
M. Raybourn, Ph.D., Sandia National Laboratories/ADL
Session Chair: Jay White, Department of Homeland Security
Initiative; Frank C. DiGiovanni, SES, Force Readiness and
Session Deputy:Thomas Archibald, IDSI
Training, OASD(R); MGen Tom Jones, USMC (Ret.), Outdoor
Antecedents of Adaptive Collaborative Learning
Odyssey
Environments (15211)  Robert A. Sottilare, Ph.D., Joan H.
Johnston, Ph.D., Anne M. Sinatra, Ph.D., ARL HRED STTC;
ED -8
TH URSD AY, 3 D ECEM BER • 1030-1200 • S3 2 0 D
Shawn Burke, Ph.D., University of Central Florida; Eduardo
Make a Mobile
Salas, Ph.D., Rice University; Heather Holden, Ph.D., Mount
Session Chair: Ellen Menaker, Ph.D., IDSI
Washington University
Session Deputy:Liz Gehr, Ph.D., The Boeing Company
Adaptable Resilience Training for Transitioning Veterans
Innovative Mobile Technologies for Assessing and Enhancing
Using Existing Technologies (15285)  Jill Sheperd, Jennifer
Soldier Performance (15082)   Krista L. Ratwani,
Ukwa, Booz|Allen|Hamilton
Ph.D., Courtney R. Dean, Camilla Knott, Ph.D., Frederick
The Changing Face of Military Learning (15327)  Sae
Dietrich, Ph.D., Aptima, Inc.; Scott Flanagan, Sophia Speira;
Schatz, Ph.D., Advanced Distributed Learning; David
Jennifer S. Tucker, Ph.D., U.S. Army Research Institute
Fautua, Ph.D., Joint Staff J7; Julian Stodd, SeaSalt Learning;
A Reference Model for Designing Mobile Learning and
Emile Reitz, Alion Science and Technology
Performance Support (15225)  Peter Berking, Jason
Haag, Advanced Distributed Learning Initiative
E D-5 WED N ES D AY, 2 DEC EMB ER • 1400- 1530 • S 3 20D
Development and Evaluation of Mobile Adaptive Training
Challenging the Game
Technologies (15231)  Rodney Long, ARL HRED STTC;
Session Chair: Adelle Lynch, Rockwell Collins
Jessie Hyland, Joanne Barnieu, ICF International
Session Deputy:Brian Stensrud, Ph.D., Soar Technology, Inc.
Game-Based Training for Human-Intelligence Skills (15067)
 % John T. Miller, II, Consortium Research Fellows
Program and Capella University; Martin L. Bink, Ph.D., U.S.
Army Research Institute
A Conceptual Model of Pedagogic Design to support Critical
Thinking in Commanders (15075)  % ñ Kia Hong Tan,
Teng Howe Lim, Boon Kee Soh, DSO National Laboratories
E D-6
WED N ES D AY, 2 DEC EMB ER • 1600- 1730 • S 3 20D
Instructors for the Force of the Future
Session Chair: Jim Threlfall, C2 Technologies, Inc.
Session Deputy:David Fautua, Joint Staff J7
Designing Instructor Support Tools for Virtual Shiphandling
Training (15133)  Martin Voshell, Ph.D., Ryan Kilgore,
Ph.D., Christopher Hogan, David Young, Charles River
Analytics
Hey, Remember to Add Motivational Design to Your
E-learning (15030)  ñ Geir Isaksen, CDR, NoD University
College; Siren Elise Frøytlog Hole, Transform AS
Achieving Educational Excellence: What do Effective
Instructors do? (15226)  Heidi Keller-Glaze, Ph.D.,
Jonathan Bryson, Ray Morath, Ph.D., ICF International; U.S.
Army Research Institute
E ME R G ING C O NC E PT S &
INNO VATIVE TE C H NO LO G IE S
SAFE TESTING OF AUTONOMOUS SYSTEMS PERFORMANCE
(15348)
EC-1
TUESD AY, 1 D ECEM BER • 1400-1530 • S3 2 0 C
Easy Button
Session Chair: John Dzenutis, The Boeing Company
Session Deputy:Susan Harkrider, RDECOM CRDEC NVESD
Automated Simulation Creation from Military Operations
Documents (15227)  John Balint, Ph.D., Jan M. Allbeck,
Ph.D., Michael R. Heib, Ph.D., George Mason University
Scheduling Training to Manage Acquisition & Decay (15340)
 Mohammed Eslami, Ph.D., Jared Freeman, Ph.D., Scott
Pappada, Ph.D., Aptima, Inc.
M&S as a Service: Paradigm for Future Simulation
Environments (15324)  ñ Robert Siegfried, aditerna
GmbH; Tom van den Berg, TNO
Ï Cyber Security
Mobile
59
EC - 2 T U ES DAY, 1 DEC EMB ER • 1400-1530 • S330C EC-5
W ED N ESD AY, 2 D ECEM BER • 083 0 -1 0 0 0 • S3 2 0 C
PAPERS AND AUTHORS
Are We There Yet?
Fidelity Matters
Session Chair: Scott Ariotti, The DiSTI Corporation
Session Deputy:Brian Overy, Diamond Visionics
Session Chair: Eric Weisel, Ph.D., Old Dominion University
Session Deputy:Paul Bogard, USAF Simulators Division
Modelling a Helicopter Training Continuum to Support
System Transformation (15165)  ñ Michael Johnstone,
Ph.D., Vu Le, Ph.D., Burhan Khan, Doug Creighton, Ph.D.,
Center for Intelligent Systems Research; Ana Novak,
Ph.D., Vivian Nguyen, Luke Tracey, Defence Science and
Technology Organisation
Reliably Assessing the Effectiveness of a Plan with Models
of Verying Fidelity and Under Time Constraints (15060) 
ñ Steven de Jong, Ph.D., Wouter Noordkamp, Nick van der
Poel, Selmar Smit, Ph.D., TNO Defence, Safety & Security
Military Vehicle Training with Augmented Reality (15180)
 Jonathan Brookshire, Ph.D., Taragay Oskiper, Ph.D.,
Vlad Branzoi, Supun Samarasekera, Rakesh Kumar, Ph.D.,
SRI International; Sean Cullen, Richard Schaffer, Lockheed
Martin Mission Systems and Training
EC - 3 T U ES DAY, 1 DEC EMB ER • 1600-1730 • S320C
Pinging and the Brain
Requirements for Future SAFs: Beyond Tactical Realism
(15193)  Robert E. Wray, Ph.D., Soar Technology, Inc.;
Heather A. Priest, Ph.D., Melissa M. Walwanis, Katherine
Kaste, NAWCTSD
Required Fidelity of simulated Wound at the Point of Injury
(15351)  h M. Beth H. Pettitt, ARL HRED STTC
EC-6
W ED N ESD AY, 2 D ECEM BER • 083 0 -1 0 0 0 • S3 2 0 F
It’s All About Veterans
Session Chair: Luis Velazquez, MARCORSYSCOM SIAT
Session Deputy:Michael Finnern, Engility Corporation
Session Chair:Brian Holmes, The AEgis Technologies Group,
Inc.
Session Deputy:Connie Perry: U.S. Army PEO STRI
Cognitive Two-Way Interactions In an Immersive Virtual
Reality Environment (15332)   Brennan D. Cox, Ph.D.,
Harvey M. Edwards, Kathrine A. Service, Pinata H. Sessoms,
Ph.D., Jose A. Dominguez, Ph.D., Weimin Zheng, Ph.D.,
Naval Health Research Center; Seth A. Reini, Ph.D., Navy
Experimental Diving Unit
Virtual Interview Training Increases Job Offers for Veterans
and Others (15013)  Dale E. Olsen, Ph.D., Laura B
Humm, SIMmersion; Matthew J. Smith, Ph.D., Michael
Fleming, M.D., Neil Jorden, Ph.D., Northwestern University
Feinberg School of Medicine
Modeling and Integrating Cognitive Agents Within the
Emerging Cyber Domain (15232)  Randolph M. Jones,
Ph.D., Ryan O’Grady, Denise Nicholson, Ph.D., Soar
Technology
Stepping Stones – An Augmented Reality Rehabilitation
Game (15181)  Stuart Armstrong, QinetiQ Inc.
The VA Virtual Medical Center: Implementing a Vision for a
Virtual Healthcare Campus for our Veterans (15358)  h
Rosalyn P. Scott, M.D., Terry L. Oroszi, Brian V. Burke,
Tablet Computer Call for Fire Simulation Proof of Concept
M.D., Cathy D. Graham, Ph.D., Dayton VA Medical Center;
Study Results (15008)  James Reynolds, USMC
Nancy Benton, Ph.D., R.N., Mann-Grandstaff VA Medical
TECOM; Craig Smith, USMC II MEF
Center; Paul T. Ingmundson, Ph.D., South Texas Veterans
Health Care System, Jenni Gallimore, Ph.D., Wright State
EC - 4 TU ES DAY, 1 DEC EMB ER • 1600 -1730 • S330C University; Helga Carabello, VA Portland Health Care
Avatars Crossing
System; Sean M. McCoy, Ph.D., Veterans Rural Health
Session Chair: James (Josh) Jackson, SAIC
Resource Center-Eastern Region; Manny Dominguez, Ph.D.,
Session Deputy:Paul Thurkettle, NATO HQ SACT
VHA Employee Education System; Mary E. Davidson,
Healthcare System of Ohio
Turn-Based Gaming for Convoy Commander Training
(15036)  ñ % Rudy Boonekamp, Tijmen Muller, TNO;
EC-7 W ED N ESD AY, 2 D ECEM BER • 103 0 -1 2 0 0 • S3 2 0 C
Jur de Vrijer, SIMCEN RNA
An Immersive Live / Virtual Bridge Approach with Ultra
Wideband Tracking Technology: Phase II (15024)  Jay
Saffold, Tovar Shoaf, Jason Holutiak, Reseach Network, Inc.;
Pat Garrity, Timothy Roberts, ARL HRED STTC
Virtualizing Humans for Game Ready Avatars (15023) 
Jay Saffold, Tovar Shoaf, Jason Houliak, Research Network,
Inc.; Timothy Roberts, Pat Garrity, ARL HRED STTC
Let’s Get Physiological
Session Chair:Jennifer Murphy, Ph.D., Quantum
Improvements Consulting, LLC
Session Deputy:Jefferson Grubb, NAVAIRSYSCOM
Visualizing fMRI Data Using Volume Rendering in Virtual
Reality (15253)  Joseph Holub, Ph.D., Eliot Winer, Ph.D.,
Iowa State University
Professional Soldier Assessment of a Rifle-Mounted Target
Hand-Off System (15039)  ñ Jerome Levesque, Ph.D.,
Katherine Banko, Ph.D., Defence Research & Development
Canada; Olaf Binsch, Ph.D., Netherlands Organization for
Applied Scientific Research
60
Ï Cyber Security
Mobile
E C-8
Relationship Between Learner and Environment: Learner
Traits in Serious Games (15092)  Marvin G. Fuller, Ph.D.,
Oglethorpe Charter School; Dennis Beck, Ph.D., University
of Arkansas
PAPERS AND AUTHORS
Empirical Support for Brain-Based Assessment in
Simulation-Based Training (15300)  Kevin B. Oden,
Ph.D., Kelly L. Phillips, Lockheed Martin Mission Systems
and Training; Kurtulus Izzetoglu, Ph.D., Patrick Craven,
Ph.D., Hasan Ayaz, Ph.D., Gabriela Hernandez, Drexel
University
Learning Stories: Design Considerations for Narrative
Elements in Serious Games (15303)  % Michael W.
Freeman, MW Freeman Solutions; Mark Friedman, Adayana,
W ED N ES D AY, 2 DEC EMB ER • 1600- 1730 • S 320C Inc.
Experience with the Experience API
Session Chair: Nick Giannias, CAE
Session Deputy:Keith Biggers, Ph.D., Texas A&M Engineering
Experiment Station
Putting Live Firing Range Data to Work Using the xAPI
(15019)  Paula J. Durlach, Ph.D., ARL HRED STTC;
Nick Washburn, Riptide Software, Inc.; Damon Regan, The
Tolliver Group, Inc./ADL Initiative
Adapting Gunnery Training Using the Experience API
(15179)  Rodney Long, ARL HRED STTC; Jennifer S.
Murphy, Ph.D., Carolyn Newton, Quantum Improvements
Consulting; Michael Hruska, Ashley L. Medford, Problem
Solutions; Tara Kilcullen, Robert L. Harvey, Jr., Raydon
Corporation
Opening Legacy Data Silos: Using Experience Data for
Educational Impact (15043)  Jonathan Poltrack, Tom
Creighton, Problem Solutions/ADL Initiative
E C-9
TH URSD AY, 3 D ECEM BER • 1330-1500 • S3 2 0 A
Droning On
Session Chair: John Aughey, The Boeing Company
Session Deputy:Joan Johnston, Ph.D., ARL HRED STTC
Procedural Reconstruction of Simulation Terrain Using
Drones (15041)  Ryan McAlinden, Evan Suma Ph.D.,
Timofey Grechkin, Ph.D., USC Institute for Creative
Technologies; Michael Enloe, National Simulation Center
Assessment of Unmanned Aircraft Platform Performance
Using Modeling and Simulation (15006)  Brent
Terwillinger, Ph.D., Dennis Vincenzi, Ph.D., David Ison,
Ph.D., Todd Smith, Ph.D., Embry-Riddle Aeronautical
University Worldwide Campus
EC-12
TH URSD AY, 3 D ECEM BER • 1330-1500 • S3 2 0 C
A Little Design Will Do Ya
Session Chair: Susan Harkrider, RDECOM CERDEC NVESD
T H U R S D AY, 3 DEC EMB ER • 0830- 1000 • S 320C Session Deputy: Ryan McAlinden, USC/ICT
3D Psycho
Session Chair: John Dzenutis, The Boeing Company
Session Deputy:Ba Duong, MARCORSYSCOM PM TRASYS
Rapid 3D Geospatially Oriented Structure Extraction from
Minimal Image Sets (15323)  R. Scott Starsman Ph.D.,
Avineon, Inc.
Extending Intelligent Tutoring Beyond the Desktop to the
Psychomotor Domain (15029)  Robert A. Sottilare, Ph.D.,
ARL HRED STTC; Joseph LaViola, Ph.D., University of
Central Florida
Delivering 3D Virtual Maintenance Training Content:
Examining the Deployment Options (15239)  Christopher
Van Duyne, Scott Ariotti, The DiSTI Corporation
E C-10
EC-11
Early Synthetic Prototyping: When We Build It, Will
They Come? (15187)  LTC Brian Vogt, USA, U.S. Army
TRADOC; Michael Megiveron, Robert E. Smith, Ph.D., U.S.
Army TARDEC
Implementation of Agile Methods within Instructional
Systems Design: A Case Study (15094)  Lisa Cooney,
Anne Little, Ph.D., Addx Corporation
Development and Evaluation of a Venipuncture and
Phlebotomy Training System (15084)  h Teresita M.
Sotomayor, Ph.D., ARL HRED STTC; Angela M. Alban,
SIMETRI, Inc.
T H U R S D AY, 3 DEC EMB ER • 1030- 1200 • S320C
To Game or Not To Game
Session Chair: Stu Armstrong, QinetiQ, Inc.
Session Deputy:Leslie Dubow, Veterans Health Administration
SimLEARN
Gamers Today, Surgeons Tomorrow? (15235)  ñ h Alyssa
Tanaka, Courtney Graddy, Roger Smith, Ph.D., Florida
Hospital Nicholson Center; Manuela Perez, Ph.D., Nancy
University Hospital
Ï Cyber Security
Mobile
61
H UMA N P E R F O R M A NC E
ANA LYS I S A N D E N G I N EE R ING
PAPERS AND AUTHORS
Gregory A. Fabiano, Ph.D., Department of Counseling,
School and Educational Psychology; Mark G. Frank, Ph.D.,
Department of Communications; Rebecca J. Houston, Ph.D.,
Research institute on Addictions; Panos Ch. Anastasopoulos,
Department of Civil, Structural and Environmental
YOU CANNOT HIT WHAT YOU DO NOT SHOOT (15209)
Engineering, The State University of New York-University at
H-1
T U ES DAY, 1 DEC EMB ER • 1400 -1530 • S320E Buffalo
This Won’t Hurt a Bit
Session Chair:Robert Hester, Ph.D., University of Mississippi
Medical Center
Session Deputy:Ingrid Mellone, Camber Corporation
Measuring Trust of Autonomous Vehicles: A Development
and Validation Study (15049)  David R. Garcia, Christine
Kreutzer, Karla A. Badillo-Urquiola, University of Central
Florida, Psychology Department
Piloting a Groundbreaking Virtual Continuing Competency
Platform: Results and Recommendations (15325)  % h
Jennifer McNamara, BreakAway Games; Paul Grace,
Margaret Bent, Ph.D., National Board for Certification in
Occupational Therapy, Inc.
Building Trust in a Human-Robot Team with Automatically
Generated Explanations (15315)  Ning Wang, Ph.D.,
David V. Pynadath, Ph.D., University of Southern California
Mobile App Design for Veterans with Physical and Cognitive
Limitations (15182)   h Nina P. Deibler, Lea G.
Blake, Serco, Inc.; Devin Harrison, William Plew, U.S.
Department of Veterans Affairs
It’s a Wild, Wild Cyber World
Structured Development of Interventions to Improve
Physician Knowledge Retention (15095)  h Lloyd
Werk, M.D., Maria Carmen Diaz, M.D., James P. Franciosi,
M.D., Tim Wysocki, Nemours Children’s Health System,
Jacksonville; Lorie Ingraham, James Crutchfield, Ph.D.,
Lockheed Martin Mission Systems and Training
Automated Performance Assessment in Cyber Training
Exercises (15044)  Ï Robert G. Abbott, Ph.D., Jonathan
McClain, Benjamin Anderson, Kevin Nauer, Austin Silva,
Chris Forsythe, Ph.D., Sandia National Laboratories
H-2 W ED N ESD AY, 2 D ECEM BER • 103 0 -1 2 0 0 • S3 2 0 E
Session Chair: Jennifer Arnold, Booz|Allen|Hamilton, Inc.
Session Deputy:Elaine Raybourn, Ph.D., Sandia National
Laboratories, ADL Initiative
Command Shift: Exploring Modern Gaming Technologies
to Create Next-Generation Offensive Cyber Operations
(OCO) Interfaces (15091)   % Ï Chad Caison, KEYW
T U ES DAY, 1 DEC EMB ER • 1600 -1730 • S320E
Corporation; Jennifer McNamara, BreakAway Games
Analyze This
Session Chair: Bill Gerber, Ph.D., WJ Gerber Consulting
Session Deputy:Matthew Hackett, Ph.D., ARL HRED STTC
Work Domain Analysis for Ecological Interface Design of
Tangible Interfaces (15130)  Michael W. Boyce, Ph.D.,
Robert A. Sottilare, Ph.D., Benjamin Goldberg, Ph.D.,
Charles Amburn, ARL HRED STTC
Adaptive Testing: Adapt and Overcome the Shortfalls of
Traditional Proficiency Assessments (15196)  Robert
McLaughlin, Steven Gunter, Ph.D., Camber Corporation; Jeff
Pearson, Veterans Benefits Administration
Cognitive Load Assessment for Intelligence Analysts through
Full Motion Video Analytics (15142)  Elizabeth Wilson,
Chenega Technical Innovations, LLC; Upesh Patel, U.S. Army
Intelligence
H-3
H -4
WED NES DAY, 2 DEC EMB ER • 0830-1000 • S320E
Do You Understand?
Session Chair: John Schlott, L-3 Communications
Session Deputy:Todd Glenn, FAAC, Inc.
Embedded Cyber-Physical Systems for Assessing Performance
in Training Simulations (15263)  Ï P. Shane Gallagher,
Ph.D., ADL Initiative; Brenda Bannan, Ph.D., Bridgett Lewis,
George Mason University; Shelly Blake-Plock, Yet Analytics
H -5
W ED N ESD AY, 2 D ECEM BER • 140 0 -1 5 3 0 • S3 2 0 E
Maintaining Your Cool
Session Chair:Elaine Raybourn, Ph.D., Sandia National
Session Deputy:Amit Kapadia, U.S. Army PEO STRI
Human Performance in Content Design for Interactive
Augmented Reality Systems (15156)  Louise Yarnall,
Ph.D., Anna Werner, Rakesh “Teddy” Kumar, Ph.D., Supun
Samarasekera, Girish Acharya, Glenn Murray, Michael
Wolverton, Zhiwei Zhu, Vlad Branzoi, Nicholas Vitovitch,
Jim Carpenter, SRI International
Using Micro and Macro Studies of Tablets to Improve
Maintenance (15279)  Robert Pokorny, Ph.D.,
Jacqueline Haynes, Ph.D, Lisa Holt, Ph.D., Intelligent
Automation, Inc.; Michael diPilla, NAWC Carderock
Multi-measure Assessment of Internal Distractions on
Driver Performance (15017)  Kevin F. Hulme, Ph.D.,
Center for Engineering Design and Applied Simulation;
Karen L. Morris, The Center for Children and Families;
62
Ï Cyber Security
Mobile
H-6
WED N ES D AY, 2 DEC EMB ER • 1600- 1730 • S 320E H -9
TH URSD AY, 3 D ECEM BER • 1330-1500 • S3 2 0 E
Experience Counts
Session Chair: Dennis Vincenzi, Ph.D., ERAU Worldwide
Session Deputy:Maureen Bergondy-Wilhelm, NAWCTSD
Session Chair: Randy Jensen, Stottler Henke Associates, Inc.
Session Deputy:Dennis Vincenzi, Ph.D., ERAU Worldwide
Helicopter Pilot’s Modeling Including the Stress Factor
(15168)  ñ Antoni Kopyt, Warsaw University of
Technology
Realism and Effectiveness of Robotic Moving Targets
(15118)  Elizabeth Uhl, Ph.D., Martin Bink, Ph.D., U.S.
Army Research Institute; David James, Northrop Grumman
Technical Services
Human-in-the-Loop Flight Simulation Study of Virtual
Constructive Representation on Live Avionics Displays
(15197)  Thomas Schnell, Ph.D., University of Iowa;
Angus L.M. Thom McLean, Ph.D., Scott Rediger, Rockwell
Collins Advanced Technology Center
Practical Recommendations for Validating Survey Apparatus
in Coalition Training Environments (15299)  Emilie Reitz,
Alion Science and Technology
H-7
T H U R S D AY, 3 DEC EMB ER • 0830- 1000 • S320E
Is That Your Final Decision?
Session Chair: Eric Jarabak, MARCORSYSCOM PM TRASYS
Session Deputy:Steve Monson, The Boeing Company
Stealth Assessment of Problem-Solving Skills from Gameplay
(15212)  % Weinan Zhao, Valerie Shute, Ph.D., Lubin
Wang, Florida State University
PAPERS AND AUTHORS
Flight Life
Soldier Physiological Changes of Shooting Performance in the
Tank Simulator (15192)  ñ Li-Wei Ko, Ph.D., Peng-Wen
Lai, Yi-Cheng Shih, Shih-Chuan Lin, Meng-Shun Yang, ChinTeng Lin, Ph.D., National Chiao Tung University; In-Chung
Chang, MND Taipei
Novice and Experience Police Officer Simulation Experience:
Guiding the Future (15370)  ñ Amanda Davies, Ph.D.,
Charles Stuart University
PO LIC Y, STA NDA R DS,
MA NA G E ME NT, A ND A C Q U ISITION
MEASURING VIRTUAL SIMULATION’S VALUE IN TRAINING
EXERCISES – USMC USE CASE (15114)
“Fixing” the Military Decision-Making Process (15220) 
Michael J. Smith, Ronald B. Sprinkle, Leidos; COL Johnny
Powers, U.S. Army PEO STRI; James Xu, Adayana; Michael
Knapp, Aptima, Inc.
P-1
TUESD AY, 1 D ECEM BER • 1400-1530 • S3 2 0 F
The Small Unit Decision Making Assessment Battery:
Development and Psychometric Analysis (15143)  Karol
G. Ross, Ph.D., Jennifer K. Phillips, Cognitive Performance
Group; Kenneth A. Knarr, II Corps Consultants, Inc.
The Live-Synthetic Training and Test & Evaluation
Enterprise Architecture (15076)   Paul Dumanoir, U.S.
Army PEO STRI; Jeff Bergenthal, David Drake, the Johns
Hopkins University Applied Physics Laboratory
Enterprise, Architecture, and Standards
Session Chair: Jan Drabczuk, JD Defense Solutions, LLC.
Session Deputy:Harry Sotomayor, U.S. Army PEO STRI
Early Adoption of Common Operating Environment (COE)
T H U R S D AY, 3 DEC EMB ER • 1030- 1200 • S 320E Standards and Guidelines (15098)  Robert Wittman,
From the Halls of Montezuma
Ph.D., Burt Grippin, Sean Barie, Chris Holmes, MITRE
Session Chair: Jerry Stahl, Cypress International
Corporation; Amit Kapadia, Paul Dumanoir, U.S. Army PEO
Session Deputy:Sherrie Jones, Ph.D., MARCORSYSCOM PM
STRI
TRASYS
P-2
TUESD AY, 1 D ECEM BER • 1600-1730 • S3 2 0 F
MarineNet User Engagement Exercise (15011) Maj Michael
A. Gavin, USMC, Marine Corps University, College of
Design, Build, Track and Train – Here and Abroad
Distance Education & Training
Session Chair:Mary Trier, Capital Communications and
Consulting
Supporting Unit Training Management Through Mobile
Session Deputy:Richard Grohs, USAF HQ Air Combat Command
Performance Assessment Tools (15034)  Courtney
Dean, Matthew Puglisi, Jared Freeman, Ph.D., Aptima, Inc.
Sejong the Great Class DDGs: How ROK Navy Trained and
Embraced Them (15032)  ñ LCDR Junho Eum, Sangyoon
Marine Corps Instructor Master Model: A Foundation
Oh, Ph.D., Ajou University; CAPT Minsoo Yang, Republic of
for Marine Faculty Professional Development (15146) 
Korea Navy Headquarters
Jennifer J. Vogel-Walcutt, Ph.D. Jennifer K. Phillips, Karol
G. Ross, Ph.D., Cognitive Performance Group, Kenneth A.
A System-Model-Centric Collaborative Environment for the
Knarr, II Corps Consultants, Inc.
Acquisition Lifecycle (15093)  James E. Coolahan, Ph.D.,
Coolahan Associates, LLC.; Jeffery J. Bergenthal, John
Hopkins University Applied Physics Laboratory
H-8
Ï Cyber Security
Mobile
63
P-3 SIMU LAT IO N
WED NES DAY, 2 DEC EMB ER • 1030-1200 • S320F
Developing Competency – The Future for M&S
PAPERS AND AUTHORS
Session Chair: Jeffrey A. Raver, SAIC
Session Deputy:Tara Kilcullen, Raydon Corporation
Improving Education, Training and Career Advancement
through Competency Portability (15117)   Robby
Robson, Ph.D., Eduworks Corporation
STEMulating: An Integrated Approach to Cultivating Our
Future (15270)  Elizabeth Biddle, Ph.D., Central FL STEM
Education Council; Carol Ann Dykes, University of Central
Florida; Shawn Harrs, Ph.D., Universal Orlando Resort;
Robert Seltzer, NAWCTSD; Abdul Siddiqui, U.S. Army PEO
STRI
Modeling and Simulation Professionals – Meeting the
Demand (15342)  Lisa Jean Bair, James J. Jackson, SAIC
CYBER OPERATIONAL ARCHITECTURE TRAINING SYSTEM –
CYBER FOR ALL (15108)
S-1 TUESD AY, 1 D ECEM BER • 140 0 -1 5 3 0 • S3 2 0 B
Cutting Edge Training
Session Chair: Angela Alban, SIMETRI
Session Deputy:Mark Soodeen, CAE
Mobile Augmented Reality for Force-on-Force Training
(15223)   h Richard Schaffer, Sean Cullen, Laura
Cerritelli, Lockheed Martin; Rakesh Kumar, Ph.D., Supun
Samarasekera, Mikhail Sizintsev, Ph.D., Taragay Oskiper,
Ph.D., Vlad Branzoi, SRI International
Emergency Medical Card Augmented Reality: Training
Evaluation (15266)  h Christina L. Lacerenza, C.
P-4
W ED NES DAY, 2 DEC EMB ER • 1400-1530 • S320F Shawn Burke, Ph.D., David S. Metcalf, Ph.D., Shannon L.
Getting Your Money’s Worth
Marlow, University of Central Florida; Christine Allen, Mark
Session Chair: Larry Rieger, SAIC
Mazzeo, ARL HRED STTC
Session Deputy:Emilie Reitz, Alion Science & Technology
Real-Time Cutting of Organs with Scissors (15333)  h
Calculating Simulation-Based Training Value: Cost
Avoidance and Proficiency (15199)  Tim Cooley, Ph.D.,
DynamX Consulting; Greg Seavers, John Roth, Jose
Rodriguez, MARCORSYSCOM PM TRASYS; Steven Gordon,
Ph.D., Georgia Tech Research Institute
Large Scale Adoption of Training Simulations: Are We There
Yet? (15256)  Angela Sadagic, Ph.D., Naval Postgraduate
School; Lt Col Floy A. Yates, Jr., USMC, MAGTFTC Battle
Simulation Center
Automated Surveys: Lowering the Respondent’s Burden
(15080)  ñ Richard Kist, Igor Franken, National
Aerospace Laboratory (NLR)
P-5
W ED NES DAY, 2 DEC EMB ER • 1600 -1730 • S320F
Edouard Poutot, Aditya Bhatia, CAE, Inc., Healthcare Division
S-2
TUESD AY, 1 D ECEM BER • 160 0 -1 7 3 0 • S3 2 0 B
Getting to the Right Scenarios & Data
Session Chair:Roy Scrudder, The University of Texas at
Austin, Applied Research Laboratories
Session Deputy:Mike Flanagan, CACI-Federal
Virtual Battlespace Scenario Encoding for Reuse (15027) 
% Capt Michael J. Eady, USMC, USMC TECOM; LTC David
W. Parkes, USA, Joint Staff J7
Multi-Federate Scenario Development and Testing: “A
Good Plan, Violently Executed” (15249)  Donald C.
Meinshausen, Lockheed Martin; Mitchell Faircloth, SAIC
The Expected Results Method for Data Verification (15020)
Cybersecurity: New Threats, New Policies, New
Solutions
 Paul Monday, Lockheed Martin
Session Chair: Robby Robson, Ph.D., Eduworks Corporation
Session Deputy:Kevin Hulme, Ph.D., University at Buffalo
S-3 Risk Management Framework (RMF) Transition Impacts in
Training Simulation Systems (15009)  Ï Graham Fleener,
Marco Mayor, U.S. Army PEO STRI; Cliff Zou, Ph.D.,
Cybersecurity Controls: Then and Now (15010)  Ï Marco
Mayor, U.S. Army PEO STRI
Cybersecurity Challenges and Resolutions for Simulator
& Training Systems (15063)  Ï Douglas E. Wedel, Ilya
Lipkin, Lt. Luis Cintron, USAF
W ED N ESD AY, 2 D ECEM BER • 083 0 -1 0 0 0 • S3 2 0 B
Convergence of Testing & Training
Session Chair: Karen Williams, U.S. Army PEO STRI
Session Deputy:Favio Lopez, Trideum Corporation
Polygone LVC: The New Paradigm for EW Training (15213)
 Lt Col Scott Case, USAF Warrior Preparation Center
Detachment 3 Germany; Ryan McLaughlin, Northrop
Grumman
Live Synthetic Training and Test & Evaluation Infrastructure
Architecture (LS TTE IA) Prototype (15099)  Paul
Dumanoir, Mike Willoughby, U.S. Army PEO STRI; Brent
Grippin, Richard Crutchfield, Rob Wittman, Ph.D., Sean
Barie, MITRE Corporation
Measuring Realism in Simulations for Training and Testing
(15206)   Jerrit Askvig, Phil Hallenbeck, The MITRE
Corporation
64
Ï Cyber Security
Mobile
S-4 W ED N ES D AY, 2 DEC EMB ER • 1030- 1200 • S 320B On the Peripheral Application of Head Mounted Display
(HMD) Devices in Infantry Simulation (15186) ñ Tomer J.
Michael, IDF GFC Battle-Lab
Session Chair: Mark Soodeen, CAE
From Reality to Simulation
Network Bandwidth’s Effect on Virtual World Simulator
Performance Optimization (15360)  Sean C. Mondesire,
Ph.D., Douglas B. Maxwell, ARL HRED STTC; Jonathan
Stevens, Ph.D., University of Central Florida
3D Immersive Environment Using X-Plane for Depth
Perception Research (15261)  Logan A. Williams, Ph.D.,
Charles T. Bullock, Marc D. Winterbottom, Ph.D., James
P. Gaska, Ph.D., Steven C. Hadley, MD, USAF School
of Aerospace Medicine; Charles J. Lloyd, Ph.D., Visual
Performance, LLC; Michael P. Browne, Ph.D., SA Photonics
S-7 PAPERS AND AUTHORS
Session Deputy:Ron Dionne, FLETC
W ED N ESD AY, 2 D ECEM BER • 1600-1730 • S3 2 0 B
Next Gen LVC
Session Chair: Robert Kleinhample, SAIC
Session Deputy:Gordon King, Intelligent Decisions, Inc.
M&S Training Transformation: Bridging the Next Generation
Joint LVC (15167)  Bruce Uphoff, Camber Corportion;
Michael Koscielniak, Los Alamos National Laboratory;
Brian Gregg, Karl Hines, Joint Staff J7; John Mizelle, Daniel
Leigeber, Intelligent Decision Systems, Inc.
Leveraging Cloud Computing Technology for LVC Training
(15101)  Paul Dumanoir, U.S. Army PEO STRI; Henry
Battle Damage Computation Server (15051)  Hung Tran,
Marshall,
ARL HRED STTC; Robert Wells, Dynamic
Tactical Systems, CAE USA
Animation Systems; Jeff Truong, Effective Applications
S-5
WED N ES D AY, 2 DEC EMB ER • 1400- 1530 • S 320B Corporation
Osseus, An Experiment in What’s Next in LVC M&S
Innovation in Environmental Modeling
Architecture (15085)  Mark Riecken, Ph.D., John
Session Chair:Roy Scrudder, The University of Texas at
Rutledge, Trideum Corporation; Derrick Franceschini,
Austin, Applied Research Laboratories
StakFrame, LLC; Scott Gallant, Effective Applications; Walter
Session Deputy:Robert Dixon, U.S. Army PEO STRI
Barge, OSD Force Readiness and Training
High Fidelity Wind Model Software for Real-Time
Simulation Platforms (15362)  ñ Jaime Sanchez, Juan
S-8
TH URSD AY, 3 D ECEM BER • 0830-1000 • S3 2 0 B
Pelaez, Ph.D., SimSpace Ingenieria, S.L.
Automated Runtime Terrain Database Correlation
Assessment (15218)  Jeremy P. Joseph, Andrew Tosh,
GameSim, Inc.; Benito Graniela, Ph.D., NAWCTSD
Automated Modelization in Terrain Database Production
(15290)  Stephen Eckman, GameSim, Inc.; Ronald
Moore, Leidos; Mark Johnson, U.S. Army PEO STRI; Jaeson
Munro, PAI
Simulation Supported Training
Session Chair: Scott Hooper, Havok, an Intel Company
Session Deputy:Lisa Jean Bair, SAIC
Embarking on a Home Station Training Revolution (15176)
 Anthony J. Cerri, U.S. Army TRADOC; Mathew N.
McMillan, Alan J. Knox, CGI Federal
Implementation of Role-Based Command Hierarchy Model
for Actor Cooperation (15166)  ñ Jungyoon Kim, Ph.D.,
S-6 WED N ES D AY, 2 DEC EMB ER • 1400- 1530 • S 320C Hee-Soo Kim, Ph.D., REALTIMEVISUAL, Republic of Korea;
Sangjin Lee, Ph.D., Samjoon Park, Ph.D., Agency for
Human Interface to Artificial Intelligence
Defense Development, Republic of Korea
Session Chair:Gerald Dreggors, Northrop Grumman
Innovative Division/Brigade Level CO Training Solution for
Corporation
Influence Operations (15107)  ñ Ariane Bitoun, Romain
Session Deputy:Brad Cope, NAWCTSD
Bosa, MASA Group; Tahar Hannachi, Université Pierre
Modeling CGF Behavior with Machine Learning Techniques:
et Marie Curie; Lionel Khimeche, Direction Générale de
Requirements and Future Directions (15128)  ñ Armon
l’Armement
Toubman, Ph.D., Gerald Poppinga, Jan Joris Roessingh,
Ph.D., National Aerospace Laboratory NLR; Ming Hou,
Ph.D., Defence Research and Development Canada; Linus
Loutsinen, Ph.D., Swedish Defence Research Agency FOI
Rikke Amilde Løvlid, Ph.D., Norwegian Defence Research
Establishment (FFI); Christophe Meyer, Ph.D., Thales; Roel
Rijken, Dutch Ministry of Defence; Micha Turcanik, Ph.D.
Armed Forces Academy Liptovský Mikulàš, Slovakia
Automatic Speech Recognition in Training Systems:
Misconceptions, Challenges and Paths Forward (15205) 
Brian Stensrud, Ph.D., Charles Newton, Soar Technology,
Inc.; Beth Atkinson, John Killilea, NAWCTSD
Ï Cyber Security
Mobile
65
S-9 T H U R S DAY, 3 DEC EMB ER • 0830-1000 • S320F Distributed Soldier Representation: Improving M&S
Representation of the Soldier (15123)  Manuel Diego,
Clayton W. Burford, ARL HRED STTC; Joseph S. McDonnell,
Session Chair: Ron Dionne, FLETC
Ph.D., Bert Davis, Gary Smith, Dynamic Animation Systems
Session Deputy:Jerry Gordon, Cubic Global Defense
Inc.; Derick Franceshini, StackFrame, LLC
Corporation
UAS Engineering Design Simulation
PAPERS AND AUTHORS
Reducing Operational Risk through Better Performance
Testing (15138)  Tom Wilson, Lockheed Martin
Design of an Educational Tool for Unmanned Air Vehicle
Design and Analysis (15086)  Brian Sanders, Ph.D., Brent
Terwilliger, Ph.D., Ken Witcher, Mark Leary, Embry-Riddle
Aeronautical University; James Ohlman, Christina Tucker,
Pinnacle Solutions
Using Simulation to Test Manned-Unmanned Teaming
(15112)  Michael J. O’Connor, Trideum Corporation;
Kenneth LeSueur, Ph.D, Mark Ebert, Sean Millich, Redstone
Test Center; Fred Ventrone, CTSi; Tom Punihaole, Scalable
Network Technologies
S-12
TH URSD AY, 3 D ECEM BER • 133 0 -1 5 0 0 • S3 2 0 B
The Business of Simulations
Session Chair:Randy Crowe, Ph.D., Lockheed Martin Mission
Systems and Training
Session Deputy:Greg Sidor, AFRL
A MBSE Approach in Modeling Systems Using Hybrid
Simulation Techniques (15368)  Asli Soyler Akbas,
Waldemar Karwowski, Ph.D., University of Central Florida
Simulator Architecture Upgrade Utilizing Virtual Machines
(VMs) (15219)  Thomas Bridgman, National Technologies
Associates; Elizabeth Gaugler, USAF Simulators Division
Improved Process for Bridging the Technology Transition
T H UR S DAY, 3 DEC EMB ER • 1030 -1200 • S320B Valley of Death (15103)  Henry Marshall, ARL HRED
STTC; Paul Dumanoir, U.S. Army PEO STRI; Robert
Preparation Through Virtualization
Wells, Dynamic Animation Systems; Bob Burch, Dignitas
Session Chair:Brent Terwilliger, Ph.D., Embry-Riddle
Technologies, LLC; Jeff Truong, Effective Applications
Aeronautical University
Session Deputy:Randy Crowe, Ph.D., Lockheed Martin Mission Corporation
S-10
Design and Development of a General Virtual Maintenance
Training Platform (15312)  ñ Xingxin Li, Ph.D., Jianping
Hao, Fei Ye, Shijiazhuang Mechanical Engineering College,
P.R. China; Xu Yang, The University of Tennessee
Virtual Environment Computer-Based Training for Bridge
and Tunnel Inspections (15276)  Steve Ianni, Engility
Corporation; Mary P. Rosik, Michael Baker International
Developing Authoring Tools for Skills Models that Enable
Adaptive Game-Based Maintenance Training (15129) 
Sean Guarino, Peter Weyhruch, Ph.D., James Niehaus,
Ph.D., Charles River Analytics, Inc.
T R A INING
DIFFERENTIATING MEASURES OF LEARNING (MOL) FROM
MEASURES OF PERFORMANCE (MOP) DURING AIRCRAFT
CARRIER LANDING PRACTICE (15210)
T-1 TUESD AY, 1 D ECEM BER • 140 0 -1 5 3 0 • S3 2 0 A
Designing for Training Effectiveness
Session Chair:Susan Coleman, Ph.D., Intelligent Decision
Systems, Inc.
Session Deputy: Graham Fleener, U.S. Army PEO STRI
Measuring Training Effectiveness of Lightweight GameS-11 T H U R S DAY, 3 DEC EMB ER • 1030 -1200 • S320F based Constructive Simulation (15007)  Jonathan Stevens,
Ph.D., Stephen Serge, Ph.D., Dean Reed, University of
Soldier Representation
Central Florida; Latika Eifert, ARL HRED STTC; Boris
Session Chair: Lisa Jean Bair, SAIC
Stilman, Ph.D., University of Colorado; Oleg Umansky,
Session Deputy:Bradley Ehrhardt, NAWCTSD
Ph.D., STILMAN Advanced Strategies
Toward Acquiring a Human Behavior Model of Competition
Validating Scenario-based Training Sequencing: The Scenario
vs Cooperation (15316)  David V. Pynadath, Ning
Complexity Tool (15102)  Robb Dunne, Ph.D., Innovative
Wang, Ph.D., Chirag Merchant, USC Institute for Creative
Reasoning, LLC; Stephen A. Sivo, Ph.D., University of
Technologies
Central Florida; Nathan Jones, MARCORSYSCOM PM
Embedded Simulation to Prevent Tactical Surprise and
TRASYS
Improve Soldier Performance (15054)  Jonathan Stevens,
Adaptive Instructor Operating Stations: Design to Decrease
Ph.D., University of Central Florida; Latika Eifert, ARL HRED
Instructor Workload and Increase Effectiveness (15336) 
STTC; Timothy Baldwin, EOIR Technologies; Oleg Umanskiy,
James A. Pharmer, Ph.D., Laura M. Milham, Ph.D.,
Boris Stilman, Ph.D., STILMAN Advanced Strategies
NAWCTSD; John A. Valaitis, John Winters, Basic Commerce
and Industries, Inc.
66
Ï Cyber Security
Mobile
T- 2
T U ES D AY, 1 DEC EMB ER • 1600- 1730 • S320A Alternative Front End Analysis for Automated Complex
Systems (15121)  Natalie Drzymala, Tim Buehner, Ph.D.,
Session Chair:Paul Lyon, Esterline Simulation Visual Systems Linda Brent, Ed.D., The ASTA Group, LLC.; M. Glenn
Cobb, U.S. Army Research Institute; John Nelson, Engility
Session Deputy:Robert Wallace, USAF Air Combat Command
Corporation
Training Effectiveness Evaluation of Augmented Virtuality
Evaluating Distributed Teams with the Team Multiple
for Call for Fire Training: Insights from a Novice Population
(15014)  Julie N. Salcedo, Ph.D., Stephen R. Serge, Ph.D., Errands Test (15264)  Jamiahus Walton, Stephen
Gilbert, Ph.D., Eliot Winer, Ph.D., Michael Dorneich, Ph.D.,
Stephanie J. Lackey, Ph.D., Jonathan Hurter, University of
Central Florida Institute for Simulation and Training; Roberto Desmond Bonner, Iowa State University
Champney, Ph.D., Design Interactive; Gino Fragomeni, ARL
T-5 TH URSD AY, 3 D ECEM BER • 0830-1000 • S3 2 0 A
HRED STTC
VR, AR, and AV for Training “Reality”
Using Augmented Reality to Tutor Military Tasks in the Wild
(15050)  Joseph J. LaViola, Ph.D., Brian M. Williamson,
Conner Brooks, Sergiu Veazanchin, University of Central
Florida; Robert Sottilare, Ph.D., Pat Garrity, ARL HRED STTC
PAPERS AND AUTHORS
Empirically Derived Recommendations for Training Novices
Using Virtual Worlds (15038)  Crystal S. Maraj, Ph.D.,
Stephanie J. Lackey, Ph.D., Karla A. Badillo-Urquiola, Sherry
L. Ogreteen, University of Central Florida, Institute for
Simulation and Training; Douglas B. Maxwell, ARL HRED
STTC
Where’d Who Go? – Operational Considerations for
LVC Training
Session Chair: William Reuter, R-Squared Solutions, LLC
Session Deputy:Daniel Cain, OPNAV N98
Exercise Management Considerations for Live, Virtual, and
Constructive (LVC) Training (15281)  Katherine P. Kaste,
Kelly Neville, Ph.D., Melissa M. Walwanis, NAWCTSD; Amy
E. Bolton, Ph.D., Office of Naval Research
Integrating Warship Bridge, Combat, and Deck Teams in LVC
Environment (15191)  Eric Phipps, Engility, Inc.; Richard
Gaughen, Camber, Inc.
T- 3 WED N ES D AY, 2 DEC EMB ER • 0830- 1000 • S 320A Capability Assessment of Test and Live Training Systems
for Real Time Casualty Assessment (15364)  Joan H.
Physician Know Thyself! Enhancing Surgical Team
Skills through Gaming and Simulation
Johnston, Ph.D., ARL HRED STTC; Margaret D. Nolan,
Session Chair: Robert Heinlein, 3D Systems
NAWCTSD; Jake Caldwell, University of Central Florida
Session Deputy:Kerri Chik, TiER1 Performance Solutions
T-6
TH URSD AY, 3 D ECEM BER • 1030-1200 • S3 2 0 A
Developing Game-Based Leadership Training for Robotic
Surgeons (15198)  h Roger Smith, Ph.D., Alyssa D. S.
Tanaka, Florida Hospital Nicholson Center; Steve McIlwan,
Brad Wilson, Applied Research Associates, Inc.
The Use of Hyper-Realistic Surgical Simulation (15244) 
Anthony J. LaPorta, M.D., FACS, Douglas Robinson, MS4,
Mark Lea, FACS, Rocky Vista University School of Medicine;
Charles Hutchinson, DO, U.S. Army; Reginald Franciose,
M.D., FACS, Lawrence Gaul, M.D. FACS, Vail Valley Medical
Center; Roy Alson, M.D., Ph.D., Wake Forest University;
Michael Czekajlo, M.D., Ph.D., Hunter Maguire VA Hospital,
Virginia Commonwealth University; Douglas Granger, Ph.D.,
Arizona State University and Salimetrics, Inc.; Alan Moloff,
D.O., USA (Ret), Aerospace Medicine; Tuan Hoang, M.D.,
FACS, Commander, U.S. Navy. San Diego
T- 4 ISR, CRM, TDM, Oh My! – Different Domains,
Universal Strategies
Session Chair: Anne Little, Ph.D., Addx Corporation
Session Deputy:Jeffrey Beaubien, Aptima, Inc.
Improving Military Crew Resource Management Using a
Commercial Strategy Game (15097)  ñ Christopher
Roos, Jelke van der Pal, Ph.D., Ghanshaam Sewnath, Johan
Meijer, National Aerospace Laboratory NLR; Lt Col Michel
de Rivecourt, Centre for Man in Aviation CML, Royal
Netherlands Air Force
Simulation and Training Challenges for Intelligence,
Surveillance and Reconnaissance Analysts (15175)  Lisa
Tripp, Ph.D., Elliott Humphrey, Christine Covas-Smith,
Ph.D., Jonathan Diemunsch, USAF Air Force Research
Laboratory; Mike Garrity, Ph.D., Cullen Jackson, Mike
W ED N ES D AY, 2 DEC EMB ER • 1600- 1730 • S 320A Keeney, Ph.D., Sterling Wiggins, Aptima, Inc.
Team Building: All for One and One for All
Session Chair: Brian Cairns, Moulage Sciences & Training
Session Deputy:Beth Pettitt, ARL HRED STTC
Assessing the Effects of Virtual Emergency Training on Mine
Rescue Team Efficacy (15119)   Cassandra Hoebbel,
Ph.D., Tim Bauerle, Brendan Macdonald, Launa Mallett,
Ph.D., The National Institute for Occupational Safety and
Health
Effectiveness of Process Level Feedback at Training Tactical
Decision Making (15201)   Meredith Carroll, Ph.D.,
Christina K. Padron, Stephanie Quinn, Ph.D., Glenn Surpris,
Brent Winslow, Ph.D., Design Interactive, Inc.; Erica
Viklund, Ph.D., Pacific Science and Engineering
Ï Cyber Security
Mobile
67
STEM
S T E M W O R K F O R C E I N I T I AT I V E
Workforce Initiative
STEM supports and promotes activities encouraging students’ interest and
pursuit in Science, Technology, Engineering and Mathematics.
STEM Today = Prepared Workforce for Tomorrow
In support of STEM and Workforce Development, I/ITSEC sponsors the following programs:
• Future Leaders Pavilion
• Students at I/ITSEC
• Post Graduate Scholarships (Masters and
Doctorate)
• Simulation Technician Scholarships
• Serious Games Showcase & Challenge
• I/ITSEC Professional Development
Workshops
• Central Florida Educators Workshop
• Continuing Education Units
• America’s Teachers at I/ITSEC
• Golf and 5K Fundraiser
69
STOP!
SEE THE FUTURE
FUTURE LEADERS/STUDENTS
Future Leaders
Pavilion
Booth
2681
Tuesday, 1 December
1200 – 1730
0930 – 1730
0930 – 1500
WEDNESDAY
SPECIAL SESSION
1030 – 1200 • S210D
AWARDS CEREMONY
1345 • Warfighters’
Corner Sta ge
Students at I/ITSEC
Thursday, 3 December • 0900 – 1400
Over the years, thousands of Central Florida high school students have participated in a unique learning experience by
visiting the Exhibitors/Exhibits. The purpose of the I/ITSEC
Student Tours is to allow students to experience first-hand,
real-world Training, Simulation, and Education solutions that
will help bridge the gap between classroom theory and the applied use of Science, Technology, Engineering and Mathematics (STEM) subjects. Annually, over 600 students, along with
200 school chaperones and volunteer I/ITSEC member escorts,
are exposed to special demonstrations and static displays of
the Simulation, Training and Education Industry. Students are
able to learn about the basic building blocks required to deliver high fidelity modeling and simulation products across
a broad range of training environments. Participating in the
I/ITSEC Student Tours on Thursday, 3 December, 2015, will
give students a complete understanding of how they can apply the STEM related skills they learn in the classroom to
highly successful careers in our Industry. To learn more about
the I/ITSEC Conference and Student Tours, please contact Dr.
Kristy Murray, murrayk@cfl.rr.com.
Learning and Leadership
are indispensable to each other.
The National Training and Simulation Association and the
members of I/ITSEC take great pleasure in welcoming you
to the Ninth Annual Future Leaders Pavilion and Special Session.
We are delighted to host secondary students from such
diverse areas as:
• Dayton, OH
• Lexington Park, MD
• Hampton, VA
• Orlando, FL
• Huntsville, AL
• Philadelphia, PA
• Latham, NY
The students who participate in the Future Leaders
Pavilion (FLP) are committed to excellence and are enrolled
in engineering, computer sciences, mathematics, or modeling
and simulation tracks. Projects presented this year will
continue the legacy of excellence built by previous Future
Leaders.
Please remember to include FLP, located in Booth 2467,
during your visits to the exhibit floor.
On Thursday at 1030, please lend support to our Future
Leaders as they present their projects during their Special
Session – “The Future is Now!”
Join us again at 1345 at the Warfighters Stage, Booth 2681
for an award ceremony acknowledging the work of our Future
Leaders.
70
America’s Teachers @ I/ITSEC
I/ITSEC has a long history of supporting the education
E D U C AT I O N
of students and teachers through visits to the
conference. Since the America’s Teachers at I/ITSEC
program began in 2008, we have hosted teachers
from Arizona, California, Florida, Georgia, Maryland,
Montana, New York, Ohio, Rhode Island, Tennessee,
Texas and Virginia. As part of I/ITSEC’s efforts to
further education in Science, Technology, Engineering
and Mathematics (STEM), teachers and administrators
from across the country have been invited to attend
the conference. The America’s Teachers at I/ITSEC
Program consists of an orientation session, attendance at the Modeling and Simulation techPATH, guided tours of the Exhibit
Hall, and attendance at tutorials, paper sessions, and special events. This program is supported by the National Training and
Simulation Association and its industry members.
Educators
Workshop to Introduce Simulation into the Physics Classroom – I/ITSEC 2015
than 125 techCAMPs have been delivered to more than
2,100 teachers and 2,500 students.
To highlight the thriving Modeling, Simulation & Training
(MS&T) sector in the Corridor, techPATH will be hosting two
programs during the I/ITSEC conference – one techCAMP
for teachers and a techCAMP program for students – to
highlight the many technologies involved in MS&T and the
related high tech careers available to students that study
STEM subjects.
Recognizing the need for a trained high tech workforce,
The teacher’s workshop will feature guided tours of the
the Florida High Tech Corridor Council established its industry exhibits and presentations from well-known experts
educational initiative—techPATH. Involving representatives in the field, including representatives from the Institute for
from a variety of academic affiliations and high tech Simulation and Training at the University of Central Florida
companies, techPATH is “cultivating tomorrow’s workforce” and the National Center for Simulation. Teachers attending
in The Corridor’s 23 counties through a number of innovative the special Educators techCAMP will utilize newly gained
programs, designed to encourage students to pursue high knowledge and experiences at I/ITSEC to build upon the
tech careers. techPATH supports the national objectives for many applications of math and science to help motivate
Science, Technology, Engineering and Mathematics (STEM).
The signature offering of techPATH is The Corridor’s
students.
Students will participate in a special techCAMP that will
techCAMP program. techCAMPs are high tech workshops introduce them to the field of robotics and demonstrate how
offered to middle and high school math, science, technology to program a LEGO robot. They will also have an escorted
and career education teachers and students, to provide tour of the convention floor where they will be introduced
information about the many STEM oriented careers that to the varied work of simulation and robotics in the military,
drive the economic growth of the region. Since 1998, more plus interact with an NAO robot.
For more information, contact Vicki Morelli at vicki.morelli@floridahightech.com
71
SERIOUS GAMES
72
STEM Pavilion: Near the Exhibit Hall Lunch Entrance
Project Based Learning Exhibitors
engages students in active exploration of real-world problems
and challenges. Though questioning, inquiry, critical thinking
and trial and error, students absorb new knowledge and
PROJECT BASED LEARNING
STEM – Tomorrow’s
Workforce, Today!
PROJECT BASED LEARNING (PBL) is a hands-on approach that
educational content in a problem-solving context. PBL
necessitates that students interweave individual learning
concepts and ideas while collaborating and communicating
with others. Studies show that not only do students retain
more of what they learn and for longer, but they are better
able to apply the knowledge in new situations. Just as
important, PBL enables students to grasp the relevance of
STEM educational content to their world both today and into
the future. This in-turn helps fill our workforce pipeline with
prospects who are better prepared, more confident and highly
engaged to take on careers in the STEM fields.
Tell me and I forget. Teach me and I remember. Involve me and I learn. Chinese Proverb
73
25th Annual RADM Fred Lewis I/ITSEC
Postgraduate Scholarship Recipients
I/ITSEC SCHOLARSHIPS
Amelia Kinsella
Doctoral Candidate
Human Factors
Psychology,
Clemson University
Michael Eady
Masters Student
Cybersecurity,
Johns Hopkins
University
RADM Fred Lewis, USN (Ret.)
President, NTSA • 1995 - 2012
These scholarships have been named
the RADM Fred Lewis Postgraduate
I/ITSEC Scholarship in honor of the
former President of the National
Training and Simulation Association
(NTSA).
IMPORTANT DATES FOR 2016
When to Apply
Applications must be postmarked by
20 June 2016. (Don’t Delay!)
How to Apply
See http://www.iitsec.org/Community/
Education/Pages/Scholarships.aspx
for complete application details.
Award Announcement
5 August 2016
Post Graduate Scholarships
Sean Osmond
Masters Student
Modeling and
Simulation,
University of
Central Florida
Looking for
Future Leaders in the Simulation,
Training and Education Community.
Learn more about the I/ITSEC
community at www.iitsec.org
Eligibility
U.S. Citizens
Full-time Masters or Doctoral students
(complete undergraduate work by Spring 2016)
See Study Disciplines at
http://www.iitsec.org/education/studentsandteachers/Pages/Scholarships.aspx
Award Amounts
$10,000 (Doctoral Candidates)
$5,000 (Masters Candidates)
Available for Fall 2016
Be our guest at I/ITSEC November 28 – December 2, 2016
Direct Further Inquiries and Provide Submissions
Lewis-I/ITSEC Scholarship Program
c/o The National Training and Simulation Association
2111 Wilson Boulevard Suite 400 • Arlington, VA 22201-3061
(703) 247-2569 or bmcdaniel@ndia.org
Simulator Maintenance Technician Scholarship Programs
To promote the study of simulation technology,
I/ITSEC continues agreements with Daytona State
College (DSC) in Daytona Beach, Florida, and Lake
Region State College (LRSC), Devils Lake, North Dakota, to provide a year's funding at each school for
a student enrolled in the Simulation Technology program. DSC and LRSC have established themselves as
leaders with accredited programs in this field.
In addition to the scholarship programs, both schools are interested in acquiring
corporate partners willing to provide used simulators, establish intern positions, or
consider other means of supporting the programs. Contact us at (703) 247-2569 or
bmcdaniel@ndia.org if you are interested in finding out more about scholarship or
partnership opportunities.
74
VADM John S. Disher, USN (Ret.)
Executive Director, NTSA • 1991-1995
These scholarships have been named
the Vice Admiral John S. Disher Simulator
Maintenance Technology Scholarship
(DSC and LRSC) in honor of the former
Executive Director of the National
Training and Simulation Association
(NTSA).
Professional Development Workshops
Orange County Convention Center, South Concourse
Date:
Friday, 4 December
Times:
0700 Breakfast and registration • AM Sessions 0800 – 1200 • PM Sessions 1300-1700
Who may attend?
All registrants of I/ITSEC are welcome to attend.
Fees:
There is no fee to attend.
PROFESSIONAL DEVELOPMENT WORKSHOPS
Location:
CEU/CLP:Paid I/ITSEC Conference registrants are eligible to receive CEU/CLP credits. If not a paid attendee,
a $45 fee will be charged only if you wish to receive the CEU credits.
Registration:Preregister via https://secure2.rhq.com/iitsec/iitsec2015/public/index.cgi?track=workshoponly
Registrations also accepted on-site during I/ITSEC registration hours.
Lunch:
On own.
Coordinated by University of Central FloridaDivision of Continuing Education.
For additional information on these seminars including topical outline and
instructor bios, please see: www.ce.ucf.edu/iitsec
UCF Continuing Education POC
Maria Cherjovsky, Assistant Director, Continuing Education & Regional Campuses
Phone Number: 407-882-0260
 All Professional Development Workshops are eligible for CEU/CLP credits. (See page 12) 
P D W1 • R o o m S 330A • 0800 – 1700
Exam Topic Outline. The CMSP exam has been completely
revised and refined over the past two years, and new applicants will now have a choice of two tracks — Technical and
Presenters:
User/Manager — and will take an entirely new exam. The
Talib Hussain, Ph.D., Senior Scientist, Raytheon BBN
workshop will be taught by charter/pioneer CMSPs who have
Technologies; Kelly Pounds, Vice President, IDEAS Learning; been involved in oversight of the CMSP program and/or creVance Souders, Producer, Janus Research
ation/revision of the CMSP exam. The workshop will not by
Participants will be introduced to key concepts, steps and itself prepare applicants to take the exam, but will provide a
processes involved in designing a serious game for learning. thorough overview of exam content and a blueprint for furThrough hands-on activities and working together in groups, ther self-study.
participants will design a learning game. Participants will experience each phase of the design process, including identifying the training requirements and learning objectives, creating
PD W 3 • Room S330C • 0800 – 1200
an effective story, determining instructional and gaming stratLive-Virtual-Constructive (LVC)
egies, and designing key game and instructional mechanics.
Interoperability Techniques
Central to our approach will be ensuring that any key
design decision addresses both gaming and instructional Presenters:
considerations. During the workshop, participants will be Edward Powell, Ph.D., Chief Architect and Program Manager
introduced to key methods to use and issues to consider when for TENA, Leidos; Randy Saunders, The Johns Hopkins
designing a learning game. Groups will share their designs University Applied Physics Lab
and discuss their decisions after each phase of design.
This workshop will provide an overview of the systems engineering issues with regard to integrating disparate military
simulations for analysis, training, testing, and other purposes.
P D W 2 • R o o m S 330B • 0800 – 1200
We will discuss the three major interoperability techniques,
the Distributed Interactive Simulation (DIS) standards, the
Certified Modeling & Simulation
High Level Architecture (HLA) for Modeling and Simulation,
Professional (CMSP) Exam Preparation
and the Test and Training Enabling Architecture (TENA), inPresenter:
cluding descriptions of their architectures and some of their
Mikel Petty, Ph.D., University of Alabama in Huntsville
use cases. Recent and planned evolution of each architecture
This workshop will provide an overview of the Certified Mod- will be explained. A discussion of how these architectures are
eling & Simulation Professional (CMSP) certification program, actually used in the real world and the process for integrating
with a particular focus on preparing prospective applicants to disparate systems in a multi-architecture environment will be
take the CMSP exam. The workshop will cover the applica- discussed. The format of the workshop will be part lecture
tion and examination process (education/ work experience and part informal discussion/question answer. Participants
requirements, application fees, how the exam is administered, are encouraged to raise specific topics any time during the
etc.), in addition to an in-depth review of the new CMSP workshop.
Serious Game Design Tutorial
75
PDW4 • S 330D • 0800 – 120 0
Modeling & Simulation for Acquisition
PROFESSIONAL DEVELOPMENT WORKSHOPS
Presenter:
Rob Lisle, Newport News Shipbuilding
Modeling and Simulation (M&S) in Department of Defense
(DoD) acquisition programs encompasses a wide variety of
technologies, organizations, processes, and best practices.
This workshop provides a practical overview of M&S for acquisition, created by professionals experienced in the largest DoD acquisition programs. The workshop begins with
a high-level presentation of organizing principles, and then
transitions to specific, real-world examples. A guidebook and
slides will be provided as handouts for this course.
P D W5 • R oom S 330E • 0800 – 1200
Seamless Mobile Learning and Simulations
Presenters:
David Metcalf, Ph.D., Director, Mixed Emerging Technology
Integration Lab, UCF Institute for Simulation and Training;
Angela Hamilton, Program Lead, Mixed Emerging
Technology Integration Lab, UCF Institute for Simulation and
Training
Participants will discuss how to promote learning and performance within a mobile workforce that is separated by time,
space, and context. The workshop will demonstrate and elaborate on the affordances of mobile simulation for promoting
seamless formal and informal learning experiences and increasing human performance. Focus will be on best practices
for design, development, and strategy.
In addition to ubiquitous mobile content delivery and
assessment channels (e-mail, voice, text messages, web, and
76
mobile apps), emerging technologies and capabilities such
as context-awareness, mobile 3D, and augmented reality are
expanding the potential applications of mobile simulation.
Participants will 1) see current government and military
examples, 2) explore key technological features and design
characteristics unique to mobile, and 3) develop their own
mobile strategy capable of bridging formal and informal
contexts. The workshop will cover information necessary
to build and implement a cohesive design and development
strategy for seamless mobile training and simulation and will
also include a hands-on exercise.
PD W 6 • Room S330F • 0800 – 1 2 0 0
Measuring the Impact and ROI of Training,
Simulation, and Education Programs
Presenter:
Timothy R. Brock, PhD, CPT, ID (S&L+), CEO, The Institute
4 Worthy Performance LLC; Associate, ROI Institute Inc.;
Practice Leader, The Institute for Performance Improvement
L3C
Training, Simulation, and Education programs offer significant value to improve military preparedness and mission
outcomes. Yet, it is now necessary to add bottom line and
ROI funding justifications to support government mandates
for decreased costs, higher value through improved efficiencies and outcomes, and expanded, sustainable capabilities to
compensate for continuing funding decreases. This workshop
introduces a ROI Methodology to show value in terms that
government, military, and corporate executives understand
and desire to make initial and ongoing funding decisions.
Exhibit Hall
Attendee Luncheon
Lunch will be served Tuesday - Thursday at 1200. You must enter & exit luncheon through the Exhibit Hall. Full
Conference registrants will receive lunch tickets with their registration materials. Exhibitors and Visitors may
HALL HAPPENINGS
purchase a ticket for $25.00 at the main Registration Station. Lunch tickets are dated; you must present the current
day’s lunch ticket for entry.
Connections Lounge & Grill
Stop by and relax in the Connections Lounge & Grill for a bite to eat or a refreshing drink and then connect to
your email or review the I/ITSEC program online to plan your next move at the conference. Connections Lounge
& Grill will be located in Booth 100, South A Hall.
Show Management Office
Room S220A • The Show Management Office will be staffed during show hours for all questions regarding booth
space, rules, regulations, exhibitor locators, security and late/early passes. Registration will not be made available
at the Show Management Office.
National Training & Simulation Association (NTSA)
Booth 1880 • The National Training and Simulation Association (NTSA) is America’s premier organization
representing the interests of the modeling and simulation community. As such, it serves as a constant point of
contact for government, academia, industry, research organizations and the military to exchange information, share
knowledge, align business interests, and in general stimulate the growth and overall dynamism of the industry.
Service Booths
PEO STRI
1433
PM TRASYS
1533
NAWCTSD
439/1439
USAF1539
International Pavilions
Canada727
Netherlands2273
European1980
Healthcare Pavilion
Society for Simulation in Healthcare
2073-2185
Recognizing that simulation represents a paradigm shift in health care education, SSH promotes improvements
in simulation technology, educational methods, practitioner assessment, and patient safety that promote better
patient care and can improve patient outcome. Other participants in the Healthcare Pavilion: Laerdal Medical,
TraumaFX, Gaumard and Meadows Medical.
77
Innovation Showcase
P
Exhibit Hall – Booth 2287
HALL HAPPENINGS
resentations within the Innovation Showcase are led by cutting-edge exhibiting companies that are knowledgeable
on the various subject matter within the M&S industry. Mark your calendar to stop by one of the 30-minute
sessions to hear what is new and exciting in M&S! Be sure to check out the official I/ITSEC website and onsite
signage for updated participants.
(As of 30 October 2015)
Monday, 30 November
1515
Mobile Target – Meet Sheldon
Caliente SimIS
1645
Innovation in Projecting the Visible Color Space
Digital Projection
Tuesday, 1 December
1230
CBRN/WMD Simulation in Live Training Environment
Argon Electronics (UK)
Ltd.
1315
Recipe for Driving Effective, Efficient and Tinely Training
BNH Expert Software
1400
Game Engine Use for Simulations: What’s New in 2015 in Unity, Unreal and More
Forge FX Simulations
1445
The Future of Making Things – Training and Simulation
Autodesk
1530
Making Motion Available to all Simulators: Optimize Motion Cueing to Augment
Trainees Simulator Adoption and Preparedness
D-Box
78
1000
Building a Beautiful World: Using Blueberry 3D and VR-Forces to Create ContentRich Terrains
VT MÄK
1130
Vortex Dynamics for Simulation-based Training for Ground Vehicles and Heavy
Equipment
CM Labs Simulations
1345
Adobe Learning 2025 – Beyond the Traditional LMS
Adobe
1430
How to Build a Simulation & Training Cloud Using Open International Standards
Pitch Technologies
Exhibitor Networking Event
B
Tu e s d a y, 1 De c e m b e r • 1700 - 1830 • E x h ib it H a lls
HALL HAPPENINGS
e sure to kick off I/ITSEC 2015 with a stop by one of the participating booths at the I/ITSEC Exhibitor
Networking Event. What a great way to view the latest technology while networking with exhibitors and
your fellow attendees. Be sure to check out the official I/ITSEC website and onsite signage for updated
participants.
Booth #
Company
313
Unreal Engine 4
429
Aptima, Inc.
449
Oakwood Worldwide
835
Alion Science and Technology
840
LSI, Inc.
1101
TRU Simulation + Training
1273SAIC
1513Barco
1620
AMSEC and Newport News Shipbuilding
1734CAE
1880NTSA
1933
Ravenswood Solutions/SRI International
1980ETSA
2009
Adayana Government Group
2039QinetiQ
2149
Esterline BVBA
2200
Soar Technology, Inc.
2401
Krauss-Maffei Wegmann GmbH & Co. KG
2411Leidos
2419
Ruag Schweiz AG, Ruag Defence
2461
The DiSTI Corporation
79
Warfighters Corner
T
HALL HAPPENINGS
hroughout the week the Warfighters Corner will serve as the stage for many events. See the list below,
and go to the referenced pages of this Program Guide for further details about the specific presentation or
program.
Monday, 30 November
1530 – 1700 DoD Small Business Programs: Mr. Kenyata Wesley, SES
Tuesday, 1 December
1300 – 1330 PEO STRI TSIS Update: Director, Field Operations, COL Bill Canaley
1330 – 1400 PEO STRI TSIS Update: PM TRADE, COL Vince Malone
1400 – 1430 PEO STRI TSIS Update: Contracting Command-Orlando, Mr. Joe Giunta
1530 – 1700 “Tip of the Spear” Warfighter Briefings
1030 – 1200
“Tip of the Spear” Warfighter Briefings
1300 – 1330 PEO STRI TSIS Update: PM ITTS, COL Rick Haggerty
1330 – 1400 PEO STRI TSIS Update: PM ITE, COL Ron Gaddy
1400 – 1430 PEO STRI TSIS Update: FMS Update, Mr. Dale Whittaker
1500 – 1530 STEM Proclamation: Orange County Mayor Teresa Jacobs
1600 – 1730 Hiring America’s Service Members: DoD Skillbridge Forum
1030 – 1200
“Tip of the Spear” Warfighter Briefings
1300 – 1330 Serious Games Awards
1345 – 1415 Future Leaders Pavilion Award
80
MONDAY, 30 NOVEMBER
1430 - 1600
DISASTER STRIKES
Addressing 23 Years of LVC Status Quo
HALL HAPPENINGS
TUESDAY, 1 DECEMBER
1600 - 1730
IMMEDIATE RESPONSE
WEDNESDAY, 2 DECMBER
1030 - 1200
COUNTERING EXPLOITATION
WEDNESDAY, 2 DECMBER
1600 - 1730
SECURING THE SKIES
1030 - 1200
OVERMATCH: FORWARD FROM
THE SEA
BOOTH 339 AND OTHER
FLOOR LOCATIONS
FL1
Operation Blended
Warrior (OBW)
I
n this Live-Virtual-Constructive (LVC) Special Event, industry members and government organizations are joining forces
to create an LVC environment for the purpose of: 1) showcasing their capabilities, and 2) collecting data on challenges
encountered during their efforts. The relative effort to establish an LVC event versus the benefit of, or the effort to conduct
the LVC event is out of kilter; the former takes considerably more time and effort, and too little is being done to address this
discrepancy.
This year, Operation Blended Warrior will provide attendees with glimpses of the state of LVC capabilities constrained to
I/ITSEC confines, as well as insights into the number, degree and priority of challenges encountered. This year’s concentration
is on standards, after action review, and traditional LVC and cyber-contested environments using a Black Swan (humanitarian
assistance) event as a backdrop. In future years, additional complexities and concentrations will be added to the event construct — both to showcase additional capabilities, and to tease out additional challenges that need to be addressed.
BY WORKING TOGETHER, NTSA, INDUSTRY AND GOVERNMENT PERSONNEL BELIEVE
SOLUTIONS CAN BE FOUND TO IMPROVE THE CREATION AND EXECUTION OF LVC EVENTS.
Check your Meeting Bags or stop by Booth 339 for a full listing of the participants and the programs to be demonstrated.
81
NTSA Sustaining Member • NTSA Regular Member • NTSA Associate Member
2015 EXHIBITORS
3D Perception 1920
4C Strategies 1917
AAADA727
Acme Worldwide Enterprises, Inc.
713
Adacel Systems, Inc. 1221
Adayana, Inc. 2009
Adobe Systems, Inc.
455
Advanced IT Concepts, Inc.
1167
Advanced Simulation Technology, Inc. 2139
Aechelon Technology, Inc. 1722
AECOM 1070
AEgis Technologies 1901
Aero Simulation, Inc.
813
Aerotronics
248, 1135
Air National Guard Trainer Development
1562
Airbus Defence and Space
1171
Alelo Inc. 2034
Alion Science and Technology 835
AlphaPixel415
American Society of Civil Engineers
2874
AMSEC a Subsidiary of Huntington Ingalls Industries
1620
AMST-Systemtechnik GmbH
2719
Applied Research Associates, Inc. 2271
Aptima, Inc. 429, 690
Argon Electronics
451
Arrington Research, Inc.
413
Aruba Networks
507
AVT Simulation
2049
BAE Systems
1961
Bagira Systems, Ltd.
1862
Barco
287, 1513
Battlespace Simulations
1249
B-Design3D2028
Beijing Sunheart Simulation Technology Co., Ltd.
417
BGI, LLC
448
Bihrle Applied Research Inc. 2641
BIONATICS2129
B-Line Medical
863
Blue Marble Geographics
2625
BNH Expert Software, Inc.
2748
Boeing Company
1700
Bohemia Interactive Simulations
2248
Bosch Rexroth
749
Bowhead/UIC Government Services
486
Brazilian Defense and Security Industries Association
1080
Bugeye Technologies
712
C2 Technologies, Inc.
701
CAE 1734
Calienté SimIS
533
Calytrix
490, 2459
Camber Corporation
1225
Canon USA, Inc.
2213
Capstone Corporation
2156
Carley Corporation
586, 2227
CAST Navigation LLC
622
Central Florida STEM Education Council
2772
CEVIANS LLC
2653
Charles River Analytics
517
Christie Digital Systems 171, 2627
82
Clear-Com1258
Close Air Solutions
1249
CM Labs Simulations 707
Cole Engineering Services, Inc. 181
Concurrent 1908
Connections Café and Lounge
100, 132
Control Products Corporation 1159
Cranfield Aerospace, Ltd. 1973
Crestron Electronics
1165
CSC275
Cubic
249, 1748
Cyber Security and Information Systems 982
Information Analysis Center (CSIAC)
Cybernet Systems Corporation 1162
D2 TEAM-Sim
2664
Da-Lite Screen Company
755
David Clark Company Incorporated 880
D-BOX Technologies, Inc. 1026
Dedicated Computing 2208
Defense News Media Group
2629
Delaware Resource Group of Oklahoma, LLC
627
Department of Defense STEM Development
2884
Program Office
Design Interactive, Inc. 621
Diamond Visionics 2101
DIGINEXT721
Digital Projection 970
Dignitas Technologies
281
Displays & Optical Technologies, Inc. 1926
DiSTI Corporation 2461
DMSCO1280
Doron Precision Systems, Inc.
433
Draken International
459
Draper, Inc.
1064
Drew Defense GmbH
1972
Drexel University MS in Medical and 2182
Healthcare Simulation Program
Driven Technologies, Inc.
1931
DRS Technologies 2133
E2M Technologies B.V R
1723
EBC Electronics Corp
560
EDM, Ltd.
960
EDN Aviation
716
Eduworks Corporation
1873
Elbit Systems, Ltd.
2000
Electric Picture
1288
Electro-Optical Imaging, Inc.
521
ELM333
Embry Riddle Aeronautical University
2735
eMDee Technology, Inc.
2633
Engility 2449
Engineering & Computer Simulations, Inc.
1463
Engineering & Manufacturing Services, Inc. (EMS)
512
Enovative Technologies
2031
Envitia2754
Epson America
2727
ESP Inc. 1914
Esri301
Esterline 2149
Krauss-Maffei Wegmann GmbH & Co. KG 2401
L-3 Communications 1449
Laerdal Medical 2073
Laser Ammo
775
Leidos 2411
Lockheed Martin 2235
Look Solutions USA, Ltd
664
Louisiana State University, Transformational 513
Technologies & Cyber Research Center
LSI, Inc.
840
M3 Solutions Technologies
807
Marathon Targets
1915
Marine Corps System Command (PM TRASYS)
1533
MASA Group
2224
Meadows Medical Supply LLC
2180
Meggitt Training Systems
1238
Melrose Center at Orange County Library System
2872
MetaVR1249
Mid Florida Tech
2665
MONCH PUBLISHING GROUP
453
Moog
471, 1900
Motion Reality, Inc.
321
MS&T Magazine – Halldale Media
2060
NAF – Central Florida NAF Academies
2789
Nakuuruq Solutions 187
NASA2473
National Center for Simulation 2160
National Defense Industrial Association (NDIA)
1880
National Research Council Canada
2026
National Training & Simulation Association (NTSA)
1880
NATO2619
NAWCTSD and U.S. Naval Academy
2769
nCASE–Materials World Modules
2765
NCS Technologies, Inc.
665
NCS/Orlando Tech
2771
Netherlands Simulation Pavilion
2273
Newport News Shipbuilding 1620
Nida Corporation
612
Northrop Grumman
1949
Nova Technologies
581
NSC2465
Oak Grove Technologies
590
Oakwood Worldwide
449
Oculus801
ODU (MSVE Department) 2172
Operation Blended Warrior (OBW)
339
OPTIS North America
2064
Orange County CTE
2785
Orange County Public Schools
2787
Orion Technologies
868
Orlando Science Center Hands-On STEM Activities
2773
Orlando Science Schools
2775
Oshkosh Speciality Vehicles 401
Panasonic862
Panel Products, Inc.
874
Paramount Panels, Inc.
554
Parsons S
1121
PatchPlus Consulting, Inc.
525
Patriot Products LLC 508
2015 EXHIBITORS
ETC 2127
E-Tech Simulations
529
ETSA1980
EUROSATORY2282
Evertz762
Extron Electronics
1127
eyevis GmbH
1171
F2Si175
FAAC, Inc.
2471
Fain Models, Simulation Systems
1973
Fidelity Technologies 1769
FIRST Robotics
2792
FlightSafety International
1401
ForgeFX Simulations
1266
Frasca International, Inc.
2649
Future Leaders Pavilion
2681
Gaumard Scientific
761
General Digital Corporation
329
General Dynamics 475, 1413
George Mason University Serious Games Institute
409
Georgia Tech Research Institute 2214
Geoweb3d2068
Global Defence Technology
Virtual
GovEvents516
Great Minds in STEM
2864
Guild Associates, Inc.
809
Hampden Engineering Corporation
1927
HapTech Inc.
561
Harris Corporation
765
Heartwood Inc. A
686
Hi-Dow514
ICF International
1814
IEEE Xplore Digital Library
1260
IHSE USA, LLC
1165
Immersive Display Solutions, Inc.
1580
INDIENOMICOM2886
Indra 1226
Industrial Smoke & Mirrors 1712
Inert Products LLC
2659
Innovation Showcase
1180
Institute for Simulation and Training 2781, 2783
Intelligent Decisions, Inc.
1036
Inter-Coastal Electronics Inc.
1925
IPKeys Technologies
2469
Israel Aerospace Industries, Ltd.
1762
ITEC606
J.F. Taylor, Inc. 1471
JHT, Inc.
1234
JRL Ventures, Inc. 2032
JRM Technologies 2015
Jupiter Systems
407
JVC Professional Visual Systems 1421
Katmai607
Kentucky Trailer Technologies 549
Key Electronics
523
KGS-TraumaFX2181
KMI Media Group 1969
Kongsberg Maritime Simulation Inc.
1000
Kratos Technology & Training Solutions 481, 1012
83
2015 EXHIBITORS
Photo Research, Inc.
864
Pitch Technologies
1020
PLEXSYS Interface Products, Inc. 2435
PLW Modelworks 548
Pocket Nurse
2184
Polhemus 2115
Power Innovations Int’l Inc.
1113
Pragmatics, Inc. 2269
Pratt & Miller Engineering
2635
Presagis 375, 821
Project Lead the Way
2770
Protobox LLC 1236
Pulau Corporation
2427
Q4 Services
2069
QinetiQ 2039
Quadrant Simulation Systems, Inc.
1909
Quantum 3D, Inc. 2161, 2170
RAFAEL2750
Rapid Prototyping Services
349
Rave Computer 1139
Ravenswood Solutions
1933
RAYDON Corporation 1048
Raytheon 1213
REALTIMEVISUAL 1032
RGB Spectrum 1158
Rheinmetall Defence 2601
Robotics Club at UCF
2880
Rockwell Collins 2201
RPA Electronic Solutions, Inc. 735
RSI Visual Systems 2120
RUAG Defence
2419
SA Photonics
806
Saab Defense and Security 1939
Safety Training Systems, Inc. 1480
SAIC 1273
Scalable Display Technologies
1170
SCALABLE Network Technologies 613
Sea Box, Inc.
2655
SeaPerch2693
Sensics1161
SensoMotoric Instruments, Inc.
764
Senspex, Inc.
2081
Serious Games Challenge
2481
Serious Simulations LLC
771
SGB Enterprises, Inc.
2759
Shephard Media
869
Shooting Range Industries, LLC
2188
SimiGon, Inc. 2749
SIMmersion LLC 1008
SimPhonics, Inc. 2109
Simsoft618
SimSTAFF Technical Services
881
Simtek, Inc.
620
Simthetiq964
Simulation and Control Technologies
501
SMART EYE AB
1264
2200
Society for Simulation in Healthcare
2183
Sonalysts 403
84
Sony Electronics, Inc. 848
Source Code B46
2890
Sterling Global Operations, Inc.
772
Stirling Dynamics 2257
Stottler Henke Associates
1009
Strategic Systems, Inc.
307
Symbolic Displays, Inc.
663
SYNERCO S.A.
1484
Tactical Communications Group
552
Tactical Micro 1058
Talon Simulations
616
Tannas Electronic Displays, Inc.
1788
TAPE 1932
TEC Simulation
515
Tech Sassy Girlz
2888
Tech Wizards, Inc.
2080
Technical Sales & Applications/Pelican Products
506
Ternion Corporation
601
TerraSim Inc.
2248
Texas Advanced Optics 872
Textron Systems
1101
Thales 2221
Theissen Training Systems, Inc. 2612
Third Dimension Technologies
1033
Tobii Technology, Inc.
1163
Trailer Transit, Inc
870
Transas Americas, Inc. 2021
TrianGraphics964
TRU Simulation + Training 1101
Turning Technologies
808
U. S. Jaclean, Inc.
861
UCF Center for Initiatives in STEM (iSTEM) 2870
UFA, Inc. 2741
United Electronic Industries (UEI) 2220
Unreal Engine 4
313
U.S. Army PEO STRI
149, 1433
U.S. Navy / NAWCTSD
439, 1439
U.S. News STEM Solutions
2752
USAA774
USAF Training Systems Product Group
1539
Veraxx Engineering Corporation 1149
VirTra Systems, Inc. 2261
Virtual Motion Labs
1262
VT MÄK 381, 827
Wacom558
WARFIGHTERS CORNER
2280
Westar Display Technologies, Inc.
980
WITTENSTEIN Aerospace & Simulation 1780
World Wide Technology
509
WorldViz2201
WUCF2791
Wyle624
ZedaSoft, Inc.
2435
Zel Technologies, LLC 1021
Committees
Conference Committee
COMMITTEES
Service Executives
CAPT Wes Naylor, USN, Commanding Officer, NAWCTSD
Col Walter Yates, USMC, Program Manager,
MARCORSYSCOM PM TRASYS
Col Dan Marticello, USAF, Director, Simulators Division, Air
Force Materiel Command
MG Jon Maddux, USA, Program Executive Officer for PEO STRI
OSD/Joint ExecutiveFrank C. DiGiovanni, Director, Force Readiness and Training
Service Principals
Diana Teel, NAWCTSD
Martin Bushika, MARCORSYSCOM PM TRASYS
Tony DalSasso, USAF Simulators Division
Traci Jones, U.S. Army PEO STRI
OSD PrincipalSae Schatz, Ph.D., Director, Advanced Distributed Learning
Conference Chair
Brent Smith, Engineering & Computer Simulations, Inc.
Deputy Conference Chair Janet Spruill, Hickory Ground Solutions, LLC (HGS)
Program ChairDavid Hutchings, Raydon Corporation
Deputy Program Chair
Elizabeth Biddle, Ph.D., The Boeing Company
Subcommittee Chairs
Education
Chris Bryant, Operation Smile
Emerging Concepts and Innovative Technologies
Steve Gordon, Ph.D., Georgia Tech Research Institute
Human Performance Analysis and Engineering
Jennifer McNamara, BreakAway, Ltd.
Policy, Standards, Management and AcquisitionTom Yanoschik, SAIC
Simulation
Matt Spruill, Engineering & Computer Simulations
Training
Felicia Douglis, FRD Solutions, LLC
Best Paper Committee Chair Susan Sherman, NAWCTSD
Tutorial Board Chair
Robert Richbourg, Ph.D., Institute for Defense Analyses
Best Tutorial Committee Chair
Katrina Ricci, NAWCTSD
Education and Training Advisor
VADM Al Harms, USN (Ret.), UCF, VP Emeritus
Scholarship Committee Chair
Ron Smits, Engility Corporation
Director for International ProgramsK. Denise Threlfall, Ph.D., Kratos Defense & Security
Solutions
Strategic Planning and STEM Committee ChairLinda Brent, Ed.D., The ASTA Group, Inc.
Special Event Coordinator
Benjamin Bell, Ph.D., Aqru Research and Technology, LLC
Website and Social Media Advisor
John Killilea, NAWCTSD
Conference Sponsor
National Training and Simulation Association
President
RADM James Robb, USN (Ret.)
Coordinator Barbara McDaniel
Exhibits and Sponsorships
Debbie Langelier, CEM
Advisor
CAPT Nelson P. Jackson, USN (Ret.)
Media Relations/Communications
John Williams
Operations
Len Kravitz, LRK Associates, Inc.
Protocol Coordinator
Steve Detro
Historians
Carol Denton and Allen Collier
Veterans Coordinator
DeLloyd Voorhees, General Dynamics Information Systems
Council of Chairs
The Council of Chairs is a special advisory group to the NTSA Sponsor and to the I/ITSEC Committee organization. The exclusive membership comprises the previous I/ITSEC Conference chairs. Drawing on their cumulative experience, these leaders
provide a unique perspective and advice for the ongoing mission of I/ITSEC.
1979A.W. Herzog (Deceased)
and G.V. (Vince) Amico
1980 Robert W. Layne
1981 Kurt Merl
1982 James A. Gardner, Ph.D.
1983 John Todd (Deceased)
1984Ralph T. Davis (Deceased)
1985 John W. Hammond
1986 Rodney S. Rougelot
1987 David P. Crane (Deceased)
1988
1989
1990
1991
1992
1993
1994
1995 1996
Thomas E. Sitterley, Ph.D.
Arthur L. Banman
Steve Selcho
Donald M. Campbell
Jerry Jerome
J.D. (Jack) Drewett
G.P. (Pres) McGee
Judith Riess, Ph.D.
Ed Ward
1997 1998
1999
2000
2001
2002
2003
2004
2005
Dennis Shockley
Jim Cooksey
Stan Aronberg (Deceased)
Ron Johnson (Deceased)
Debbie L. Berry
Paul Bernhardt
Bill Walsh
Buck Leahy
Steve Swaine
2006 2007
2008
2009
2010
2011
2012
2013
2014
Steve Detro
Amy Henninger, Ph.D.
Don Currie
DeLloyd Voorhees, Jr.
Jim Wall, Ph.D.
Mike Genetti, Ph.D.
Amy Motko
Cyndi Turner
Ron Smits
85
Education
Chair:
COMMITTEES
Christopher Bryant
Operation Smile
Deputy Chair:
Jan Brown
CAE USA
Emerging Concepts
& Innovative
Technologies
Chair:
Steve Gordon, Ph.D.
Georgia Tech Research
Institute
Deputy Chair:
nya Andrews, Ph.D.
A
Erudition Corporation
Human
Performance
Analysis and
Engineering
Chair:
Jennifer McNamara
BreakAway, Ltd.
Deputy Chair:
Kelly Hale, Ph.D.
Design Interactive, Inc.
86
Thomas Archibald, IDSI
Martin Bink, Ph.D., U.S. Army Research Insititute
Chuck Breed, Zenetex
Anthony Carbonari, MARCORSYSCOM PM
TRASYS
Sally Carter, Air Force Air Education & Training
Command
Claudia Clark, Ph.D., NETC
Ed Degnan, Ph.D., AFAMS
Nina Diebler, Serco Inc.
David Fautua, Joint Staff J7
Mark Friedman, Adayana Government Group
Liz Gehr, Ph.D., The Boeing Company
Lewis Harris, Booz|Allen|Hamilton, Inc.
Scott Harris, MARCORSYSCOM PM TRASYS
Jeff Kissinger, Ph.D., Rollins College
Adelle Lynch, Rockwell Collins
Pres McGee, ZedaSoft, Inc.
Ellen Menaker, Ph.D., IDSI
Klaine Nedoroscik, Camber Corporation
David Olsen, Ph.D., Engility Corporation
Kara Orvis, Ph.D., Aptima
Tiffany Parrish, NAWCTSD
Robert “Buddha” Snyder, WBB, Inc.
Robert Sottilare, Ph.D., ARL HRED STTC
Brian Stensrud, Ph.D., Soar Technology, Inc.
Suzy Sutton, Air Force Air Education & Training
Command
Jim Threlfall, C2 Technologies, Inc.
Brian Vogt, U.S. Army TRADOC
JoAnn Wesley, MARCORSYSCOM PM TRASYS
Jay White, Department of Homeland Security
Scott Ariotti, The DiSTI Corporation
Stu Armstrong, QinetiQ
John Aughey, The Boeing Company
Keith Biggers, Ph.D., Texas A&M Engineering
Experiment Station
Paul Bogard, USAF Simulators Division
Aaron Burciaga
Leslie Dubow, Veterans Health Administration
SimLEARN
Ba Duong, MARCORSYSCOM PM TRASYS
John Dzenutis, The Boeing Company
Michael Finnern, Engility Corporation
Nick Giannias, CAE
Kent Gritton, JTIEC/NAWCTSD
Jeff Grubb, NAVAIRSYSCOM
Susan Harkrider, RDECOM CERDEC NVESD
Brian Holmes, The AEgis Technologies Group, Inc.
James (Josh) Jackson, SAIC
Joan Johnston, Ph.D., ARL HRED STTC
Ryan McAlinden, USC/ICT
Matthew Morse, USMC MAGTFTC
Jennifer Murphy, Ph.D., Quantum Improvements
Consulting, LLC
Brian Overy, Diamond Visionics
Constance Perry, U.S. Army PEO STRI
Luis Pineiro, AFRL
Susan Sherman, NAWCTSD
Dennis Shockley, Motion Analysis Corporation
Paul Thurkettle, NATO HQ SACT
Luis E. Velazquez, MARCORSYSCOM SIAT
Jeff Wakefield, USAF 505th Distributed Warfare
Group
Eric Weisel, Ph.D., Old Dominion University
Jennifer Arnold, Booz|Allen|Hamilton, Inc.
Kristen Barrera, Air Force Research Laboratory
Maureen Bergondy-Wilhelm, NAWCTSD
Mary Driskel, NAWCTSD
Fred Fleury, ZedaSoft
Michael Genetti, Ph.D. Rockwell Collins
Simulation & Training Solutions
Bill Gerber, Ph.D., WJ Gerber Consulting
Todd Glenn, FAAC, Inc.
Matthew Hackett, Ph.D., ARL HRED STTC
Robert Hester, Ph.D., University of Mississippi
Medical Center
Glenn Hodges, Ph.D., JAMSD
Eric Jarabak, MARCORSYSCOM PM TRASYS
Randy Jensen, Stottler Henke Associates, Inc.
Sherrie Jones, Ph.D., MARCORSYSCOM PM
TRASYS
Amit Kapadia, U.S. Army PEO STRI C3T
Ingrid Mellone, Camber Corporation
Mike Merritt, NAWCTSD
Steve Monson, The Boeing Company
Susan Myers, Ph.D., ManTech International
Corporation
Paul Phillips, ASC/WNS
Elaine Raybourn, Ph.D., Sandia National
John Schlott, L-3 Communications
Jerry Stahl, Cypress International
Kendy Vierling, Ph.D., USMC TECOM
Dennis Vincenzi, Ph.D., ERAU Worldwide
Alice Wakefield, USAF 705th Combat Training
Squadron
Chair:
Tom Yanoschik
SAIC
Deputy Chair: Chuck Secard
Lockheed Martin
Mission Systems and
Training
Randal Allen, Ph.D., Lone Star
Peter Bavesaro, MARCORSYSCOM AC ALPS
Phil Brown, D.M., NORAD-USNORTHCOM J7
Craig Dorrans, NAVAIR PMA 205
Gregory Dougherty, NAWCTSD
Jan Drabczuk, JD Defense Solutions, LLC
Sam Fragapane, AFAMS
Richard Grohs, USAF HQ Air Combat Command
Cynthia Harrison, U.S. Army PEO STRI
Kevin Hulme, Ph.D., University at Buffalo
Tara Kilcullen, Raydon Corporation
John Lynch, MARCORSYSCOM PM TRASYS
Pete Marion, TMST Consultants
Simulation
Rob Matthews, NAWCTSD
Karen Pogoloff, MTS Technologies, Inc.
Jeffrey A. Raver, SAIC
Emilie Reitz, Alion Science & Technology
Larry Rieger, SAIC
Robby Robson, Ph.D., Eduworks Corporation
Elizabeth Root, i Solutions and Results, Inc.
Ramona Shires, Education Programs & Support, Inc.
Craig Siefert, USAF Simulators Division
Harry Sotomayor, U.S. Army PEO STRI
Brett Telford, MCMSMO
Mary Trier, Capital Communications & Consulting
Paul Watson, U.S. Army PEO STRI
Angela Alban, SIMETRI
Lisa Jean Bair, SAIC
Chair:
Brad Cope, NAWCTSD
Matt Spruill
Engineering & Computer Randy Crowe, Ph.D., Lockheed Martin Mission
Simulations, Inc.
Steve Detro
Ron Dionne, FLETC
Deputy Chair:
Robert Dixon, U.S. Army PEO STRI
Carla Cropper
Gerald Dreggors, Northrop Grumman Corporation
Rockwell Collins
Simulation & Training Michael Eady, USMC TECOM
Solutions
Bradley Ehrhardt, NAWCTSD
Mike Flanagan, CACI-Federal
Luis Garcia, MARCORSYSCOM PM TRASYS
Jerry Gordon, Cubic Global Defense Corporation
Toni Hawkins-Scribner, The Air University, USAF
Scott Hooper, Havok, an Intel Company
Gordon King, Intelligent Decisions, Inc.
Bob Kleinhample, SAIC
Favio Lopez, Trideum Corporation
David Picinich, SOCOM
James Reynolds, MARCORSYSCOM SIAT
Tim Ringler, AFLCMC/WNSEB
Roy Scrudder, The University of Texas at Austin,
Applied Research Laboratories
Gregory Sidor, AFRL
Mark Soodeen, CAE
Teresita Sotomayor, Ph.D., ARL HRED STTC
Brent Terwilliger, Ph.D., Embry Riddle
Aeronautical University
Karen Williams, U.S. Army PEO STRI
Training
Brian Kemper, U.S. Army PEO STRI
Anne Little, Ph.D., Addx Corporation
Paul Lyon, Esterline Simulation Visual Systems
Amy Motko, Carley Corporation
Michael O’Connor, Trideum Corporation
Koren Odermann, MARCORSYSCOM PM TRASYS
Beth Pettitt, ARL HRED STTC
Sowmya Ramachandran, Ph.D., Stottler Henke
Associates, Inc.
Damon Regan, Ph.D., The Tolliver Group
Bill “Roto” Reuter, R-Squared Solutions, LLC
Don Snyder, NAWCTSD
Lisa Tripp, Ph.D., USAF 711 Human Performance
Wing
Robert Wallace, USAF Air Combat Command
Chair:
Felicia Douglis
FRD Solutions, LLC
Deputy Chair:
Eliot Winer, Ph.D.
Mike Aldinger, Northrop Grumman Corporation
Jeffrey Beaubien, Ph.D., Aptima, Inc.
Amy Bolton, Ph.D., Office of Naval Research
Daniel Cain, OPNAV N98
Brian Cairns, Moulage Sciences & Training, LLC
Tom Casey, USAF Simulators Division
Kerri Chik, TiER1 Performance Solutions
Susan Coleman, Ph.D., Intelligent Decision
Systems, Inc.
Javier "Jeff" Covelli, Ph.D., CMSP, PMP,
Computer Sciences Corporation
Gabriel Diaz, USMC Warfighting Laboratory
Graham Fleener, U.S. Army PEO STRI
Bob Heinlein, 3D Systems Corporation
Nathan Jones, MARCORSYSCOM PM TRASYS
COMMITTEES
Policy, Standards,
Management, and
Acquisition
87
International Programs
SPECIAL TEAMS
Director
K. Denise Threlfall, Ph.D., Kratos
Defense & Security Solutions
Deputy Coordinator
Cathy Matthews, Matthews Systems
Engineering, Inc.
Member
Michael Weber, Arore Corporation
Operations/Protocol
Chair
Deputy Chairs Operations
Jim Pohlen, Pulau Corporation
Bruce Schwanda, B.A.S. Associates, LLC
Deputy Chair Protocol
Steve Detro
Members
Mike Armstrong, Pulau Corporation
Jan Baka, Electronic Consulting Services,
Inc.
Lee Barnes
Richard Boyd, Szl.it, Inc.
Catherine Emerick, QinetiQ Training and
Simulation, Inc.
Carol Denton
Charlie Frye, Camber Corporation
Jim Godwin, The Tolliver Group, Inc.
Steve Golberg, Ph.D.
Marge Hadbavny
Bill Hornsby, A. Harold & Associates, LLC
Zach Johnson, Booz|Allen|Hamilton, Inc.
Ed Kulakowski, OT Training Solutions, Inc.
Mike Motko, QinetiQ Training and
Simulation, Inc.
Annie Patenaude, AMP Analytics
Skip Vibert, Innovation Transformation
Consulting
Sam Worrell
Serious Games Showcase
& Challenge IPT
Chair
Lisa Scott Holt, Ph.D., Intelligent
Automation, Inc..
Deputy Chair
Wendy Williams, NAWCTSD
Members
Jennie Ablanedo, STTC, University of
Central Florida
Stu Armstrong, QinetiQ, Inc.
Brian Chan, University of Central Florida
Karen Cooper, Ph.D., NAWCAD
Seth Crofton, SqwishLand, Inc.
Leslie Dubow, Veterans Health
Administration SimLEARN
Mark Friedman, Adayana Government
Group
88
Dolly Rairigh Glass, Capital
Communications & Consulting
Gregg Lagnese, Autodesk
Adelle Lynch, Rockwell Collins
David Martz, BroadReach Strategies
Jennifer McNamara, BreakAway Games
Stacy Pierce, Rockwell Collins
Elaine Raybourn, Ph.D., Sandia National
Laboratories/ADL Initiative
Trey Reyher
Erik Sand, Junyo
Pete Schrider, Schrider Consulting
Scott Shiffert, Hewlett-Packard
Steve Slosser, JTIEC/NAWCTSD
Juliana Slye, Government Business
Results, LLC
Brent Smith, Engineering & Computer
Simulations, Inc.
Peter Smith, Ph.D., University of Central
Florida
Vance Souders, JANUS Research Group
Matt Spruill, Engineering & Computer
Simulations, Inc.
Stephen Stewart, Evviva Games
Shane Taber, Engineering & Computer
Simulations, Inc.
K. Denise Threlfall, Ph.D., Kratos
Defense & Security Solutions
Mary Trier, Capital Communications &
Consulting
Greg Trnka, Booz Allen Hamilton
Michael Woodman, Ph.D., Bohemia
Interactive Simulations
Special Events Committee
Chair
Benjamin Bell, Ph.D., Aqru Research and
Technology, LLC
Deputy Chair
Kara Orvis, Ph.D., Aptima
Members
Warfighters’ Corner
DeLloyd Voorhees, General Dynamics
Information Technology
I/ITSEC Fellows
Robert Lutz, The Johns Hopkins
University/Applied Physics Lab
Robert Richbourg, Ph.D., Institute for
Defense Analyses
Margaret Loper, Ph.D., Georgia Tech
Research Institute
Michael Genetti, Ph.D., Rockwell Collins
Simulations Training Solutions
Operations Liaison
STEM Committee
Chair
Linda Brent, Ed.D., The ASTA Group,
LLC; NTSA, Strategic Planning
Members
Serious Games
Future Leaders Pavilion
Ann Friel
Students at I/ITSEC
Kristy Murray, Ed.D, Summit Strategic
Consulting
Scholarships
Ron Smits, Engility Corporation
CEU/Professional Development Workshops
Debbie Berry, Lockheed Martin
Maria Cherjovksy, University of Central
Florida Continuing Education
Jennifer McNamara, BreakAway Games
America’s Teachers at I/ITSEC
Research Institute
STEM Pavilion Project Based Learning
Exhibits
Robert Seltzer, NAWCTSD
Teacher Tours and Training
Robert Seltzer, NAWCTSD
TechPATH
Lynn Sand, Skybridge Tactical
Eileen Smith, University of Central
Florida
Tutorial Board
Chair
Robert Richbourg, Ph.D., Institute for
Defense Analyses
Deputy Chair
David Milewski, Booz|Allen|Hamilton
Members
Charles Cohen, Ph.D., Cybernet Systems
Corporation
James Coolahan, Ph.D., Coolahan
Associates, LLC
Catherine Emerick, QinetiQ
Luis Miguel Encarnação, Ph.D., ACT, Inc.
Michael Freeman, Ed.D., MW Freeman
Solutions
Lisa Scott Holt, Ph.D., Intelligent
Automation, Inc.
Zach Johnson, Booz|Allen|Hamilton, Inc.
Research Institute
Robert Lutz, The Johns Hopkins
University/Applied Physics Lab
Thomas Mastaglio, Ph.D., MYMIC LLC
Denise Nicholson, Ph.D., Soar
Technology Inc.
Katrina Ricci, Ph.D., NAWCTSD
Leah Rowe, Ph.D., AFRL
Don Sine, Ph.D., Dickieson Projects, Inc.
Jim Wall, Ph.D., Texas A&M Engineering
Experiment Station
Conference Information
About Registration
WHAT DO THE REGISTRATION FEES COVER?
International registrants will have special registration stations.
More details will be provided to each group, but be sure and
In addition to Tutorials, Papers, Special Events and Workshops, watch for signage pointing to these areas.
R E G I S T R AT I O N I N F O R M AT I O N
registration fees cover Continuing Education Units (CEUs),
Lunches (T-W-Th), Coffee Breaks (T-W PM, W-Th AM), Conti- Registration outside of the Orange County Convention Center.
nental Breakfasts (W-Th), and the Thursday Banquet. A meet- I/ITSEC Full Service Satellite Registrations will be located
ing bag with conference materials (including an Abstract book at the Hyatt Regency and Hilton from Sunday noon through
and CD ROM of the current papers) is included. The fees also Tuesday. These stations will be staffed to assist you whether
cover administrative expenses incurred.
you need to start your registration from scratch or just need
I/ITSEC Registration Services for 2015
Hyatt Regency location: Convention Level, near the Grand Ball-
to pick up you nametag.
We strive to minimize the time spent in line so you can move on room as you pass from the Hyatt to OCCC.
to the conference events or the exhibit floor. Our goal is to make Hilton location: Lobby Level at the entrance of the skywalk
your I/ITSEC experience a pleasant one even before you enter leading from the hotel to the OCCC.
the OCCC. Avoid that line and move on to what you came to
Parking
I/ITSEC to do!
EXHIBITOR PARKING:
Traditional Registration Stations. Located in S220 of the South
Concourse Registration area, traditional walk-up registration
will be available for Full Service Registration, on-site payments, changes/edits to name badges, multiple badge pickups, or just because you prefer dealing one-to-one with a real
person.
Alternate Registration Stations within the Orange County Convention Center. Specific stations at the Main Registration Station
will be open Friday and Saturday to handle, especially, Exhibitor Registration. Conference Attendees are encouraged to
wait until Sunday afternoon or use the Self Badging/Self Registration kiosks.
Self-badging printing stations will be available for those
who pre-registered and received a confirmation number. To
$15 per Day – For regular vehicles with re-entry privileges
each day. Exhibitor must show badge and receipt for repeat entries.
$ 25 per Day – For oversized vehicles with re-entry privileges each day. Exhibitor must show badge and receipt for
repeat entries.
ATTENDEE PARKING:
$15 per Day – For regular vehicles per entry.
$25 per Day – For oversized vehicles per entry.
AFTER 5PM:
$10 per Day – For regular vehicles. Same stipulations as
above.
$15 per Day – For oversized vehicles. Same stipulations as
above.
complete your registration at this station, you must be paid in
full with no outstanding balance or questions remaining about
your registration.
VIPs, Speakers (including Paper Presenters), Media, and
ACCEPTED PAYMENT METHODS:
Cash, Traveler’s Checks, American Express, MasterCard &
Visa
Dress Code
BRANCH
CONFERENCE AND GENERAL SESSIONS
BANQUET
Army
ACUs or Duty Uniform
Army Blue (Army Evening Mess optional)
Marine Corps
Service “C”
Evening Dress (Dress Blue “B” or Service “A” optional)
Navy
Service Khaki, Navy Service Uniform
Air Force
Short or Long Service Blues
Service Dress Blue with tie and jacket (Mess Dress optional)
Coast Guard
Tropical Blue Long
Civilian
Business attire
Black tie (optional)
89
LODGING
The National Training and Simulation Association has blocked
rooms with the Orlando hotels listed below. Make your lodging arrangements either on-line or by phone through Travel
Planners, our official Housing Partner, through 24 November
2015. Hotel phone numbers will be posted on the I/ITSEC
web at that time for your convenience in making last minute
changes or arrangements. (Current room rates may apply after 24 November.) Travel Planners, Inc. is our official housing
partner and the only company authorized to represent I/
ITSEC and NTSA. If you are contacted by other companies
who present themselves as representing the Conference or
Association, please report to bmcdaniel@ndia.org.
Note: TravelPlanners is merging with onPeak this year. Your correspondence may reference one or the other company. Our contacts
have not changed, and you may feel safe with either identifier.
On-Line: Go to http://www.iitsec.org/attendees/Pages/Local Accommodations.aspx, select Lodging, select whether you are a corporate or government attendee and the program attached will
lead you through the process from location, to hotel selection,
to special needs, to payment and confirmation.
By Phone: If you prefer to book via telephone, friendly and
knowledgeable agents are ready to take your calls Monday
through Friday from 9:00AM – 7:00PM ET at 800-221-3531 or
212-532-1660.
More Information about Lodging Arrangements:
• Some Room Rates are subject to change, based on the government per diem rate. Those listed with an * are the most
likely to change.
• Government Rate Room Reservations: Rooms shown in the
“Gov’t Rate” column are to be assigned to those with appropriate ID, to be presented at the hotel desk upon check-in.
Please do not reserve unless you are eligible to do so.
• Be aware that some hotels may charge an additional Resort
Fee as well as applicable taxes.
• Additional hotels may be added at a later date.
• The individual hotels are not authorized to accept reservations directly for this conference. You may state your hotel
preference when making your reservations.
•
Attendees must identify themselves as being with the
I/ITSEC to receive the rates shown.
• The Conference is being held at the Orange County Convention Center, located between the Hyatt Regency and the
Hilton Hotels. The majority of the I/ITSEC 2015 activities
will be located in the South Concourse.
• Shuttle buses/vans will be available throughout the conference.
• To help defray conference management costs, an assessment is included in the room rates shown with these hotels. We encourage you to make your lodging arrangements
within the designated housing package established.
Hyatt Regency
u Homewood Suites
(Conference Headquarters)
8745 International Drive
(407) 248-2232
(407) 352-4000
Industry: $138 • Government: $127*
iHyatt Place Convention Center
q Castle Hotel, Autograph Collection
(407) 370-4720
(407) 245-1511
One Rate: $129
One Rate: $127*
o Rosen Centre Hotel
w Days Inn Convention Center
(407) 996-9840
(407) 352-8700
One Rate: $73
aRosen Inn at the Pointe Orlando eDoubletree by Hilton at Sea World
(407) 996-8505
(407) 352-1100
One Rate: $83
sRosen Plaza Hotel
r Embassy Suites I-Drive
(407) 996-9700
(407) 352-1400
Industry: $184 • Government: $127
dRosen Shingle Creek
tHampton Inn Convention Center
9399 Universal Boulevard
(407) 996-9939
(407) 354-4447
fWestin Orlando Universal Boulevard
yHilton Orlando
6001 Destination Parkway
(407) 233-2200
(407) 313-4300
90
To get from your hotel to the South Concourse of the OCCC, you have several choices of transportation.
• I/ITSEC will provide Shuttle Bus service to all properties listed. (Schedules will be available at the hotels and at the entrance
to the conference registration area.)
GETTING AROUND
• Very reasonable Public Transportation is available on the I-Ride trolley bus along International Drive.
Check http://www.iridetrolley.com or your hotel for schedules.
• Your own or a rented vehicle. See page 85 for detailed parking information.
• Most of the hotels are within walking distance (wear comfortable shoes).
The National Training and Simulation Association has arranged for the Hertz Company to be
the official car rental agency for I/ITSEC with the special rates below. You can also make your
reservations on-line through the I/ITSEC website (Lodging/Travel). Vehicles may be returned
to any Hertz location in Florida at no additional charge.
CAR CLASS
DAILY
WEEKEND
WEEKLY
TO RECEIVE SPECIAL MEETING RATES
A
Economy
$39
$19
$169
Call Hertz at 1-800-654-2240 or 405-749-4434
B
Compact
$42
$21
$174
or your nearest Hertz reservation center, your
C
Midsize
$45
$23
$184
corporate travel department, or your travel agent
D
Standard 2/4-Door
$49
$25
$194
and give the agent CV#04860008.
F
Full Size 4-Door
$55
$33
$205
Rates are guaranteed from November 23 -
G
Premium
$69
$69
$345
December 10, 2015 subject to car availability.
I
Luxury
$89
$89
$399
Government surcharges, taxes, tax reimbursment,
Q4
Midsize SUV
$62
$62
$299
title and license fee reimbursement and optional
L
Standard SUV
$74
$74
$339
items such as refueling or additional driver fees,
R
Minivan 2WD
$79
$79
$399
are extra. Advance reservations are (strongly)
U
Convertible
$72
$72
$359
recommended. Minimum rental age is 20 (age
T
Large SUV
$109
$109
$549
differential for age 20-24 applies). Standard rental
T6
PRM XCAP SUV
$119
$119
$599
conditions and qualifications qualify.
Make all the difference in your
trip to Orlando by filling your
down time with magical moments. Whether it’s spending
Client Events & Discounts to
Dining, Nightlife, Attractions,
Golf, etc.!
a truly unforgettable evening Orlando Convention Aid has partnered
with an old friend or sharing with I/ITSEC to help you arrange for the perfect restaurant for your client
a dazzling nighttime sky with or staff dinner/event. We have a relationship with 60 local venues to
a new contact, magical expe- provide this service on a complimentary basis, and we will provide you
riences reign supreme in the with availability, pricing, and options, normally at a discount, within
Walt Disney World® Theme 24 hours! Please also visit our website by clicking on the golden tickParks. To get additional information and order et graphic at http://www.iitsec.org/attendees/planningyourstay/Pages/
OrlandoConnections.aspx to make dinner reservations, buy discounted
tickets go online at:
attraction tickets, book tee times, and so much more! This web site will
help you plan your time in Orlando and SAVE MONEY! Be sure to check
http://www.iitsec.org/attendees/planningyourstay/
out the coupons available and get your coupon book at registration —
Pages/OrlandoConnections.aspx
jam packed with thousands of dollars of savings!
or call 407-566-5600.
91
P U B L I C AT I O N S & M E D I A
Advertising Opportunities:
Official Publications of
I/ITSEC
Now more than ever, with the
increased challenges facing the
defense and security marketplace, you need to keep your
organization’s message in front
of its target audience. Reach
the leading decision-makers at
the world’s largest simulation,
training and modeling event
of the year by advertising your
products and services in the Official
Publications of I/ITSEC.
Advertising in these publications is
an excellent way to stand out in the
crowd and invite the attendees to visit
your exhibit, product demonstration and/or website. Then after the event has ended, these publications are used by many
as desk-references, so your advertisement will reach the decision-makers long after the conference is over.
I/ITSEC Proceedings (Three ways to purchase)
ONLINE REPOSITORY (PAPERS FROM 1966 – 2015)
The I/ITSEC knowledge Repository provides a valuable link
to the I/ITSEC training, simulation and education community. Access the online papers repository at http://www.iitsecdocs.
com.
YEARLY PROCEEDINGS (PAPERS FROM 1995 – 2015)
Individual CDs for the years shown above are available for
$30.00 each. Each CD includes all accepted papers from that
year. Orders can be placed by calling (703) 247-9471.
(2002, 2004, 2005, 2007 sold out)
I/ITSEC COMPENDIUM (PAPERS FROM 1966 – 2000)
Great resource for students, librarians, researchers! The full
record of papers published from1966 through 2000 is available for order (or at I/ITSEC) as a two-CD set for $300.00. You
may place an order through the NTSA office (703) 247-9471,
or on the I/ITSEC registration form. Papers from the pre-electronic era have been reviewed, scanned and provided with
keywords, making ALL papers searchable electronically. Orders can be placed by calling (703) 247-9471. Limited supply.
Stay in Touch
The National Training and Simulation Association’s
Annual Simulation & Training Trends and Technology Free Wireless hot spots. E-mail/Internet Kiosks.
Complimentary internet, WiFi and email access in the
Review – I/ITSEC Exhibitor Directory
This publication will be available to all the attendees, exhibitors, and exhibit visitors at I/ITSEC. It will be placed in the
attendees’ conference bags and available at registration, and
other locations at the convention center. As an added bonus,
your ad will also appear in the December Issue of National
Defense Magazine — exposure beyond the walls of the convention center. National Defense is sent to over 81,000 BPA
audited readers, including the members of NTSA. (Directory
section will not appear in National Defense Magazine).
I/ITSEC is the premier annual event of its kind, attendance
by the mainstream and specialist trade press is heavy, resulting in coverage that reaches your key marketing targets.
The I/ITSEC Show Daily
Our media staff stands ready to assist you in achieving
Advertise in this year’s Daily and be noticed by your customers and potential partners who are attending I/ITSEC. The I/
ITSEC Show Daily informs the simulation & training community on breaking events & happenings on-site at I/ITSEC. It
is printed overnight and distributed daily at the conference
center, choice hotels, and uploaded to the I/ITSEC website.
The daily has evolved into a vital part of I/ITSEC; a “must
read” while attending the conference.
maximum exposure during your time at I/ITSEC. Corporate
Use both to give your company Unequalled Exposure
Special packages have been created so your organization can
take advantage of both opportunities!
Web Banners
92
lobby
(look for signage). Internet kiosks are also available at no
charge in the Connections Lounge and Grill located inside
South Exhibit Hall 100 Aisle. If you need access outside of the
complimentary stations, all of OCCC is now WiFi enabled for
a modest user fee.
representatives are invited to bring their marketing materials to the Media Room for distribution as early as possible
after the opening of registration. The Media Room area will
also include a separate facility for briefings/ presentations
with a capacity of approximately 30 persons. We strongly
recommend early bookings for this room, which will be in
demand. Additional exhibitor presentations will be made
available inside the exhibit hall at the Innovation Showcase,
Booth 1080.
Prior to the conference, contact John Williams at (703)
A limited number of banner ad spaces are available on the
I/ITSEC website.
362-7005 or jwilliams@ndia.org; check out more details on the
For more information on advertising in these publications, contact
Dino Pignotti at (703) 247-2541 or dpignotti@ndia.org.
dia Room is S210E, phone (407) 685-6107.
I/ITSEC News page of http://www.iitsec.org. The I/ITSEC Me-
National Training and Simulation Association
The NTSA, an affiliate of NDIA, represents and promotes the
business interests of companies in the simulation, training,
NTSA’s 200 corporate and 500 individual members enjoy reduced fees on all NTSA
events and services, as well as a monthly e-newsletter (Training Industry News)
and National Defense magazine. Sustaining and Regular Corporate members receive early space selection and discounts on exhibit space at I/ITSEC. Individual
memberships are also available. For membership information, call (703) 247-9471 or
NOVEMBER 28 –
DECEMBER 2 2016
Orange County Convention
Center • West Concourse
Orlando, FL
visit the NTSA website http://www.trainingsystems.org.
Exhibit Information
National Defense Industrial Association
Debbie Langelier, CEM
Based in Arlington, Virginia, the National Defense Industrial
Director of Exhibits, NTSA
Association (NDIA) is a non-profit, educational association
Phone: (703) 247-9480
representing industry, government, and all the military ser-
FAX: (703) 243-1659
vices. About 1,600 companies and 87,000 individuals rely on NDIA for networking, knowledge, and business development opportunities. As the nation’s leading
E-mail: dlangelier@ndia.org
defense industry association promoting national security, NDIA advocates cut-
Visit http://exhibits.iitsec.org
ting-edge technology and superior weapons, equipment, training, and support for
to view the current floor plan,
the warfighter and first responder. Through events, working divisions, local chap-
exhibitor list, and
ters, and four affiliate organizations, NDIA connects the government and defense
industry to create a vigorous and ethical forum of information exchange leading to
S P O N S O R I N G A S S O C I AT I O N
mission planning/rehearsal, and support services industry.
SAVE THE DATE FOR
I/ITSEC 2016!
sponsorship opportunities.
greater support for national security.
For NDIA membership information visit www.ndia.org or contact Mike Kibler at
mkibler@ndia.org.
Women In Defense,
A NATIONAL SECURITY ORGANIZATION
EARNING THE CMSP
DESIGNATION WILL:
• Demonstrate expertise in
the field of M&S to your
employer and the larger
TO LEARN MORE ABOUT THE
REQUIREMENTS AND TO APPLY,
PLEASE VISIT
M&S community
• Provide opportunities for
professional advancement
Requirements include 3-8 years
WWW.SIMPROFESSIONAL.
of work experience (depending on
ORG
level of highest collegiate degree),
3 professional letters of reference,
OR CONTACT PATRICK ROWE AT and successful completion of an
PROWE@NDIA.ORG.
online examination.
CMSP Applicants now have a choice between
CMSP-Technical and CMSP-Management exams.
Celebrating its 30th
year, Women In Defense
(WID), A National Security Organization, cultivates and supports the
advancement and recognition of women
working in all sectors of national security. Established in 1985, Women In Defense became an affiliate of the National
Defense Industrial Association (NDIA)
in 2004. WID Membership is open to
women and men whose primary professional activities relate to the United
States’ national security. WID members
are from defense companies and related
small, medium and large business, all
branches of the U.S. Armed Forces, government agencies, academia, associations and professional services. Women
In Defense provides women a dynamic
community for developing leadership
skills as well as mentoring, networking,
and educational programs with CEU’s
and CLP’s for programs available in
2016. WID National provides many educational and professional development
programs in addition to the many programs of WID’s 20 Chapters across the
country.
93
SAFETY & SECURITY
FOR LIFE THREATENING EMERGENCIES DIAL 911
SECURITY HOTLINE DURING I/ITSEC: (407) 685-6111
Security Training Before The Conference
Technology collection directives contain mandates requiring exhibitors and presenters to receive a Counterintelligence (CI)
briefing from their CI support staff prior to I/ITSEC. Contractors with classified contracts may contact their Defense Security Service Special Agents. To avoid security breaches, I/ITSEC presenters and exhibitors should ensure that the required
briefing has been received. A list of CI support agencies follows. Please contact your security officer/manager and ensure
that an appropriate briefing for yourself and your colleagues is arranged. Providers of the briefings are:
902 Military Intelligence
Army
Navy, USMC, Coast Guard
Naval Criminal Investigative Service
Air Force Office of Special Investigation
Air Force
Defense Security Service
Contractors
Personal Security
The most important thing to protect, of course, is yourself. Pay attention to your surroundings. Report suspicious behavior
or security breaches to a security person or NTSA staff. Familiarize yourself with emergency procedures and exits at your
hotel and the Convention Center. Conference Security Office will be located in the South Lobby Registration Area and
inside the Exhibit Hall.
Emergency Medical Services
EMT and/or paramedics will be on-site during I/ITSEC (including hall build-up and tear-down). During
I/ITSEC 2015, they will be located on the same level as Registration, near the escalators between S220 and
S230. See the layout on page 13 for the exact location. Dial 911 for life threatening emergencies. For non
emergencies within the center, dial 5-9809 or on your cell dial (407) 685-9809. Or alert any security or
I/ITSEC staff member with a radio.
Bags and Briefcases
Bags and Briefcases may be carried in by those wearing Conference Attendee or Exhibitor badges. Exhibit Visitors
(those who are only visiting the exhibits) WILL NOT be allowed to carry in bags or briefcases. A check
room will be available in the main registration area. A small purse or fanny pack is allowed, but is subject
to search. Additional security restrictions may be posted on http://www.iitsec.org and on signage at the
conference. Conference Management reserves the right to adjust security levels as deemed necessary during
the conference.
Presentations
Recording devices will not be permitted in the presentation rooms, unless authorized by the conference
management. Presenters and Exhibitors should review their company’s policy documents and those of the
government agencies with whom you contract regarding open distribution, limited distribution, restricted
distribution and sharing limitations.
Cameras
Exhibitors have the right to limit photographs and videos of their displays. Please respect this right by asking before photographing or videotaping. Participants found taking photos or videos without the consent of
the subject presentors or exhibitors will be dealt with according to security procedures, to possibly include
confiscation of materials and removal from the premises.
Inquiries (before the conference)
Registration (702) 798-8340 • Exhibit/Sponsorship (703) 247-9480 • All other inquiries (703) 247-2569
94
Earle L. Denton Memorial Golf Tournament
Organized by Central Florida Chapter NDIA • Sunday, 29 November OR Monday, 30 November
GOLF TOURNAMENT
Rosen Shingle Creek Golf Club
9939 Universal Blvd, Orlando, FL 32819 • 407-996-9933 • www.shinglecreekgolf.com
Deadlines
On-Line Registration
Golf
23 November • Register and/or select sponsorship at www.iitsec.org.
Sponsorship
19 November • Register one to four players at a time.
• Select day of play (Sunday or Monday).
Tournament Times
Sunday
1100 Registration
1230 Start
Fees
Monday
0630 Registration
0730 Start
$90 per player (green fees, range balls, cart, lunch)
Point of Contact
Debbie Berry
Coordinate club rentals directly with the Rosen Shingle
407-748-3807 Creek Golf Club pro shop.
debbie.berry@lmco.com
Two Day Sponsorships
Format
Details available at www.iitsec.org
Captain’s Choice/Scramble
Tee Box
$500
Putting Green
$500
Pairings & Requests
Driving Range
$500
Final assignments and pairings will be made by the tourna-
GPS$750
ment coordinator. Priority is based upon receipt of payment.
Beverage Cart
$2,500
Hole-in-One$2,500
NOTE: Requested team pairings cannot be guaranteed if not
Boxed Lunch
$3,000
registered together. Requests noted under comments when
registering will be considered but cannot be guaranteed.
Sponsors
If requesting pairing with player(s) registering separately,
Send your logos via email to debbie.berry@lmco.com no
specify all names in the comment area when registering.
later than 19 November. Do not bring your own sign
Cancellations
*Scholarships and additional qualified initiatives supported
Must be received via email to debbie.berry@lmco.com by
through tournament proceeds; for a full list of initiatives
close of business 20 November to receive 50% refund.
(STEM, Wounded Warriors, etc.) supported, please visit:
No refunds thereafter.
http://www.ndia-cfl.org.
95
5K RUN
ANNUAL I/ITSEC 5K
RUN/WALK/ROLL 2015
W ED N ES DAY, D E C EM B ER 2 , 2 0 1 5
Orange County Convention Center
5:30AM Packet Pickup • 6:45AM Start Time
WEBSITE:
http://www.iitsec.org/attendees/planningyourstay
www.facebook.com/iitsec5k
All registered runners will receive custom race tech shirt,
custom race metal, swag bag, race bib and official
timing by Milestone Race Authority, pre- and post-race
refreshments. Tax deductible registration.
IRD
EARISLYTRB
ATION
REG
$
25
until October
31
Registration $35
November 1 - November 29
Late Registration $45
November 30 & December 1
CHARITIES THE
5K WILL SUPPORT
TITLE SPONSOR
Camaraderie Foundation
IITSEC STEM Initiative
Email Sean Osmond for Race Information at iitsec5k@gmail.com or
Debbie Langelier for Sponsorship information at dlangelier@ndia.org
96
I/ITSEC 2016
Save the Date!
Pushing the Training Envelope
Live • Virtual • Constructive
R E G I S T R A T I O N T HI NE FWOO R
R LM
D ’AS T LI AORNG E S T
I/ITSEC
MODELING & SIMULATION EVENT
INTERSERVICE/INDUSTRY TRAINING,
SIMULATION & EDUCATION CONFERENCE
PUSHING THE TRAINING ENVELOPE
LIVE • VIRTUAL • CONSTRUCTIVE
e!
Sa v e th e Da t
No v emb e r 28016
Dec emb e r 2, 2
w w w.ii ts ec.o r g
NOVEMBER 28-DECEMBER 2
u
WWW.IITSEC.ORG
u
ORLANDO, FLORIDA
THE WORLD’S LARGEST MODELING & SIMULATION EVENT
CALL FOR PAPERS
AND TUTORIALS
Pushing the Training Envelope
I/ITSEC 2016
Live • Virtual • Constructive
ABSTRACT DEADLINE: 22 FEBRUARY 2016
ON-LINE ABSTRACT SUBMITTAL
SUBCOMMITTEES/CATEGORIES
• Education
• Emerging Concepts & Innovative Technologies
• Human Performance, Analysis and Engineering
• Policy, Standards, Management & Acquisition
• Simulation
• Training
TUTORIALS
Information on core M&S, training, and education
topics suitable for management and technical
personnel.
The submission process for the I/ITSEC Papers
and Tutorials coincide. Submittal details will vary
slightly, but the milestones will match.
Follow the
Papers/Tutorials Completion Process
for 2016 Abstract Submittal which
will be posted in December.
http://www.iitsec.org/authors
I/ITSEC 2016 Program Chair
Elizabeth Biddle, Ph.D., CMSP
The Boeing Company
Phone: 407-249-3562
Email: elizabeth.m.biddle@boeing.com
I/ITSEC 2016 Tutorial Board Chair
David Milewski
Booz | Allen | Hamilton
Phone: 757-784-2865
Email: milewski_david@bah.com
NOVEMBER 28-DECEMBER 2
u
WWW.IITSEC.ORG
u
ORLANDO, FLORIDA
Abstracts
2015 I/ITSEC Abstracts
FELLOWS
2015 FELLOWS PAPER: SIMNET AND BEYOND: A HISTORY OF THE DEVELOPMENT OF
DISTRIBUTED SIMULATION ................................................................................................................. 8
BEST PAPERS
STRESS EXPOSURE TRAINING FOR THE DISMOUNTED SQUAD: THE HUMAN
DIMENSION ................................................................................................................................................ 9
DIFFERENTIATING MEASURES OF LEARNING (MOL) FROM MEASURES OF
PERFORMANCE (MOP) DURING AIRCRAFT CARRIER LANDING PRACTICE ........................ 9
YOU CANNOT HIT WHAT YOU DO NOT SHOOT.............................................................................10
SAFE TESTING OF AUTONOMOUS SYSTEMS PERFORMANCE .................................................11
CYBER OPERATIONAL ARCHITECTURE TRAINING SYSTEM – CYBER FOR ALL ..............11
MEASURING VIRTUAL SIMULATION’S VALUE IN TRAINING EXERCISES - USMC USE
CASE ............................................................................................................................................................12
EDUCATION
STUDENT RETENTION IN STEM CAREER PATHS: PRIMARY INFLUENCES ON THE
DECISION TO STAY OR LEAVE ...........................................................................................................13
OVERCOMING THE CHALLENGE OF EVALUATING SKILLS TRANSFER FROM TRAINING
TO JOB ........................................................................................................................................................13
MEASURING A MOVING TARGET: VALIDATING DEPLOYED TRAINING COURSES ..........14
ENHANCING GOOD STRANGER SKILLS: A METHOD AND STUDY ..........................................15
CURRICULUM GPS: AN ADAPTIVE CURRICULUM GENERATION AND PLANNING
SYSTEM ......................................................................................................................................................15
MARS GAME: CREATING AND EVALUATING AN ENGAGING EDUCATIONAL GAME .......16
THE SECRET FOR STEM SUCCESS: EMPLOYING TECHNOLOGY FOR MATH
PROFICIENCY ...........................................................................................................................................17
INQUIRY AND DESIGN APPROACH TO STEM EDUCATION USING PROJECT-BASED
LEARNING .................................................................................................................................................17
ANTECEDENTS OF ADAPTIVE COLLABORATIVE LEARNING ENVIRONMENTS ................18
ADAPTABLE RESILIENCE TRAINING FOR TRANSITIONING VETERANS USING
EXISTING TECHNOLOGIES ..................................................................................................................19
THE CHANGING FACE OF MILITARY LEARNING .........................................................................19
GAME-BASED TRAINING FOR HUMAN-INTELLIGENCE SKILLS .............................................20
A CONCEPTUAL MODEL OF PEDAGOGIC DESIGN TO SUPPORT CRITICAL
THINKING IN COMMANDERS ..............................................................................................................21
DESIGNING INSTRUCTOR SUPPORT TOOLS FOR VIRTUAL SHIPHANDLING
TRAINING...................................................................................................................................................22
HEY, REMEMBER TO ADD MOTIVATIONAL DESIGN TO YOUR E-LEARNING .....................22
ACHIEVING EDUCATIONAL EXCELLENCE: WHAT DO EFFECTIVE
INSTRUCTORS DO? .................................................................................................................................23
Papers are available on the 2015 I/ITSEC CD ROM included in the Conference Attendee meeting bag, or visit the
I/ITSEC Website (www.iitsec.org) for ordering information. (Limited numbers of CDs from 1998-2014 are also
available.) All papers from 1966 through 2000 are available in the I/ITSEC Compendium. Individual papers from
1966 through 2015 may also be ordered through the www.iitsec.org portal.
1
INNOVATIVELY APPLYING SKILL ACQUISITION THEORY TO THE ARMY LEARNING
MODEL ........................................................................................................................................................24
TRANSMEDIA (SOCIAL) LEARNING IN THE WILD: EXPLORING A CONTINUUM OF
SUPPORT FOR TRANSITIONING SERVICE MEMBERS .................................................................24
INNOVATIVE MOBILE TECHNOLOGIES FOR ASSESSING AND ENHANCING SOLDIER
PERFORMANCE........................................................................................................................................25
A REFERENCE MODEL FOR DESIGNING MOBILE LEARNING AND PERFORMANCE
SUPPORT ....................................................................................................................................................26
DEVELOPMENT AND EVALUATION OF MOBILE ADAPTIVE TRAINING
TECHNOLOGIES ......................................................................................................................................26
EMERGING CONCEPTS & INNOVATIVE TECHNOLOGIES (ECIT)
AUTOMATED SIMULATION CREATION FROM MILITARY OPERATIONS
DOCUMENTS .............................................................................................................................................27
SCHEDULING TRAINING TO MANAGE ACQUISITION & DECAY..............................................28
M&S AS A SERVICE: PARADIGM FOR FUTURE SIMULATION ENVIRONMENTS .................28
MILITARY VEHICLE TRAINING WITH AUGMENTED REALITY ...............................................29
MODELLING A HELICOPTER TRAINING CONTINUUM TO SUPPORT SYSTEM
TRANSFORMATION ................................................................................................................................30
TOWARD COGNITIVE TWO-WAY INTERACTIONS IN AN IMMERSIVE VIRTUAL
REALITY ENVIRONMENT .....................................................................................................................31
MODELING AND INTEGRATING COGNITIVE AGENTS WITHIN THE EMERGING
CYBER DOMAIN .......................................................................................................................................31
TABLET COMPUTER CALL FOR FIRE SIMULATION: PROOF OF CONCEPT STUDY
RESULTS .....................................................................................................................................................32
TURN-BASED GAMING FOR CONVOY COMMANDER TRAINING .............................................33
AN IMMERSIVE LIVE / VIRTUAL BRIDGE APPROACH WITH ULTRA WIDEBAND
TRACKING TECHNOLOGY: PHASE II ...............................................................................................33
VIRTUALIZING HUMANS FOR GAME READY AVATARS ............................................................34
RELIABLY ASSESSING THE EFFECTIVENESS OF A PLAN USING MODELS OF
VARYING FIDELITY AND UNDER TIME CONSTRAINTS ..............................................................35
REQUIREMENTS FOR FUTURE SAFS: BEYOND TACTICAL REALISM ....................................35
REQUIRED FIDELITY OF SIMULATED WOUNDS AT THE POINT OF INJURY .......................36
VIRTUAL INTERVIEW TRAINING INCREASES JOB OFFERS FOR VETERANS AND
OTHERS ......................................................................................................................................................37
STEPPING STONES – AN AUGMENTED REALITY REHABILITATION GAME ........................37
THE VA VIRTUAL MEDICAL CENTER: IMPLEMENTING A VISION FOR A VIRTUAL
HEALTHCARE CAMPUS FOR OUR VETERANS ...............................................................................38
VISUALIZING FMRI DATA USING VOLUME RENDERING IN VIRTUAL REALITY ...............39
PROFESSIONAL SOLDIER ASSESSMENT OF A RIFLE-MOUNTED TARGET HAND-OFF
SYSTEM ......................................................................................................................................................39
EMPIRICAL SUPPORT FOR BRAIN-BASED ASSESSMENT IN SIMULATION-BASED
TRAINING...................................................................................................................................................40
2
Papers are available on the 2015 I/ITSEC CD ROM included in the Conference Attendee meeting bag, or visit
the I/ITSEC Website (www.iitsec.org) for ordering information. (Limited numbers of CDs from 1998-2014 are
also available.) All papers from 1966 through 2000 are available in the I/ITSEC Compendium. Individual papers
from 1966 through 2015 may also be ordered through the www.iitsec.org portal.
PUTTING LIVE FIRING RANGE DATA TO WORK USING THE XAPI .........................................41
ADAPTING GUNNERY TRAINING USING THE EXPERIENCE API..............................................41
OPENING LEGACY DATA SILOS: USING EXPERIENCE DATA FOR EDUCATIONAL
IMPACT .......................................................................................................................................................42
RAPID 3D GEOSPATIALLY ORIENTED STRUCTURE EXTRACTION FROM MINIMAL
IMAGE .........................................................................................................................................................43
EXTENDING INTELLIGENT TUTORING BEYOND THE DESKTOP TO THE
PSYCHOMOTOR DOMAIN .....................................................................................................................43
DELIVERING 3D VIRTUAL MAINTENANCE TRAINING CONTENT: EXAMINING THE
DEPLOYMENT OPTIONS........................................................................................................................44
GAMERS TODAY, SURGEONS TOMORROW? ..................................................................................45
RELATIONSHIPS BETWEEN LEARNER AND ENVIRONMENT: LEARNER TRAITS IN
SERIOUS GAMES ......................................................................................................................................45
LEARNING STORIES: DESIGN CONSIDERATIONS FOR NARRATIVE ELEMENTS IN
SERIOUS GAMES ......................................................................................................................................46
PROCEDURAL RECONSTRUCTION OF SIMULATION TERRAIN USING DRONES ................47
ASSESSMENT OF UNMANNED AIRCRAFT PLATFORM PERFORMANCE USING
MODELING AND SIMULATION ............................................................................................................47
EARLY SYNTHETIC PROTOTYPING: WHEN WE BUILD IT, WILL THEY COME? ................48
IMPLEMENTATION OF AGILE METHODS WITHIN INSTRUCTIONAL SYSTEMS DESIGN:
A CASE STUDY ..........................................................................................................................................49
DEVELOPMENT AND EVALUATION OF A VENIPUNCTURE AND PHLEBOTOMY
TRAINING SYSTEM .................................................................................................................................49
HUMAN PERFORMANCE ANALYSIS AND ENGINEERING (HPAE)
PILOTING A GROUNDBREAKING VIRTUAL CONTINUING COMPETENCY PLATFORM:
RESULTS AND RECOMMENDATIONS................................................................................................50
MOBILE APP DESIGN FOR VETERANS WITH PHYSICAL AND COGNITIVE
LIMITATIONS ...........................................................................................................................................51
STRUCTURED DEVELOPMENT OF INTERVENTIONS TO IMPROVE PHYSICIAN
KNOWLEDGE RETENTION ...................................................................................................................51
WORK DOMAIN ANALYSIS FOR ECOLOGICAL INTERFACE DESIGN OF TANGIBLE
INTERFACES .............................................................................................................................................52
ADAPTIVE TESTING: ADAPT AND OVERCOME THE SHORTFALLS OF TRADITIONAL
PROFICIENCY ASSESSMENTS .............................................................................................................53
COGNITIVE LOAD ASSESSMENT FOR INTELLIGENCE ANALYSTS THROUGH FMV (FMV)
ANALYTICS ...............................................................................................................................................53
MULTI-MEASURE ASSESSMENT OF INTERNAL DISTRACTIONS ON DRIVER
PERFORMANCE........................................................................................................................................54
HOW HUMANS TRUST AUTONOMOUS VEHICLES: A STUDY IN MEASUREMENT
DEVELOPMENT AND PERSONALITY DIFFERENCES ...................................................................55
BUILDING TRUST IN A HUMAN-ROBOT TEAM WITH AUTOMATICALLY GENERATED
EXPLANATIONS .......................................................................................................................................55
3
AUTOMATED PERFORMANCE ASSESSMENT IN CYBER TRAINING EXERCISES ................56
COMMAND SHIFT: EXPLORING MODERN GAMING TECHNOLOGIES TO CREATE
NEXT-GENERATION OCO INTERFACES ...........................................................................................57
EMBEDDING CYBER-PHYSICAL SYSTEMS FOR ASSESSING PERFORMANCE IN
TRAINING SIMULATIONS .....................................................................................................................57
HUMAN PERFORMANCE IN CONTENT DESIGN FOR INTERACTIVE AUGMENTED
REALITY SYSTEMS .................................................................................................................................58
USING MICRO AND MACRO STUDIES OF TABLETS TO IMPROVE MAINTENANCE ...........59
HELICOPTER PILOT’S MODELING INCLUDING THE STRESS FACTOR .................................60
HUMAN-IN-THE-LOOP FLIGHT SIMULATION STUDY OF VIRTUAL CONSTRUCTIVE
REPRESENTATION ON LIVE AVIONICS DISPLAYS .......................................................................60
PRACTICAL RECOMMENDATIONS FOR VALIDATING SURVEY APPARATUSES IN
COALITION TRAINING ENVIRONMENTS.........................................................................................61
STEALTH ASSESSMENT OF PROBLEM-SOLVING SKILLS FROM GAMEPLAY .....................62
“FIXING” THE MILITARY DECISION MAKING PROCESS (MDMP) ...........................................62
THE SMALL UNIT DECISION MAKING ASSESSMENT BATTERY: DEVELOPMENT
AND PSYCHOMETRIC ANALYSIS .......................................................................................................63
MARINENET USER ENGAGEMENT EXERCISE ...............................................................................64
SUPPORTING UNIT TRAINING MANAGEMENT THROUGH MOBILE PERFORMANCE
ASSESSMENT TOOLS ..............................................................................................................................64
MARINE CORPS INSTRUCTOR MASTERY MODEL: A FOUNDATION FOR MARINE
FACULTY PROFESSIONAL DEVELOPMENT ....................................................................................65
REALISM AND EFFECTIVENESS OF ROBOTIC MOVING TARGETS .........................................66
EEG/ECG OSCILLATION OF SOLDIER’S MENTAL STATES ON THE SHOOTING
TASKS IN THE TANK SIMULATOR .....................................................................................................66
NOVICE AND EXPERIENCED POLICE OFFICER SIMULATION EXPERIENCE –
GUIDING THE FUTURE ..........................................................................................................................67
POLICY, STANDARDS, MANAGEMENT, AND ACQUISITION (PSMA)
THE LIVE-SYNTHETIC TRAINING, TEST AND EVALUATION ENTERPRISE
ARCHITECTURE ......................................................................................................................................68
EARLY ADOPTION OF COMMON OPERATING ENVIRONMENT (COE) STANDARDS
AND GUIDELINES ....................................................................................................................................68
THE SEJONG THE GREAT CLASS DDGS: HOW ROK NAVY EMBRACED AND TRAINED
THEM ...........................................................................................................................................................69
A SYSTEM-MODEL-CENTRIC COLLABORATIVE ENVIRONMENT FOR THE
ACQUISITION LIFECYCLE....................................................................................................................70
IMPROVING EDUCATION, TRAINING AND CAREER ADVANCEMENT THROUGH
COMPETENCY PORTABILITY .............................................................................................................71
STEMULATING: AN INTEGRATED APPROACH TO CULTIVATING OUR FUTURE...............71
MODELING AND SIMULATION PROFESSIONALS – MEETING THE DEMAND ......................72
CALCULATING SIMULATION-BASED TRAINING VALUE: COST AVOIDANCE AND
PROFICIENCY ...........................................................................................................................................73
4
LARGE SCALE ADOPTION OF TRAINING SIMULATIONS: ARE WE THERE YET? ...............73
AUTOMATED SURVEYS: LOWERING THE RESPONDENT’S BURDEN .....................................74
RISK MANAGEMENT FRAMEWORK (RMF) TRANSITION IMPACTS IN TRAINING
SIMULATION SYSTEMS .........................................................................................................................75
CYBERSECURITY CONTROLS: THEN AND NOW ...........................................................................75
CYBERSECURITY CHALLENGES AND RESOLUTIONS FOR SIMULATOR & TRAINING
SYSTEMS ....................................................................................................................................................76
SIMULATION
MOBILE AUGMENTED REALITY FOR FORCE-ON-FORCE TRAINING ....................................77
USING AUGMENTED REALITY TO TRAIN COMBAT MEDICS: AN EVALUATION ................77
REAL-TIME CUTTING OF ORGANS WITH SCISSORS ...................................................................78
SIMULATION SCENARIO ENCODING FOR REUSE ........................................................................79
MULTI-FEDERATE SCENARIO DEVELOPMENT AND TESTING: “A GOOD PLAN,
VIOLENTLY EXECUTED” ......................................................................................................................79
THE EXPECTED RESULTS METHOD FOR DATA VERIFICATION .............................................80
POLYGONE LVC: THE NEW PARADIGM FOR EW TRAINING ....................................................81
LIVE SYNTHETIC TRAINING AND TEST & EVALUATION INFRASTRUCTURE
ARCHITECTURE (LS TTE IA) PROTOTYPE ......................................................................................81
MEASURING REALISM IN SIMULATIONS FOR TRAINING AND TESTING .............................82
NETWORK BANDWIDTH'S EFFECT ON VIRTUAL WORLD SIMULATOR PERFORMANCE
OPTIMIZATION ........................................................................................................................................83
3D IMMERSIVE ENVIRONMENT USING X-PLANE FOR DEPTH PERCEPTION
RESEARCH .................................................................................................................................................83
BATTLE DAMAGE COMPUTATION SERVER ...................................................................................84
HIGH FIDELITY WIND MODEL SOFTWARE FOR REAL-TIME SIMULATION
PLATFORMS ..............................................................................................................................................85
AUTOMATED RUNTIME TERRAIN DATABASE CORRELATION ASSESSMENT ....................85
AUTOMATED MODELIZATION IN TERRAIN DATABASE PRODUCTION ................................86
MODELING CGF BEHAVIOR WITH MACHINE LEARNING TECHNIQUES:
REQUIREMENTS AND FUTURE DIRECTIONS .................................................................................87
AUTOMATIC SPEECH RECOGNITION IN TRAINING SYSTEMS: MISCONCEPTIONS,
CHALLENGES AND PATHS FORWARD .............................................................................................87
ON THE PERIPHERAL APPLICATION OF HMD DEVICES IN INFANTRY SIMULATION......88
M&S TRAINING TRANSFORMATION: BRIDGING THE NEXT GENERATION
JOINT LVC .................................................................................................................................................89
LEVERAGING CLOUD COMPUTING TECHNOLOGY FOR LVC TRAINING ............................89
OSSEUS, AN EXPERIMENT IN NEXT GENERATION LVC M&S ARCHITECTURE .................90
EMBARKING ON A HOME STATION TRAINING REVOLUTION .................................................91
IMPLEMENTATION OF ROLE-BASED COMMAND HIERARCHY MODEL FOR ACTOR
COOPERATION .........................................................................................................................................91
5
INNOVATIVE DIVISION/BRIGADE LEVEL COMMANDING OFFICER TRAINING
SOLUTION FOR INFLUENCE OPERATIONS .....................................................................................92
REDUCING OPERATIONAL RISK THROUGH BETTER PERFORMANCE TESTING ..............93
DESIGN OF AN EDUCATIONAL TOOL FOR UNMANNED AIR VEHICLE DESIGN AND
ANALYSIS ...................................................................................................................................................93
USING SIMULATION TO TEST MANNED-UNMANNED TEAMING .............................................94
DESIGN AND DEVELOPMENT OF A GENERAL VIRTUAL MAINTENANCE TRAINING
PLATFORM ................................................................................................................................................95
VIRTUAL ENVIRONMENT COMPUTER-BASED TRAINING FOR BRIDGE AND TUNNEL
INSPECTIONS ............................................................................................................................................95
DEVELOPING AUTHORING TOOLS FOR SKILL MODELS THAT ENABLE ADAPTIVE
GAME-BASED MAINTENANCE TRAINING .......................................................................................96
TOWARD ACQUIRING A HUMAN BEHAVIOR MODEL OF COMPETITION VS.
COOPERATION .........................................................................................................................................97
EMBEDDED SIMULATION TO PREVENT TACTICAL SURPRISE AND IMPROVE
SOLDIER PERFORMANCE .....................................................................................................................97
DISTRIBUTED SOLDIER REPRESENTATION: IMPROVING M&S REPRESENTATION OF
THE SOLDIER............................................................................................................................................98
A MBSE APPROACH IN MODELING SYSTEMS USING HYBRID SIMULATION
TECHNIQUES ............................................................................................................................................99
SIMULATOR ARCHITECTURE UPGRADE UTILIZING VIRTUAL MACHINES (VMS) ...........99
IMPROVED PROCESS FOR BRIDGING THE TECHNOLOGY TRANSITION VALLEY OF
DEATH .......................................................................................................................................................100
TRAINING
MEASURING TRAINING EFFECTIVENESS OF LIGHTWEIGHT GAME-BASED
CONSTRUCTIVE SIMULATION ..........................................................................................................101
VALIDATING SCENARIO-BASED TRAINING SEQUENCING: THE SCENARIO
COMPLEXITY TOOL .............................................................................................................................101
ADAPTIVE INSTRUCTOR OPERATING STATIONS: DESIGN TO DECREASE INSTRUCTOR
WORKLOAD AND INCREASE EFFECTIVENESS ............................................................................102
METRICS ASSESSMENT TOWARD A TRAINING EFFECTIVENESS EVALUATION OF
AUGMENTED VIRTUALITY FOR CALL FOR FIRE TRAINING: INSIGHTS FROM A NOVICE
POPULATION ..........................................................................................................................................103
EMPIRICALLY DERIVED RECOMMENDATIONS FOR TRAINING NOVICES USING
VIRTUAL WORLDS ................................................................................................................................103
USING AUGMENTED REALITY TO TUTOR MILITARY TASKS IN THE WILD ......................104
DEVELOPING GAME-BASED LEADERSHIP TRAINING FOR ROBOTIC SURGEONS ..........105
THE USE OF HYPER-REALISTIC SURGICAL SIMULATION ......................................................105
ASSESSING THE EFFECTS OF VIRTUAL EMERGENCY TRAINING ON MINE RESCUE
TEAM EFFICACY ...................................................................................................................................106
ALTERNATIVE FRONT END ANALYSIS FOR AUTOMATED COMPLEX SYSTEMS .............107
EVALUATING DISTRIBUTED TEAMS WITH THE TEAM MULTIPLE ERRANDS TEST ......107
6
EXERCISE MANAGEMENT CONSIDERATIONS FOR LIVE, VIRTUAL, AND
CONSTRUCTIVE (LVC) TRAINING....................................................................................................108
INTEGRATING WARSHIP BRIDGE, COMBAT, AND DECK TEAMS IN LVC
ENVIRONMENT ......................................................................................................................................109
CAPABILITY ASSESSMENT OF TEST AND LIVE TRAINING SYSTEMS FOR REAL-TIME
CASUALTY ASSESSMENT ....................................................................................................................109
IMPROVING MILITARY CREW RESOURCE MANAGEMENT USING A COMMERCIAL
STRATEGY GAME ..................................................................................................................................110
SIMULATION AND TRAINING CHALLENGES FOR INTELLIGENCE, SURVEILLANCE
AND RECONNAISSANCE ANALYSTS ................................................................................................111
EFFECTIVENESS OF PROCESS LEVEL FEEDBACK AT TRAINING TACTICAL DECISION
MAKING....................................................................................................................................................111
TABLE OF AUTHORS ............................................................................................................................113
PLEASE NOTE: TUTORIALS ARE INCLUDED ON THE CD
7
2015 FELLOWS PAPER: SIMNET AND BEYOND:
A HISTORY OF THE DEVELOPMENT OF DISTRIBUTED SIMULATION
2015 IITSEC Paper No. IF1501
Duncan C. Miller, Sc.D.
Why Is SIMNET Important?
SIMNET stands for SIMulator NETworking. Initiated in 1983, it was the first “shared virtual
reality” distributed simulation system, which continues to have significant influences. It was
sponsored by DARPA, the Defense Advanced Research Projects Agency, the Department of
Defense’s principal high-risk, high-payoff research and development organization, established in
1958. In 1991, a study of various DARPA initiatives by the Potomac Institute for Policy Studies
listed SIMNET as one of six programs that have had the most profound effects on the DoD. To put
this in perspective, the other five were the ARPANET (the predecessor of the Internet); the
individual computer workstation; phased array radar; the stealth technology used to make aircraft
such as the F-117 fighter and the B-2 bomber “invisible” on radar; and ATACMS, the low cost,
long-range tactical artillery rocket system used successfully in Desert Storm to destroy enemy
surface-to-air missile sites and other targets (Potomac Institute for Policy Studies, 1991). That’s
quite a distinguished list in which to be included!
WEDNESDAY, 2 DECEMBER, 2015 ROOM S320A
BP-1 - Education, Training, & Human Performance Analysis & Engineering
1030
Stress Exposure Training for the
Dismounted
Squad: The Human Dimension
(15150) (Education)
1100
Differentiating Measures of Learning
(MOL) from
Measures of Performance (MOP)
During Aircraft Carrier
Landing Practice (15210) (Training)
1130
You Cannot Hit
What You Do Not Shoot (15209)
(Human Performance Analysis and
Engineering)
Notes
8
STRESS EXPOSURE TRAINING FOR THE DISMOUNTED SQUAD: THE HUMAN DIMENSION
2015 IITSEC Paper No. 15150
Dr. Jay Brimstin
Squad Overmatch Study Team
U.S. Army, Maneuver Center of Excellence
PEO STRI, MCoE, ARL, STTC, MITRE, CPG
Ft Benning, GA
Orlando, FL and Ft. Benning, GA
Today’s soldiers face a complex, unpredictable, and fluid operational environment encountering more
stressors and trauma than ever before. Mental disorders account for more hospitalizations of U.S. service
members than any other diagnostic category. The Army is aggressively pursuing programs to address these
challenges, including revising the Comprehensive Soldier and Family Fitness Program and launching the
Ready and Resilient Campaign to improve the performance, resilience, and readiness of soldiers.
While most of the emphasis has been on post-event treatment, preventative resilience training that focuses on
the human dimension is a key priority for the Army. The Army Study Program Office provided funding for
the Squad Overmatch Study in 2013 and 2014 as its top priority program. This study is focused on
investigating how to improve existing training methodologies and technologies to better develop cognitive
skills and mental resilience at the squad level with more combat realistic exercises and experiences.
The vision for the Squad Overmatch Study is to optimize performance by enhancing existing training—from
basic individual skills to unit training prior to, during, and post-deployment—through early and continuous
Stress Exposure Training (SET) to reduce post-traumatic stress and to better prepare soldiers for the stressful
situations that are a natural part of combat operations. In June of 2014 the study team conducted a
demonstration of graduated SET, as well as cognitive and situational awareness skills training. Of the soldiers
who participated, 90% agreed that the use of virtual technologies is effective for training situational
awareness and resilience and 100% agreed that training that provides realistic scenarios are helpful in
preparing for stressful combat situations. This paper will describe the Squad Overmatch Study objectives, the
squad-based SET gaming, virtual and live scenario exercises and technologies used, and present results of the
2014 demonstration at Fort Benning, Georgia.
DIFFERENTIATING MEASURES OF LEARNING (MOL) FROM MEASURES OF PERFORMANCE (MOP)
DURING AIRCRAFT CARRIER LANDING PRACTICE
Jeffrey M. Beaubien & E. Webb Stacy
Sterling L. Wiggins
Aptima, Inc.
Aptima, Inc.
Woburn, MA
Fairborn, OH
Amy E. Bolton
LCDR Jefferson D. Grubb
Melissa M. Walwanis,
Office of Naval Research,
Naval Aviation System
Heather Priest
Code 34
Program Office
Naval Air Warfare Center
Arlington, VA
(PMA-205)
Training Systems Division
Patuxent River, MD
Orlando, FL
Michael J. Keeney
Aptima, Inc.
Washington, DC
Christian S. Riddle
Naval Air Systems
Command, Manned Flight
Simulator
Patuxent River, MD
Measures of performance collected during initial skill acquisition can be misleading indicators of long-term
retention or transfer (Soderstrom & Bjork, 2013). For example, previous research demonstrates that learning can
occur in the absence of visible performance gains, and temporary performance gains can occur in the absence of
long-term retention or transfer (Singer & Edmondson, 2006; Soderstrom & Bjork, 2013). Therefore, it is critical
that authors clearly differentiate between Measures of Learning (MOLs) and Measures of Performance (MOPs) in
their research. While this distinction was frequently made in the psychological literature until the 1950’s, it has
been somewhat forgotten since then (Schmidt & Bjork, 1992). As part of a larger study on the effects of simulator
cue fidelity on aircraft carrier landing skills, we collected both MOLs and MOPs. The sample included fifteen Navy
F/A-18 pilots (8 novices, 7 experts), each of whom flew 24 landing passes in a high-fidelity simulator over two
consecutive days. MOPs were calculated for each pass, and were operationalized as deviations (measured in
degrees) from the ideal angle of attack, glide slope, and center line. The data were then aggregated across all 24
passes. In contrast, MOLs were operationalized as changes in performance over time. The two sets of analyses –
learning vs. performance – provide very different interpretations of the data. In this paper, we describe the
conceptual differences between MOLs and MOPs; show how the choice of analysis can have profound implications
for interpreting the results; and provide the reader with actionable guidelines that they can use in their own work to
better differentiate learning from performance.
9
YOU CANNOT HIT WHAT YOU DO NOT SHOOT
Martin L. Bink & Elizabeth Uhl
U.S. Army Research Institute
Fort Benning, GA
David James
Northrop-Grumman Corp.
Columbus, GA
A training system can only be effective if it is appropriately utilized, regardless of whether the training system
is a sophisticated full-motion simulator or steel targets on a small-arms range. However, without understanding
how trainees use a training system and without clear performance feedback, it is not likely that desired training
outcomes will be met. A recent example of training-system underutilization impacting training performance
comes from the U.S. Army Sniper School (USASS). In the USASS, sniper teams spend a considerable amount
of time at the beginning of the course conducting “data confirmation.” Data confirmation is accomplished by
engaging static targets at varying distances on an unknown distance range. So, in the case of data confirmation,
the training system is very simple: a small arms range with static targets at varying distances. It was observed
over several iterations of USASS that shooters rarely engaged targets at distances beyond 600 m during data
confirmation and that, when engaged, the hit percentage of targets over 600 m was very low. The consequence
of failing to shoot at far targets (i.e., over 600 m) during data confirmation was low hit percentages on far
targets in the record fire event that was a graduation requirement. An intervention was introduced to increase
engagements with far targets that required USASS instructors to record and analyze individual shot data. By
requiring instructors to document data, the instructor was able to determine if the shooter was spending too
much time at closer distances (i.e., not fully utilizing the training system) and to intervene if necessary. The
result was increased record-fire performance on far targets. Even though the intervention and results may seem
intuitive, the need for such an intervention highlights the importance of trainer engagement to ensure proper
training-system utilization and the importance of providing performance feedback during training.
BP-2 Emerging Concepts and Innovative Technologies; Simulation; & Policy
Standards Management and Acquisition
1400
Safe Testing of
Autonomous Systems Performance
(15348) (Emerging Concepts and
Innovative Technologies)
1430
Cyber Operational Architecture
Training System –
Cyber for All (15108) (Simulation)
1500
Measuring Virtual Simulation’s
Value in Training Exercises –
USMC Use Case (15114) (Policy
Standards Management and
Acquisition)
Notes
10
SAFE TESTING OF AUTONOMOUS SYSTEMS PERFORMANCE
David Scheidt, Robert Lutz, William D’Amico, Dean Kleissas, Robert Chalmers, Robert Bamberger,
Johns Hopkins University Applied Physics Laboratory, Laurel, MD
The role of unmanned platforms is rapidly expanding across a wide range of defense and homeland security
missions. Currently operational unmanned vehicles are “tele-operated”, using a command and control link to a
remotely located pilot. However, operational complexity, operational pace, and a need to function in
communication denied environments necessitate a trend toward autonomous unmanned vehicles. Autonomous
systems that make independent decisions in complex engagements, such as the Navy’s Autonomous Aerial
Cargo Unmanned System, are currently under development and will require development and operational testing
within the next 3-5 years.
Testing of autonomous systems presents some unique and vexing challenges. For instance, the infinite number
of variations of test conditions that can exist to stimulate autonomous behaviors and the complexity of the
interactions that can occur among multiple autonomous systems combine to make comparative measurement of
autonomous system performance extremely difficult. Also, the inherent unpredictability of decision making by
autonomous systems may result in decisions that are considered unsafe by managers of live test ranges.
Advanced test and evaluation techniques that focus on the unique challenges of autonomy represent a clear and
increasing need within the DoD.
The Safe Testing of Autonomy in Complex, Interactive Environments (TACE) Program is a research initiative
to develop an advanced test infrastructure that can measure the performance of autonomous systems operating in
complex Live-Virtual-Constructive (LVC) environments while ensuring that the autonomous system does not
violate range safety policy. This paper will provide an overview of the TACE hardware and software
architecture and will highlight the LVC testing that has been performed at the Aberdeen Test Center to validate
TACE capabilities. A discussion of anticipated transition activities with DoD partner programs will also be
provided.
CYBER OPERATIONAL ARCHITECTURE TRAINING SYSTEM – CYBER FOR ALL
Dr. David “Fuzzy” Wells, IPA, CMSP
USPACOM J81 / Cyber War Innovation Center
Camp H.M. Smith, H
Derek Bryan
USPACOM J81 / Ingenia Services, Inc.
Camp H.M. Smith, H
Current methods for conducting cyber training are incompatible with the traditional, simulation-based
training architectures used to conduct battlestaff training. As a result there is little to no interaction
between the cyber domain and the traditional warfighting domains during exercises. This situation does
not accurately reflect the current operational environment nor does it address the Secretary of Defense’s
(SECDEF) and the Chairman of the Joint Chiefs of Staff’s (CJCS) directives and guidance for
incorporating realistic cyberspace conditions into major Department of Defense (DoD) exercises. The
Cyber Operational Architecture Training System (COATS) is a U.S. DoD Modeling & Simulation
Coordination Office (M&SCO) High-Level Task (HLT) that integrates existing cyber range environments,
traditional simulation architectures, operational networks, and cyber emulations to safely and securely
synchronize and deliver realistic cyber effects to the entire battlestaff – cyber for all. In doing so COATS
provides an integrated and contested training environment where operators plan, execute and experience
realistic cyberspace operations and conditions in all domains. This paper describes the key components of
the COATS architecture, including the application of network guards and the first draft of a cyber data
exchange model, lessons learned from the demonstration and employment of COATS during three U.S.
Forces Korea exercises, and recommendations for future cyber and traditional modeling and simulation
capability research, development, test and evaluation.
11
MEASURING VIRTUAL SIMULATION’S VALUE IN TRAINING EXERCISES USMC USE CASE
Nathan Jones & Greg Seavers
Christin Capriglione
MCSC PM TRASYS
NAWCTSD
Orlando, FL
Orlando, FL
In 2013, Lieutenant General (LTGen.) John A. Toolan, former Commanding General (CG) of First Marine
Expeditionary Force (I MEF), requested incorporating previously non-interoperable and ‘stove-piped’
virtual and constructive Training Aids, Devices, Simulators and Simulations (TADSS) at I MEF’s First
Marine Expeditionary Brigade’s (1st MEB’s) Large Scale Exercise 2014 (LSE-14) to demonstrate that
Live, Virtual, Constructive (LVC) TADSS could collectively stimulate a Marine Air Ground Task Force
(MAGTF) Commander’s Common Operational Picture (COP). The expected outcome would be an
operationally-effective MEF with capabilities to conduct full-spectrum military operations with a COP
stimulated with data feeds from LVC entities; while providing training to both the primary (battlestaff) and
secondary (supporting unit) training audiences. The objective of this assessment was to measure the
training value gained. A measurable training value of utilizing virtual TADSS in a live exercise could
have the potential to impact the historical and traditionally biased paradigm to train everything live
whenever possible within the Marine Corps.
This paper presents the results of the training value assessment of augmenting the live training event with
virtual TADSS. It provides impacts of virtual integration on training efficacy achieved for primary and
secondary training audiences. Included is the training value construct, defined assessment approach,
limitations, results (both immediate and post event impacts), and efficiencies in terms of cost plus cost
avoidance. Recommendations and discussions focus on: (1) identified needs for improvements in exercise
planning and tools to facilitate more efficient satisfaction of training objectives for primary and secondary
training audiences, (2) develop training-related human performance measures in TADSS to measure
performance against training objectives, and (3) define an encompassing methodology for assessing
training value of training solutions to inform requirements and acquisition decision makers.
TUESDAY, 1 DECEMBER, 2015 ROOM S320D
ED-1 Assessment and Evaluation; Isolating Effects
1600
Student Retention in STEM Career
Paths: Primary Influences
on the Decision to Stay or Leave
(15018)
1630
Overcoming the Challenge of
Evaluating Skills Transfer from
Training to Job (15151)
1700
Measuring a Moving Target:
Validating Deployed Training
Courses (15189)
NOTES
12
STUDENT RETENTION IN STEM CAREER PATHS:
PRIMARY INFLUENCES ON THE DECISION TO STAY OR LEAVE
Jennifer Winner
Air Force Research Laboratory
Wright Patterson Air Force Base,
Dayton, OH
Michael D. Coovert,
University of South Florida
Tampa, FL
LT Christopher Faxon
United States Air Force
Wright Patterson Air Force Base,
Dayton, OH
Extracurricular programs for science, technology, engineering and mathematics (STEM) content for
middle and high school students are growing in number and are distributed utilizing in-class learning, after
school clubs and activities and summer internship programs. It is often the case that anecdotal evidence of
an internship program’s effectiveness is plentiful but quantitative data to support this evidence is lacking.
In this paper we discuss how the existing organizational turnover literature may be leveraged to explore
student interest and retention in STEM career paths. This approach was inspired by the summer modeling
and simulation (M&S) program conducted by the Air Force Research Laboratory’s (AFRL) Gaming
Research Integration for Learning Laboratory (GRILLTM). We describe the basic features of the
GRILL™ program and classify dimensions in behavioral change that have emerged over four years of the
program. By interpreting these data through the theoretical framework of retention and turnover,
specifically focused on organizational commitment, job satisfaction, fit, stress, and career intentions, we
develop a more complete picture of factors associated with student entrance into STEM disciplines,
continuance through educational and training programs, and entry into the STEM workforce.
OVERCOMING THE CHALLENGE OF EVALUATING SKILLS TRANSFER FROM TRAINING TO JOB
Ms. Toumnakone (Annie) Hester & Dr. Jay Brimstin
Maneuver Center of Excellence
Fort Benning, GA
One of the long-standing challenges in the academic community has been evaluating the transfer of
learning from the classroom to on the job performance (McDonald, 2010). Although billions of dollars are
spent on training annually, some studies suggested that only 10-15% of training content is transferred to
behavioral changes on the job (Lancaster, Milia, & Cameron, 2013). This has also been a challenge in the
military training setting. The 19D One Station Unit (OSUT) training course is the initial entry training for
the Army’s Cavalry Scouts that combines both basic combat training and advanced individual training.
Among the skills taught to these new soldiers is land navigation, which is a fundamental skill necessary for
successful performance of all Cavalry Scouts. The land navigation training provided to these soldiers in
OSUT was significantly changed in late 2014 in an effort to improve the competence of graduating
soldiers. An evaluation of this new training curriculum showed a significant improvement in soldier
competence, but the challenge was to determine whether the improved learning outcomes resulted in
improved job performance for these soldiers once they arrived in their first unit of assignment. This paper
describes the methodology used to evaluate the degree to which newly assigned Cavalry Scouts were able
to apply the land navigation skills learned in initial entry training in their first unit of assignment. The
methodology applied in the evaluation consisted of establishing a baseline assessment, provided by unit
leaders of newly assigned soldiers in selected units, and then using the same measures and leaders to assess
newly arrived soldiers who had experienced the improved land navigation curriculum. Further, this paper
reports on the degree to which the improved curriculum resulted in improved soldier performance in their
first unit of assignment and the challenges soldiers encountered in applying their skills.
13
MEASURING A MOVING TARGET: VALIDATING DEPLOYED TRAINING COURSES
Timothy R. Brock, PhD, CPT, ID(S&L+) & Denise R. Stevens, EdD
General Dynamics Information Technology
Orlando, FL
The Veteran Benefits Administration implemented a new requirement to validate the effectiveness of new or revised elearning courseware after deploying it to the field using the total population as samples of the target audience. In the
past, the validation effort occurred in a controlled environment using a small sample of the population prior to fielding
the course. The methodologies used with the small sample population were the U.S. Army’s Sequential Validation or
Fixed Validation. Because of the push to deploy the required entry-level and refresher/recurring training quicker (as
well as cheaper without sacrificing quality), course effectiveness validation is now conducted post-deployment. This
mandate poses several challenges, one of which is how to determine whether a training course is effective when it is
deployed to the field and completed by government employees expected to simultaneously meet their fast-paced daily
production requirements in a high-stress work environment. This paper reports how an argument-based approach is
being assessed as an alternative courseware validation process that provides practical evidence to allow reasoned, datadriven interpretations and conclusions regarding the effectiveness of a deployed course. The approach uses both
qualitative and quantitative data to establish reasoned arguments to make the evidence-based interpretations of the
data. This paper discusses how this argument-based framework for measuring, analyzing, and reporting validation
results is evolving to make reasoned determinations about the effectiveness of deployed e-learning products conducted
in uncontrolled work environments.
TUESDAY, 1 DECEMBER, 2015 ROOM S320D
1600
Enhancing Good Stranger Skills: A Method and
Study (15071)
1630
Curriculum GPS: An Adaptive Curriculum
Generation and Planning System (15369)
Notes
14
ENHANCING GOOD STRANGER SKILLS: A METHOD AND STUDY
Robert Hubal, Mike van Lent, Bob Marinier, Chris Kawatsu, Bob Bechtel
Ann Arbor, MI
A good stranger (GS) is a professional who can effectively integrate tact and tactics, in order to create
positive outcomes in difficult social encounters. For military personnel, creating positive social outcomes
enhances mission effectiveness and force security, and supports broader strategic and tactical objectives.
Some evidence suggests that military personnel may come into situations with preconceived ideas, or
frames, about how to behave, not all of which involve GS tactics. Deliberate training in a variety of such
situations is required to gain more effective control of people and situations. As part of a large DARPAfunded program maximizing especially high-risk, high-consequence interactions occurring in unfamiliar
social terrain, we investigated how to train military personnel on GS skills in order to adapt to and
successfully manage these interactions. Training was based on a theoretical structure for GS skills-based
interaction; generally this flow maps to the basic sequencing for most interactions that produce positive
end states: An approach, a period of framing, orientation and sensemaking, followed by engagement in the
evolving business of the encounter. This engagement often involves necessary rapport-building, trouble
recovery, and appropriate departure. We conducted an experiment with students at the Infantry Basic
Officers Leader Course at Ft. Benning using a browser-based tool developed under the DARPA funding.
We presented 32 students with a series of storylines, some having multiple injects, and asked the students
to demonstrate their perception of relevant cues in a scene as they observed the interaction depicted by the
storyline. We found this training to have a positive effect in increasing behaviors associated with a GS
frame. In this paper we detail the training approach, describe our study, and offer recommendations on
improving GS skills training in military personnel.
CURRICULUM GPS: AN ADAPTIVE CURRICULUM GENERATION AND PLANNING SYSTEM
Mustafa İlhan Akbaş, Prateek Basavaraj
Michael Georgiopoulos
Özlem Garibay, Ivan Garibay
Department of Electrical Engineering
Office of Research and Commercialization
and Computer Science
Orlando, Florida
Orlando, Florida
In educational systems, there has been an increasing interest for the innovative applications such as educational
data mining and predictive analytics. These applications are utilized by the institutions for fulfilling academic
missions and for improving the utilization of institutional resources. In this paper, we propose the “Curriculum
GPS”, an adaptive curriculum generation and planning system, to provide a quantitative model and an
interactive system that helps to grow and maintain programs with high retention and satisfaction rates in college,
military or corporate education. The Curriculum GPS is composed of three main components: Curriculum
analysis, historical data mining and an adaptive course sequence generation. The existing literature demonstrates
how curricular efficiency correlates to student academic success in terms of graduation and retention rates.
Therefore we first use an approach from the literature to analyze the curriculum under discussion as a directed
graph by considering the conditions among courses such as prerequisite requirements. We conduct network
analysis in this graph and compare our results with the catalog of courses currently in use. Then we combine this
analysis with the historical data of the students and courses to train our model and develop our system’s
database. The resulting system uses this training to create a set of quantitative recommendations for each student
depending on her individual data such as passed/remaining courses, grades and time to graduate. The system
also allows running what-if scenarios to test the outcomes of different choices by students. Therefore it is
advantageous for students, instructors and advisors. The system is being developed for the Information
Technology based departments of one of the largest universities in US by using the curricula and student
datasets from the last thirty semesters. Initial results suggest this novel system provides both insight and
improvement for the institutional education.
15
WEDNESDAY, 2 DECEMBER, 2015 ROOM S320D
0830
Mars Game: Creating and Evaluating
an Engaging Educational Game
(15105)
0900
The Secret for STEM Success:
Employing Technology for
Math Proficiency (15145)
0930
Inquiry and Design Approach to
STEM Education Using
Project-based Learning (15238)
Notes
MARS GAME: CREATING AND EVALUATING AN ENGAGING EDUCATIONAL GAME
Kevin Dill & Spencer Frazier
Lockheed Martin Mission Systems &
Training
Burlington, MA
Barbara Freeman
Standards Work / UC Berkeley
Quincy, MA
Juan Benito
Cooperative Entertainment Inc.
Raleigh, NC
Games have been studied for some time as a possible supplement to classroom-based learning and training,
yet questions remain about how best to create the content. How can learning and gaming be merged in a
way that does not diminish the positive aspects of either? Games that poorly integrate the educational
component suffer for failing to teach, and games that poorly design the entertainment component suffer
because they fail to engage the student. The content, and its comprehension by students, must then be
evaluated to determine whether learning actually occurred. This paper describes the design,
implementation, and evaluation of a game – the Mars Game prototype – that has been crafted from the
beginning with the intent of emphasizing the educational content and the player’s engagement (i.e. the
“fun”) in equal measure. The game teaches 9th and 10 th grade math and programming concepts, and aligns
to the Common Core State Standards for Mathematics. The results of a randomized control study
performed with students from a U.S. high school are provided. The study evaluated the effect of the Mars
Game prototype on students learning and engagement. The study demonstrated that the game statistically
significantly improved learning outcomes against a comparison group. The study also showed that the
treatment group scored significantly higher than the control group on engagement and deep immersion in
the gameplay. These evaluations provide encouraging results reinforcing the design goals set out for the
game as well as the efficacy of game-based learning – when you have a truly engaging game.
16
THE SECRET FOR STEM SUCCESS: EMPLOYING TECHNOLOGY FOR MATH PROFICIENCY
Edward P. Harvey, Jr.
Marvin G. Fuller, Ph.D.
Edward P. Harvey, III
ATLT, LLC
Oglethorpe Charter School
Harvard University
Virginia Beach, VA
Savannah, GA
Cambridge, MA
This paper proposes an effective, affordable, and practical approach to increasing the number of successful Science,
Technology, Engineering and Mathematics (STEM) high school and college graduates. Currently, the primary barrier
for U.S. students pursuing STEM degrees is math proficiency, and the problem is ubiquitous among secondary schools
systems and colleges. The paper presents experiences and use cases that provide an insight as to why the math
proficiency problem exists. The paper highlights considerations as to why the U.S. is ranked 27 out of 34 Organization
of Economic Co-operation and Development (OECD) countries in math proficiency; why only 40% of U.S. high
school graduates who attend college are prepared for college-level math; and why only 13% of math proficient middle
school students become college graduates with a STEM degree. The paper also addresses costs to students, schools,
and the government in terms of time, tuition, and subsidies for students to earn STEM degrees. It then presents an
educational strategy for the integrated use of serious games, software tutoring agents, and webbased technologies to
improve math instruction, review, and remediation.
This paper provides an in-depth discussion on the design, development, testing, validation, and deployment of a
comprehensive algebra readiness program. The program integrates serious game technologies, software tutoring
agents, online math content, and web-based student performance reports. The program was developed by integrating
state and national mathematics standards with commercial game and U.S. DoD training system technologies. It
includes 130 foundational arithmetic and algebraic skills students must master to achieve and maintain STEM
proficiency. Efficacy study results presented in the paper highlight use of the program by middle school, high school,
and college students. Finally, lessons learned are described that can assist in the development of a set of best practices
for developing and using technology-based STEM educational tools.
INQUIRY AND DESIGN APPROACH TO STEM EDUCATION USING PROJECT-BASED LEARNING
Danielle McNeely & Robert
Seltzer
NAWCTSD
Orlando, FL
Susan Nelson
AUVSI Foundation
Arlington, VA
Dr. Stephen Priselac & Dr. Nancy
Priselac
nCASE
Uniontown, Pennsylvania
Eileen Smith
Orlando, FL
Heather Norton & Alicia Frascati
Orlando Science Center
Orlando, FL
Abdul Siddiqui
PEO STRI
Orlando, FL
In the 21st century it is time to effectively apply the Chinese Proverb, “Tell me and I’ll forget; show me and I
may remember; involve me and I’ll understand” to Science Technology Engineering and Mathematics (STEM)
K-12 education (Edwards and Muir, 2004). An effective means to apply this proverb to STEM learning is
through the use of combining Project-Based Learning (PBL) with Inquiry and Design (I&D) Instruction. PBL is
accomplished by conducting a comprehensive study of a particular topic by means of engaging at several
different decision making junctures (Moursund, Kafai, Sandoval, Enyedy, Nixon, Herrera, and Stewart, 2002).
Currently PBL activities are used extensively in post-secondary academia and by industry (internships).
Learners who are exposed to PBL environments not only actively apply engineering and science understanding,
but also tend to obtain a more concrete foundation of science and mathematical knowledge (Lou, Shih, Diez,
and Tseng, 2011). Research indicates it is essential to engage a learner’s interest in a technical career path by
establishing a link between the theoretical knowledge and its application to solve real life problems early on in
the learning experience (Verma, 2011). I&D, which is attracting interest among STEM educators nationwide,
emulates the scientific method in the classroom. I&D is a student-centered approach emphasizing the integration
of inquiry (science, technology and mathematics) and design (engineering) elements. The intent of this paper is
to illustrate how three existing PBL programs can be successfully combined with the I&D teaching methodology
to effectively teach K-12 STEM education curricula. The three programs are: 1) Engineering our Future, 2)
Materials World Modules, and 3) SeaPerch. From these exemplar programs, schools can better decide with
confidence how to adopt these or similar STEM programs into their formal and informal STEM programs.
17
ED-4 Getting In Front of It
1030
Antecedents of Adaptive
Collaborative Learning
Environments (15211)
1100
Adaptable Resilience Training for
Transitioning Veterans
Using Existing Technologies
(15285)
1130
The Changing Face of Military
Learning (15327)
Notes
ANTECEDENTS OF ADAPTIVE COLLABORATIVE LEARNING ENVIRONMENTS
Robert A. Sottilare, Ph.D.,
Joan H. Johnston, PhD.,
& Anne M. Sinatra, Ph.D.
US Army Research Laboratory
Orlando, FL
Shawn Burke, Ph.D.
University of Central
Florida
Orlando , FL
Eduardo Salas,
Ph.D.
Rice University
Houston, TX
Heather Holden, Ph.D.
Mount Washington
University
Manchester, NH
This paper explores the effect of critical precursors to realizing successful collaborative instructional
environments in terms of their interaction within the learning effect model (Sottilare, 2012; Fletcher &
Sottilare, 2013; Sottilare, 2013; Sottilare, Ragusa, Hoffman & Goldberg, 2013) and measurable
relationships to team learning in the literature. We evaluated potential antecedents of successful
collaborative instruction in the literature through a large-scale meta-analysis. Adaptive collaborative
learning environments are group or team instruction where the challenge level of the learning experience is
driven by the shared states (e.g., cognitive, affective, physical) and team performance. Independent of the
computer technology, the methodology we used examined team behaviors which included, but were not
limited to: cognition, communications, coordination, conflict resolution, cooperation, coaching, and
leadership. Recommendations about which team behaviors are critical antecedents to the optimal selection
of instructional strategies, tactics, and techniques (policies) during adaptive training and educational
experiences are also discussed with respect to their effect on team learning. This research is important to
the development of effective software-based agents for adaptive systems (e.g. Intelligent Tutoring
Systems) where these agents are responsible for planning and executing actions based on the needs of each
unique team.
18
ADAPTABLE RESILIENCE TRAINING FOR TRANSITIONING VETERANS USING EXISTING
TECHNOLOGIES
Ms. Jill Shepherd & Ms. Jennifer Ukwa
Booz Allen Hamilton
McLean, VA
Suicide rates among Military Service Members (SM) and Veterans remain a crisis despite the efforts of the
Pentagon and Department of Veteran Affairs (VA) to connect these individuals to support, prevention, and
emergency resources. According to the 2012 VA Suicide Data Report, roughly 8,000 veterans are thought to die
by suicide each year. Many contributing factors are directly associated with military experiences such as battle
wounds, deployment stress, grief, anxiety, depression, post-traumatic stress and other mental disorders. This
paper describes how, recognizing that transitioning SMs need a personalized set of resilience skills that change
as their circumstances change, the Defense Suicide Prevention Office (DSPO) and Transition to Veterans
Program Office (TVPO) collaborated to develop training that reaches SMs at one of the most suicide-vulnerable
points in their careers, the transition to Veteran status. Without an option to deliver this sensitive content in a
face-to-face setting, DSPO and TVPO had to work within the constraints of deploying their resilience training
on an existing Sharable Content Object Reference Model (SCORM)-compliant Learning Management System
(LMS). The team aspired to create training that could adapt to individuals’ needs, while also adapting to the
audience needs over time. The paper describes how the designers created a Resilience Gauge that, when taken at
the beginning of the course, customizes the content to the learner’s needs. The paper describes the research
behind the Resilience Gauge, which was developed with experts in both resilience and survey design. The
Resilience Gauge is a tool that allows learners to self-report their current resilience skill by answering 25
scenario-based multiple-choice questions, then assembles a custom set of content based on four domains of
resilience (mind, body, spirit, social) and three proficiency levels (beginner, intermediate, advanced). The paper
describes how designers can repurpose this gauge as an efficient way to create adaptable content.
THE CHANGING FACE OF MILITARY LEARNING
Sae Schatz, Ph.D
ADL Initiative
. Orlando, FL
David Fautua, Ph.D.
Joint Staff J7, Joint
Training
Suffolk, VA
Julian Stodd
SeaSalt Learning
Bournemouth, UK
Emilie Reitz
Alion
Suffolk, VA
Globalization, social media, ever-increasing computing power, and the proliferation of low-cost advanced
technologies have created a level of worldwide complexity and rapid change never before seen. To remain
competitive in this environment, the US Department of Defense and our coalition allies must identify new
ways to empower our forces. In this paper, we assert that part of that solution includes increased
investments in our Human Dimension. Specifically, we argue that military personnel require an expanded
set of competencies, higher levels of nuanced skills such as critical thinking and emotional intelligence,
and more efficient and agile pathways to expertise, and that achieving these outcomes depends, at least in
part, on revising the military learning enterprise.
Towards this end, we outline a vision for the future of military learning, painting a picture of the “art of the
possible” and proposing a roadmap that outlines five enabling conditions needed to achieve this future
vision. The conditions include: (1) Cultivate ubiquitous learner-centric, technology-enabled instruction; (2)
Build upon the foundations of data-driven learning; (3) Foster a learning culture at the organizational level;
(4) Encourage and empower social learning; and (5) Draw upon deliberate practices and the evidencebased body-of-knowledge from learning science. Enacting any one of these conditions will pose significant
challenges, and particular science or technology gaps associated with each condition create additional
hurdles. Nonetheless, we argue that the time is right, in terms of understanding and demand, to take action.
One major step in that direction is to agree upon a shared grand strategy, that is a vision for our Human
Dimension and the military learning system that empowers it. That is the professional dialog this paper
attempts to help inform and encourage.
19
ED-5 Challenging the Game
1400
Game-Based Training for Human-Intelligence
Skills (15067)
1430
A Conceptual Model of Pedagogic Design to
Support Critical Thinking in Commanders
(15075)
Notes
GAME-BASED TRAINING FOR HUMAN-INTELLIGENCE SKILLS
John T. Miller, II
Consortium Research Fellows Program and
Capella University
Fort Benning, GA
Martin L. Bink
Fort Benning, GA
The U.S. Army increasingly relies on game-based training as a tool for skill development in nontraditional
areas such as moral–ethical decision-making, social–cultural awareness, and cognitive reasoning. The use
of game-based exercises is nonetheless a novel approach for training human-intelligence tasks. Humanintelligence tasks are the actions related to collecting information from people and other sources (i.e.,
social networks, print and visual media) to identify elements, intentions, composition, strength,
dispositions, tactics, equipment, personnel and capabilities. In U.S. Army, human-intelligence skills are
taught in a week-long resident course called the Attack the Network (AtN) course. In order to determine
the extent to which game-based training provides a meaningful and effective contribution to the
development human-intelligence skills, two forms of scenario-based practical exercises were compared in
the AtN. Course performance and perceptions of training were compared across students who completed
traditional paper-based practical exercises to students who completed game-based practical exercises in the
Army’s Enhance Dynamic Geo-social Environment (EDGE) desktop training environment. The EDGE
practical exercises did no better in increasing end-of-course test scores than did traditional paper-based
practical exercises. In addition, the paper-based practical exercises were perceived as more beneficial to
learning and course outcomes as compared to the EDGE practical exercises. These results add to the
growing literature that fails to find a relative advantage of game-based training. However, these data as
well as insights from AtN instructors were used to determine how EDGE may have a greater impact on
human-intelligence skills. These insights may have wider applicability for increasing game-based training
effectiveness in other contexts.
20
A CONCEPTUAL MODEL OF PEDAGOGIC DESIGN TO SUPPORT CRITICAL THINKING IN
COMMANDERS
Kia Hong Tan, Teng Howe Lim & Boon Kee Soh
DSO National Laboratories
Singapore
The use of serious games as a potential training solution has found traction in the militaries of many
countries to meet the complex training needs of soldiers. Games repurposed as training platforms have
become more immersive, emotionally engaging, and significantly less expensive as compared to fullfledged simulators. They can also be highly customizable, allow for easy and consistent repetition, and
when designed well, create intrinsic motivation in users. Coupled with advances in cognitive and learning
sciences which shed light on how humans learn as an individual and in a team, it is therefore timely for us
to leverage on these emerging technologies for the next generation of training and learning systems in the
military.
In this paper, a conceptual model of pedagogic design is proposed to support the development of a serious
game targeted at critical thinking for commanders. The paper explains how the model is used to represent
theoretical approaches and to support game design. Key components to train critical thinking are distilled
from the review of learning theories and game design recommendations. This establishes the starting point
towards selecting appropriate processes, tools and resources to develop the game and learning scenarios.
Components of the learning scenarios will be described and related to the appropriate theoretical
approaches. Our assertion is that application of pedagogical processes, tools and techniques is useful to
support a serious game development that is targeted at critical thinking for senior commanders.
ED-6 Instructors for the Force of Future
1600
Designing Instructor Support Tools
for Virtual Shiphandling
Training (15133)
1630
Hey, Remember to Add
Motivational Design to Your
E-learning (15030)
1700
Achieving Educational Excellence:
What do Effective Instructors do?
(15226)
Notes
21
DESIGNING INSTRUCTOR SUPPORT TOOLS FOR VIRTUAL SHIPHANDLING TRAINING
Martin Voshell, Ryan Kilgore, Christopher Hogan, Timothy R. McEwen, David Young
Charles River Analytics
Cambridge, MA
Practice and experience are fundamental to seamanship and shiphandling training, but are significantly
limited by platform availability and at-sea time. With hands-on experience and at-sea assignments
available only to a subset of personnel at a given time, virtual environments (VEs) have the potential to
provide a complementary and cost- effective method to support task mastery without putting lives and
platforms at risk. VE training systems, such as the Conning Officer Virtual Environment (COVE)
currently used by the US Navy for training ship-handling skills, have great potential value to increase
training exposure. However, even these virtual systems are resource-intensive because they require highly
trained instructors to closely monitor individual students’ progress and provide targeted coaching and
feedback. The growing popularity of VE-based training approaches is rapidly outpacing the number of
available instructors, who need better tools to support their delivery of high-quality training to larger
numbers of students. In this paper, we describe ongoing research and development efforts to extend the
Navy’s VE training capabilities by creating work-support tools and dashboard displays that enable COVE
instructors to efficiently monitor and manage larger numbers of students with VE training. Based on
design principles that support attention management, we have created a series of linked alerting displays to
support improved instructor supervision across multiple student training sessions. We also present our
analytic approach, provide design implications and initial instructor support concepts, and discuss how our
approach and initial results are generalizable to other VE-based instructional settings.
HEY, REMEMBER TO ADD MOTIVATIONAL DESIGN TO YOUR E-LEARNING
Geir Isaksen
Commander, NoD University College
Oslo, Akershus Fortress
Siren Elise Frøytlog Hole
Project Manager, Transform AS
Oslo
Student motivation is an essential component of all educational and learning processes. Without
motivation, students lack cognitive presence resulting in little, if any, learning. In the traditional classroom
setting, it’s up to the teacher to facilitate and maintain student motivation. In an e-learning course however,
there is less teacher or facilitator presence and the learner is left alone to interact with the instruction
mostly alone. E-learning designers and developers must integrate appropriate motivational elements to
ensure the learner sustains his/her motivation throughout the entire instruction to maximize the learning
outcome. Over the past few years the Norwegian Armed Forces (NoAF) has incorporated motivational
design elements focused on promoting and sustaining motivation into our e-learning courses based on John
Keller's ARCS Model of Motivational Design. This paper outlines the rationale, methodology, and
resulting implementation.
22
ACHIEVING EDUCATIONAL EXCELLENCE: WHAT DO EFFECTIVE INSTRUCTORS DO?
Heidi Keller-Glaze,
Jonathan Bryson & Ray Morath
William R. Bickley
With the publication of the Army Learning Model (ALM; U.S. Army Training and Doctrine Command, 2011),
the Army seeks to shift the nature of instruction from instructor-centric to learner-centric and integrate
technology into training and education to a greater extent than has been done in the past. Specifically, the ALM
-solving approaches in the classroom, with instructors
Increase the use of interactive technology in learning.
Recent research efforts to develop a framework to select, develop, and evaluate Army instructors revealed a lack
of requirements for instructors of adult learners in an environment of interactive, engaging, and learner-centric
education. Through a review of military and education literature and a workshop with subject matter experts, a
definition of an effective instructor was developed initially. Subsequently, 13 work behaviors and 32 knowledge
elements, skills, abilities, and other characteristics (KSAOs) were identified as being necessary for an instructor
to be effective in a learner-centric environment. These behaviors and KSAOs were then used to generate tasks
performed by instructors in a learner-centric classroom. Data were gathered from instructors regarding the
importance of these tasks, the frequency with which they are performed, and the effectiveness of instructor
training in teaching these tasks. The behaviors, KSAOs, and tasks can be used in the selection, development, and
evaluation of instructors who can effectively implement learner-centric practices and technology into their
instruction. The framework developed through the research and analysis will be described along with the next
steps to identify training for Army instructors. Follow on work with the behaviors, KSAOs, and tasks is focused
on identifying training for Army instructors to prepare them to implement the directives of ALM. This training
is intended to complement the training Army instructors currently receive.
THURSDAY, 3 DECEMBER, 2015 ROOM S320D
ED-7 It’s All About the Learning
0830
Using a Skill Acquisition Theory as a Framework for the
Army Learning (15125)
0915
Transmedia (Social) Learning in the Wild: DoD
SkillBridge for Transitioning Service Members (15162)
Notes
23
INNOVATIVELY APPLYING SKILL ACQUISITION THEORY TO THE ARMY LEARNING MODEL
LTC Glenn A. Hodges, Ph.D.
Human Dimension Division (HDD)
Army Capabilities Integration Center (ARCIC)
Training and Doctrine Command (TRADOC)
FT Eustis, Virginia
The Army Learning Concept (ALC) for 2015 discusses a continuous adaptive learning model and multiple 21 st century
competencies that have been described as critical for U.S. Army Soldiers if the Army is to maintain its competitive
advantage into the future. Unfortunately, neither the ALC for 2015 nor the Army Training Concept (ATC) for 20122020 provide an instantiation of a model or a description of how to integrate or employ new ideas or approaches into
training and education programs and activities to obtain the desired 21st century competencies. Attempts at
transitioning training and learning environments from traditional “brick and mortar” settings to the “point of need”
leveraging new multi-media technologies and approaches are numerous and ongoing. These efforts have not been
coordinated, synchronized or similarly assessed due to the lack of an integrated framework. This paper conveys a
possible solution derived using experiential learning theory and skill acquisition research conducted by Dreyfus and
Dreyfus (1980). The Dreyfus model is inlaid into a current military career, creating the framework for the development
and use of experiential learning inventories (ELI) and competence-based assessments (CBA). Examples of existing
CBA are provided and recommendations supporting the development of ELI are discussed. The data implications of
this proposal are acknowledged. It is believed that if ELI and CBA are used and punctuate a career (e.g. upon initial
entry, upon arrival to and exit from duty assignments, pre- and post- training/education, prior to promotions, etc.) they
will help to inform the development and refinement of new and existing instructional methods and technologies useful
for training and educating the current and future force. Additionally, the information obtained from ELI and CBA will
help to support the efforts of human resource managers and commanders in their efforts to manage their human capital
talent.
TRANSMEDIA (SOCIAL) LEARNING IN THE WILD:
EXPLORING A CONTINUUM OF SUPPORT FOR TRANSITIONING SERVICE MEMBERS
Elaine M. Raybourn, Ph.D.
MajGen Tom Jones, USMC (Ret)
Sandia National Labs* / ADL
DASD, Force Readiness & Training
Outdoor Odyssey
Orlando, FL
Washington, DC
Boswell, PA
While social learning is not a new construct, we now have the ability to extend learning across place and time
instantaneously with social media and digital tools. As we interact in new ways with multiple media we discover new
approaches like transmedia learning, which leverages both social learning and social media. Transmedia learning is
engagement-driven, learner-centric, unfolds across multiple media, and is designed to promote social learning. An
I/ITSEC paper published in 2013 utilized the example of Warrior-Diplomat and was the first in a series introducing
transmedia learning strategies to meet the demands of next generation learning. The present paper is the second in the
series and expands on theories discussed in the 2015 I/ITSEC tutorial “Transmedia Learning in the Wild.” While the
tutorial uses Warrior-Athlete as an example, the present paper departs from the tutorial and the 2013 paper by honing
in on one aspect of transmedia learning—the practice of social learning as it applies to two approaches for preparing
Veterans and transitioning Service members for the civilian workforce. In the spirit of the definition of transmedia
learning provided above, conference participants who review the 2013 paper and attend both the 2015 paper and
tutorial presentations will obtain unique and complementary information from each presentation about transmedia
(social) learning in the wild, or as it naturally occurs in a cultural context. The present paper is divided into three
sections. The first section introduces the need to retrain transitioning members of the Force and provides a description
of the Instruction (DoDI) 1322.29 issued in 2014, titled "Job Training, Employment Skills Training, Apprenticeships,
and Internships (JTEST-AI) for Eligible Service." DoD SkillBridge implements this instruction and is a Deputy
Assistant Secretary of Defense (DASD) Force Readiness and Training initiative to connect transitioning Service
members with civilian training opportunities. Semper Fi Odyssey, a 6-day intensive transition-assistance and career
advancement program for injured veterans, is also presented to illustrate a range of social learning experiences
presented as a continuum. The next section introduces three theories from learning science, cognitive psychology, and
communication that support a social learning continuum. The theories support 5 key design features: Learning context,
culture & community, calibration, and connections. The last section suggests a data collection plan for future
measurement of digital engagement as it applies to transmedia (social) learning in the wild.
24
THURSDAY, 3 DECEMBER, 2015 ROOM S320D
ED-8 Make a Mobile
1030
Innovative Mobile Technologies for
Assessing and Enhancing
Soldier Performance (15082)
1100
A Reference Model for Designing
Mobile Learning and
Performance Support (15225)
1130
Development and Evaluation of
Mobile Adaptive Training
Technologies (15231)
Notes
INNOVATIVE MOBILE TECHNOLOGIES FOR ASSESSING AND ENHANCING SOLDIER
PERFORMANCE
Krista L. Ratwani &
Courtney R. Dean
Aptima, Inc.
Washington, DC
Scott Flanagan
Sophia Speira
Carthage, NC
Camilla Knott &
Frederick Diedrich
Aptima, Inc.
Washington, DC
Jennifer S. Tucker
Army Research
Institute
Ft. Benning, GA
A key element of the Army’s Human Dimension Concept is the need to prepare Soldiers to thrive in
conditions of uncertainty while they contend with ambiguous and amorphous threats. To prepare for such
conditions, advanced talent management strategies are needed to facilitate Soldier development across the
cognitive, physical, and social domains. Comprehensive talent management systems must ultimately
leverage assessment tools to gather large amounts of data to enable a detailed determination of Soldier
strengths and weaknesses and facilitate continuous learning. The question remains, however, about how
best to achieve this goal. This paper reports lessons learned from research with the Army Reconnaissance
Course (ARC), Ft. Benning, GA, to assess and track student performance over time in both performance
outcomes (e.g., fundamental skills, understanding information needs) and leader attributes (e.g.,
anticipation, accountability). The final ARC performance assessment system included a mobile application
to record student observations, a method to link those observations to key competencies, and a method for
presenting trends over time. The trending method enabled student data to be aggregated across instructors
and over classes to demonstrate larger changes in performance over time. In this paper, we present the
methodology for developing this assessment system, results from an evaluation of the system, and
reactions to employing the full assessment system during a course. The findings reflect the results from the
in situ testing and use of the assessment system to include additional features which facilitate future utility
and promote usability. Implications of the research are discussed to provide suggestions and future
research questions to inform the creation of a comprehensive Soldier assessment system as the Army
strives toward effective talent management strategies.
25
A REFERENCE MODEL FOR DESIGNING MOBILE LEARNING AND PERFORMANCE SUPPORT
Peter Berking, Jason Haag
Advanced Distributed Learning (ADL) Initiative
During recent years of increased smartphone and tablet adoption there has been a growing interest in how
to improve training and performance opportunities with mobile learning and mobile performance support.
Increasingly, instructional designers and developers of traditional eLearning are realizing that the design
paradigms for mobile learning are significantly different. Results from a needs assessment conducted for
the Advanced Distributed Learning (ADL)'s Mobile Training Implementation Framework (MoTIF) project
in 2014 identified a strong demand for a mobile learning design model that can effectively inform, situate,
and invite consideration of tactical learning approaches, mobile usage patterns, and mobile affordances.
This paper is based on nearly three years of research findings on mobile learning and performance support
as part of the MoTIF project. The findings led to the development of a reference model that could improve
the design of training and performance support solutions for mobile devices. The reference model
components were substantiated by the quantitative and qualitative data collected during the needs
assessment and will be iteratively refined and evaluated for improvements in the future. While the model
will continue to capture new considerations as an innovative mobile learning design strategy, it can
actually be leveraged and adopted by DoD education and training initiatives today as either a conceptual
framework or decision support tool.
DEVELOPMENT AND EVALUATION OF MOBILE ADAPTIVE TRAINING TECHNOLOGIES
Rodney Long
Army Research Laboratory
Orlando, FL
Jessie Hyland & Joanne Barnieu
ICFI International
Fairfax, VA
This research involves development and evaluation of adaptive training strategies. The current prototype mobile
training technology was designed to include an adaptive feature within one instructional cycle (at the Terminal
Objective or Module level). Based on pre-assessment questions, the learner’s sequence within the instructional cycle
will be adapted accordingly. Should certain learning objectives be considered mastered via determined pre-requisite
knowledge, those associated lessons will be collapsed in the curriculum yet accessible if desired by the learner.
Additionally, after completing a lesson, the learner can receive guidance from the system if he or she answers the
intermittent assessment item (Check on Learning) incorrectly and is also prompted to return to the lesson content
should the guidance not assist the learner in answering the assessment item. To examine the effectiveness of the
adaptive training, we conducted an experiment to compare students receiving the adaptive version of the prototype
(i.e., treatment condition) versus those receiving the non-adaptive version (i.e., control condition). Specifically,
participants were compared on the following dimensions: learner reactions; training efficiency; and training
effectiveness. On learner reactions, we found some preliminary descriptive evidence that participants in the treatment
condition were more engaged and held more favorable perceptions of the training adaptability than those in the control
condition but the difference was not statistically significant. Evidence on the potential unintended negative
consequences of the adaptive training was inconclusive. On training efficiency, we found the adaptive training to be
more efficient than its non-adaptive counterpart, as expected. On training effectiveness, participants in the treatment
condition performed as well as their counterparts in the control condition on an independent hands-on performance
test. Our findings are encouraging but highlight the need to continue robust research in tandem with the development
and integration of new technologies in order to realize the full potential of adaptive training.
26
TUESDAY, 1 DECEMBER, 2015 ROOM S320C
EC-1 Easy Button
1400
Automated Simulation Creation
from Military Operations
Documents (15227)
1430
Scheduling Training to Manage
Acquisition & Decay (15340)
1500
M&S as a Service: Paradigm for
Future Simulation
Environments (15324)
Notes
AUTOMATED SIMULATION CREATION FROM MILITARY OPERATIONS DOCUMENTS
John Balint, Jan M. Allbeck, Michael R. Hieb
George Mason University
Fairfax, VA
The creation of virtual reality simulations for training or analysis is an arduous process requiring
specialized knowledge. Graphical models and even animated, articulated figures can now be obtained from
websites or hired artists. Even after these assets are obtained, putting scenes together and authoring
character behaviors can be a lengthy process. Furthermore, ensuring that character behaviors will be
successfully performed in a virtual environment is often a trial-and-error process. Automating the creation
of these behaviors and facilitating their modification by Subject Matter Experts (SMEs) – as opposed to
technicians – will shorten the time required and reduce costs.
This paper presents a framework (VerbsEye) for using descriptive texts, such as military operations
documents to semi automate simulation creation. While previous research, such as the WordsEye system,
have created static scenes from natural language inputs, our framework further automates the process and
includes the generation of agent behavior scripts from the text. Specifically, we present a text-to-scene
system that generates 1) scene scripts and 2) agent behavior scripts for virtual environments. The spatial
information required for the scenes is obtained both explicitly through prepositions found in the input text
and implicitly from the described agent behaviors. Motion data used to depict agent behaviors is exploited
to provide additional spatial constraints and assure the behaviors will be possible.
Automated scene creation is challenging and unlikely to result in perfection. The VerbsEye framework is
evaluated in terms of how well sample military operations documents can be used to generate scenes and
behaviors. The specific metrics used are the percentage of scenes and behaviors in the sample operation
documents successfully processed. Our framework shows how additional automation can be used to enable
SMEs and technicians to better and more quickly create training tools and environments.
27
SCHEDULING TRAINING TO MANAGE ACQUISITION & DECAY
Mohammed Eslami, Ph.D. & Jared Freeman, Ph.D.
Aptima, Inc.
Washington, DC
Scott Pappada, Ph.D.
Aptima, Inc.
Dayton, OH
The accelerating effects of adaptive training systems are well established (Lesgold, 2012; Cohn &
Fletcher, 2010). This power might be enhanced further by scheduling training to accelerate acquisition and
scheduling re-training to reduce decay. Models of acquisition and decay have been available to support
scheduling since the early days of memory research (Ebbinghaus, 1913). But these models are derived
mainly from laboratory tasks that are learned and executed over seconds or minutes, and performed in
isolation from competing tasks. The models are much less explanatory or predictive over real world tasks
that are complex, learned and executed over hours and days, and situated in a river of daily assignments
that impose the scientifically acknowledged cause of skill decay: interference with memory retrieval (Farr,
1987; Arthur, et al., 1998). In this paper, we propose a new approach to acquisition and decay modeling to
make the science of skill acquisition and decay more useful and usable. The approach applies machine
learning techniques to model skill acquisition and decay. We apply these methods to a large dataset from a
game-ified working memory exercise, compare the performance of these methods with a conventional
technique, and present the argument for applying these methods to predict learning and schedule training
for realistically complex tasks such as system diagnosis and corrective maintenance.
M&S AS A SERVICE: PARADIGM FOR FUTURE SIMULATION ENVIRONMENTS
Robert Siegfried
aditerna GmbH
Riemerling/Munich, Germany
Tom van den Berg
TNO
The Hague, Netherlands
This paper presents results of NATO activities in the area of “M&S as a Service” (MSaaS). It illustrates potential
benefits with regards to quality, efficiency, and interoperability that may be achieved by MSaaS and provides an
insight how some of the existing challenges are currently being addressed.
As M&S products are highly valuable to NATO and military organizations it is essential that M&S products, data and
processes are conveniently accessible to a large number of users as often as possible. This requires a new “M&S
ecosystem” that has to support stand-alone use as well as integration of multiple simulated and real systems into a
unified simulation environment whenever the need arises. Due to many factors, service-based approaches are
considered to be very promising for realizing future simulation environments. This idea is known as “Modeling &
Simulation as a Service” (MSaaS).
NATO Modeling and Simulation Group 131 (“Modelling and Simulation as a Service: New concepts and Service
Oriented Architectures”) has investigated the idea of “M&S as a Service” as a 1-year Specialist Team. MSG-131
defined a consistent MSaaS terminology and placed MSaaS into the wider context of the NATO C3 Classification
Taxonomy. Second, an exhaustive overview about service-based approaches used in the M&S domain in NATO and
Partner Nations was produced. Third, a comprehensive overview of existing service-oriented (reference) architectures
in the M&S domain was produced.
A more detailed investigation of MSaaS and first steps towards an incremental implementation of a “Federated M&S
Eco-System” are objectives of MSG-136 (“Modelling and Simulation as a Service - Rapid deployment of
interoperable and credible simulation environments”) which started its 3-year term in November 2014. This paper
presents results of MSG-131 and current work done by MSG-136.
28
EC-2 Are We There Yet?
1400
Military Vehicle Training with Augmented Reality
(15180)
1445
Modelling a Helicopter Training Continuum to Support
System Transformation (15165)
Notes
MILITARY VEHICLE TRAINING WITH AUGMENTED REALITY
Jonathan Brookshire, Taragay Oskiper, Vlad Branzoi,
Supun Samarasekera & Rakesh Kumar
SRI International
Princeton, NJ
Sean Cullen & Richard Schaffer
Lockheed Martin Mission Systems and Training
Burlington, MA
In order to be effective in the field, the military trains warfighters to operate its many ground vehicles. The goals
of training are for the warfighter to learn vehicle and weapon operations and dynamics (e.g., how the vehicle and
gun turret work and “feel”) in live tactical situations. Additionally, because many vehicles require multiple
operators (e.g., a gunner and driver), team coordination is an important element of the tactical training.
The military employs both live and virtual reality training to achieve these goals. Live training, especially
gunnery, requires significant facilities and range infrastructure and is also limited to specific sites due to safely
restrictions. Such training events generally require travel/transportation to CTCs and ranges. Unfortunately, live
training is expensive. In this paper, an augmented reality based vehicle training system is presented. The trainees
are able to drive on physical terrain and engage virtual entities for tactical and gunnery training. By augmenting
the real world using virtual entities and effects, along with existing training aids and devices, training anywhere
and anytime is enabled.
The details of the vehicle-borne augmented reality system for augmenting both the driver’s periscope and the
gunner’s remote weapon sight are presented. The system relies on inertial measurements, cameras, and GPS to
provide jitter free, robust and real-time 6-DOF (degree of freedom) pose estimation. These poses are used to
render synthetic targets (e.g., dismounts, technical, target) to the driver and gunner. An iPad style instructor
interfaces controls the augmented engagement and provides student scores.
The system is evaluated on an Army Stryker vehicle operating in a real range. The consistency and quality of
target insertions between the driver’s three augmented periscopes and the gunner’s augmented weapon sights are
compared. The importance of each sensor is evaluated by removing its input and comparing.
29
MODELLING A HELICOPTER TRAINING CONTINUUM TO SUPPORT SYSTEM TRANSFORMATION
Michael Johnstone, Vu Le, Burhan Khan,
& Doug Creighton
Centre for Intelligent Systems Research
Geelong, Vic
Ana Novak, Vivian Nguyen, & Luke Tracey
Defence Science and Technology Organisation
Melbourne, Vic
This study investigates the role of system dynamics (SD) modeling to support strategic decision making
for an aviation training continuum that is going through major change. The Australian helicopter training
continuum (HTC) is currently undergoing transformation, with restructure and consolidation of training
schools and training platforms across multiple services. In this research, we introduce a novel SD-based
HTC simulation architecture to facilitate the discovery of relationships between student and instructor
development and flow dynamics. The proposed simulation architecture employs hybrid push – pull flow
control to quantify transience and estimate recovery time after a policy change or disturbance. This
architecture allows for multiple student and instructor types, and their respective intake levels and pass
rates. Here the instructor variables include availability, specialization and experience. Enos (2011)
successfully explored the application of SD modeling to understand the behavior for combat aviation
training in an individual school. This research employs a similar modeling philosophy, but takes a higher
level view of the system by looking across multiple training schools, which introduces complexity due to
pooling, latency and the amplification of affects across the system. The ability to identify causal
relationships allowed stakeholders to develop a deeper understanding of the underlying systemic problems,
such as delayed transitions between schools and instructor shortages, whilst the hybrid “push-pull” design
allowed us to quantify the pooling of students between schools.
EC-4 Pinging and the Brain
1600
Cognitive Two-Way Interactions In
an Immersive Virtual
Reality Environment (15332)
1630
Modeling and Integrating Cognitive
Agents Within the
Emerging Cyber Domain (15232)
1700
Tablet Computer Call for Fire
Simulation Proof of Concept
Study Results (15008)
Notes
30
TOWARD COGNITIVE TWO-WAY INTERACTIONS
IN AN IMMERSIVE VIRTUAL REALITY ENVIRONMENT
Brennan D. Cox, Harvey M. Edwards, Kathrine A. Service, Pinata H.
Seth A. Reini
Sessoms, Jose A. Dominguez, & Weimin Zheng
Navy Experimental Diving Unit
Naval Health Research Center
Panama City Beach, FL
San Diego, CA
The use of Immersive Virtual Reality Environments (iVREs) for training and rehabilitation purposes is growing
in popularity. An emerging topic in iVRE program development, particularly programs aimed at cognition
enhancement, is implementing two-way interactions between the user’s cognitive state and the iVRE to achieve
maximal effects. This process incurs several outstanding challenges. For example, although recent advances in
electroencephalography (EEG) have revealed a number of neural correlates/signatures for a variety of
operationally-relevant cognitive states (e.g., attentiveness, fatigue), most if not all of these neuromarkers have
not been validated in iVREs. The current paper addresses this gap by describing efforts to achieve high quality
EEG signals in an iVRE with millisecond-time synchronization between the two systems. To achieve these
goals, we evaluated several mobile EEG systems, incorporated off-the-shelf hardware, and developed custom
software to effectively implement the EEG devices into the Physical and Cognitive Operational Research
Environment (PhyCORE), an iVRE located at the Naval Health Research Center in San Diego, California. As a
result, the PhyCORE can now provide cognitive information about human subjects through on-line monitoring
of brain activity patterns. Equipped with this capability, the PhyCORE is ready for further development of
individualized training and rehabilitation programs based on the subjects’ cognitive states as assessed in realtime and in a real-life environment. The technical challenges and solutions described herein can be easily
generalized and adapted for other iVREs, and represent a critical step toward optimization of the humanmachine interaction.
MODELING AND INTEGRATING COGNITIVE AGENTS WITHIN THE EMERGING CYBER DOMAIN
Randolph M. Jones, Ryan O'Grady
& Denise Nicholson
Soar Technology
Ann Arbor, MI
Robert Hoffman, Larry Bunch &
Jeffrey Bradshaw
IHMC
Pensacola, FL
Ami Bolton
Office of Naval Research
Arlington, VA
One of the elements missing from virtual environments in the emerging cyber domain is an element of
active opposition. For example, in a training simulation the instructor assigns the student a task or
objective, and the student then practices within the environment (the “cyber range”) until they feel
comfortable with the task or are able to demonstrate the requisite level of mastery. The environment may
have static defenses, such as access control or firewalls, or a fixed set of intrusion methods to defend
against, but it typically lacks any active opposition that might adapt defensive or offensive actions (e.g.,
monitor logs, blocked connections, exploit switching or information gathering). This is akin to training
fighter pilots against adversaries who know how to use their weapons, but do not have any tactical or
strategic goals beyond that. This is unfortunate for two reasons: 1) it trains cyber operators to behave as
though opponents do not have a tangible existence or do not have higher-level goals, and 2) it ignores an
opportunity to tailor the student’s learning experience through adjustable adversary behavior. Cognitive
agents have the potential to transform the cyber operations training experience. The application of
cognitive agents to the roles of cyber offense and defense would provide a more complete cyber ecology
for training purposes and thus a more realistic training experience for the student. There are two key
challenges to creating such cyber agents: 1) modeling the complex, and continually evolving, processes of
cyber operations within a cognitive architecture, and 2) defining the tools and data standards to enable
cognitive agents to interoperate with networks in a portable way. This paper discusses novel models of
cyber offensive and defensive behavior based on observation and elaboration of human expertise, as well
as an approach to the creation of software adapters that translate from task-level actions to network-level
events to support agent-network interoperability.
31
TABLET COMPUTER CALL FOR FIRE SIMULATION: PROOF OF CONCEPT STUDY RESULTS
James Reynolds, USMC
Marine Corps Systems Command
MCB Quantico, VA
Craig Smith, USMC
II Marine Expeditionary Force
Camp Lejeune, NC
Call for fire (CFF), the coordination of indirect artillery and mortar fires by a ground observer, is an ideal
mission set for virtual environment (VE) training. CFF is a United States Marine Corps (USMC) core
competency and is a perishable skill that requires frequent reinforcement training. The requirement to
expose Marines to initial and recurrent CFF training is hampered by the expensive and time-consuming
nature of live indirect fire training. The USMC currently has a CFF simulation training capability, but
access is limited by the fixed site nature of the simulations.
This paper presents the results of a proof-of-concept study that developed and tested a tablet-based CFF
training simulation. The objective of this study was to investigate the comparative value of tablet-based
CFF VE training. The research team designed and developed a tablet-based CFF prototype and then
executed a user feedback experiment that compared the tablet solution to the USMC’s current personal
computer (PC) based CFF simulation, ObserverSim. The comparison focused on end user opinions
regarding the training value and effectiveness of the tablet’s multifunction interface relative to
ObserverSim’s traditional mouse and keyboard interface.
End users with and without previous CFF experience registered an overwhelming preference for the CFF
tablet prototype (p=0.002). While the tablet prototype was primitive and of much lower fidelity than
ObserverSim, participants liked the tablet’s ability to mimic real world physical motion, its ease of use,
and shallow learning curve. These study results offer the modeling and simulation community important
lessons learned and a realistic example of how to exploit the tablet’s multifunction user interface to further
training simulation development efforts.
EC-4 Avatars Crossing
1600
Turn-Based Gaming for Convoy
Commander Training
(15036)
1630
An Immersive Live / Virtual Bridge
Approach with Ultra
Wideband Tracking Technology:
Phase II (15024)
1700
Virtualizing Humans for Game
Ready Avatars (15023)
Notes
32
TURN-BASED GAMING FOR CONVOY COMMANDER TRAINING
Rudy Boonekamp, Tijmen Muller
Jur de Vrijer
TNO
SIMCEN, Royal Netherlands Army
Soesterberg, The Netherlands
Amersfoort, The Netherlands
With the increasing complexity of current-day military operations, effective education and training of military
commanders is of vital importance. Commanders need to perform within a broader range of conflicts; unpredictable
threats and civil-military interaction place a great demand on their decision making skills. Because defense is
transitioning towards a leaner organization, efficient and innovative tools are needed to provide better training value.
The possibility to train frequently and learn from experience is indispensable.
The Royal Netherlands Army sees potential in the use of serious games to meet these demands. This paper presents the
results of a research project that explores the use of ‘turn-based gaming’ for training convoy commanders. In a turnbased serious game, a scenario is played in rounds which have a distinct planning and execution phase, making it
possible to control time compression and time pressure. The advantages of this concept are that 1) the trainee can gain
experience controlling a large number of units; 2) less experienced trainees can focus on tactical decision making as
the complicating real-time factor is removed; and 3) trainees can plan and reflect on tactical decisions while staying
immersed in the game.
A prototype game was developed using VBS2, implementing the functionality and user interface for a turn-based
convoy scenario. In a pilot session, seven logistics trainees played through three scenarios of increasing difficulty. The
potential advantages of turn-based gaming were assessed using a questionnaire.
The results indicate that the participants gain relevant experience, insight in effective communication and that turnbased gaming helps them learn by experience through fast loops of planning, execution and reflection. However,
technical limitations and the limited scope of the experiment keep us from final judgment whether turn-based
mechanics help trainees stay immersed in the game.
AN IMMERSIVE LIVE / VIRTUAL BRIDGE APPROACH WITH ULTRA WIDEBAND TRACKING
TECHNOLOGY: PHASE II
Jay Saffold, Tovar Shoaf
& Jason Holutiak
Research Network, Inc
Kennesaw, GA
Pat Garrity & Timothy Roberts
U.S. Army Research Laboratory-Human Research and Engineering
Directorate, Simulation and Training Technology Center (ARL-HRED STTC)
Orlando, FL
The U.S. Army Research Laboratory-Human Research and Engineering Directorate, Simulation and Training
Technology Center (ARL-HRED STTC) performs research and development in the field of live/virtual and immersive
technology with real-time Ultra-WideBand (UWB) tracking technology. This technical challenge has been thoroughly
researched for many years and recently UWB technologies have become more mature. The basis of these studies is
that live soldiers must be accurately located while virtual soldiers must stay immersed all within a common real
environment. A novel integrated system approach previously developed has been updated to take better advantage of
new UWB tracking systems, inertial measurement units, and global positioning system sensors. These redundant
tracking sensors with uncorrelated error sources have been intelligently fused in real-time and combined with existing
inverse kinematic technologies related to immersive systems developed by STTC, to provide a fast update rate
tracking solution with full body articulation. The UWB component has also been optimized to allow for faster update
rates and more intelligent responder choosing algorithms with transitioning between responder zones in the physical
area; with the benefit of reducing the total UWB infrastructure requirements. This paper discusses extending these
ongoing efforts to a more simplified system design and initial experimentation to demonstrate an improved soldier
tracking and telemetry system which offers seamless indoor/outdoor tracking capabilities for live/virtual bridging with
sufficient accuracy for high fidelity demonstration at the STTC facility, Military Operations for Urban Terrain, and
other physical locations applicable for dismount training. The solution to real-time 3D location with high accuracy (<
1 ft) suitable for augmented reality over operational environments requires redundant systems with equivalent
accuracy (when available), uncorrelated error sources to provide at least one tracking modality in denied conditions,
and a high update rate for real-time systems.
33
VIRTUALIZING HUMANS FOR GAME READY AVATARS
Jay Saffold, Tovar Shoaf
& Jason Holutiak
Research Network, Inc
Kennesaw, GA
Timothy Roberts & Pat Garrity
U.S. Army Research Laboratory-Human Research and Engineering
Directorate, Simulation and Training Technology Center (ARL-HRED STTC)
Orlando, FL
The U.S. Army Research Laboratory-Human Research and Engineering Directorate, Simulation and Training
Technology Center (ARL-HRED STTC) performs research and development in the field of creating realistic,
individualized virtual avatars from live subjects that retain the physical characteristics and appearance of the
subject including height, weight, skeletal dimensions, body morphology and facial/body appearance. While
photogrammetric extraction technologies are maturing there are a number of additional steps which must be
performed to “virtualize” live humans into game ready avatars. Game ready in this context means the mesh
stretches properly with motion, there are sufficient level-of-detail options, and the number of polygons is
optimized for computer rendering in real-time on commercial graphics adapters and central processing units.
Photogrammetric algorithms which extract mesh information from 3D subjects also do not inherently include
the underlying bone structure (rigging) required for avatars to move in virtual environments. A novel integrated
system approach developed leverages low-cost data capture systems and targets automation of all the steps
necessary to go from live human to a high-fidelity game-ready avatar. This paper discusses the different trade
spaces associated with various photogrammetric techniques/algorithms, commercial software packages, data
capture approaches, subject lighting, frame occupancy, motion during data collection impacts, and converting
what is originally a very dense mesh through “retopologization” into optimized levels-of-detail which are
properly weighted to a virtual bone system. Each step in the process is discussed along with approaches for
automation and the associated trade spaces which affect the quality of the outcome.
WEDNESDAY, 2 DECEMBER, 2015 ROOM S320C
EC-5 Fidelity Matters
0830
Reliably Assessing the Effectiveness
of a Plan with Models
of Verying Fidelity and Under Time
Constraints (15060)
0900
Requirements for Future SAFs:
Beyond Tactical Realism
(15193)
0930
Required Fidelity of simulated
Wound at the Point of Injury
(15351)
Notes
34
RELIABLY ASSESSING THE EFFECTIVENESS OF A PLAN USING MODELS OF VARYING FIDELITY
AND UNDER TIME CONSTRAINTS
Steven de Jong, Wouter Noordkamp, Nick van der Poel, Selmar Smit
TNO Defence, Safety & Security, The Netherlands
Assessing the effectiveness of a plan, given multiple potential scenarios, is a common problem for
analysts, espe-cially in the military domain. This problem can seriously impact the safety of the people that
are involved in planned missions. More precisely, the availability of multiple models, with varying levels
of fidelity, leads to the complex task of selecting the best model(s) to assess the effectiveness of a plan.
Under time constraints, optimal model selec-tion depends not only on the fidelity of the models at hand,
but also on the nature of the possible scenarios the plan applies to, such as the potential presence of
stochastic variables and the number of different scenarios that have to be evaluated in order to obtain a
reliable estimate of the true effectiveness of the plan. In this paper, two algorithms are presented to
maximize the reliability of the obtained plan effectiveness under time constraints. To this end, the algorithms select the best model(s) as well as the most appropriate scenarios. Both algorithms have been tested
on syn-thetic data as well as on two Navy-related use cases. Results show that both algorithms reach a
higher level of relia-bility within the given amount of time than conventional approaches. Thus, they allow
analysts to better assess the effectiveness of their plans and therefore they increase the safety of everyone
involved in planned missions.
REQUIREMENTS FOR FUTURE SAFS: BEYOND TACTICAL REALISM
Robert E. Wray, PhD
Soar Technology, Inc. NAWC TSD
Ann Arbor, MI
Heather A. Priest, Melissa M. Walwanis, & Katherine Kaste
NAWC TSD
Orlando, FL
A key component of realistic and effective training in simulation is the behavior of semi-automated forces
(SAFs). SAFs provide opponents, friendly forces, and other dynamic entities within the simulation. In most
cases today, SAFs are designed and implemented to be tactically realistic; that is, they take actions that carry out
good tactical decisions. As a result, SAFs are typically evaluated in terms of the realism or “fidelity” of their
actions to the tactical situation and not with regard to training effectiveness.
We contend SAF tactical realism is a necessary but incomplete requirement for cost-effective and trainingeffective deployment of SAFs for simulation-based training. SAF behavior should also be modulated by
scenario/exercise goals and also by the learning needs of individual trainees. In practice, these additional
requirements tend to surface during delivery of training, requiring human instructor/operator teams to intervene.
Interventions both increase the cost of simulation-based training and potentially lower the aggregate
effectiveness of that training: delivering an appropriate experience at an apt time to the trainee is contingent on
the attention and action of the instructional team.
Further, as SAFs are increasingly used in mixed live-virtual-constructive training situations, SAFs that consider
only tactical decisions will further limit scalability and increase the operational cost of LVC training.
In response, we suggest that imbuing the training system with the capability to understand and support scenario
goals and individual training needs can make SAFs more practical for everyday training. We present examples
of adaptation and variation that may be important for training but that are not typically embedded in a tactical
SAF. We discuss the implications of these missed requirements and outline suggestions for incorporating
interpretations of learning context in future simulation systems based on experience researching and developing
such a capability. Finally, we outline methods for verifying and validating SAFs designed to meet these
additional requirements.
35
REQUIRED FIDELITY OF SIMULATED WOUNDS AT THE POINT OF INJURY
M. Beth H. Pettitt
Human Research and Engineering Directorate
Simulation and Training Technology Center
Orlando, Florida
At the point of injury, critical medical tasks include finding and identifying the injury as well as applying
the appropriate initial care. A considerable amount of research and development has already occurred to
increase the fidelity of simulated wounds for training, primarily at the point of injury. As material and
moulage techniques mature, and as more relevant data is collected on tissue properties, it is worth
examining what fidelity is really required for training at the point of injury. This effort will explore the
current state of wound simulation and propose a basic test methodology to assess what fidelity is adequate.
Secondly, this effort will analyze the differences in technology effectiveness of two and three dimensional
(2D and 3D) wound moulage. Other factors that will be examined including cost comparisons between the
average 2D wound and silicon-based 3D wound, as well as the time to apply each type of moulage.
Finally, conclusions will be discussed on the training effectiveness of the two types of moulage and
recommendations will be made on the appropriate use of each in medical training.
WEDNESDAY, 2 DECEMBER, 2015 ROOM S320F
EC-6 It’s All About Veterans
0830
Virtual Interview Training Increases
Job Offers for Veterans
and Others (15013)
0900
Stepping Stones – An Augmented
Reality Rehabilitation
Game (15181)
0930
The VA Virtual Medical Center:
Implementing a Vision for a
Virtual Healthcare Campus for our
Veterans (15358)
Notes
36
VIRTUAL INTERVIEW TRAINING INCREASES JOB OFFERS FOR VETERANS AND OTHERS
Dale E. Olsen, PhD &
Laura B. Humm, BS
SIMmersion
Columbia, Maryland
Matthew J. Smith, PhD, LCSW,MPE;
Michael Fleming, MD, MPH & Neil Jorden, PhD
Northwestern University Feinberg School of Medicine
Chicago, Illinois
Morris D. Bell, Ph.D.,
ABPP
Yale School of Medicine,
Dept. of Psychiatry
It is difficult for Veterans with Post Traumatic Stress Disorder (PTSD) and others with disabilities to enter the
workforce, resulting in a low employment rate. The job interview presents a critical barrier for obtaining
employment. To improve job interview skills and the employment prospects of people with disabilities, we
developed Molly, a virtual human resource manager. The simulation provides repeated job interview practice
with extensive feedback and accommodates a variety of special needs. This paper will focus on the methodology
and steps used to develop the simulation, and then report on four single-blind controlled studies and four field
validations of the training solution.
During each virtual interview, Molly asks trainees questions about their skills and experiences. Using
information provided on a job application, she randomly selects questions tailored to the trainee’s needs from a
database of 1,200 options. The trainees practice until they master the skills at three difficulty levels.
The four studies included veterans with PTSD (n=33), people with mood disorders (n=37), people on the autism
spectrum (n=26), and people with Schizophrenia (n=32). Those who used the simulation (treatment)
demonstrated significantly greater improvement than the control group during live role-play interviews showing
efficacy. They were also more confident in their interview abilities. Separate follow-up studies surveyed people
from these study groups after six months. For each follow-up study, people from the PTSD and the Mood
Disorder Cohorts were combined. The data analysis used logistic regression to adjust for known covariates and
to estimate the odds of receiving a job offer. For each study group the estimated odds of receiving a job offer
were about 8-9 times greater for the treatment group than the control group.
STEPPING STONES – AN AUGMENTED REALITY REHABILITATION GAME
Stuart Armstrong
QinetiQ Inc.
Orlando, Florida
Gait disabilities are highly prevalent in veteran populations and include a wide range of symptoms, often
caused by trauma or disease. Recent experimental techniques suggests that rapid advancement of
Augmented Reality (AR) and hybrid virtual reality (VR) technologies have the potential for simulation of
sensorimotor training in gait rehabilitation. This paper describes the development of a game based AR
system to support the rehabilitation of lower limb amputees.
The AR system was built was designed to be a standalone wearable system that can be used outside of a
clinical setting. Initial trials were held at the Providence Veteran Administration Medical Centers Gait and
Motion Analysis Laboratory and the AR system was identified as a novel tool that can be used for gait
rehabilitation in the clinic and the home.
37
THE VA VIRTUAL MEDICAL CENTER:
IMPLEMENTING A VISION FOR A VIRTUAL HEALTHCARE CAMPUS FOR OUR VETERANS
Rosalyn P. Scott, Brian V. Burke, Cathy D.
Nancy Benton
Jennie Gallimore
Graham & Terry L. Oroszi
Veterans Health
Wright State University
Veterans Health Administration
Administration
Dayton, OH
Dayton, OH
Spokane, WA
Helga Carabello
Mary E. Davidson
Paul T.
Sean C. McCoy
Manny
Veterans Health
Veterans Health
Ingmundson
Veterans Health
Dominguez
Administration
Administration
Veterans Health
Administration
Veterans Health
Portland, OR
Cleveland, OH
Administration
Gainesville, FL
Administration
San Antonio, TX
Orlando, FL
The Veterans Health Administration is the largest integrated health system in the world serving Veterans in both
urban and rural environments. To enhance clinical outcomes and education, a VA Virtual Medical Center
(VMC) has been launched as a collaborative care and learning environment. Resources can be accessed anytime
and anywhere. Capabilities include virtual clinics with the integration of current telehealth technologies,
cybraries for patients and healthcare team members with electronic resources and searchable medical content,
serious medical games, e-learning platforms and conference venues. A full range of learning technologies,
including virtual patient and standardized patient-based platforms are fully integrated into the environment. Our
implementation strategy leverages ways in which the VMC can be synergistic with existing care models;
decrease repetitive staff activities; increase dissemination of and participation in educational interventions;
provide more effective education; capture productivity measures; and, be easy to navigate. Input from human
factors engineers, clinicians, educators, and technology experts has been critical. Initial implementation includes
five pilot projects characterized by the need for educational interventions for patients or/and healthcare team
members as well as clinical interventions to optimize Veteran health outcomes in key clinical areas. The clinical
areas include diabetes, sleep disturbances, congestive heart failure management, obesity, and palliative care.
Interventions include staff training for new protocols, peer and professional coaching for patients with chronic
diseases, shared medical appointments, training and resources for uisng at home equipment such as CPAP
machines. Assessment strategies are comprised of a global assessment of the technologies in place and project
specific ones tracking outcomes. Our newest generation of Veterans is very tech savvy and embraces virtual
world technologies. The VMC will allow geographically separated staff and patients to interact in a rich avatarbased environment. In-world opportunities can provide important care resources and rich educational
experiences for all learners.
EC-7 Let’s Get Physiological
1030
Visualizing fMRI Data Using
Volume Rendering in Virtual
Reality (15253)
1100
Professional Soldier Assessment of a
Rifle-Mounted Target
Hand-Off System (15039)
1130
Empirical Support for Brain-Based
Assessment in Simulation-Based
Training (15300)
Notes
38
VISUALIZING FMRI DATA USING VOLUME RENDERING IN VIRTUAL REALITY
Joseph Holub, Eliot Winer
Ames, Iowa
Medical imaging technology has changed patient diagnosis since the first x-ray in 1895 (Rontgen, 1896). Powerful
imaging technologies like Computed Tomography (CT), Ultrasound, and Magnetic Resonance Imaging (MRI) are now
used daily. One study showed preoperative imaging for potential appendicitis reduced unnecessary surgeries by 87%
(Raman et al., 2008). With the 2015 Defense Budget including $47.4 billion for the Military Health System (Overview
United States Department of Defense Fiscal Year 2015 Budget Request Office of the Under Secretary of Defense
(Comptroller)/ Chief Financial Officer, 2014), enhanced use of imaging for improved patient care and cost reduction is
critical. More recently, functional MRI (fMRI) technology was developed to extend medical imaging beyond 3D static
models to capture physiological changes over time. Currently, fMRI is used for applications from examining beating
hearts to mapping brain activity in real-time. fMRI has the potential to dramatically change how illnesses are
diagnosed, planned for, and treated.
Methods created for visualizing fMRI data in the academic realm have rarely made their way into commercial
software toolsets. For example, there are no software libraries available for researchers to create their own fMRI
visualization tools. Another consideration needs to be the visual manner (i.e., 2D, 3D, or 3D stereo) in which these
visual representations are created. Previous research on visualizing medical data has demonstrated improved
understanding of spatial relationships when using stereoscopic 3D over traditional 2D representations. This indicates
that virtual reality may be a superior medium for visualizing fMRIs.
This paper presents research to: 1) make readily available fMRI software libraries and 2) use these libraries to
visualize fMRI data in immersive VR. The method was tested on a desktop computer as well as a large multi-walled
VR system running off a cluster of computers. Preliminary results have indicated that visualizing fMRI data in VR can
be done in a computationally efficient manner. Multiple fMRI datasets were used for evaluation by measuring load
times and frame rates.
PROFESSIONAL SOLDIER ASSESSMENT OF A RIFLE-MOUNTED TARGET HAND-OFF SYSTEM
Jerome Levesque, Katherine Banko
Defence Research & Development Canada
Ottawa, Canada
Olaf Binsch
Netherlands Organization for Applied Scientific Research
The miniaturization of digital image acquisition and processing hardware, positional sensors, and batteries has enabled
the creation of assisted targeting systems light enough to be integrated onto small firearms to increase the probability
of soldiers detecting and hitting targets. As well, the technology allows soldiers to share target locations, thereby
increasing tactical situational awareness and enabling target prioritization and target hand-off. We investigated how
these new technologies might impact operational effectiveness by testing the concepts using human-in-the-loop
simulation in a virtual environment. Two conditions examined the tool usage (no target hand-off vs. target hand-off).
Within these conditions we added patrol and attack variants (no enemy, inaccurate enemy and accurate enemy). Each
condition was repeated 8 times for a total of 64 randomized trials. Combat effectiveness measures quantifying blue
casualties and the disruption of enemy activity were augmented with physiological indicants of stress and self-report
measures of self-efficacy, performance and cognitive load. Null hypothesis significance testing applied to the combat
effectiveness measures did not detect any statistically significant improvement in the combat effectiveness of the
section as a result of using the target hand-off system. A Bayesian analysis was conducted to determine the probable
size of an undetected effect. The human factor measures indicated differences between the simulated high and low
threat conditions. Self-report measures combined with physiological measures did not reveal increases in stress when
high and low levels of threat were compared. While participants evaluated the target hand-off system positively, the
ability of the new technology to decrease cognitive load and therefore increase combat effectiveness measures remains
unconfirmed. Simulations have limitations, particularly when exploring the benefits of target hand-off functionality
(i.e. weapons effects and risks encountered in combat cannot be fully represented for safety and ethical reasons). And,
combat stress is difficult to produce in an experimental setting. However, despite the small number of participants (n =
8), it was possible to estimate the probability distribution for the actual effect size.
39
EMPIRICAL SUPPORT FOR BRAIN-BASED ASSESSMENT IN SIMULATION-BASED TRAINING
Kevin B. Oden, Ph.D., Kelley L. Phillips
Lockheed Martin
Mission Systems and Training
Orlando, FL
Kurtulus Izzetoglu, Ph.D., Patrick Craven, Ph.D.,
Hasan Ayaz, Ph.D., Gabriela Hernandez
Drexel University
Philadelphia, PA
Individualized training solutions are increasingly important to military training programs. To effectively
adapt training to each student requires valid and reliable measures of human performance. Historically, the
military has relied on behavioral and subjective reporting methods to assess and evaluate human
performance (e.g., speed, accuracy, reported workload); such measures are effective for simple, wellcontrolled tasks with a strong behavioral element, but they lack the diagnostic sensitivity required to
measure meaningful differences in individuals’ performance on complex tasks that stress cognitive
performance.
Brain-based measures of functional brain activity in naturalistic settings may lead to improved
understanding of a trainee’s progress. Specifically, functional Near-Infrared Spectroscopy (fNIRS) is an
optical brain imaging technology that can be used in the context of simulation-based training to measure
changes in brain activity related to executive cognitive functions. Previous work describes how these
changes could be used to measure the transition from novice to expert performance by accurately and
reliably assessing the transition of cognitive skills out of working memory into automaticity.
In this paper, the authors describe how brain-based metrics derived from fNIRS and task performance data
were developed for a realistic pilot flying task in a simulation environment. Results from this study
demonstrate that fNIRS can be used to quantify meaningful differences as novice pilots learn to navigate a
prescribed flight path. Behavioral performance data confirmed that navigational proficiency improved
across trials, while the average oxygen concentrations declined in several areas of the prefrontal cortex, as
hypothesized. Furthermore, there were statistically significant correlations between the neural and
behavioral data., These results show that neurological data may provide a powerful complement to existing
behavioral measures by allowing instructor pilots not only to observe trainee outward behavior, but also to
gain a perspective of neurological changes occurring in the brain itself.
EC-8 Experience with the Experience API
1600
Putting Live Firing Range Data to
Work Using the xAPI (15019)
1630
Adapting Gunnery Training Using
the Experience API (15179)
1700
Opening Legacy Data Silos: Using
Experience Data for
Educational Impact (15043)
Notes
40
PUTTING LIVE FIRING RANGE DATA TO WORK USING THE XAPI
Paula J. Durlach
Orlando, FL
Nick Washburn
Riptide Software, Inc.
Oviedo, FL
Damon Regan
The Tolliver Group, Inc. Inc.
Orlando, FL
The past decade has seen advances in instrumentation of live training ranges. For example, when combined, the
Location of Hit and Miss (LOMAH) system and the Targetry Range Automated Control and Recording
(TRACR) system send virtually immediate feedback on marksmanship performance to trainers, via a tablet
computer. However, despite digital availability, the performance data are not used for individual feedback,
analyzed, nor automatically shared with any other training management or readiness systems. A proof-ofprinciple prototype system was developed, which demonstrated how the use of the Experience Application
Programming Interface (xAPI) could be used to collect valuable training data and support (1) individual
feedback, (2) aggregated data views for trainers and range operations personnel, (3) flexible data views for
training researchers, and (4) automated availability of qualification data to the Army Training Management
System. The xAPI was developed to allow the collection of learner data from different types of learning
experiences, and to make the data available to other applications. The LOMAH-TRACR data were converted to
xAPI statements, which were sent via an encrypted wireless network to a Learning Record Store (LRS). Using a
pin number, individual trainees could access a visualization of their own data on a mobile device, and be given a
link to learning content, personalized by the software’s analysis of their individual shot group pattern; however,
no actual Soldier testing occurred as part of the project. Trainers and range operations personnel could also view
data, and filter it according to their needs. An unanticipated benefit was the ability of range personnel to identify
operational defects in LOMAH targets. A third “researcher” dashboard was created to allow for analysts to
select data and export for further analysis. A future benefit will be the ability to integrate data from simulation
and live training, in order to determine the most efficient and cost-effective combination to achieve desired
levels of performance.
ADAPTING GUNNERY TRAINING USING THE EXPERIENCE API
Rodney Long
United States Army Research
Laboratory
Human Research and Engineering
Directorate Simulation and Training
Technology Center, Orlando, FL
Michael Hruska
Ashley Medford
Problem Solutions
Johnstown, PA
Jennifer Murphy,
Carolyn Newton
Quantum Improvements
Consulting
Orlando, FL
Tara Kilcullen,
Robert L. Harvey Jr
Raydon Corporation
Port Orange, FL
The Army’s Training and Doctrine Command (TRADOC) has described plans for modernizing Army
training in documents such as the Army Learning Model (ALM, TRADOC PAM 525-8-2). The ALM calls
for increasing the personalization of the soldier learning process so that training is tailored to the individual
soldier throughout his/her career. To accomplish this goal, a persistent representation of soldier
performance across a variety of technology-based training systems is required. Currently, performance
data throughout the live, virtual, constructive, and gaming (LVCG) spectrum is not maintained, nor is it
used to adapt future training for soldiers or their units. However, advances in data interoperability have
recently made development of complex student models using this performance data a possibility. The
Experience API (xAPI) is one such innovation. As part of our research, we have used the xAPI to capture
interoperable performance data for unstabilized gunnery simulators. Using this performance data, we have
developed an adaptive training curriculum in which crew training is adapted based on prior individual
performance on a gunnery simulator. This paper describes the development of interoperable performance
data for unstabilized gunnery simulators using the xAPI specification as well as the findings of an
experiment to demonstrate gains in learning and training efficiency. The results can be used to inform the
Army in its training modernization goals, as well as the simulation-based training community as a whole.
41
OPENING LEGACY DATA SILOS: USING EXPERIENCE DATA FOR EDUCATIONAL IMPACT
Jonathan Poltrack
ADL Initiative Contractor with Problem Solutions
Alexandria, VA
Tom Creighton
ADL Initiative Contractor with Aquate Corp
Alexandria, VA
The Sharable Content Object Reference Model (SCORM) afforded major benefits to the learning and training
industry by creating an environment of interoperability for e-learning content and systems. However, the data
that resulted from a learner experiencing SCORM content was often stored in proprietary data stores. As a
result, potentially important data was locked away and unable to be used.
Recently, emerging trends in big data, predictive analytics and data visualization renewed interest in accessing
massive amounts of learning experience data. Paradata and correlations can be evaluated to provide learner
recommendations for relevant content, to present visualizations to teachers so they can see how their content is
being used, and to view meaningful analytics that among other things, can be used to refine and improve
learning content. But how can this be accomplished when the requisite data is locked in proprietary learning
management systems?
This paper will discuss a novel method of intercepting SCORM communications and translating to standard
Experience API (xAPI) ‘statements’. The xAPI is an emerging technology that allows tracking of experiential
data and provides secure access to data once stored. After applying this solution, SCORM run-time data is stored
in a learning record store (LRS) allowing secure access for analysis and visualization. It is possible to apply this
solution in two distinct ways: content or server-side updates. Both of these are viable, and in some cases almost
automatable solutions to exposing vast amounts of SCORM data.
This paper will explore both methods for removing legacy data silos, will discuss the pros and cons of both
content and server side updates, will report on the feasibility of these methods by describing software proofs-ofconcept, and will illustrate several use cases and examples of the value of leveraging SCORM e-learning data
once it is available en masse.
THURSDAY, 3 DECEMBER, 2015 ROOM S320C
EC-9 3D Psycho
0830
Rapid 3D Geospatially Oriented
Structure Extraction from
Minimal Image Sets (15323)
0900
Extending Intelligent Tutoring
Beyond the Desktop to the
Psychomotor Domain (15029)
0930
Delivering 3D Virtual Maintenance
Training Content: Examining the
Deployment Options (15239)
Notes
42
RAPID 3D GEOSPATIALLY ORIENTED STRUCTURE EXTRACTION FROM MINIMAL IMAGE
R. Scott Starsman, PhD
Avineon, Inc. Director, Defense Systems
McLean, VA
Traditional approaches to 3D scene reconstruction require very large image sets, are extremely processor
intensive, and perform poorly when faced with surfaces with limited features. The work described in this
paper builds upon an approach presented at I/ITSEC 2013 that greatly increases quality of 3D
reconstruction, requires a minimal set of images, reduces model storage size, and is resilient in the face of
low-feature surfaces. While the work presented in 2013 demonstrated the successful reconstruction of a 3D
model from a small set of images, it suffered from several problems including long processing time, low
success rate for arbitrary structures (meaning that it worked well on specific types of buildings/structures
but not on the vast majority), sensitivity to misidentification of commonly present elements, such as logos
and signs, and correspondence point noise. This paper details the methods used to address these
deficiencies and achieve the full promise of rapid scene reconstruction in the face of a limited number of
images. Two elements of the image processing pipeline were identified that led to the performance issues
and replacement algorithms and processes were developed and integrated into the model. The replacement
of those components dramatically improved performance and supported the generation of arbitrary
structures from an image set. Key issues that have been resolved include: extremely long processing times,
sensitivity to structures with few surface features, sensitivity to repeated features, and sensitivity to
correspondence point noise. A description and derivation of these new approaches is discussed and as a
final demonstration, the system is used to generate a 3D reconstruction of a city block with the results
capable of being viewed in a tool such as Google Earth. This work is pertinent in the military, security,
simulation, and disaster response scenarios.
EXTENDING INTELLIGENT TUTORING BEYOND THE DESKTOP TO THE PSYCHOMOTOR
DOMAIN
Robert A. Sottilare, Ph.D.
US Army Research Laboratory
Orlando, Florida
Joseph LaViola, Ph.D
Orlando, Florida
Today, Intelligent Tutoring Systems (ITSs) are generally authored to support desktop training applications with the
most common domains involving cognitive problem solving tasks (e.g., mathematics and physics). In recent years,
implementations of game-based tutors based on the Generalized Intelligent Framework for Tutoring (GIFT), an opensource tutoring architecture, provided tailored, militarily-relevant training experiences in desktop applications (e.g.,
Virtual Battlespace and Virtual Medic). However, these game-based desktop tutors have been limited to adaptive
training for cognitive tasks (e.g., problem solving and decision-making), whereas the military requires adaptive
training to extend beyond the desktop to be compatible with the physical nature of many tasks performed by soldiers.
This paper examines how commercial smart glass technologies could be adapted to support tailored, computer-guided
instruction in the psychomotor domain for military training in-the-wild, locations where no formal training
infrastructure is present. We evaluated the usability and system features of 10 commercial smart glasses including
Atheer One, CastAR, Epson Moverio BT-200, GlassUp, Google Glass, LaForge Icis, Laster See-Through, Meta Space
Glasses, Optinvent ORA-S, and Vuzix M-100. Smart glasses were selected as the focus of this study over handheld
mobile devices to promote a hands-free experience during a training task where the hands are needed to accomplish
the task (e.g., climbing and maneuvering over uneven terrain). Each set of smart glasses was evaluated not with
respect to each other, but with respect to their capabilities to support adaptive instruction in-the-wild and at the
learner’s point-of-need. We examined a wide range of smart glass features and capabilities, and evaluated their
compatibility with a representative military task, land navigation, to answer the question: what system design features
(e.g., usability and interaction) are needed to support adaptive training for this individual psychomotor task beyond
desktop applications so it can be taught anywhere (in-the-wild)?
43
DELIVERING 3D VIRTUAL MAINTENANCE TRAINING CONTENT: EXAMINING THE DEPLOYMENT
OPTIONS
Christopher Van Duyne, Scott Ariotti
The DiSTI Corporation
Orlando, FL
Over the past several years, the U.S. Armed Forces have been significantly expanding their adoption of
virtualized solutions for use in maintenance training applications. This growing adoption is coupled with a
growing interest in expanding the ways in which these training materials are consumed. Unlike traditional
operational trainers that teach students on fixed hardware, the virtualized training material for maintainers
lends itself well to providing innovative training beyond typical brick and mortar schoolhouses; so long as
the content can be effectively delivered. These graphically intense 3D virtual environments often levy
hefty requirements on the type of computer capable of delivering an immersive interactive 3D experience.
This paper focuses on the latest options available for delivering these virtual environments to the training
consumer along with the pros and cons of each option and key lessons learned for two different types of
training consumers; Classroom users and External users.
The deployment options in this paper compare and contrast traditional desktop use with technologies that
incorporate mobile client applications, fixed server rendering solutions, and newly emerging cloud-based
application rendering services. The review includes relative cost comparisons, barriers to entry, the
consumer access experience, application development considerations, and necessary hardware utilizing
real-world examples that encompass both military aircraft and commercial automotive training devices.
The paper also introduces discussion topics on information assurance and security considerations for each
deployment option.
EC-10 To Game or Not To Game
1030
Gamers Today, Surgeons
Tomorrow? (15235)
1100
Relationship Between Learner and
Environment: Learner
Traits in Serious Games (15092)
1130
Learning Stories: Design
Considerations for Narrative
Elements in Serious Games (15303)
Notes
44
GAMERS TODAY, SURGEONS TOMORROW?
Alyssa Tanaka, M.S., Courtney Graddy, M.S. & Roger Smith, Ph.D.
Florida Hospital Nicholson Center
Celebration, FL
Manuela Perez, M.D., Ph.D
Nancy University Hospital
Nancy, FR
Faced with an age of reliance on technology and innovative advances, surgeons are using cutting-edge robotic systems
to perform complex procedures and virtual reality simulators for specialized skill training. The virtual environment
and controllers in surgical simulators are reminiscent of those in videogames. So, can playing video games develop
skills similar to those used in robotic surgery?
This paper compares the performance of video gamers, medical students, and “lay people” to expert robotic surgeons
on a robotic surgery simulator. Participants recruited from the UCF College of Medicine, UCF FIEA, and Florida
Hospital completed a demographic questionnaire. The subjects then performed three computer-based perceptual tests
and participated in two warm-up tasks on the Mimic dV-Trainer to familiarize themselves with the system. The
experiment then measured their performance over eight trials of two core simulated exercises. After completing these
trials, participants completed a post-questionnaire about their experience.
Analysis of the data did not verify differences between the groups for the perceptual tests except for the time to
complete scores in the Flanker and subsidizing tasks, in which expert surgeons took significantly longer than other
groups. Significant differences were found between the groups for the first and eighth trials of the simulated exercises,
with surgeons performing better than other groups. All groups improved significantly from trial one to trial eight, with
surgeons performing better than all groups. Gaming console type positively correlated with Overall Score in the Ring
& Rail exercise, as well as Time and Economy of Motion in the suturing exercise. No other correlations were found.
The results are in contrast with prior literature on video game experience in laparoscopic surgery, suggesting that
gaming abilities do not translate to all surgical modalities. Future research is necessary to further examine the impact
alternative skillsets may have on surgical skills.
RELATIONSHIPS BETWEEN LEARNER AND ENVIRONMENT:
LEARNER TRAITS IN SERIOUS GAMES
Marvin G. Fuller, Ph.D.
Oglethorpe Charter School
Savannah, Georgia
Dennis Beck, Ph.D.
University of Arkansas
Fayetteville , Arkansas
When investigating learning, it is commonly discussed that there is a strong relationship between learning
and the interactions between the learner, environment, and content. It is important to understand these
relationships regarding serious games because they may provide many learning and training advantages,
including maximizing the efficiency of achievement of learning targets. Despite the recognized potential of
serious games, few researchers have explored the relationships between specific patterns of behaviors and
types of game-based learning environments.
This paper presents a practical underlying theory that relates how patterns of learner characteristics and
behavior can be used to improve serious game design and promote learning effectiveness. It also presents
research results showing relationships between the learner and engagement in serious game environments.
The case study used for this research involved high school and college-level math students using a
commercial 3D adventure-quest math game. The data and statistically significant results of the study show
the relationship between gamer behavior, gender, and age-band, to time-on-task and learning performance.
Important conclusions presented at the end of this paper include a range of principles useful for serious
game designers, developers, and educators. Recommendations for future research provided at the end of
the paper will provoke interest in furthering basic principles for learner engagement across a broad range
of serious game applications.
45
LEARNING STORIES: DESIGN CONSIDERATIONS FOR NARRATIVE ELEMENTS IN SERIOUS
GAMES
Michael W. Freeman, EdD
MW Freeman Solutions
Fayetteville, NC
Mark Friedman
Adayana Inc
Suffolk, VA
Serious games are powerful tools for providing direct experience and concrete contexts in military training
environments. They depend on stories, or narratives, to provide the basis for effective, engaging learning experiences.
However, there are few research-based guidelines to support design of learning narratives or account for why they
include specific characters, environments, or activities. While the fundamentals of good instructional design and
learning are enduring, narratives for serious games require careful design to leverage the great promise and inherent
power of the serious game.
This paper proposes an inclusive model for understanding and designing serious game narrative. The paper starts with
a working definition of serious game narrative and continues with a distillation of a review of the literature to propose
the components that make up an effective narrative for serious games. The paper continues by proposing specific
design considerations based on a review of learning theories and best practices for each of these components. This is
intended to recommend to serious game designers and developers a common lexicon of terms to describe narrative and
to enable a purposeful process for designing the game narrative and experience. The paper concludes with
recommendations for implementation and future study.
THURSDAY, 3 DECEMBER, 2015 ROOM S320A
EC-11 Droning On
1330
Procedural Reconstruction of Simulation Terrain
Using Drones (15041)
1415
Assessment of Unmanned Aircraft Platform Performance
Using Modeling and Simulation (15006)
Notes
46
PROCEDURAL RECONSTRUCTION OF SIMULATION TERRAIN USING DRONES
Ryan McAlinden, Evan Suma, Timofey Grechkin
USC Institute for Creative Technologies
Los Angeles California
Michael Enloe
National Simulation Center
Ft. Leavenworth, Kansas
Photogrammetric techniques for constructing 3D virtual environments have previously been plagued by
expensive equipment, imprecise and visually unappealing results. However, with the introduction of lowcost, off-the-shelf (OTS) unmanned aerial systems (UAS), lighter and capable cameras, and more efficient
software techniques for reconstruction, the modeling and simulation (M&S) community now has available
to it new types of virtual assets that are suited for modern-day games and simulations. This paper presents
an approach for fully autonomously collecting, processing, storing and rendering highly-detailed geospecific terrain data using these OTS techniques and methods. We detail the types of equipment used, the
flight parameters, the processing and reconstruction pipeline, and finally the results of using the dataset in
a game/simulation engine. A key objective of the research is procedurally segmenting the terrain into
usable features that the engine can interpret – i.e. distinguishing between roads, buildings, vegetation, etc.
This allows the simulation core to assign attributes related to physics, lighting, collision cylinders and
navigation meshes that not only support basic rendering of the model but introduce interaction with it. The
results of this research are framed in the context of a new paradigm for geospatial collection, analysis and
simulation. Specifically, the next generation of M&S systems will need to integrate environmental
representations that have higher detail and richer metadata while ensuring a balance between performance
and usability.
ASSESSMENT OF UNMANNED AIRCRAFT PLATFORM PERFORMANCE USING MODELING AND
SIMULATION
Brent Terwilliger, Dennis Vincenzi, David Ison, and Todd Smith
Embry-Riddle Aeronautical University, Worldwide Campus
Unmanned aircraft systems (UAS) can provide significant enhancement to capability, when used in a
manner best aligning inherent design characteristics to requirements of a given application. However, wide
variability in designs, configurations, and operational attributes requires the performance of thorough
investigation to appropriately identify suitable platforms. Failure to perform sufficient examination can
lead to expensive cost overruns, diminished capability, and degraded safety. Assessing the capabilities and
performance associated with categorized UAS platforms through experimentation and analysis can
produce valuable insight regarding propriety for application. The use of modeling and simulation (M&S)
provides the means to identify limitations, benefits, and considerations necessary to aptly employ UAS.
Understanding how to best select, configure, and apply this rapidly advancing technology is anticipated to
support increased innovation, safety, efficiency, and effectiveness; elements essential to achieving
successful integration into the National Airspace System (NAS) for use across government, industry, and
academia. This paper contains a description of continued work from an experimental research project
featuring use of M&S to identify, observe, and investigate critical factors of UAS platform application in
an efficient and expedient manner. Operational design attributes (i.e., published and derived metrics) of
282 commercially-off-the-shelf (COTS) platform configurations were identified, classified, and analyzed
to create category representative UAS performance models. These models were employed in 30
experimental trials and subsequent statistical analysis. The results led to the development of a theory of
operation, selection requirements for use of UAS in aircraft rescue and fire fighting (ARFF), and an
expanded series of UAS category performance models. Future anticipated research, including
improvement of performance models, expanded simulation trials, and further refinements will also be
discussed.
47
EC-12 A Little Design Will Do Ya
1330
Early Synthetic Prototyping: When
We Build It, Will
They Come? (15187)
1400
Implementation of Agile Methods
within Instructional
Systems Design: A Case Study
(15094)
1430
Development and Evaluation of a
Venipuncture and
Phlebotomy Training System
(15084)
Notes
EARLY SYNTHETIC PROTOTYPING: WHEN WE BUILD IT, WILL THEY COME?
LTC Brian Vogt
U.S. Army, Army Capabilities Integration Center
Fort Eustis, VA
Mr. Michael Megiveron & Dr. Robert E Smith
U.S. Army, TARDEC
Warren, MI
One of the challenges facing the Army today is the ability to explore innovative concepts and capabilities in a
resource constrained environment to develop materiel, doctrinal, and organizational solutions for the future
force. Early Synthetic Prototyping (ESP) is a process and a set of tools that will enable Soldiers and
technologists to rapidly assess how technologies might be employed within a game environment. ESP is
envisioned to be a persistent game network that allows Soldiers to play scenarios and provide experiential
feedback to concept and capability developers. An operational test has been conducted that leveraged VBS3 to
explore four future concepts: Virtual Pointer, Counter Unmanned Aerial System, Aerial Resupply, and the Next
Generation Close Combat Vehicle A total of 76 Soldiers participated in the test. The test allowed the
prototyping of ESP tools and processes, along with answering three primary questions: (1) What games do
Soldiers play in their off-duty time and what devices do they use? (2) What would motivate Soldiers to
participate in ESP on their own time? (3) How valuable is their qualitative feedback and game data to concept
and capability developers? The results of this test were encouraging and overwhelmingly positive. More than
85% of Soldiers play military-themed games in their off duty hours; more than half of the Soldiers play more
than 10 hours of military-themed games each week. Most Soldiers stated they would participate in ESP on their
off duty hours and their biggest motivation to participate is knowing they are helping shape the future of their
Army. The qualitative feedback indicates Soldiers are able to provide insightful feedback about materiel,
organizational, and doctrinal solutions. The results of this study show there is strong potential that ESP will not
only be a great way for concept and capability developers to gain meaningful feedback from end users but is
also an environment Soldiers relate with and can innovate solutions. This paper describes the test procedure,
analysis of the results, lessons learned, and recommendations for future development. Insights are generalizable
to understand how to engage service members in their off-duty hours through gaming solutions. The results of
this test gained the attention and support of senior leaders in ARCIC and TARDEC. Continued research needs to
refine the ESP method and to determine the best way to extract and visualize both qualitative and quantitative
data from the composite data collected from thousands of Soldiers.
48
IMPLEMENTATION OF AGILE METHODS WITHIN INSTRUCTIONAL SYSTEMS DESIGN: A CASE
STUDY
Lisa Cooney & Anne Little, PhD
Addx Corporation
Alexandria, VA
Today’s economic environment of shrinking budgets demands training that aligns with the needs of the
workforce by improving specific employee behaviors. When organizations identify a workforce deficiency there
often are critical implications for their operations, so they typically respond with training solutions to correct the
deficiency.
During the fall of 2014, the authors were tasked with a course design and development effort for a two-week
instructor-led program management course. The traditional instructional design methods used for previous
versions of this course relied on a locked-down front-end design, classic linear processes, and evaluation
methods that provided feedback late in the development cycle. The previous version of the course was heavily
based on another Federal agency’s models and did not meet the needs of the students. Additionally, policies
needed for inclusion in the class were in flux. The time available for development was tight; two full-time
instructional designers and three part-time subject matter experts needed to create a 10-day instructor-led course
in time to deliver a class offering in less than seven months.
To create the new course, the development team incorporated the principles of Agile software development.
This paper will review Agile software development, the ADDIE (Analysis, Design, Development,
Implementation, Evaluation) instructional design method, and explain how the team applied the principles of
Agile development to instructional systems design. We will discuss the implementation process, organizational
tools, team dynamics, and customer involvement. Finally, we will illustrate the potential cost savings of this
method by comparing a summary of the resources utilized to industry training development metrics.
DEVELOPMENT AND EVALUATION OF A VENIPUNCTURE AND PHLEBOTOMY TRAINING
SYSTEM
Teresita M. Sotomayor, Ph.D.
Angela M. Alban
U.S. Army Research Laboratory HRED-STTC
SIMETRI, Inc.
Orlando, Florida
Winter Park, Florida
The U.S. Army has invested significantly in manikin technology to train procedural skills associated with
military medical training in diverse simulated environments. Training equipment needs to be rugged and
reliable to endure austere conditions but refined enough to provide training solutions with appropriate
fidelity. A manikin or Part-Task Trainer (PTT) possessing those qualities that accurately trains
venipuncture and injection procedures has historically been a challenge. The goal of the U.S. Army
Medical Simulation Training Centers (MSTCs) is to provide Army personnel with more effective
technology, tools, and techniques for training Army personnel. As a result, the U.S. Army Research
Laboratory Human Research and Engineering Directorate (ARL-HRED) Simulation and Training
Technology Center (STTC) was sponsored by the U.S. Army Program Executive Office for Simulation,
Training and Instrumentation (PEO STRI) to develop a next-generation venipuncture and injection PTT,
that is more realistic, durable, and cost effective to teach these lifesaving skills. The primary objective is to
develop a proof of concept device that demonstrates the viability of the materials, the electrical/mechanical
design, and the technical approach. The research focused on identifying innovative technologies, technical
risks of the approach, costs, and benefits associated with development and demonstration of the prototype.
Additionally, a usability study was conducted with first responders to gather feedback and assess whether
the initial prototype met training requirements. This paper will discuss in detail how training requirements
impacted the design of the training system and also explore the criteria used to develop the overall design,
as well as the identification of specific capabilities. In addition, it will explain how subject matter expertise
was utilized to develop requirements and performance metrics used to evaluate the feasibility of the
concept. Finally, it will review results from usability evaluations and lessons learned from the development
and implementation of this project.
49
TUESDAY, 1 DECEMBER, 2015 ROOM S320E
H-1 This Won’t Hurt a Bit
1400
Piloting a Groundbreaking Virtual
Continuing Competency
Platform: Results and
Recommendations (15325)
1430
Mobile App Design for Veterans
with Physical and Cognitive
Limitations (15182)
1500
Structured Development of
Interventions to Improved
Physician Knowledge Retention
(15095)
Notes
PILOTING A GROUNDBREAKING VIRTUAL CONTINUING COMPETENCY PLATFORM:
RESULTS AND RECOMMENDATIONS
Jennifer McNamara
BreakAway Games
Hunt Valley, MD
Paul Grace and Margaret Bent, Ph.D. OTR
National Board for Certification in Occupational Therapy Inc
Gaithersburg, MD
The National Board for Certification in Occupational Therapy (NBCOT®), the national certification body for
occupational therapy professionals in the United States, embarked upon a novel project to employ a virtual continuing
competency platform. The genesis for the innovative virtual product was the result of a practice analysis study - the
goal of which was to gain evidence-based direction for individualized programs of continuing professional
development. The study identified six key areas for focus: providing client-centered care, working in interprofessional
teams, employing evidence-based practice, applying quality improvement, utilizing informatics, and promoting
professional responsibility. The virtual platform targets certificants’ needs related to maintaining knowledge for
current practice as well as supporting career enhancement and growth. With neither an existing platform nor content to
meet its needs, NBCOT took on the task of designing, developing, pilot testing, and delivering the virtual platform and
all of its supporting content from initial concept through deployment. The live system includes a web-based
assessment delivery engine, certificant dashboard, and interfaces that support self-reflective assessments, multiplechoice practice knowledge assessments called mini practice quizzes, animated case simulations, and games as
educational experiences. Prior to the full implementation of the new virtual continuing competency platform, a pilot
test including 512 unique testers accessing 6,561 assessment tools was conducted. This paper will introduce the
program at a high level and discuss the design process to frame discussion and then share the descriptive results of the
user pilot study. While this specific program targets occupational therapy certificants, the virtual platform, focus areas,
and lessons learned regarding use of a large scale virtual assessment program apply to other domains. The team will
share generalized recommendations for future design and development of advanced technology-enabled assessment,
certification, and educational experiences based upon our findings.
50
MOBILE APP DESIGN FOR VETERANS WITH PHYSICAL AND COGNITIVE LIMITATIONS
Nina P. Deibler, Lea G. Blake
Serco Inc.
Pittsburgh, PA, Richmond, VA
Devin Harrison, William Plew
U.S. Department of Veterans Affairs
Atlanta, GA, Nashville, TN
Pressure ulcers are a significant cause of morbidity and mortality among hospitalized, institutionalized, and
mobility-compromised Veterans, and the prevalence of pressure ulcers has been an ongoing challenge for the
U.S. Department of Veterans Affairs (VA). With the VA currently serving World War II- through Post-911-era
Veterans with Service-, age-, and illness-related physical and cognitive limitations, the VA sought an innovative
solution to educate and support the full spectrum of Veterans at risk of developing pressure ulcers. They selected
the VA Pressure Ulcer Resource (VAPUR) project to design a hybrid Mobile Application (App) providing this
“just-in-time” education and performance support for Veterans and their Caregivers.
In addition to educational content structured as frequently asked questions with graphics and videos, the App
allows Veterans to securely self-report wound data to the VA using a simply worded form. This reporting
capability enhances the VA’s ability to monitor Veterans who live too far from specialized wound care facilities
to get regular pressure ulcer care. The App supports Veterans with cognitive limitations by automating functions
like setting reminders for daily tasks, dialing help and locating resources, and communicating with medical
providers. The VA Human Factors Team tested the App with Veterans to ensure usability heuristics and
industry-wide standards were focal points in the design. The VA Section 508 Accessibility Team also tested the
App to ensure it optimizes the accessibility features in current operating systems and fully complies with all
Section 508 requirements.
Because standard App interfaces and traditional educational approaches were insufficient for the diverse target
audience, this paper discusses the unique human-computer interface design considerations made for users with
physical and cognitive limitations. It also discusses how the resulting design can be reused for other Apps,
particularly for conditions like COPD, heart disease, and diabetes. VAPUR will be deployed in August 2015.
STRUCTURED DEVELOPMENT OF INTERVENTIONS TO IMPROVE PHYSICIAN KNOWLEDGE
RETENTION
Lloyd Werk, Maria Carmen Diaz,
Lorie Ingraham, James Crutchfield
James P. Franciosi, Tim Wysocki
Lockheed Martin
Nemours Children’s Health System
Mission Systems and Training
Jacksonville, FL
Orlando, FL
Military and civilian healthcare is undergoing radical transformations in almost every aspect of patient care from
diagnosis to treatment. Along with increased complexity in the technology of delivery systems and procedures,
medical knowledge is expanding at an ever-increasing rate, and yet clinicians are expected to retain knowledge and
remain proficient in their fields. Frequency of exposure to specific clinical problems and processes are known
contributors to physicians’ decay of clinical knowledge and proficiencies. For example, while deployed, military
physicians may experience less demand for specific clinical skills and are, therefore, at risk for knowledge decay. A
systematically applied knowledge retention program integrated with continuous training is one possible response.
However, institutionalizing standardized training at fixed intervals for all may not be the most cost-effective nor
efficient solution. This paper discusses the progress of a research study tasked to develop and validate efficient
interventions to mitigate physician knowledge decay that address both increased domain complexity and lower
frequencies of exposure.
The process of intervention selection is based on the analysis of elements of the care for nine targeted clinical
problems that reveal physician knowledge decay with decreasing frequency of exposure to those clinical problems.
Once the most critical elements of the care process have been identified, we apply a structured approach for selecting,
developing, and evaluating possible interventions geared towards choosing those that specifically address identified
knowledge needs and align with the organization’s learning goals, infrastructure and operating budgets.
Recommendations for a systematic, yet flexible, method for evaluating, weighing and scoring multiple knowledge
decay mitigation alternatives are included, supporting interventions ranging from static job aids to immersive learning
simulations. In summary, this paper proposes a comprehensive selection model for continuing medical education
programs committed to prevent skill decay, aid knowledge retention and improve overall physician and organizational
performance.
51
TUESDAY, 1 DECEMBER, 2015 ROOM S320E
H-2 Analyze This
1600
Work Domain Analysis for
Ecological Interface Design of
Tangible Interfaces (15130)
1630
Adaptive Testing: Adapt and
Overcome the Shortfalls of
Traditional Proficiency Assessments
(15196)
1700
Cognitive Load Assessment for
Intelligence Analysts through
Full Motion Video Analytics
(15142)
Notes
WORK DOMAIN ANALYSIS FOR ECOLOGICAL INTERFACE DESIGN OF TANGIBLE INTERFACES
Michael W. Boyce, Robert A. Sottilare, Benjamin Goldberg & Charles R. Amburn
Human Research and Engineering Directorate Simulation and Training Technology Center
Orlando, FL
This research developed a Work Domain Analysis (WDA) to help in guiding the design of a user interface for the
Augmented REality Sandtable (ARES). ARES combines the traditional military sandtable used in sandtable exercises
with projected topography. It leverages commercial off the shelf software in an effort to provide affordable simulation
technology. In order to create an interface that is usable, valid, representative of the environment, and free from
unnecessary elements, there is a need to perform a top to bottom domain analysis that can be validated by experts. This
assists in deciding interface grouping, visual mapping, and ease of learning in the operational environment. WDA
takes the relationships that exist between interface components and translates those into tangible interface design
specifications. WDA, a foundation for ecological interface design (EID), will be leveraged in future usability
experiments as well as the incorporation of a tutor to ARES. The WDA uncovered common functionality and
unexpected relationships between the interface components to better support the tasks of land navigation, and military
tactics training to support mission needs. Detailed breakdown of these domains can help to serve as guidance for other
projects looking for a structured basis for interface design.
52
ADAPTIVE TESTING: ADAPT AND OVERCOME THE SHORTFALLS OF TRADITIONAL
PROFICIENCY ASSESSMENTS
Robert “Mac” McLaughlin, Dr. Stephen Gunter
Camber Corporation
Orlando, FL
Jeff Pearson
Veterans Benefits Administration
Orlando, FL
Most military tests direct outcomes to a simple pass or fail; a Go/No Go model in which the individual must meet
specific standards of performance. Another testing approach can assess critical job competencies and current
proficiency levels across a continuum, which ranges from entry level to mastery. When implemented for occupational
specialties and skill areas for which it is suited, such a model offers significant long-term advantages in maximizing
training budgets, and developing skill mastery throughout a warfighter’s career.
Specific proficiency levels may be designated as minimum standards for different pay-grades, or for testing similar
jobs sharing common competencies at different required levels of proficiency. Unlike a binary pass/fail approach, this
identifies specific areas and the degree of remedial training required to meet standards. It also identifies gradual skill
decay and offers targeted remediation before it reaches the point of certification failure.
Over the long term, this can reduce training expenses by identifying specific training requirements. Standard remedial
training approaches are often very broad, with participants sitting through hours of training on standards, which they
may actually meet, waiting for the specific training content in which they are deficient. It may also allow for faster
advancement to skill mastery, as targeted remediation means additional training hours are available for skill
advancement beyond minimum certification standards. Identifying areas of skill decay provides for a just-in-time
training approach that can reduce future test failures and the impact of individuals taken away from their primary job
for corrective training.
Adaptive test engines, which use a branching logic to adjust test question difficulty at multiple points during the test,
offer an effective means to achieve this outcome. This paper presents an overview of the benefits of a diagnostic
testing model, steps required for its implementation, and experiences designing such a test..
COGNITIVE LOAD ASSESSMENT FOR INTELLIGENCE ANALYSTS THROUGH FMV (FMV)
ANALYTICS
Elizabeth Wilson
Chenega Technical Innovations, LLC
Dumfries, VA
Upesh Patel
US Army Intelligence and Information Warfare Directorate
Aberdeen Proving Ground, MD
The U.S. military has made a significant investment in fielding a wide variety of airborne and ground Full-motion
Video (FMV) electro-optical and infrared sensors to provide superior situational awareness and persistent surveillance
of the battlefield. These sensors collect an increasingly unmanageable amount of data, up to terabytes per hour from a
single wide area motion imagery sensor. Even with conventional FMV sensors, the data being produced far exceed the
number of intelligence analysts available to manually exploit the data. Together, the U.S. Army CommunicationsElectronics Research, Development and Engineering Center, U.S. Army Intelligence and Information Warfare
Directorate (I2WD), and the Joint Improvised Explosive Device Defeat Organization (JIEDDO) are working to
address this operational need. The project to provide an initial material capability to meet these requirements is named
Advanced Video Activity Analytics (AVAA). The AVAA is maturing a video processing exploitation framework
(VPEF), a video data model (VDM), a video annotation web service (VAWS), and integrating computer vision
analytic algorithms as plug-ins. The framework provides standardization, integration, and parallelization of computer
vision algorithms (CVAs), making them interoperable and testable. The system processes large-scale data and
manages the results using a video data model. This paper describes the formulation for testing and evaluation
conducted at the Army Intelligence Center of Excellence at Fort Huachuca, AZ, to measure AVAA’s ability to
improve video data processing and to reduce the cognitive load on analysts while providing the building blocks for
improved knowledge discovery across Intelligence domains. The techniques can be applied to understand and refine
cognitive load on training. Quickly processed full-motion imagery data can also facilitate population of simulation
data for an experimentation or training event.
53
WEDNESDAY, 2 DECEMBER, 2015 ROOM S320E
H-3 Do You Understand?
0830
Multi-measure Assessment of
Internal Distractions on
Driver Performance (15017)
0900
Measuring Trust of Autonomous
Vehicles: A Development
and Validation Study (15049)
0930
Building Trust in a Human-Robot
Team with Automatically
Generated Explanations (15315)
Notes
MULTI-MEASURE ASSESSMENT OF INTERNAL DISTRACTIONS ON
DRIVER PERFORMANCE
Kevin F. Hulme
Center for Engineering Design
and Applied Simulation (CEDAS)
Buffalo, NY
Karen L. Morris
The Center for Children and
Families (CCF)
Buffalo, NY
Gregory A. Fabiano
Department of Counseling, School,
and Educational Psychology
Buffalo, NY
Mark G. Frank
Department of Communications
Buffalo, NY
Rebecca J. Houston
Research Institute on Addictions
Buffalo, NY
Panos Ch. Anastasopoulos
Department of Civil, Structural,
and Environmental Engineering
Buffalo, NY
The primary objective of this effort is to employ a high fidelity simulator for a small pilot study to assess the impact of
internal distractions on traffic safety. While all vehicle distractions have the potential to endanger driver, passenger,
and bystander safety, distractions internal to the driver (i.e., mindlessness, being lost-in-thought, mind wandering) can
be defined as “the decoupling of attention from the task at hand coincident with a shift in focus to internal thought
processes.” Recent studies estimate that internally-distracted driving is the least understood and most deadly form of
distracted driving: 62% of all driving fatality cases involving distractions are “internal.” By contrast, the second
deadliest source of distraction, cell phone usage, accounts for 12% of fatalities. Internal Distraction is often
unintentional, and can last from a split second to numerous minutes, and while driving, has been shown to occur most
frequently during low-stimulus drives. Regardless of content, length, or intensity, whenever perception and attention
are decoupled, the risk of “looking but not seeing” increases, along with the likelihood of driver error.
Previous research in this area has documented impairments in driver performance while internally distracted, however
the reliability with which internal distraction was “induced” in simulation remains a point of contention. Most
simulator-based research that has analyzed the topic employs a “straight road, car following” model to induce mind
wandering. In this study, we employ a Route Familiarity scenario coupled with an Unusual Uses Task (UUT) to
induce a state of internal distraction while driving. Our novel multi-measure assessment includes: self-report, evaluator
observation, and simulator performance measurement (e.g., lane position, speed, following distance). Physiological
metrics (e.g., facial expression, eye pupil dilation) with on-board cameras are captured for future analysis. Ultimately,
the outcomes of this investigation could lead to countermeasures (e.g., vehicle technologies, improved practices in
road geometry, signage, targeted training) that mitigate negative driving outcomes resulting from internal distraction.
54
HOW HUMANS TRUST AUTONOMOUS VEHICLES: A STUDY IN MEASUREMENT DEVELOPMENT
AND PERSONALITY DIFFERENCES
David R. Garcia, Christine Kreutzer, and Karla A. Badillo-Urquiola
University of Central Florida, Psychology Department
Orlando, Florida
Recent advances in technology have improved the ability of vehicles to act autonomously, thereby
enabling the implementation of these systems into the lives of the everyday consumers. For example, in the
past three years several major vehicle manufacturers, suppliers, and technology companies have
announced projects involving autonomous vehicles (AVs). While the notion of AVs has been popular
within the military, the urgency to make them commonplace has gathered pace as companies outside the
auto industry have illustrated the feasibility and benefits that AVs offer. However, in order to predict user
adoption of these autonomous features, attitudes towards them must be understood. Thus, the purpose of
the present work is to develop and validate a scale to quantify trust towards autonomous vehicles. The data
was subjected to a factor analysis with Promax rotation, yielding two factors. A number of correlations
between trust towards autonomous features and personality were also identified. Finally, differences in
trust between autonomous levels were identified.
BUILDING TRUST IN A HUMAN-ROBOT TEAM WITH AUTOMATICALLY GENERATED
EXPLANATIONS
Ning Wang, David V. Pynadath
University of Southern California
Los Angeles, CA
Susan G. Hill
U.S. Army Research Laboratory
Aberdeen Proving Ground, MD
Technological advances offer the promise of robotic systems that work with people to form human-robot
teams that are more capable than their individual members. Unfortunately, the increasing capability of
such autonomous systems has often failed to increase the capability of the human-robot team. Studies have
identified many causes underlying these failures, but one critical aspect of a successful human-machine
interaction is trust. When robots are more suited than humans for a certain task, we want the humans to
trust the robots to perform that task. When the robots are less suited, we want the humans to appropriately
gauge the robots’ ability and have people perform the task manually. Failure to do so results in disuse of
robots in the former case and misuse in the latter. Real-world case studies and laboratory experiments
show that failures in both cases are common. Researchers have theorized that people will more accurately
trust an autonomous system, such as a robot, if they have a more accurate understanding of its decisionmaking process. Studies show that explanations offered by an automated system can help maintain trust
with the humans in case the system makes an error, indicating that the robot’s communication transparency
can be an important factor in earning an appropriate level of trust. To study how robots can communicate
their decision making process to humans, we have designed an agent-based online test-bed that supports
virtual simulation of domain-independent human-robot interaction. In the simulation, humans work
together with virtual robots as a team. The test-bed allows researchers to conduct online human-subject
studies and gain better understanding of how robot communication can improve human-robot team
performance by fostering better trust relationships between humans and their robot teammates. In this
paper, we describe the details of our design, and illustrate its operation with an example human-robot team
reconnaissance task.
55
H-4 It’s a Wild, Wild Cyber World
1030
Automated Performance
Assessment in Cyber Training
Exercises (15044)
1100
Command Shift: Exploring Modern
Gaming Technologies to Create NextGeneration Offensive Cyber Operations
(OCO) Interfaces (15091)
1130
Embedded Cyber-Physical Systems
for Assessing Performance
in Training Simulations (15263)
Notes
AUTOMATED PERFORMANCE ASSESSMENT IN CYBER TRAINING EXERCISES
Robert G. Abbott, Jonathan McClain, Benjamin Anderson, Kevin Nauer, Austin Silva & Chris Forsythe
Sandia National Laboratories
Albuquerque, NM USA
Cyber threats have become ubiquitous as criminals extend their reach and cyber becomes a front in conflicts between
different peoples and a major source of revenue for criminal organizations. Personnel responsible for cyber defense are
becoming increasing critical. However, there is a shortfall between the number of individuals training to enter cyber
security and the projected demand for these skills. Consequently, methods and technologies are needed to enhance and
accelerate the training of cyber security personnel.
Previous research has demonstrated the benefits of automated performance assessments as a means to target training to
the specific needs of individual students. The current paper describes an extension of these capabilities to cyber
security training exercises. In these exercises, students are placed in teams and must work together, using appropriate
software tools and online resources, to conduct forensic analysis for cyber crimes. Individual and team performance is
assessed on the basis of successfully solving individual challenges and applying information from individual
challenges to correctly ascertain an overall picture of the who, what and why of the crimes.
The current paper describes a framework for conducting cyber security training exercises with an emphasis on
instrumentation to enable automated performance assessment. Instrumentation captures students’ computer-based
transactions in a log that is time-synched with the game-server used to deliver challenges and register student
responses. Analyses were conducted to better understand the factors that distinguish more or less effective student
performance and techniques developed to automatically parse logs of student activities into meaningful blocks of taskoriented activity. These capabilities are a prerequisite for the development of real-time automated assessment of
student performance within the context of cyber security exercises.
56
COMMAND SHIFT: EXPLORING MODERN GAMING TECHNOLOGIES TO CREATE NEXTGENERATION OCO INTERFACES
Chad Caison
KEYW Corporation
Hanover, MD
Jennifer McNamara
BreakAway Games
Hunt Valley, MD
Offensive Cyber Operations (OCO) are complex tasks involving abstract and logical concepts that are difficult
for end users to synthesize and engage with. Typical user engagement involves using a keyboard and mouse, and
working with Command Line or Graphical User Interfaces (CLIs and GUIs). While these tools let users
command tools, they do not provide robust situational awareness and they require extensive training and
experience to achieve competence. This paper describes the analysis, design, and development of a new cyber
interface that uses input and visualization methods borrowed from the game industry to fully immerse the
operator more naturally in the cyber battlespace. This type of tool has never been used in the OCO space and has
neither been openly welcomed nor understood within the community. Until now, stakeholders have not seen a
prototype to demonstrate the potential of this more natural type of interface, which gives the operator a better
understanding of the abstract environment to facilitate better decision making and reduce human mistakes. To
create this new user interface, two organizations collaborated: one representing current OCO training and stateof-the-art OCO tools, and the other representing game design. Together, these organizations designed a new
cyber interface focused on three primary goals: to reduce complexity and training time, to improve situational
awareness, and to reduce human error. This paper discusses the standard OCO work environment and its
challenges for end users, the results of our user analysis that drove the design process, the game-based hardware
and software considerations used by the team, and the prototype interface itself, along with informal playtest
reactions from end users.
EMBEDDING CYBER-PHYSICAL SYSTEMS FOR ASSESSING PERFORMANCE IN TRAINING
SIMULATIONS
P. Shane Gallagher
ADL
Alexandria, VA
Brenda Bannan & Bridgette Lewis
GMU
Fairfax, VA
Shelly Blake-Plock
Yet Analytics
Baltimore, MD
As a result of next-generation networking and the Internet of Things (IoT) technologies, big data analysis is possible
and has been shown to have a positive impact on areas of national significance yet requires new tools to deal with the
variety and quantity of data multiplying at an exponential rate.. Concurrently, IoT technologies are rapidly becoming a
mainstream data source. Training simulations have historically been limited either to computer-based simulations or
live human-observable field-based simulations ;however, IoT technologies can open up innovative, hybrid digitalphysical opportunities both for delivering and for understanding the outcomes of training in a much more dynamic and
comprehensive way. The feasibility of IoT technologies in training has historically been limited by interoperability and
scale. However, Advanced Distributed Learning’s Experience Application Programming Interface (xAPI) allows
interoperability and scale in next-generation training environments and provides a way to standardize the formative
data of human experience captured through digital context. It also provides a way to capture information and formalize
human experience from multiple and varied networked devices into standardized, human-readable statements. These
can inform both human and machine learning through leveraging big data analysis and interoperability of the IoT
technologies. By leveraging the xAPI and IoT technologies as a cyber-physical system embedded in virtual and live
training scenarios, it is possible to capture and measure real-time team performance for immediate analysis and
remediation or for post hoc analysis in after action reviews. This paper discusses the application of learning analytics
and design for an IoT context through describing the implementation of 1) a live action medical simulation as part of
the Global Smart Cities Challenge (sponsored by the NIST and the OSTP) and 2) the proposed capture and analysis of
communication performance data and measures within specific coalition training scenarios supporting the 2015 Bold
Quest Assessment sponsored by the Joint Fires Division of the Joint Staff.
57
H-5 Maintaining Your Cool
1400
Human Performance in Content Design for Interactive
Augmented Reality Systems (15156)
1430
Using Micro and Macro Studies of Tablets to Improve
Maintenance (15279)
Notes
HUMAN PERFORMANCE IN CONTENT DESIGN FOR INTERACTIVE AUGMENTED REALITY
SYSTEMS
Louise Yarnall, Sara Vasquez, Anna Werner, Rakesh (Teddy) Kumar, Supun Samarasekera, Girish
Acharya, Glenn Murray, Michael Wolverton, Zhiwei Zhu,Vlad Branzoi, Nicholas Vitovitch, & Jim Carpenter
SRI International
Menlo Park, CA and Princeton, NJ
The military is exploring the use of new technologies to improve the training of a large pool of personnel to maintain
and repair a variety of complex equipment. Achieving technology integration in maintenance training requires
developers to clear three human performance hurdles: physical usability, learning content efficacy, and a path to
integration with existing learning methods. Augmented reality technologies smoothly project explanatory visuals (e.g.,
text, directional arrows, videos, and 2D and 3D animations) over the workspace. Interactive technologies verbally
dictate steps to the user and respond to users’ spoken commands (e.g., “Computer, repeat step.”). In developing ARMentor, an innovative maintenance training technology that combines augmented reality and interactive dialogue
technologies, a team of engineers and education researchers encountered and responded to each of the human
performance hurdles.
In this paper, we show how these human performance hurdles were addressed and how they informed the refinement
of the AR-Mentor in two rounds of system development and testing. The AR-Mentor system provides a Heads-up and
Hands-free experience to permit a user to train with real equipment. The AR-Mentor system consists of a compact
computer, head worn cameras, microphone, ear-buds, and augmented reality eyewear.
The learning content addressed in the two rounds of testing focused on both basic training in maintenance procedures
and more advanced training in troubleshooting. The performance evaluation measured usability, time to learn, and the
relative learning achievement in procedural knowledge and troubleshooting reasoning between business-as-usual
instruction and technology-assisted learning conditions. The paper concludes by presenting key human performance
concepts for trainers and vendors of complex equipment systems to consider when designing technological content for
presentation with newer automated instructional technologies such as AR-Mentor.
58
USING MICRO AND MACRO STUDIES OF TABLETS TO IMPROVE MAINTENANCE
Robert Pokorny, Ph.D., Jacqueline Haynes, Ph.D. &
Lisa Holt, Ph.D.
Rockville, Maryland
Michael diPilla
Naval Surface Warfare Center, Carderock Division
Philadelphia, Pennsylvania
Navy maintenance is becoming increasingly difficult with more complicated systems, reduced staffing, and efforts to
reduce expensive training time. To improve maintenance readiness, the Navy is testing tablets by which technicians
receive performance aids and directly connect with the larger Navy maintenance enterprise system. Technicians’
performance can be improved by (1) micro controlled experiments which investigate interface details of accessing and
presenting content via tablets and (2) macro field tests that illustrate the effect of technological tools in their deployed
state. A comprehensive approach to improve productivity and decrease cost through tools is best informed by both
micro and macro studies, and to integrate the results of both to create and promote Navy goals.
The micro element of our approach is the study of how a tablet-based presentation of procedures can be structured to
provide technicians the support needed to maximize performance. We will report results from one such study,
identifying difficulties introduced by tablets and how they can be overcome, and the capabilities now possible with
interactive tablets.
The macro element of our comprehensive approach is the study of how the Navy maintenance technicians can benefit
when connected to enterprise resources. Technician benefits include an ability to order components when technicians
are in the field, access updated technical documentation, and automatically collect work performance data which
reduces redundant paperwork and enables big-data analytics to identify interesting trends of previously unknown
efficiencies and performance difficulties. We will report results from a recent field test that includes lessons learned
from connecting technicians to the enterprise system.
Micro studies provide scientific verification of principles used to develop the solution, and macro studies reveal how
well the solution improves work flow and productivity in the Navy context.
H-6 Flight Life
1600
Helicopter Pilot’s Modeling
Including the Stress Factor
(15168)
1630
Human-in-the-Loop Flight
Simulation Study of Virtual
Constructive Representation on Live
Avionics Displays (15197)
1700
Practical Recommendations for
Validating Survey Apparatus
in Coalition Training Environments
(15299)
Notes
59
HELICOPTER PILOT’S MODELING INCLUDING THE STRESS FACTOR
Antoni Kopyt
Warsaw University of Technology
Warsaw, Poland
In the modeling and simulation domain the human is often considered as an inherent system element. In
most studies his/her model remains unchanged due to the external factors. Concerning the wide studies on
human performance, workload impact, psychological aspects of human behavior, such an assumption
might be too far of a simplification. The present study proves, that a relationship between the mental stress
and human dynamics cannot be neglected. The dynamic characteristics of the operator’s model, change in
the function of external stimuli i.e. mental stress must be considered. The aim of this study was to present
identification of a mathematical human model and measurement methodology of the mental stress level
during various work conditions. 20 pilots form Polish Air force Academy were involved. Pilots performed
a slalom maneuver task on a SW-4 helicopter flight simulator. Subjects had to repeat slalom maneuver
three times, each time, the work conditions were different. The simulator software allowed the registration
of flight parameters during the experiment. The analysis of collected data were used to assess the flight
efficiency of each task. Pilot’s mental stress level was measured with NASA Task Load Index survey.
Additionally, to determine the level of pilot’s response to external stimuli, the electrocardiography (ECG)
and skin impedance methods were applied. Finally, base on registered data, the typical dynamic models of
each pilot have been identified. Consequently the models obtained from various flight conditions were
compared with the stress level respectively. The comparison of model parameters and detailed analysis
identified some tendencies in models. The presented paper proves that human susceptibility to external
factors directly transfers into dynamic models. The study shows that using more complex models that
includes stress factors is much closer to the real human behavior.
HUMAN-IN-THE-LOOP FLIGHT SIMULATION STUDY OF VIRTUAL CONSTRUCTIVE
REPRESENTATION ON LIVE AVIONICS DISPLAYS
Dr. Tom “Mach” Schnell
Operator Performance Lab
University of Iowa
Iowa City, IA
Dr. Angus “Mac” McLean, Mr. Scott Rediger
Advanced Technology Center (ATC)
Rockwell Collins
Cedar Rapids, IA
The integration of virtual and constructive elements into live training not only opens new training avenues, but also
raises concerns about flight safety as live aircraft trainees need to be able to differentiate between live and virtual
entities and threats. Current fourth-generation fighter aircraft lack an integrated avionics methodology to provide this
Live Virtual Constructive (LVC) specific situation awareness (SA). A flight simulation study was performed to assess
fighter pilot behaviors in an air-to-air context involving virtual (V) and constructive (C) red force representations.
LVC enabled avionics systems were used involving a networked federation comprised of two fast jet flight simulators,
a fighter trainer jet aircraft configured as an aircraft-in-the-loop (AIL) simulator, a Ground Control Intercept (GCI)
station, and a Next Generation Threat System (NGTS) semi-automated forces (SAF) generator. Participants were nonactive duty fighter pilots. The objective of the study was to attempt to detect and quantify specific, important attributes
of aircrew condition and performance and to show the relationship to VC-enabled training situations. The results
indicate that that red air entity count significantly drove fighter pilot workload and engagement. We also found a
statistically non-significant trend that V red air entities tended to generate a higher workload than C red air.
60
PRACTICAL RECOMMENDATIONS FOR VALIDATING SURVEY APPARATUSES IN COALITION
TRAINING ENVIRONMENTS
2015 IITSEC Paper No.15299
Emilie Reitz
Alion S&T
Norfolk, VA
If a training event happens and no one builds a record of its gains and outcomes, does it matter? How do you know
that the gains and outcomes you recorded, or the tools you used to make that record, are even valid and generalizable
to other situations? Are you really improving human performance, or just inferring that you improved it? It’s a
challenge faced by all communities of research (Teijlingen & Hundley, 2002), whether attempting to solicit survey
data in support of human factors assessments or training effectiveness analyses. This challenge is increased in multinational events, where results contribute to a shared end state for the coalition. To create a valid new measurement
apparatus, reliability and validity must be established, and correlations should be built between subscales. Nonetheless,
that takes time, results measured from a comparable apparatus or repeated tests, and access to audiences that many
researchers lack. During Bold Quest 15.1, two apparatuses were run for precisely this testing and validation purpose
and presented to the multinational training audience under one of two circumstances: uncommented testing of the
apparatuses or careful explanation of the validation and verification purpose. Two-hundred and seven participants
provided over 1600 free text responses which were taken as indicators of their engagement with each apparatus,
compared against a non-pilot-tested survey. The pilot-tested apparatuses that were actively administered, elicited
significantly more productive responses from the participants than the passive administration groups.
Recommendations focus on optimizing apparatuses that cannot be translated into a native language due to constraints,
and provide suggestions to bolster both pilot tested and non-pilot tested apparatuses.
THURSDAY, 3 DECEMBER, 2015 ROOM S320E
H-7 Is That Your Final Decision?
0830
Stealth Assessment of ProblemSolving Skills from Gameplay
(15212)
0900
“Fixing” the Military DecisionMaking Process (15220)
0930
The Small Unit Decision Making
Assessment Battery:
Development and Psychometric
Analysis (15143)
Notes
61
STEALTH ASSESSMENT OF PROBLEM-SOLVING SKILLS FROM GAMEPLAY
Weinan Zhao, Valerie Shute, Lubin Wang
Florida State University
Tallahassee, FL
Stealth assessment represents a promising way to address the needs of validly measuring and supporting
important 21st century competencies (e.g., creativity, problem solving) within interactive digital environments
(e.g., video games). The assessment is woven into the environment such that it becomes invisible to students,
which is conducive to eliciting targeted competencies (Shute, 2011). Stealth assessment also runs dynamically,
enabling real-time support. We use ECD (Evidence-Centered Design, Almond, & Lukas, 2003) as our
assessment design framework for creating stealth assessments that capture far more information related to
multiple competencies compared to traditional forms of assessment, which typically report a single summative
score, and/or or judgments of right or wrong. To date, we have developed a number of stealth assessments for
use in different games to examine various competencies. For example, we have designed three stealth
assessments to measure various cognitive and noncognitive variables in a game called Physics Playground
(Shute & Ventura, 2013). The focal competencies included persistence (Ventura, Shute, & Small, 2014),
qualitative physics knowledge (Shute, Ventura, & Kim, 2013), and creativity (Kim & Shute, in press). From
these design and development efforts, we have learned a number of useful lessons about developing and
applying stealth assessment. This will comprise the focus of our paper—lessons learned and best practices
related to the design process of stealth assessment. We will demonstrate the process of designing stealth
assessment using a research project that assesses problem solving skill in the popular game Plants vs. Zombies
2. Results from our evaluation study show that our game-based assessment is promising, correlating with the
external measures of problem solving: Raven’s progressive matrices (r = .40, p < .01) and MicroDYN (r = .48, p
< .01). However a larger sample size is needed to establish definite claims about its validity.
“FIXING” THE MILITARY DECISION MAKING PROCESS (MDMP)
Michael J. Smith,
Ronald B. Sprinkle
Leidos, Inc.
Orlando, FL
LTC(P) Johnny Powers
PEO STRI
Orlando, FL
James Xu
Adayana
Falls Church, VA
Michael Knapp
Aptima, Inc.
Orlando, FL
The U.S. Army Military Decision Making Process (MDMP), used for planning operations, is a deliberate,
time intensive, manual process. Critics state that MDMP Course of Action (COA) Analyses take too long
to arrive at a single plan. COA analysis and Running Estimate (real time comparison of a running
operation against the plan) require data to measure and compare combat actions. Many assert that the only
viable way to automate and measure proposed COAs is to use data produced by simulation. Historically,
simulations have been difficult to setup, require specially trained personnel and separate computing
hardware to operate, making their application impractical in a tactical environment.
To address these problems, we developed a concept prototype and architecture to make practical use of
simulation to support the MDMP. We believe that “fixing” the MDMP means increasing its speed through
rapid automated decision support. During development of the prototype, we explored the technical barriers
and military planning process updates that would help automate the MDMP with simulation support.
U.S. Army simulations require several major modifications to be practical in a Mission Command
Information System (MCIS) environment. First, technical support requirements must be eliminated.
Second, an interface that supports the input of plans and operations by Warfighting planners is needed.
Third, Warfighters must be able to specify measurement of COAs, plans, and operations. In addition,
recognizing the human/machine boundaries in the decision-making process, we must be mindful that
simulation systems cannot present conclusions that can only be fully developed by experienced
warfighters. This paper shows how these things can be done and addresses primary MDMP criticisms.
62
THE SMALL UNIT DECISION MAKING ASSESSMENT BATTERY: DEVELOPMENT AND
PSYCHOMETRIC ANALYSIS
Karol G. Ross, Jennifer K. Phillips
Cognitive Performance Group
Orlando, Florida
Kenneth A. Knarr
II Corps Consultants, Inc.
Quantico, Virginia
The U.S. Marine Corps Training and Education Command is developing a requirement for a set of
measures to assess cognitive abilities in support of small unit decision-making challenges. Prior
assessment efforts in this area of research have failed to address several key issues, including the
multidimensionality of decision making and the ability to predict decision performance across a range of
operational settings. This paper reports on the development of a Small Unit Decision Making Assessment
Battery that treats decision performance as a multidimensional construct supported by competencies, such
as problem solving and attentional control, and cognitive and relational skills, such as perspective taking
and resilience. Candidate battery instruments were selected or developed based on their face validity and
existing psychometric properties as measures of 15 constructs hypothesized to enable small unit decision
making. The instruments were subjected to comprehensive testing with a large population from The Basic
School to assess their psychometric properties and to finalize each instrument. Analyses were performed at
item, battery, and relationship levels to identify the most meaningful items and improve internal
consistency reliability, examine the factor structure of each instrument, and identify the constructs most
predictive of decision-making proficiency. Results indicate the predictive ability of the battery and the
ability of the battery to distinguish levels of performance by correctly, significantly classifying participants
into different performance levels. This research furthers the community’s understanding of decision
making as a multidimensional construct. Plans for future research and application are discussed.
H-8 From the Halls of Montezuma
1030
MarineNet User Engagement
Exercise (15011)
1100
Supporting Unit Training
Management Through Mobile
Performance Assessment Tools
(15034)
1130
Marine Corps Instructor Master
Model: A Foundation
for Marine Faculty Professional
Development (15146)
Notes
63
MARINENET USER ENGAGEMENT EXERCISE
Major Michael A. Gavin, USMC
College of Distance Education & Training
Quantico, VA
The Marine Corps Distance Learning Network (MarineNet) is the United States Marine Corps’ enterprise level
Learning Management System. MarineNet is employed to increase operational readiness by improving training quality
and accessibility for individual Marines. The Marine Corps University’s (MCU) College of Distance Education and
Training (CDET) is the entity responsible for managing MarineNet. Like many technology heavy organizations,
CDET has encountered several challenges in adapting and aligning organizational practices with emerging
technologies and evolving user needs. This paper details CDET’s efforts to mitigate these challenges through the
conduct of the MarineNet User Engagement Exercise (MUE2).
The MUE2 was executed as an instructor led discussion and survey. Participants were drawn from the I, II, and III
Marine Expeditionary Forces, Marine Forces Reserve, and MCU’s Professional Military Education resident
schoolhouses. The problem the MUE2 research addressed focused on the development of a procedural method to tap
into the range and depth of knowledge available within the MarineNet end user community. Based on the principles of
human-centered design, the MUE2 is a requirements elicitation project that directly engaged the MarineNet end user
community as an exploitable systems design asset. The objective of the MUE2 was to give voice to the MarineNet end
user population and to provide CDET with a contextually based understanding of the concerns held by the end user
community. The purpose of the MUE2 was to provide CDET with a data-driven decision support methodology on
which the architecture changes designed to improve MarineNet’s capabilities could be validated and appropriately
prioritized.
SUPPORTING UNIT TRAINING MANAGEMENT THROUGH MOBILE PERFORMANCE ASSESSMENT
TOOLS
Courtney Dean
Aptima, Inc.
Woburn, MA
Matthew Puglisi, Jared Freeman, Ph.D.
Aptima, Inc.
Washington, D.C.
The Marine Corps Training Information Management System (MCTIMS) warehouses performance information
regarding all mission essential tasks Marine Corps units must perform in order to execute across the full range of
military operations. MCTIMS currently has a requirement for a mobile application. This application enables Marine
leaders to collect unit performance data digitally in the field and quickly upload those data into MCTIMS, avoiding
laborious manual input of each result. The application’s concept of operations is as follows: 1) download performance
evaluation criteria from MCTIMS; 2) provide leaders with inputs to capture performance ratings; 3) display results
immediately following an exercise to enable After Action Reviews (AARs) and; 4) upload results to MCTIMS to
support tracking of trends across the force. A prototype application, MCTIMS Mobile, was developed for testing and
feasibility assessment. The application was tested in two live-fire exercises where usability and utility metrics were
captured. The goals of the field tests were to informally evaluate, with Subject Matter Experts (SMEs) and proposed
users, the ability of the MCTIMS Mobile tool to support productive trainee assessment and efficient data collection
and measurement. The field study demonstrated support for the tool, with clear directions for improvements. The
results show that the tool works as expected, but certain features can be made more intuitive and easier to use. The
quantitative results were interpreted very strictly and users provided substantial constructive feedback. Consideration
of the feedback received from the users led to redesign and modification of the mobile tool. MCTIMS Mobile is
intended to capture more data concerning Marine performance, better data (because it is captured in real time), and
data that persist in MCTIMS to improve assessments of Marines and the training the Marine Corps provides.
64
MARINE CORPS INSTRUCTOR MASTERY MODEL:
A FOUNDATION FOR MARINE FACULTY PROFESSIONAL DEVELOPMENT
Jennifer J. Vogel-Walcutt, Jennifer K. Phillips, Karol G. Ross
Cognitive Performance Group
Orlando, Florida
Kenneth A. Knarr
II Corps Consultants, Inc.
Quantico, Virginia
This paper reports on the creation of a USMC Instructor Mastery Model and its utilization for setting
performance standards and assessing instructor performance. The Mastery Model is derived from the
Dreyfus and Dreyfus model of cognitive skill acquisition. It makes the path to mastery explicit by
specifying how individuals progressively develop into high performers and what indicators can be
observed and assessed during each of five stages of development. To customize the model for Marine
instructors, reviews of the literature and other services’ instructor development approaches were
conducted. Semi-structured interviews were conducted with 93 highly skilled instructors at 15 USMC
learning institutions. Thematic analysis and card sorts were employed to understand what a skilled Marine
instructor does and define 10 Key Performance Areas (KPAs). A second analysis examined how an
instructor performs at the five stages of development and yielded performance indicators for each KPA at
each stage. Outcomes were examined against teacher competencies from the literature to compare military
and civilian instruction. The final set of KPAs has been adopted by TECOM and included in the Train the
Trainer Training and Readiness (T&R) Manual in the form of five T&R events and five appended learning
outcomes. The next step is to produce an Instructor Assessment Battery and observation rubric to enable
assessment of individuals’ progressive skill development as well as program- and policy-level impact
assessments.
H-9 Experience Counts
1330
Realism and Effectiveness of
Robotic Moving Targets
(15118)
1400
Soldier Physiological Changes of
Shooting Performance in the
Tank Simulator (15192)
1430
Novice and Experience Police
Officer Simulation Experience:
Guiding the Future (15370)
Notes
65
REALISM AND EFFECTIVENESS OF ROBOTIC MOVING TARGETS
Elizabeth Uhl Ph.D., Martin Bink Ph.D.
Fort Benning, GA
David James
Northrop-Grumman Technical Services
Columbus, GA
For the vast majority of U.S. Army Soldiers, the first opportunity to engage a realistic moving target with
small arms is in combat. Even Infantry Soldiers and special-skill Soldiers (e.g., Snipers) have very limited
opportunities to train realistic moving-target engagements. Current capabilities are limited to targets fixed
to rail systems or silhouette targets mounted on pickets that can be walked by Soldiers in a firing-range
target pit. Without the opportunity to practice engaging realistic moving targets, the Soldier is not able to
develop the correct perceptual and motor tuning to adequately engage live moving targets. One solution for
the lack of moving target training capabilities is the use of robotic human-type targets (RHTTs). RHTTs
can present a realistic three-dimensional human-sized target that can freely move with semi-autonomous
control. Furthermore, RHTTs can be programmed to react to events (e.g., flee after another RHTT is hit)
and to move in groups in order to provide more complex training scenarios. Even though RHTTs provide a
significant increase in training capability, the realism of the RHTTs will ultimately determine the training
effectiveness of the capability. In a sense, the training effectiveness question is a matter of human-robot
interaction. RHTTs are designed to emulate human beings moving over terrain, moving in a defined area,
and reacting to scenario events. If the RHTTs are perceived as freely moving and acting humans, then
training can be optimized. Perceptions of realism, shooting performance metrics, and training capabilities
inventories were collected from Soldiers training with one type of RHTT to determine the level of target
realism. Overall, the RHTT was determined to provide a realistic representation of human targets.
However, several factors detracted from realism in certain scenarios. Both the factors that contributed to
and the factors that detracted from realism provide insights for developing more effective RHTTs.
EEG/ECG OSCILLATION OF SOLDIER’S MENTAL STATES ON THE SHOOTING TASKS IN THE
TANK SIMULATOR
Li-Wei Ko, Peng-Wen Lai, Yi-Cheng Shih
Shih-Chuan Lin, Meng-Shun Yang, Chin-Teng Lin
Hsinchu, Taiwan R.O.C.
In-Chung Chang
Taipei, Taiwan R.O.C.
Soldiers executing military missions, especially shooting tasks that involving visual attention are high level cognitive
processes, which is highly correlated to the soldiers’ survival and fighting abilities in military operations. Current
shooting simulation training in the military mainly focuses on the performance of target shooting and does not
consider the soldier’s mental states to evaluate their performance. Therefore, this study intends to explore soldier’s
mental states of visual attention corresponding to EEG oscillations when performing the simulated tank-shooter task.
Soldiers are required to finish (5-times target shooting practice) with time constraint in the realistic tank-inspired cabin
simulator. Physiological signals including 32-channel EEG signals and lead-II ECG signals are simultaneously
recorded while performing shooting tasks. Using shooting scores and physiological measurements, we can further
investigate the cross correlation with each other and figure out when will be the better time period to execute military
mission. According to the preliminary study, in time-frequency domain, this study observed the correlation between
task performance and the difference of frequency bands activities in each brain regions from junior and senior
shooters. In terms of frequency domain, the study found that novices and skilled subjects have differences in prefrontal lobe, motor areas and occipital lobe. Based on these results, we constructed biomedical-signals models with
attention, decision. A comprehensive analysis of shooting performance and physiological signals will be applied to
construct a prediction model of the better-shooting state. The constructed model can effectively reflect the soldier’s
mental states and could be considered as one of the evaluation indicators for target training performance in the future.
66
NOVICE AND EXPERIENCED POLICE OFFICER SIMULATION EXPERIENCE – GUIDING THE
FUTURE
Dr Amanda Davies
Charles Sturt University
Goulburn, NSW, Australia
In the education and simulation design communities the deliberations continue which are centered on how much
fidelity is enough or too much. This paper presents findings from a research project which explored the way in
which fidelity influences the sense of immersion and presence and the subsequent perceived benefit to the
transfer of learning to the field of application in police training for high-risk high-stakes decision-making. The
unique feature of the study is the inclusion of two case studies, one of which utilizes low physical fidelity and
high psychological fidelity with participants who are seasoned field based police operatives. The second
simulation based learning exercise environment and scenario embraced high levels of physical and
psychological fidelity with participants who have nil or limited operational experience in the real world of
policing. The common criterion for the two simulation exercises is a pivotal catalyst which requires decisionmaking in providing a police response to a high risk incident. The findings suggest that a key design feature in
the development and application of simulation-based learning environments and exercises is the level of prior
real world experience the learner has with the simulated environment. This case study offers insight into the
value learners place on the simulation characteristics in representing the real world environment and how this
influences the application of knowledge and skills in their real world of policing the streets. Understanding the
influence on field based application of simulated learning environments offers a valuable contribution to the
instructional design endeavours for creating authentic situated learner experiences.
TUESDAY, 1 DECEMBER 2015 ROOM S320F
P-1 Enterprise, Architecture, and Standards
1400
The Live-Synthetic Training and Test & Evaluation
Enterprise Architecture (15076)
1445
Early Adoption of Common Operating Environment
(COE) Standards and Guidelines (15098)
Notes
67
THE LIVE-SYNTHETIC TRAINING, TEST AND EVALUATION ENTERPRISE ARCHITECTURE
Paul Dumanoir
PEO STRI
Orlando, FL
Jeff Bergenthal, David Drake
The Johns Hopkins University Applied Physics Laboratory
Laurel, MD
Cross community and interservice Live-Synthetic (Virtual-Constructive-Gaming) initiatives often fail due to the
lack of formalized governance, as stated by Frank DiGiovanni, Director, Force Readiness and Training in the
Office of the Deputy Assistant Secretary of Defense (Readiness), during the 2014 Interservice/Industry Training,
Simulation and Education Conference (I/ITSEC). In March 2014, the US Army initiated a cross community
Research and Development (R&D) initiative to investigate the feasibility of establishing a common LiveSynthetic approach for the Training and Test & Evaluation (T&E) communities, called the Live-Synthetic
Training and Test & Evaluation Enterprise Architecture (LS TTE EA).
This paper reports on the progress of this initiative in establishing the objective framework for the enterprise
architecture (EA) that includes: (1) the initial governance approach, (2) the business architecture, and (3) the
reference architecture. The governance approach provides agreed-upon practices and interactions for the
formalized collaboration between organizations to build and deploy services that are useful and sustainable for
the EA. The framework for the development and evolution of the governance approach for the LS TTE EA is
outlined, including how the governance approach is being applied to current prototyping activities. The business
architecture provides a common understanding to align Army strategic objectives and tactical Training and T&E
demands. Business architecture artifacts and the results of a quick-look cost benefit analysis are discussed. The
reference architecture is the authoritative source of information that guides the implementation of EA solutions.
The reference architecture layers and initial documentation in Department of Defense Architecture Framework
(DoDAF) viewpoints are shown. Finally, the paper discusses the way forward for application of the EA
objective framework to the Army’s Integrated Training Environment (ITE), Integrated Live-VirtualConstructive Test Environment (ILTE), and Synthetic Training Environment (STE), and applicability to the
Defense Training Environment (DTE).
EARLY ADOPTION OF COMMON OPERATING ENVIRONMENT (COE) STANDARDS AND
GUIDELINES
Robert Wittman, PhD
MITRE Corporation
McLean, VA
Amit Kapadia
US Army PEO STRI –
PdM OneSAF
Orlando, FL
Burt Grippin, Sean Barie,
Chris Holmes
MITRE Corporation
Orlando, FL
Paul Dumanoir
US Army PEO STRI –
PdM WTI
Orlando, FL
This paper presents the driving requirements, results, and user-oriented use cases behind the Enterprise
After Action Review (EAAR) prototype. The activity, funded in FY14 by Program Manager Integrated
Training Environment (PM ITE), focused on producing a reusable AAR architecture with transferrable
technologies and lessons learned for early adoption of the Common Operational Environment (COE)
across PM ITE and PEO STRI. At project initiation it was clear the successful execution of a prototype
depended on collaboration between the PM ITE team and engineers supporting the Command Post
Computing Environment (CP CE) within PM Mission Command (MC). This paper starts by introducing
earlier efforts that provided critical insight and software that led to the feasibility of the EAAR prototype
investigation using COE constructs. It continues by highlighting common AAR requirements across the
training and operational MC community. This sheds light on the potential for AAR system lifecycle costsavings through shared AAR component development and use across the two communities. The paper then
explores the specific COE design constructs and technologies employed as part of the EAAR prototype
effort. Finally, the paper concludes with a section on technology transfer progress and a listing of the COE
and other standards-based technologies employed and their training or operational MC source.
68
TUESDAY, 1 DECEMBER, 2015 ROOM S320F
P-2 Design, Build, Track and Train – Here and Abroad
1600
Sejong the Great Class DDGs: How ROK Navy Trained
and Embraced Them (15032)
1645
A System-Model-Centric Collaborative Environment for
the Acquisition Lifecycle (15093)
Notes
THE SEJONG THE GREAT CLASS DDGS: HOW ROK NAVY EMBRACED AND TRAINED THEM
LCDR Junho Eum, ROKN
Ajou University
Suwon, Republic of Korea
CAPT Minsoo Yang, ROKN
Republic of Korea Navy
Busan, Republic of Korea
Sangyoon Oh
Ajou University
Suwon, Republic of Korea
The Aegis combat system has been widely considered as one of the most powerful weapon systems for surface
ships due to its high level of war-fighting capability. In 2009, Republic of Korea (ROK) Navy became the fifth
country in the world that operates Aegis combat system-equipped ships through the KDX-III program, and a
total of three Sejong the Great class DDGs (Destroyers with Guided missile) are currently in fleet operations.
These ships have performed an excellent level of war-fighting capabilities such as successful tracking missions
for the missile provocations by North Korea and ROKS Sejong the Great’s top-gun award-winning at the firing
competition during multi-national exercise. These are empirical supports showing outperformance of ROK
Navy’s DDGs in spite of its relatively short period of operations as less than six years when compared to other
countries operating similar Aegis equipped ships. In this paper, we introduce ROK Navy’s strategy, plan and
efforts to realize these accomplishments in the KDX-III program, mainly focusing on how ROK Navy
accomplished and settled a high level of war-fighting readiness in such a short period from the ship’s training
and familiarization perspectives. Primarily, ship crew’s individual capability and team work as an entire ship
force were considered as the fundamental for proper operation and maintenance, which precede the state-of-theart hardware such as system and weapons.
We introduce ROK Navy’s systematic approach applied to training and familiarization as a core factor to
maximize ship’s performance and readiness in this paper. From the beginning of the KDX-III program, a phased
approach for ship crew was applied to develop skills from the basic to advance. Based on the programmatic
foundation, we present plans and achievements by ROK Navy Headquarters and fleet operations, which
provided various opportunities including on-board familiarization and utilization of the Aegis Operation and
Maintenance Center (AOMTC) – ROK Navy’s own education and training facility for DDGs. At the end, we
conclude with lessons learned and proposals to utilize these efforts for force improvements in the future.
69
A SYSTEM-MODEL-CENTRIC COLLABORATIVE ENVIRONMENT FOR THE ACQUISITION
LIFECYCLE
James E. Coolahan, Ph.D.
Jeffery J. Bergenthal
Coolahan Associates, LLC
Johns Hopkins University Applied Physics Laboratory
Ellicott City, MD
Laurel, MD
Since the mid-1990s, the U.S. Department of Defense has sought ways to improve the acquisition process
by applying modeling and simulation (M&S) tools and data in a collaborative fashion involving
government customers and industry providers. Originally known as (Distributed) Simulation Based
Acquisition, over the past decade, these concepts have evolved under various monikers, including Model
Based Systems Engineering, originated by the International Council on Systems Engineering, and Model
Based Engineering, a term that has been used by the Office of the Secretary of Defense (OSD) and the
National Defense Industrial Association (NDIA). OSD has recently introduced a concept for a (Digital)
System Model that evolves throughout the acquisition lifecycle, and the U.S. Air Force has coined the
terms Digital Thread and Digital Twin that complement the OSD concept.
As these concepts have evolved, the M&S Committee of NDIA’s Systems Engineering Division has
performed two technical studies that help to describe the application of M&S tools/data during the
acquisition lifecycle. This paper provides a summary of the results of these two studies. The first study, the
identification of M&S capabilities across all acquisition lifecycle phases, resulted in the development of a
multi-spreadsheet workbook that links activities in each lifecycle phase with M&S capabilities that support
performing those activities, along with example tools that can be used to implement those M&S
capabilities. The second study, the identification of essential elements of the system model, instantiates the
high-level system model concept by defining the data/information needed for its implementation, linked to
the acquisition activities and M&S capabilities identified during the first study.
Finally, the paper presents a potential way ahead for integrating these concepts and studies to formulate a
high-level approach for an overarching collaborative M&S environment centered on the system model
concept.
P-3 Developing Competency – The Future for M&S
1030
Improving Education, Training and
Career Advancement through
Competency Portability (15117)
1100
STEMulating: An Integrated
Approach to Cultivating Our
Future (15270)
1130
Professionals – Meeting the
Demand (15342)
Notes
70
IMPROVING EDUCATION, TRAINING AND CAREER ADVANCEMENT THROUGH COMPETENCY
PORTABILITY
Robby Robson
Eduworks Corporation
Corvallis, Oregon
“Competencies” – which in this paper include skills, knowledge, abilities, learning objectives, and
outcomes – play a fundamental role in education, training and career advancement. This is reflected in the
many standardized lists of competencies ranging from occupational standards (MOS and O*Net) and task
lists (UJTL) to educational standards (Common Core Standards) and industry standards (NIMS, OPITO,
etc.). It is also reflected in the emergence of organizations such as the Competency-Based Education
Network (CBEN), and in initiatives sponsored by federal agencies such as the Departments of Labor,
Defense, and Education, and the White House.
Nonetheless, competencies are still not managed or exchanged in a standardized or interoperable way.
Training systems cannot access lists of competencies through an Application Programming Interface
(API), training packages define their own tasks and outcomes rather than use existing ones, and
competency-based records of achievement are rarely transportable across military-civilian barriers. The
consequences are severe: Jobs go unfilled by qualified unemployed workers (especially veterans) and
billions of dollars are wasted because of unnecessary or ineffective training.
Multiple efforts aim to change this. These include standardization efforts, technology development
sponsored by the government and by private industry, and government initiatives. This paper provides an
overview of competencies, reports on competency-related efforts, and discusses the implications for the
training community.
STEMULATING: AN INTEGRATED APPROACH TO CULTIVATING OUR FUTURE
Elizabeth Biddle,
Ph.D., CMSP
CFSEC
Orlando, FL
Carol Ann Dykes
UCF Business
Incubator
Orlando, FL
Shawn Harrs, Ph.D.
Universal Orlando
Resort
Orlando, FL
Robert Seltzer
NAWCTSD
Orlando, FL
Abdul Siddiqui
PEO STRI
Orlando, FL
For over 50 years, the need to increase the number of students who pursue STEM (science, technology, engineering
and math) has been acknowledged. Yet the US continues to fall behind in student performance in STEM fields and
pursuit of STEM degrees (US Bureau of Labor Statistics, 2009; Galloway, 2008; Rothwell, 2014; National Research
Council Committee on Science, Engineering Education Reform, 2006;). The STEM talent pool impacts not only the
industries that drive the US economy but also those that comprise the US defense industrial base. Now more than ever,
STEM underpins the Department of Defense’s (DoD’s) ability to defend the Nation. While there are many factors that
impact STEM education, a key element of increasing the STEM workforce is stronger local and regional partnerships
among industry, academia, government, and nonprofits (Achieve, 2010). The Central Florida STEM Education
Council (CFSEC) is a model of such a regional partnership with a long term objective of encouraging and preparing
pre-college students to enter STEM fields of study and to pursue employment in the Central Florida workforce. The
CFSEC targets primarily three audiences– parents, students and teachers – through communications and events. The
CFSEC’s goals are to 1) advocate for and raise awareness of the importance of STEM, 2) connect individuals and
organizations with resources and each other, and 3) coordinate activities and partnerships that increase the awareness
and availability of STEM education opportunities. This paper will provide an overview of the future outlook for the
STEM workforce that highlights the compelling need for STEM initiatives to address the projected shortages (Bayer
Corporation, 2014; Morones, 2013). The paper will describe the vision and governing model of the CFSEC and how it
can serve as a model that other regions and states can adopt and tailor to implement a collaborative STEM community.
Finally, the paper will conclude with the challenges encountered in establishing a CFSEC-like organization as well as
best practices and lessons learned.
71
MODELING AND SIMULATION PROFESSIONALS – MEETING THE DEMAND
Lisa Jean Bair
SAIC
Norfolk, VA
James J. Jackson
SAIC
Virginia Beach, VA
The development and appropriate use of modeling and simulation (M&S) technology relies on
professionals with a skill sets that run the gamut from computational science, software engineering, and
analysis to domain knowledge found in instructional design, physics, engineering, health sciences, military
sciences and more. To support the needs for developing the skilled workforce required to grow the M&S
industry, which provides well-paying jobs and bolsters the economy, universities have developed
curricula, certificate, and degree programs to meet this niche. However, fulfilling these M&S skill sets is
not the exclusive domain of M&S programs. Furthermore, time constraints may require specialization
among the many M&S-related topics for a Certified Modeling and Simulation Professional (CMSP). It
may not be possible for a new graduate to develop proficiency sufficient to meet workforce demands. This
paper builds upon earlier work surveying Domains, Skills, Knowledge, and Applications of the M&S
Professional. Assuming the stated restrictions exist, we survey professionals, program managers, and
academics and examine areas of emphasis in the various M&S programs available. We evaluate the degree
to which M&S specific programs are able to meet industry demands and assess whether those demands are
being met by M&S graduates or graduates with other specialties. We end with observations about the
M&S workforce with recommendations for the M&S Community at large.
P-4 Getting Your Money’s Worth
1400
Calculating Simulation-Based
Training Value: Cost
Avoidance and Proficiency (15199)
1430
Large Scale Adoption of Training
Simulations: Are We There
Yet? (15256)
1500
Automated Surveys: Lowering the
Respondent’s Burden (15080)
Notes
72
CALCULATING SIMULATION-BASED TRAINING VALUE: COST AVOIDANCE AND PROFICIENCY
Dr. Tim Cooley
DynamX Consulting
Castle Rock, CO
Mr. Greg Seavers, Mr John Roth &
Mr. Jose Rodriguez
USMC/PM-TRASYS
Orlando, FL
Dr. Steven Gordon
Georgia Tech Research Institute
Orlando, FL
For over a decade, leaders in the Department of Defense (DoD) have asked “What is the value of
simulation based training?” or “Why should I spend one more dollar on simulation?”. The easiest answer
may be the qualitative benefits of simulation that pertain to areas like safety, availability, flexibility in
scenarios, and protection of operational plans. But leaders want quantitative measures such as Return on
Investment (ROI), and ROI for simulation has two major facets: cost and benefit (often called results). The
authors’ research has focused on answering the DoD questions with cost avoidance computations because
cost avoidance is a key component of ROI calculations, and with proficiency improvement evaluations to
determine if use of simulation has a quantitative benefit. The authors will present their methods of
calculating cost avoidance over the last four years for many simulation-based training systems managed by
the United States Marine Corps Program Manager for Training Systems (PM-TRASYS). The research has
shown cost avoidance of over $2B across the PM-TRASYS systems analyzed, but the authors are also
developing methods to refine these measures of cost avoidance. These more realistic measures are linked
to live training requirements allowed to be conducted in simulation and/or to proficiency increases due to
use of simulation. These same cost avoidance methods are now being applied to large scale exercises to
show the value of using a mix of live-virtual-constructive systems in these scenarios. This paper will also
discuss the management data that is gathered and depicted as part of the cost avoidance studies. This data
captures use statistics by system and site, and includes data such as the number of Marines trained and
munitions used. As will be shown in the paper, this study provides quantitative measures of simulation
cost avoidance and results.
LARGE SCALE ADOPTION OF TRAINING SIMULATIONS: ARE WE THERE YET?
Dr. Amela Sadagic
Naval Postgraduate School
Monterey, CA
Maj Floy A. Yates Jr.
MAGTFTC Battle Simulation Center, USMC
Twentynine Palms, CA
Computer-supported training simulations have been recognized for the potential and the benefits they have in
supplementing the training needs of the military, yet we still do not see evidence of large-scale deployment and
adoption of these systems by users in this domain. The current challenging budgetary situation suggests that the
Return on Investment (ROI) will be more scrutinized than ever before, forcing communities to abandon
underutilized and underperformed Modeling and Simulation (M&S) solutions. Such developments are also
likely to affect global decisions related to future investments in these types of technologies. This paper presents
the design and results of a study that included collection of comprehensive data on the adoption and use of
training simulations in the military domain. The analysis of this data set suggests that the reasons for low use of
simulations had little to do with the overall quality of hardware and software (although they were mentioned as
factors), and that a myriad of other factors were found to influence the outcome to a greater extent. The
understandings collected in this and other studies all attest that military training is a complex, multilayered
domain that is only partially defined by the type and technical characteristics of systems being used to achieve
that goal. Our work and experience in this domain give us a firm basis to hypothesize that a well selected set of
strategic approaches could bring much greater results in this domain, even with a modest investment made to
support that change. The findings and recommendations are highly applicable to all DoD services and other
communities that plan to use these types or solutions in their training and learning practices. The study also
offers a contribution towards a better understanding of general diffusion and adoption of other technical
innovations in the military domain.
73
AUTOMATED SURVEYS: LOWERING THE RESPONDENT’S BURDEN
Richard Kist MSc., Igor Franken MSc.
National Aerospace Laboratory (NLR)
Amsterdam, The Netherlands
Measuring effectiveness and the operational utility of new techniques, technologies and training through
feedback of the Warfighter is critical. Paper questionnaires and interviews have merits. However, offering
a relevant, compact questionnaire after each mission to test and exercise participants with different roles
and backgrounds can be made more efficient with an automated survey. This paper discusses the
challenges faced in survey data collection in an operationally realistic environment, and lessons learned
during years of survey data collection. During the Bold Quest cycle, a survey effort was set up using such
an automated tool (Questionnaire for Utility Evaluation and Survey Tool, QUEST). The concept of
employment for automated surveys described here addresses many of the concerns associated with webbased surveying techniques (Sills & Song, 2002). Use of clear and concise questions and other measures
lowering the burden for the respondents yield the best results from a survey effort. An automated tool
ideally should work locally on any laptop or handheld device, as well as in networked conditions. It should
be tailored to military environments, offering questionnaires for all phases (pre/post exercise, daily post
mission). By tooling the questions to reduce burden, analyzing the language used, and taking steps to
assure the relevance of the questions to each participants, more efficient data collection can occur – the
kind of data collection which provides 100,000s of survey responses helping to determine the effectiveness
of new military developments.
P-5 Cybersecurity: New Threats, New Policies, New Solutions
1600
Risk Management Framework
(RMF) Transition Impacts in
Training Simulation Systems
(15009)
1630
Cybersecurity Controls: Then and
Now (15010)
1700
Cybersecurity Challenges and
Resolutions for Simulator
& Training Systems (15063)
NOTES
74
RISK MANAGEMENT FRAMEWORK (RMF) TRANSITION IMPACTS IN TRAINING SIMULATION
SYSTEMS
Graham Fleener & Marco Mayor
U.S. Army PEO STRI
Orlando, FL
Dr. Cliff Zou
Orlando, FL
The Department of Defense (DOD) Information Assurance Certification and Accreditation Process
(DIACAP) is undergoing its first transition and update since 2007. The new process is titled Risk
Management Framework (RMF) and there are significant changes in the new guidance. Given the
transition there are a number of implications for the training and simulation community for ensuring
training systems maintain both their certification and their information security posture. Guidance for the
transition has been evolving slowly with each the agencies initiating RMF implementation individually.
The Program Executive Office for Simulation, Training, and Instrumentation (PEO STRI) follows Army
guidance for the transition. This paper will define the formal requirements, new terminology, and discuss
how the RMF risk assessment is determined. Additionally, we will capture the transition and migration of
how PEO STRI will implement the Risk Management Framework. This paper will describe the tools that
support the RMF implementation, such as the Knowledge Service (KS) and the Enterprise Mission
Assurance Support Service (eMASS). We will describe the transition impacts for PEO STRI stakeholders
such as contractors doing business with PEO STRI, system users, and Project Managers (PM). Each of the
stakeholders will have unique concerns, impacts, and questions during the transition. There will be a
number of challenges associated with transitioning to a new process that will be discussed. To conclude,
we’ll provide guidelines to help the training and simulation community make the transition to RMF.
CYBERSECURITY CONTROLS: THEN AND NOW
Marco Mayor
U.S. Army PEO STRI
Orlando, FL
The phase out of the Department of Defense (DOD) Information Assurance Certification and Accreditation
Process (DIACAP) is leading to a new process called Risk Management Framework (RMF). This new process
was mandated by DOD Instruction 8500.01, which also mandated the adoption of the term “cybersecurity” to be
used throughout DOD instead of the term “information assurance (IA).” RMF will follow a set of security
controls inherited from the National Institute of Standards and Technology (NIST). These controls are
specifically located in the Special Publication (SP) 800-53. The NIST SP 800-53 controls will replace the
existing DOD Instruction (DODI) 8500.2 controls and have been updated to reflect the evolving technologies
while addressing new cybersecurity threats. Given the transition, there are a number of implications for the
training and simulation community for ensuring training systems comply with these new controls and maintain
their information security posture. Guidance for the transition has been developing gradually and each of the
DOD agencies are handling it individually at the implementation level. The Program Executive Office for
Simulation, Training, and Instrumentation (PEO STRI) is following DOD and specifically Army guidance to
ensure the NIST control implementation gets executed in the most efficient manner possible.
This paper will first provide some background on the legacy DOD 8500.2 controls and an overview of the
transition to the NIST SP 800-53 controls. It will then discuss the formal requirements, new terminology,
implementation and guidance driving this transition. This paper will analyze the framework of the NIST SP 80053 RMF controls and how they compare to DIACAP controls. It will discuss the security control overlays, and
the assessment procedures. To conclude, this paper will describe the transition impacts for PEO STRI
stakeholders, which include DOD contractors, system users, and Project Managers (PM). This paper will layout
the fundamental idea and challenges PEO STRI faced on a particular use case, while handling the transition
from the DODI 8500.2 DIACAP controls to the NIST SP 800-53 RMF controls.
75
CYBERSECURITY CHALLENGES AND RESOLUTIONS FOR SIMULATOR & TRAINING SYSTEMS
Douglas E. Wedel
Defense Security Service, IOFND
Beavercreek, OH
Dr. Ilya Lipkin
USAF AFMC AFLCMC/WING
Wright-Patterson AFB, OH
Lt. Luis Cintron
USAF AFMC AFLCMC/WNSEB
Cybersecurity (CS) requirements and considerations have increasingly been impacting special-purpose systems with
embedded Information Technology (IT) such as simulator and training systems in recent years. This is primarily
driven by increased insider threats, proliferation of network interconnections, and the rise of mobile computing
(smartphones/tablets) as well as increased capabilities of nation states, organized crime, and political activists to gather
and exploit information about current capabilities. In the past CS measures have been applied through either Risk
Avoidance “shutting down a capability until the risk is eliminated” or Risk Ignorance, “operating a system without
regard to the risk because of a perceived functional or operational need”. However, through the use of Risk
Management, CS can balance these two areas by assuring the mission and protecting the systems, networks and
information by properly categorizing the system and the information through a risk based assessment process. To
avoid mission impact previous policy was compliance based and risk was typically avoided or waived rather than
mitigated. The DoD Risk Management Framework (RMF) (DoDI 8500.01, 2014) seeks to address the shortfalls that
compliance management imposed on systems. However, a clear understanding of how to apply risk is needed to
provide a balanced approach to CS. To support CS requirements this paper will present an approach for assessing risk
to simulator and training systems and outline the steps necessary to overcome and mitigate said issues through a
process that focuses on applicability, compliance, mitigation, and reduction of impact. This paper is not a description
of the DoD RMF, but seeks to provide a process to assess CS requirements by addressing the “Spirit and Intent” of the
CS requirement, its applicability, probability, and impact of applying or not applying that requirement, and identifying
solutions that resolve the finding or reduces the impact to an acceptable level for authorization. This paper will strive
to provide a practical approach to assessing system risk by providing initial framework examples that will demonstrate
its applicability to manage new technology insertions, network connectivity, existing program limitations and mobile
computing impacts to existing simulator and training systems.
TUESDAY, 1 DECEMBER, 2015 ROOM S320B
S-1 Cutting Edge Training
1400
Mobile Augmented Reality for
Force-on-Force Training
(15223)
1430
Emergency Medical Card
Augmented Reality: Training
Evaluation (15266)
1500
Real-Time Cutting of Organs with
Scissors (15333)
Notes
76
MOBILE AUGMENTED REALITY FOR FORCE-ON-FORCE TRAINING
Richard Schaffer, Sean Cullen, Laura Cerritelli
Rakesh Kumar, Supun Samarasekera, Mikhail
Lockheed Martin
Sizintsev, Taragay Oskiper, Vlad Branzoi
Burlington, Massachusetts
SRI International
Princeton, NJ
Live field training against a thinking human opposing force – force-on-force training – is highly valued by
commanders. However, a limitation of current force-on-force training is the lack of battlefield effects, such
as mortar or artillery detonations. This prevents fully employing indirect fires as part of combined arms
operations in these exercises. In particular, forward observers have no means to adjust fire if they cannot
observe impacts. We describe the development of a prototype system that provides mobile forward
observers the visual feedback they need to conduct these operations.
The key emerging innovative technology that enables this training is precision mobile augmented reality.
Augmented reality inserts virtual elements into views of real environments. In this application, a forward
observer’s position and look direction must be precisely tracked in real-time in order for battlefield effects
to appear stably in the correct location. This precision must be maintained as the observer moves between
positions. In addition, the effects must be rendered realistically, so they appear to be part of the
environment and reflect local conditions, including wind and obscuration by terrain. Forward observers
routinely use binoculars to locate targets and adjust fire. Consequently, augmented reality capable
binoculars are also required for this task. As an additional challenge, the tracking and rendering for both
naked eye and binocular views must be performed on a small, lightweight, body-worn computer
compatible with field use. Finally, the system must integrate with an existing LT2 force-on-force training
system.
This paper describes the key advances needed to produce the prototype system. We focus in particular on
the challenges of extending an earlier prototype designed for use only from fixed positions and not
connected to any live training system. The results of initial demonstrations at MCB Quantico are also
presented.
USING AUGMENTED REALITY TO TRAIN COMBAT MEDICS: AN EVALUATION
Christina N. Lacerenza, C. Shawn Burke, David S.
Metcalf, Shannon L. Marlow, Luke Read
University of Central Florida, IST
Orlando, FL
Christine Allen, Mark Mazzeo
ARL-HRED STTC
Orlando, FL
This study aims to evaluate the training effectiveness of the augmented reality enabled version of the
Combat Medic cards and to assist the Army in determining future development and implementation plans
for augmented reality (AR) training in conjunction with the Emergency Medical Care cards or other
similar training products. The University of Central Florida’s Institute for Simulation and Training (UCF
IST) has previously developed emergency medical training cards in an effort to design effective simulation
for Army medics. Additionally, two digital supplementary versions of the cards were developed: an online
Flash version and an iOS mobile eversion supporting flash card study and self-assessment with integration
of study scheduling to assist with scheduling of material for transfer into long-term memory. Moreover, an
augmented reality solution for the Combat Medic card deck, which will launch videos of procedures after
the user scans the card was developed, with an existing third-party augmented reality toolkit that uses
image recognition as a trigger. The evaluation compares learning, speed of learning, usability, perceived
utility, level of engagement, and perceived speed of access to information between the augmented reality
enabled Combat Medic cards and the Combat Medic app. The evaluation should provide a validated
methodology for integrating AR into existing training print and/or digital training materials which can: 1)
serve to expand the toolkit for Army instructional designers and trainers and 2) facilitate and continue to
improve an active learning process already under development which has been well received and has
already demonstrated training utility on a small scale.
77
REAL-TIME CUTTING OF ORGANS WITH SCISSORS
Matthew Hackett, Kevin Fefferman
ARL HRED-STTC
Orlando, FL
Steve McIlwain, Bradley Willson
Applied Research Associates, Inc.
Raleigh, NC
The ability to perform random cuts to organs in real-time is crucial in a virtual reality based surgical
simulator. Nowadays, organs are commonly simulated using the Finite Element Method (FEM) on
tetrahedral meshes. In many cases, cuts can be approximated as singular sections. Thus, the method
presented in this paper was designed to perform subdivisions of tetrahedrons accurately along the path of
the blades following a set of pre-computed patterns. To ensure the resulting meshes remain suitable for
simulation, the subdivisions are performed in such a way that adjacent tetrahedrons will weld and not
overlap. Care is taken to minimize the subdivision of tetrahedrons in the instances where blades traverse
the mesh along the boundaries of elements, in which case application of the basic patterns would have
generated degenerated tetrahedrons. This paper also describes the way in which the surface mesh is
extracted from the tetrahedral mesh after the cut is performed, in order to update the data for collision
management and visualization.
TUESDAY, 1 DECEMBER, 2015 ROOM S320B
S-2 Getting to the Right Scenarios & Data
1600
Virtual Battlespace Scenario
Encoding for Reuse (15027)
1630
Multi-Federate Scenario
Development and Testing: “A
Good Plan, Violently Executed”
(15249)
1700
The Expected Results Method for
Data Verification (15020)
Notes
78
SIMULATION SCENARIO ENCODING FOR REUSE
Captain Michael J. Eady, USMC
Marine Corps Training and Education Command
Quantico, Virginia
Lieutenant Colonel David W. Parkes, USA
Joint Staff J7
Suffolk, Virginia
The United States Army and United States Marine Corps employ the Virtual Battlespace 3 (VBS3)
commercial game for first-person small-unit training and have invested significantly in training scenarios
constructed using proprietary tools and data formats. Open standards data structures need to be utilized in
order to move toward improved interoperability, address the statutory intent for open competition and
affordability, and protect investments made in models, terrain, and other elements of scenarios that are
separate and distinct from the game engine source coding. Expanding capabilities for open scenario
interchange will improve scenario reuse while creating greater opportunities for simulation data
interchange and open competition for future virtual training capabilities. This paper describes and
demonstrates initial application of Extensible Markup Language (XML) technologies to represent and
interchange simulation scenario data. Design of XML data structures to capture a subset of a VBS2
scenario’s data content is successfully demonstrated, and the capability to transform content from the
XML model back to the VBS2 scenario data formats utilizing an Extensible Stylesheet Language
Transformation (XSLT) document is discussed. Proposed extensions to existing and developing simulation
standards are made in order to accommodate the set of data used in VBS2 scenarios. The research provides
a foundation for future efforts to determine the feasibility of creating an open XML schema that addresses
all critical aspects of a simulation scenario, which will enable open competition for the first-person “games
for training” requirement while preserving investments in proprietary data structures.
MULTI-FEDERATE SCENARIO DEVELOPMENT AND TESTING:
“A GOOD PLAN, VIOLENTLY EXECUTED”
Donald C. Meinshausen
ARCIC / JAMSD / Lockheed Martin
Ft Eustis, VA
Mitchell Faircloth
MCoE / MBL / EEB / SAIC
Ft Benning, GA
The Battle Lab Collaborative Simulation Environment (BLCSE) annually hosts a variety of multi-federate,
entity based, advanced concepts and systems simulation experiments. Technical challenges, evolving
scenarios, changing requirements and even good ideas, often create moving targets for scenario developers
preparing for the events. Challenges and opportunities include: structural division, replication and
distribution; automation, referential integrity, nomenclature, command and control, assignment, attrition
and lay down, composition/behavioral impact analysis, configuration management, semantic mapping,
operator/player usability and division of labor.
As collaborating members within the BLCSE Community of Practice, the Maneuver Battle Lab (MBL)
and the Joint and Army Models and Simulations Division (JAMSD) are forging methods and employing a
variety of tools, including the BLCSE ForceBuilder, to accelerate the development, quality and testing of a
variety of formats for scenario and supporting deliverables. This paper describes the rapid construction,
adaptation and testing of futuristic task organizations from the basic Table of Organization and Equipment
(TO&E) to the networked, human-in-loop, real-time, 60,000-entity war game. Lessons learned and
innovations, including a new ten stage ORBAT development process and our initial utilization of tools
such as the Web-based Military Scenario Development Environment (WebMSDE), in preparation for
BLCSE simulation exercises for 2014 and 2015, will be covered in detail.
79
THE EXPECTED RESULTS METHOD FOR DATA VERIFICATION
Paul Monday
Lockheed Martin
Radcliff, KY
The credibility of US Army analytical experiments using distributed simulation depends on the quality of the
simulation, the pedigree of the input data, and the appropriateness of the simulation system to the problem. The
second of these factors is best met by using classified performance data from the Army Materiel Systems
Analysis Activity (AMSAA) for essential battlefield behaviors, like sensors, weapon fire, and damage
assessment.
Until recently, using classified data has been a time-consuming and expensive endeavor: it requires significant
technical expertise to load, and it is difficult to verify that it works correctly. Fortunately, new capabilities, tools,
and processes are available that greatly reduce these costs. This paper will discuss these developments, a new
method to verify that all of the components are configured and operate properly, and the application to recent
Army Capabilities Integration Center (ARCIC) experiments.
Three recent developments have focused improving the process to load the data. OneSAF has redesigned their
input data file formats and structures so that they correspond exactly with the Standard File Format (SFF)
defined by AMSAA, ARCIC developed a library of supporting configurations that correlate directly to the
AMSAA nomenclature, and the Entity Validation Tool was designed to quickly execute the essential models
with a test-jig approach to identify problems with the loaded data.
The missing part of the process is provided by the new Expected Results Method. Instead of the usual subjective
assessment of quality, e.g., “It looks about right to me”, this new approach compares the performance of a
combat model with authoritative expectations to quickly verify that the model, data, and simulation are all
working correctly.
Integrated together, these developments now make it possible to use AMSAA classified performance data with
minimal time and maximum assurance that the experiment's analytical results will be of the highest quality
possible.
WEDNESDAY, 2 DECEMBER, 2015 ROOM S320B
S-3 Convergence of Testing & Training
0830
Polygone LVC: The New Paradigm
for EW Training (15213)
0900
Live Synthetic Training and Test &
Evaluation Infrastructure
Architecture (LS TTE IA) Prototype
(15099)
0930
Measuring Realism in Simulations
for Training and Testing
(15206)
Notes
80
POLYGONE LVC: THE NEW PARADIGM FOR EW TRAINING
Lt Col Scott Case
Ryan McLaughlin
United States Air Force
Northrop Grumman
Warrior Preparation Center Detachment 3 Germany
Orlando, FL
By the end of 2015, the Multinational Aircrew Electronic Warfare Tactics Facility (MAEWTF) known as
Polygone range, based on a tri-national agreement between France, Germany, and the United States will
establish itself as a world-class Live-Virtual-Constructive (LVC) training range. Through working together with
German Air Force Command, French Air Force Command, United States Air Forces in Europe-Air Forces
Africa (USAFE-AFAFRICA), and several LVC industry leaders, Polygone will offer a first-of-kind capability
that promises to usher a new paradigm of Electronic Warfare (EW) training. The initial operational capability
reuses or repurposes existing Polygone range infrastructure while leveraging innovations made by other ranges,
particularly the Joint Pacific Alaska Range Complex (JPARC). By avoiding a "home grown" approach,
Polygone managed to save approximately $300 million in acquisition costs and yield a similar training
capability.
The baseline of the Polygone LVC project, also known as Multinational Aviation LVC Training System
(MALTS), provides a mobile LVC range capable of bringing advanced EW training to major exercises
worldwide. The initial phase included developing innovative virtual surface-to-air missiles (SAMs) leveraging
the expertise of professional German Air Force live SAM operators, improving training capabilities for future
Special Operations Forces (SOF) and Joint Terminal Attack Controller (JTAC) mission readiness training, and
establishing the framework required for a worldwide distributed training audience. The lack of modern advanced
SAM training assets, increasingly prohibitive live training restrictions, and alarming increases in potential
adversary air defense capabilities have led to the Polygone's development of virtual SAMs. These virtual SAM
operator stations enable trained threat operators to utilize modern and advanced SAM techniques at virtual
single and double digit SAM stations providing affects to live aircraft flying training missions at Polygone. This
paper will discuss the MALTS LVC project, development of virtual SAMs for EW training, and how Polygone
leveraged JPARC LVC innovations to advance Coalition training in Europe.
LIVE SYNTHETIC TRAINING AND TEST & EVALUATION INFRASTRUCTURE ARCHITECTURE (LS
TTE IA) PROTOTYPE
Paul Dumanoir, Mike Willoughby
U.S. Army PEO STRI
Orlando, FL
Burt Grippin, Richard Crutchfield, Rob Wittman & Sean Barie
MITRE Corporation
Orlando, FL
This paper describes the Live Synthetic Training and Test & Evaluation Infrastructure Architecture (LS TTE IA)
prototype. The LS TTE IA was funded by PEO STRI and AMSO in FY14 with the intent of providing a
technology insertion into LVC-IA, replacing the existing infrastructure with a cloud-enabled service-oriented
architecture (SOA). This SOA infrastructure is being developed with the expressed goal of supporting both the
Training domain and the Test & Evaluation community. It will be compliant with the Common Operating
Environment (COE) and suitable for hosting within the COE Data Center/Cloud Computing Environment. The
prototype architecture was developed in collaboration with Johns Hopkins University’s LS TTE Enterprise
Architecture (LS TTE EA) research which explored the business case and governance strategy for managing a
SOA environment. The LS TTE EA is described, as well as the relationship of the LS TTE IA to the reference
architecture of the LS TTE EA.
This paper explains the SOA prototype layered architecture, and the initial services developed in FY14. Two
concurrent projects under development in FY14 developed services that operate on the LS TTE IA
infrastructure. This paper will briefly discuss those projects and their successful use of the infrastructure.
Finally, the paper will discuss the planned FY15 infrastructure improvements and the forward looking
implementation strategy.
81
MEASURING REALISM IN SIMULATIONS FOR TRAINING AND TESTING
Jerrit Askvig, Phil Hallenbeck
The MITRE Corporation
McLean, Virginia
Many publications and presentations on training or testing mention the need for “realism,” or the responsibility
of trainers or developers to provide a “realistic training environment” or “a realistic simulation.” Yet, none
appear to define realism or describe how it might be judged such that choices could be made among competing
investments, or the realism of a given simulation environment improved.
We present here a doctrine-based model to rigorously and repeatably assess the realism of a simulation
environment. It provides a framework to reason about, assess, and communicate realism for tasks and systems
spanning all the Army’s Warfighting Functions; and a software application to repeatably yet quickly and easily
assess the specific components of realism as they impact specific tasks or systems.
The model is based on the Army’s Mission Variables of METT-TC (Mission, Enemy, Terrain, Troops, Time,
and Civil considerations), plus the Immersive environment (the visual, aural, and other factors that lead a
participant to believe that he or she is in a “real” environment and should behave accordingly). Framing the
assessment in terms of Army risk management doctrine (where both the likelihood and the severity of hazards
are considered) leads to the straightforward concept of assessing risks to realism in terms of “METT-TC+I.”
The model decomposes the Enemy (E) and Troops (T) variables into fine-grained capabilities suitable for
assessment; and each of the other variables (mission, terrain, and so forth) into factors (such as weapon lethality,
or visual indications of weapons fire) that may impact capability achievement and therefore event outcome.
Each capability’s and factor’s risk to realism is calculated based on its probable impact, and the likelihood it
would be inadequately simulated. Users of the software application may easily modify these calculations or their
results to reflect risk mitigation steps, or the expert judgment of the user.
S-4 From Reality to Simulation
1030
Network Bandwidth’s Effect on
Virtual World Simulator
Performance Optimization (15360)
1100
3D Immersive Environment Using
X-Plane for Depth
Perception Research (15261)
1130
Battle Damage Computation Server
(15051)
Notes
82
NETWORK BANDWIDTH'S EFFECT ON VIRTUAL WORLD SIMULATOR PERFORMANCE
OPTIMIZATION
Sean C. Mondesire
Orlando, FL
Jonathan Stevens
Orlando, FL
Douglas B. Maxwell
Orlando, FL
The United States Department of Defense (DoD) employs virtual and game-based training simulators to train its
servicemembers on both individual and collective cognitive and psychomotor tasks. The current employment of these
training simulators is typically conducted in a stand-alone manner, with distributed simulation remaining the exception
due to interoperability challenges. The OpenSimulator (OpenSim) is a popular open-sourced virtual world simulator
that currently provides a persistent three-dimensional social community for its users. Under the Army Research
Laboratory's (ARL) Military OpenSimulator Enterprise Strategy (MOSES) research program, OpenSim is being
developed to serve as a prototypical distributed military virtual training environment for tactical operations.
Virtual worlds for military training is an emerging domain. As such, detailed analysis of critical architecture
parameters is required in order to optimize the performance of both the simulator's servers as well as the multitude of
client connections. Unfortunately, due to a lack of extensive virtual world performance analysis, OpenSim server
administrators often make arbitrary resource allocations to support their environments and training scenarios. Negative
consequences to this approach are that typically too few resources are allocated to an overwhelmed server, resulting in
an unresponsive environment, while too many resources are allocated to an underutilized server, when those resources
could be more effectively applied elsewhere. In this paper, we analyze network bandwidth's effect on virtual world
simulator performance so as to support the future creation of a predictive model that will determine the optimal
amount of resources required to support a target number of concurrent users in the virtual world. This analysis, and the
future development of our predictive model, will provide the OpenSim developer community with the knowledge
required to best allocate resources to support expected server load.
3D IMMERSIVE ENVIRONMENT USING X-PLANE FOR DEPTH PERCEPTION RESEARCH
Dr. Logan A.Williams, Charles T. Bullock, Dr. Marc D. Winterbottom,
Dr. James P. Gaska, Dr. Steven C. Hadley
711th Human Performance Wing, United States Air Force School of
Aerospace Medicine
Dr. Charles J.
Lloyd
Visual
Performance LLC
Ellisville, MO
Dr. Michael P.
Browne
SA Photonics
Los Gatos, CA
Game-based flight simulation software has been shown to provide a reliable, low-cost, virtual environment
able to facilitate a wide range of training and research objectives. In this work, which is part of the U.S.
Air Force School of Aerospace Medicine Operational Based Vision Assessment program, game-based
simulation software was used to render an immersive three-dimensional constructive environment within a
helmet mounted display (HMD) for weapons platform specific vision research and to quantify the impact
of aircrew vision on selected operational tasks. In this work, an operationally relevant MH-60 call-tolanding task was simulated to provide data relevant to the applicability of U.S. Air Force Flying Class III
depth perception standards. The specific simulation system consisted of a high-resolution (1920x1200) 55°
field-of-view binocular HMD with infrared head tracking, in which two instances of X-Plane were
stereoscopically rendered to the HMD using separate PCs, both incorporating Intel i7 processors and
Quadro K4200 video cards with Quadro Sync. This paper details the overall design, implementation, and
validation of the virtual environment used to simulate the MH-60 call-to-landing task, including
stereoscopic rendering using game-based simulation software, hardware/software stereo rendering
limitations, HMD warping, and head-tracker integration. The minimum perceived stereo threshold
capabilities of this system are also quantified, including discussion of its applicability to simulated tasks
requiring precise depth discrimination. This work will provide an example simulation framework for future
stereoscopic virtual immersive environments applicable to both research and training.
83
BATTLE DAMAGE COMPUTATION SERVER
Hung Tran
Tactical Systems, CAE USA
Tampa, FL
This paper will present a new approach for handling the battle lethality computation in the context of a Distributed
Interactive Simulation (DIS) network training scenario where participants in an exercise are required to broadcast their
current damage status. Usually, each simulation handles the computation and the assessment of damage differently,
resulting in an “unfair fight” between the participants. Rather than having each simulation performs this computation,
an alternate approach would be to delegate the lethality computation to a common processing task implemented on a
server. The design of the battle damage server will be described and discussed in this paper.
The advantage of the battle damage computation approach described in this paper is twofold: participants within the
network would use a common mathematical model, and simulations being freed from the burden to compute the battle
lethality. This approach will help to eliminate the interoperability variances in lethality results and achieve a “fair
fight” weapon effect.
S-5 Innovation in Environmental Modeling
1400
High Fidelity Wind Model Software
for Real-Time
Simulation Platforms (15362)
1430
Automated Runtime Terrain
Database Correlation
Assessment (15218)
1500
Automated Modelization in Terrain
Database Production
(15290)
Notes
84
HIGH FIDELITY WIND MODEL SOFTWARE FOR REAL-TIME SIMULATION PLATFORMS
Jaime Sanchez, Dr. Juan Pelaez
SimSpace Ingenieria S.L.
Madrid, Spain
Atmospheric conditions can be a threat to flight safety. Specifically, wind conditions can be particularly critical
in ground-proximity, low altitude operations or flying near aerodynamic wakes. Since flight simulation is
intended to train pilots in the most difficult flight situations under the most extreme conditions it is necessary to
include a detail wind model as part of the flight simulator, allowing pilots to train safely in complex operation
procedures in a cost-effective manner.
This paper presents a methodology for modeling detail physics based winds in real-time. The method is
designed to be flexible and can simulate wind conditions in different types of environments such as large
mountainous landscapes, urban areas, offshore oil rigs or aerodynamic wakes of moving ships and airplanes.
The methodology presented consists of three steps. First, given a particular environment and after its geometry
is defined, the winds are computed using a mesh-free Computational Fluid Dynamics (CFD) solver, very
adequate for complex geometries and quick turnaround times, which models turbulence using Large Eddie
Simulation (LES), very appropriate for highly separated flows. In each environment multiple CFD computations
are required for different wind conditions (i.e. wind directions and speeds) in order to characterize the real-time
operating conditions. Second, the resulting CFD solutions are compressed and optimized. Third, the post process
optimized databases are used by a software module which generates steady and turbulent wind vectors in realtime.
The method’s flexibility to simulate wind conditions in different environments will be presented through
different case studies: aerodynamic winds due to a simplified building structure, low level wind shear and
turbulent effects in Gibraltar airport, aerodynamic wakes of moving a transport airplane and a ship frigate.
Finally, the effect of the detail wind model in a helicopter pilot’s workload operating near a ship will be
presented.
AUTOMATED RUNTIME TERRAIN DATABASE CORRELATION ASSESSMENT
Jeremy P. Joseph, MS, CMSP, Andrew Tosh, MBA
GameSim Inc
Orlando, FL
Benito Graniela, PhD
Naval Air Warfare Center Training Systems Division
(NAWCTSD), Orlando, FL
As the US Navy and US Marine Corps move toward integrating existing flight simulators into common
training environments, the importance of having a correlated, correct environmental representation is vital
for achieving a fair fight and a high training value to the warfighter. Many of these simulators are
operating off different versions of source data and using different image generator (IG) vendors, which can
result in interoperability problems. Although correlation between visual terrain databases and simulation
terrain databases have been investigated in the past, there is a lack of research on correlation between large
synthetic environments using runtime visual and sensor databases in Navy and Marine Corps flight
simulators. Many current practices involve manual inspection and limited area of interest (AOI) testing to
determine correlation, resulting in ineffective correlation assessments, which may cause negative training.
In an effort to address this gap, preliminary research has been conducted to develop a tool that can perform
automated correlation and integrity assessments on runtime formats, including visual and sensor databases,
using standard interfaces such as the Common IG Interface (CIGI) within a distributed simulation
environment. Utilizing these standard interfaces along with standard data formats, such as the U.S. Navy
Naval Air Systems Command (NAVAIR) Portable Source Initiative (NPSI), the research framework
facilitates tests to identify integrity and correlation conditions that may negatively affect training. The
details of the investigation, its outcomes, and future research are reported.
85
AUTOMATED MODELIZATION IN TERRAIN DATABASE PRODUCTION
Stephen Eckman
GameSim
Orlando, FL
Ronald Moore
Leidos
Orlando, FL
Mark Johnson
U.S Army PEOSTRI
Orlando, FL
Jaeson Munro
PAI
Orlando, FL
The feature content requirements for terrain databases used in the Modeling and Simulation industry
continue to grow. The requested database geographic extents are expanding. Database feature densities and
complexities are increasing. New simulation systems are leveraging game engines, enabling this growth in
database content requirements. Unfortunately, the cost of producing 3D cultural models, that reflect these
increased complexities, using traditional commercial 3D modeling tools, by hand, is quickly becoming
unsustainable, and maintaining and updating 3D cultural models libraries are becoming unaffordable. The
process of assigning 3D model references to vector features in a geospatial database, identified as
modelization, has been automated by the SE Core program to significantly reduce database production
costs, improve database quality and consistency and increase run-time database performance. This paper
describes the processes and tools used in the Automated Modelization Process in the production of the
Synthetic Environment Core (SE Core) terrain databases. When applied, these tools and processes results
in a cost effective approach to automatically generate 3D cultural models that are designed for the
specified run-time requirements, saving considerable development time when compared to using
commercial 3D modeling tools and manual processes.
S-6 Human Interface to Artificial Intelligence
1400
Modeling CGF Behavior with
Machine Learning Techniques:
Requirements and Future Directions
(15128)
1430
Automatic Speech Recognition in
Training Systems:
Misconceptions, Challenges and
Paths Forward (15205)
1500
On the Peripheral Application of
Head Mounted Display
(HMD) Devices in Infantry
Simulation (15186)
Notes
86
MODELING CGF BEHAVIOR WITH MACHINE LEARNING TECHNIQUES: REQUIREMENTS AND
FUTURE DIRECTIONS
Armon Toubman, Gerald Poppinga,
Jan Joris Roessingh
National Aerospace Laboratory NLR
Amsterdam, Netherlands
Rikke Amilde Løvlid
Norwegian Defence Research
Establishment (FFI)
Kjeller, Norway
Ming Hou
Defense Research and
Development Canada
Toronto, Canada
Christophe Meyer
Thales
Palaiseau, France
Linus Luotsinen
Swedish Defence Research Agency
FOI
Stockholm, Sweden
Roel Rijken
Dutch Ministry of
Defence
Utrecht, Netherlands
Michal Turčaník
Armed Forces Academy
Liptovský Mikuláš,
Slovakia
Commercial/Military-Off-The-Shelf (COTS/MOTS) Computer Generated Forces (CGF) packages are
widely used in modelling and simulation for training purposes. Conventional CGF packages often include
artificial intelligence (AI) interfaces, with which the end user defines CGF behaviors. We believe Machine
Learning (ML) techniques can be beneficial to the behavior modelling process, yet such techniques seem
to be underused and perhaps underappreciated. This paper aims at bridging the gap between users in
academia and the military/industry at a high level when it comes to ML and AI. Also, specific user
requirements and how they can be addressed by ML techniques are highlighted with the focus on the
added ML value to CGF packages. The paper is based on the work of the NATO Research Task Group
IST-121 RTG-060 ‘Machine Learning Techniques for Autonomous Computer Generated Entities’.
AUTOMATIC SPEECH RECOGNITION IN TRAINING SYSTEMS: MISCONCEPTIONS, CHALLENGES
AND PATHS FORWARD
Brian Stensrud, Charles Newton
Orlando, FL
Beth Atkinson, John Killilea
NAWCTSD
Orlando, FL
Over the past decade, we have seen moderate demand for simulation-based training systems to include
automatic speech recognition (ASR). Like commercially available services such as Apple's Siri and
Google Now, ASR gives training systems the capability to interpret human speech and react to that speech
with appropriate actions (e.g. executing a spoken command) and responses (e.g. replying to a human with
confirmation or requests for clarification). Introduction of this capability is designed to address instructormanning limitations and improve the fidelity of the training experience. However, ASR successes within
simulation-based training systems have been modest, historically. We contend that this lack of widespread
usage and success stems primarily from a fundamental misunderstanding of (and thus lack of investment
in) the components necessary to achieve more effective ASR. In this paper, we describe the essential
functions of ASR: (1) Recognition is when the audio of the spoken utterance is translated into text. (2)
Understanding attempts to glean meaning from the text – whether they denote, for example, a new
directive, a response to a previous query, or a request for new information. (3) Behavior refers to the
functions the system is responsible for after receiving a recognized speech utterance. (4) Some training
systems also employ dialogue when continuous interaction with humans is required. Finally, we outline
current ASR research and development, discuss typical implementations, and introduce potential strategies
to improve specific ASR functions and the capability as a whole to provide better support for future
training systems.
87
ON THE PERIPHERAL APPLICATION OF HMD DEVICES IN INFANTRY SIMULATION
Tomer J. Michael
IDF GFC Battle-Lab
Tel-Aviv, Israel
Yaniv Minkov
IDF GFC COR
Tel-Aviv, Israel
The purpose of this paper is to present the results of an attempt carried out at the IDF Ground Forces
Command Battle-Lab to integrate a Head Mounted Display (HMD) device as part of a peripheralequipment simulator for infantry. The Battle-Lab is a research oriented simulation environment, where
combat scenarios with many multiple human participants can be run to examine the effects novel concepts
or technologies could have on scenario outcomes.
Previously (Michael et. Al, 2014) an attempt was made at the Lab to evaluate an HMD’s effectiveness as
an exclusive display for infantry simulation. At that time, while the device tested was found to have had a
positive impact on a participant’s motivation and spatial awareness, it was found lacking in the field of
visual fidelity, as well as responsible for an increased incidence of simulation sickness among its wearers.
As a result of the previous evaluation it was decided to proceed with the integration of the device, but only
in supplementary peripheral simulators. These included a pair of binoculars made available to an infantry
soldier for use concurrently with a standard flat-screen first-person infantry simulation.
However, given the device’s reputation for causing simulation sickness, and our previous experience with
the phenomena, it was decided to monitor the participants’ experience closely. This task was accomplished
through simple after-action self-review supplemented by a more detailed daily debriefing with the
Simulation Sickness Questionnaire.
In this paper are presented the results of this monitoring throughout a series of scenarios carried out at the
Battle-Lab in 2014, conclusions from the gathered data, as well as lessons learned from the process of both
building and studying simulation sickness in the use of peripheral simulators with HMD integration.
S-7 Next Gen LVC
1600
M&S Training Transformation:
Bridging the Next Generation
Joint LVC (15167)
1630
Leveraging Cloud Computing
Technology for LVC Training
(15101)
1700
Osseus, An Experiment in What’s
Next in LVC M&S
Architecture (15085)
Notes
88
M&S TRAINING TRANSFORMATION: BRIDGING THE NEXT GENERATION JOINT LVC
Mr. Bruce
Uphoff
Camber
Corporation
Leavenworth, KS
Mr. Michael
Koscielniak
Los Alamos National
Laboratory
Los Alamos, NM
Mr. Brian Gregg
Mr. Karl Hines
Joint Staff J7
Suffolk, VA
Mr. John Mizelle
Intelligent Decision
Systems, Inc.
Suffolk, VA
Mr. Daniel Leigeber
Intelligent Decision
Systems, Inc.
Huntsville, AL
To enable training of U.S. Forces, Korea (USFK) to “Fight Tonight” (deter/defeat aggression), their Korea
Battle Simulation Center (KBSC) has evolved to the Joint Training Transformation Initiative (JTTI) and Korean
Simulation System (KSIMS) (JTTI+K) federation of models. This federation is a “one off” solution within DoD,
especially with regard to the Joint Staff J7 simulation federation (Joint LVC). Why is this important? Aside from
the obvious that in a day of diminishing resources it is difficult to justify and maintain “one-off” approaches,
there is the fact that as USFK transforms to US Korea Command (KORCOM) it will fall under the J7 for
training capability support.
The JLVC and JTTI+K federations are divergent. To support the command structure change and resulting
training capability gap, the SECDEF directed an enterprise DoD M&S architecture. It was determined that the
JS J7 would provide a software bridge between the JLVC and JTTI+K federations to achieve the SECDEF
directive. Los Alamos National Laboratory is working development of the KORCOM Bridge, linking the JLVC
and JTTI+K, as part of the next generation JLVC 2020 v0.6 development cycle and faces several challenges.
The JTTI+K federation is aggregate based, time managed, connected, reliable delivery and uses the HLA 1.3
NG Pro RTI. JLVC is essentially entity-based, non-time managed, non-connected, best effort delivery, and not
previously integrated with KSIMS. The KORCOM Bridge has addressed these challenges and is delivering an
initial (production level) capability in FY15. The J7 has conducted a series of KORCOM Bridge integration tests
with the JTTI+K at the Joint Digital Integration Facility (JDIF) in Orlando, FL, and with the JTTI+K at the
KBSC. This paper captures results of design and integration testing through the initial FY15 delivery of JLVC
2020 v0.6 and how the KORCOM Bridge helps forge future Joint/Coalition M&S capabilities.
LEVERAGING CLOUD COMPUTING TECHNOLOGY FOR LVC TRAINING
Paul Dumanoir
Army, Program Executive
Office for Simulation,
Training and Instrumentation,
Project Manager, Integrated
Training Environment
Orlando FL
Henry Marshall
Human Research and
Engineering Directorate
Technology Center
Orlando, FL
Robert Wells
Dynamic
Animations
Systems
Orlando, FL
Jeff Truong
Effective
Applications
Corporation
Orlando, FL
Over the past years the Department of Defense and the Army have been working to accelerate toward wide-scale
adoption of cloud computing for the potential cost saving and enhanced mission capabilities that it brings. The
Live, Virtual, Constructive – Integrating Architecture (LVC-IA) is a Program of Record (POR), under the Army
Program Executive Office for Simulation, Training, and Instrumentation (PEO STRI) Project Manager for
Integrated Training Environment (PM ITE), which provides a net-centric linkage for existing Training Aids,
Devices, Simulations, and Simulators in an ITE. To date the LVC-IA architecture utilizes a series of servers
which are physically located in one or more fielding sites. Each instance of an LVC-IA system has to be
individually installed and maintained at each site. Building and running these on site systems is complex and
expensive. With each instantiation of LVC-IA for the new training sites, the capital and operating expenditures
would simply multiply. A research effort was sponsored by PM ITE to evaluate the feasibility of leveraging
cloud computing for LVC-IA. This paper summarizes the analysis conducted, architecture design, and prototype
implemented from this research effort. The paper dives into the Information Assurance issues encountered and
touches on processes from other Army programs as they relate to the Common Operating Environment (COE)
Data Center/Cloud (DC/C) Computing Environment (CE). The paper also reports the comparative analysis
results between an ITE with co-located LVC-IA versus an ITE with LVC-IA in the cloud. Finally the paper
reports the challenges uncovered, lessons learned, and recommended way forward.
89
OSSEUS, AN EXPERIMENT IN NEXT GENERATION LVC M&S ARCHITECTURE
Mark Riecken
Trideum
Corporation
Orlando,
Florida
Derrick
Franceschini
StackFrame,
LLC
Sanford, Florida
Scott Gallant
Effective
Applications
Orlando,
Florida
John Rutledge
Trideum
Corporation
Huntsville,
Alabama
Walter Barge
Director, Joint Assessment and Enabling
Capability (JAEC)
OSD Force Readiness and Training (FRT)
Alexandria, Virginia
Live, Virtual, and Constructive (LVC) technologies provide a powerful range of capabilities that the distributed
training community enjoys in support of its broad spectrum of training needs. The fundamentals of this LVC capability
were built over two decades ago at a time when the modeling and simulation (M&S) community was arguably ahead
of commercial information systems, especially in terms of distributed computing and networking. With the advent of
many web-based technologies the LVC community now finds itself attempting to integrate new technologies with
legacy architectures. Due to the flexibility of the new technologies as well as the inventiveness of the LVC
community, this approach has had some success, but these approaches continue to require specialized skillsets and can
be costly to establish and maintain. The Defense M&S Coordination Office (DMSCO) has sponsored an effort to
describe and prototype selected features of a next generation architecture that leverages recent and emerging
technology more directly. This paper describes a framework called Osseus to accomplish these goals. Osseus
incorporates desired next generation characteristics such as 1) more open and flexible interoperability between
disparate systems; 2) the ability for relatively untrained users to fill in functionality gaps between available systems by
dynamically injecting behavioral changes into the LVC environment; 3) the ability to connect services with granularity
smaller than an application to increase the capability of the environment; 4) a more accessible means of composing
distributed training capabilities for an educated, but non-specialist trainer; 5) data filtering to optimize or reduce data
transmission over the network; and 6) centralized data management to facilitate tools such as visualization, data
collection, and analysis. This paper discusses architectural aspects of Osseus and selected prototype results which
include the integration of OneSAF with an example virtual system.
THURSDAY, 3 DECEMBER, 2015 ROOM S320B
S-8 Simulation Supported Training
0830
Embarking on a Home Station
Training Revolution (15176)
0900
Implementation of Role-Based
Command Hierarchy Model
for Actor Cooperation (15166)
0930
Innovative Division/Brigade Level
CO Training Solution for
Influence Operations (15107)
Notes
90
EMBARKING ON A HOME STATION TRAINING REVOLUTION
Anthony J. Cerri
TRADOC, G27 OE TSC
Newport News, Virginia
Alan J. Knox & Mathew N. McMillan
CGI Federal
Newport News, Virginia
As the US Army has been at war for over a decade, the art of developing and conducting Home Station Training
(HST) has not been a focal point for our next generation of leaders. Units have become accustomed to having their
training provided to them by others prior to overseas deployment. This diminished capability has made it difficult to
develop and execute effective HST when that training includes live, virtual and constructive (LVC) environmental
considerations. To fully understand the scope of the problem, this paper articulates the processes in which HST is
currently designed and developed, and how that training could be supported by Mission Training Centers (MTCs). The
process, while not the same for all units, demonstrates the genesis of how training is developed at home station. To
help overcome the challenges, the Training Brain Repository-Exercise Design Tool (TBR-EDT), a collaborative, webbased repository and exercise design tool has been created. It enables commanders and staffs to reuse storylines,
events, and other exercise related items from within the repository. It allows units to work closely with MTCs to
automate and modernize HST lifecycles. The TBR-EDT revolutionizes the current mainstream exercise design
process; reducing the time required to develop an exercise and its associated training support package (TSP), due to
the enhanced collaborative opportunities between the training unit and the supporting MTC. The TBR-EDT currently
supports simulations with start of the exercise (STARTEX) data, provided in multiple machine readable order of battle
service, extensible markup language (OBS XML) versions for import into simulation systems. In the future, more
robust STARTEX conditions such as terrain, C2, and parametric data will further enhance the simulations
environment, and better replicate the operational environment. Finally, the paper will present the TBR-EDT as a game
changer for Army and Joint trainers; significantly shortening the design process for realistic and economical HST
exercises.
IMPLEMENTATION OF ROLE-BASED COMMAND HIERARCHY MODEL FOR ACTOR
COOPERATION
Jungyoon Kim, Hee-Soo Kim, Jihyun Jang
REALTIMEVISUAL
Seoul, Republic of Korea
Sangjin Lee, Samjoon Park
Agency for Defense Development
Daejeon, Republic of Korea
Many approaches to agent collaboration have been introduced in military war-games, and those approaches address
methods for actor- (agent-) collaboration within a team to achieve given goals, where the team’s abstract mission is
translated into concrete tasks for each actor. To meet fast-changing battlefield situations, an actor must be 1) loosely
coupled with their tasks and be 2) able to take over the role of other actors if necessary to reflect role handovers
occurring in real combat. Achieving these requirements allows the transfer of tasks assigned one actor to another actor
in circumstances when that actor cannot execute its assigned role, such as when destroyed in action. Tight coupling
between an actor and its tasks can prevent role handover in fast-changing situations. Unfortunately, existing
approaches and war-game software strictly assign tasks to actors during design, therefore they prevent the loose
coupling needed for successful role handover. To overcome these shortcomings, we have defined Role-based
Command Hierarchy (ROCH) model that dynamically assigns roles to actors based on their situation at runtime. In the
model we devise “Role” to separate actors from their tasks. Described in this paper, we implement the ROCH model
as a component that uses a publish-subscribe pattern to handle the link between an actor and the roles of its
subordinates (other actors in the team). Therefore, an actor can indirectly send a message (order or report) to another
actor without knowing which actor is recipient. The sender actor is only required to know the relevant roles. The
model has been implemented and tested in a military project, and we briefly show the outcomes in this paper.
91
INNOVATIVE DIVISION/BRIGADE LEVEL COMMANDING OFFICER TRAINING SOLUTION FOR
INFLUENCE OPERATIONS
Ariane Bitoun & Romain Bosa
MASA Group
Paris, France
Tahar Hannachi
Université Pierre et Marie Curie
(Paris VI)
Paris, France
Lionel Khimeche
DGA/DS/CATOD
Arcueil, France
Asymmetric conflicts, involving populations that support armed groups, represent an increasingly significant
proportion of all armed conflicts. Influence Operations, which deal directly with this type of warfare, is therefore a
sector for which Commanding Officers (COs) at division- or brigade-level must now train. Influence Operations
training tools, mostly based on virtual simulations, are today mainly focused on individual skills improvement and
increased cultural awareness. Therefore they are not suited to train COs who need high-level information for the
planning and successful running of operations. The main challenges regarding these objectives involve i) modeling,
simulation and visualization of the abstract concepts of Influence Operations, ii) the definition of operational training
scenarios.
This paper introduces a solution that departs from classic Influence Operations training solutions, by rising to the
challenge of training division- or brigade-level COs and offering a high-level simulation. This solution consists of an
innovative platform that integrates an existing constructive simulation, already used to train division- or brigade-level
COs, with a new simulation component capable of representing Influence Operations. Indeed simulation-based
training platforms, built on constructive simulations, are an efficient and proven means to train at the level we are
targeting in traditional military combat. The resulting platform addresses the operational expectations for the training
of Influence Operations COs with vital components (preparation, gaming, supervision and analysis) and includes highlevel models to simulate the abstract concepts required to represent Psychological Operations (PSYOP), Key Leader
Engagement (KLE) and Civil-Military Cooperation (CIMIC) operations (life-cycle of messages, dynamics of
acceptance amongst populations, Target Audiences).
Finally, the paper presents an implementation of this solution using an existing aggregated constructive simulation
integrated with a dedicated Influence Operations simulation component. This platform is validated using operational
training scenarios that highlight Influence Operation challenges.
THURSDAY, 3 DECEMBER, 2015 ROOM S320F
S-9 UAS Engineering Design Simulation
0830
Reducing Operational Risk through
Better Performance
Testing (15138)
0900
Design of an Educational Tool for
Unmanned Air Vehicle
Design and Analysis (15086)
0930
Using Simulation to Test MannedUnmanned Teaming
(15112)
Notes
92
REDUCING OPERATIONAL RISK THROUGH BETTER PERFORMANCE TESTING
Tom Wilson
Lockheed Martin
Orlando, Florida
One of the more difficult simulations to implement is an accurate load test. In order for a load test to
mimic reality, it must reflect the system's activities in function, data, and timing; properly representing the
data component is often the most challenging. Nonetheless, load testing must be more than a realistic
simulation. It must provide robustness so it can give accurate insight into the system's behavior when other
workloads are encountered.
This paper describes several methods to improve the robustness of load testing so that it not only gives
insight into performance of a system in current operations, but also gives insight into the performance
behaviors of the system for workloads yet to be encountered. Key features of the testing include:
operationally-representative scenarios, easy-to-change workloads, scalable workloads, scalable data,
random parameter generation, and repeatable tests. Since these features are provided independently of the
tool used to implement the load test, the methods are easily applied to the load testing of any system. Once
applied, a basic requirements verification exercise is transformed into a sophisticated operational riskreduction strategy.
DESIGN OF AN EDUCATIONAL TOOL FOR UNMANNED AIR VEHICLE DESIGN AND ANALYSIS
Brian Sanders, Brent Terwilliger, Ken Witcher & Mark Leary
Embry-Ride Aeronautical University
Daytona Beach, Florida
James Ohlman, Christina Tucker
Pinnacle Solutions
Huntsville, Alabama
Offering laboratories and team projects present significant challenges for delivering Science, Technology,
Engineering, and Mathematics (STEM) courses in the online (asynchronous) modality. These interactive
workspaces are important attributes since they provide forums for students to more deeply explore
fundamental principles, exercise teamwork and planning to jointly overcome problems, and gain critical
experience. The employment of online environments and interactive activities hold the potential to change
how fundamental student outcomes measured by accreditation organizations are incorporated and treated
in curricula, potentially improving the quality of the overall educational experience. To address this need
Embry-Riddle Aeronautical University has teamed with Pinnacle Solutions to develop a realistic
unmanned aircraft system (UAS) development, application, and evaluation simulation that educators can
integrate into program curriculum. The research contained in this paper addresses simulation development
and application starting with identification of basic educational objectives driving the need and how the
simulation tool is envisioned to satisfy learning objectives. This will be followed by a description and
examples of a multi-environment simulation framework designed to meet those needs. The first is a
component test environment where students can investigate basic technical principles of operation and key
performance metrics of standalone UAS components such as sensors, communications, and propulsion
elements. The second is an integration facility, where students are provided the capability to apply
knowledge gained in the previous laboratory to select and combine appropriate elements into a unified
subsystem to meet prescribed mission parameters. The third is a flight test environment, where students
experiment with development and execution of simulated flight profiles over common terrain
environments (i.e., mountainous) to measure operational performance attributes of the completed UAS.
The design is anticipated to provide the flexibility to implement each environment sequentially, as
described above, or independently; ensuring a solution applicable to a broad range of courses, objectives,
outcomes, and student capabilities.
93
USING SIMULATION TO TEST MANNED-UNMANNED TEAMING
Michael J. O’Connor
Trideum Corporation
Huntsville, AL
Kenneth LeSueur, Mark
Ebert, Sean Millich
Redstone Test Center
Huntsville, AL
Fred Ventrone
CTSi
Lexington Park, MD
Tom Punihaole
Scalable Network
Technologies
Huntsville, AL
Manned-Unmanned Teaming (MUM-T) allows helicopter pilots to link with Unmanned Aircraft Systems (UAS) and
receive video feeds, control payloads, and direct UAS movements. Robust communications protocols are used for this
process, however, communications loss does occur due to signal strength, terrain, weather, and jamming. Using live
testing to create all of these conditions is difficult and has potential safety and cost issues. To address the need for
repeatable testing of these issues, a simulation environment was created to replicate the conditions using live aircraft
on the ground.
The Joint Unmanned Aircraft Systems Mission Environment (JUAS-ME) is a Central Test and Evaluation Investment
Program (CTEIP) for creating test environments for UAS testing. The Army’s portion of the program is focused on
MUM-T testing.
Two key technologies were developed to support this testing, Hardware-In-the-Loop (HWIL) tactical network
simulation and MIL-STD-1553 bus extension. A real-time network simulation that emulates the Standard Common
Data Link (SCDL) was built to perturb the data links between the helicopter and UAS. The communications effects
server can operate in a physics based mode, fault inject mode, and a hybrid mode. A key test requirement was to
exercise the use of the UAS’s onboard lasers that receive commands over a MIL-STD-1553 bus. These non-eye safe
lasers could not be fired in the UAS HWIL facility, resulting in the need to move the UAS sensor to another test lab.
This extension necessitated the development of a MIL-STD-1553 bus extender to link the test facilities that are 4 miles
apart.
This paper describes the issues encountered in creating a tactical network simulation with HWIL interfaces and the
MIL-STD-1553 bus extender. In particular, several approaches were tried with the network simulation before a
workable solution was found. The techniques describe in this paper can be applied to other programs with similar
requirements.
S-10 Preparation Through Virtualization
1030
Design and Development of a
General Virtual Maintenance
Training Platform (15312)
1100
Virtual Environment ComputerBased Training for Bridge
and Tunnel Inspections (15276)
1130
Developing Authoring Tools for
Skills Models that Enable
Adaptive Game-Based Maintenance
Training (15129)
NOTES
94
DESIGN AND DEVELOPMENT OF A GENERAL VIRTUAL MAINTENANCE TRAINING PLATFORM
Xingxin Li, Jianping Hao, Fei Ye
Shijiazhuang Mechanical Engineering College
Shijiazhuang, Hebei, P.R.China
Xu Yang
The University of Tennessee
Knoxville, TN
Since the 1990s, Chinese researchers have initiated a series of virtual maintenance training technology
studies and system development projects sponsored either by government or industry. Most of them
employed similar methodology and developed a training system for the selected equipment and system. In
fact, rare of theses software achievements were used and improved continuously. In 2006, a proposal for a
general desktop virtual maintenance training platform suitable for diverse user entities and varied training
requirements was approved and sponsored by government and enterprise. The resultant software system
was put into use in 2009 and with continuous improvements, so far as 2012, the research work was
generally recognized and concluded that initial expectations have been met. Further research and
development work was sponsored again.
The paper describes this general virtual maintenance-training platform (GVMTP) from several aspects.
The introduction part addresses the background and a brief description of the research. The design part
puts emphasis on the problem identification, potential requirements, goal definition, and relatively detailed
platform framework design. The development part mainly describes the platform components, and the
realization of three key elements. The application part provides a general applications summary and the
effects analysis result. The paper finally presents lessons learned and the further work. GVMTP provides a
normalized and systematic process to facilitate and manage all activities throughout the lifecycle of virtual
maintenance training product, and also makes the data share and reuse easier during this process. GVMTP
is not only a set of training tools, but also a training product generation studio.
VIRTUAL ENVIRONMENT COMPUTER-BASED TRAINING FOR BRIDGE AND TUNNEL
INSPECTIONS
Steve Ianni
Engility Corporation
Virginia Beach, Virginia
Mary P. Rosick, P.E.
Michael Baker International
Pittsburgh, Pennsylvania
While physical field trips to bridge and tunnel sites are a critical learning component to the Federal
Highway Administration (FHWA) National Highway Institute’s (NHI) training courses, they are difficult
to arrange due to many factors such as safety, time constraints, weather, and costly logistics. In response
for an alternative to a physical inspection field trip, bridge engineers at Michael Baker Jr., Inc., a Michael
Baker International company, and technical developers at Engility Corporation worked together with
FHWA/NHI to create three-dimensional (3D) Virtual Inspection Computer Based Training (CBT).
This paper explains several key design, development and implementation elements of the 3D Virtual
Inspection. It details the process used to create a comprehensive 3D CBT structured for a blended learning
environment. The paper examines the elements used to create approximately 40 interactive checkpoints
across three virtual environments that respond to the inspector’s standard toolset. It explains how the CBT
incorporates reference documentation and instructor “teachable moments”. This paper also highlights how
experts from multiple disciplines (engineering, classroom instruction, 3D modeling, computer
programming, etc.) came together to create a unique and successful training tool for the modern classroom.
Since its pilot in 2012, NHI successfully delivered the award winning CBT 55 times to approximately
1400 participants. After using the 3D Virtual Inspection CBT as an alternative to the field trip, the virtual
environment is now the primary training tool for the safety inspection of in-service bridges and tunnels
courses.
95
DEVELOPING AUTHORING TOOLS FOR SKILL MODELS THAT ENABLE ADAPTIVE GAME-BASED
MAINTENANCE TRAINING
Sean Guarino, Peter Weyhrauch & James Niehaus
Charles River Analytics Inc.
Cambridge, MA
While game-based maintenance training provides a powerful, personalized approach to address individual
training needs, it can be costly to update immersive game engines to address new training objectives.
Challenges lie not only in the incorporation of new technology that must be trained, but also in the
construction of surrounding training materials—curriculums, performance metrics, and optimal training
methods—to address procedures for the new technology. In ongoing work with the Air Force Research
Laboratories (AFRL), the authors are developing a modeling framework and editing tools for subject
matter experts to translate new technology and Technical Orders (TOs) into training objectives, scenarios,
and content for existing virtual game-based trainers. The Methodology for Annotated Skill Trees (MAST)
provides a formalism that organizes training goals and associated performance metrics, skill decay models,
scaffolding models, and effective training methods. This paper discusses the application of this modeling
framework to maintenance training for the F-15E aircraft, and the associated development of editing tools
to adapt content both in MAST and in the immersive game engine. This paper also describes an approach
to improving training by adapting training objectives to support focused repetition of maintenance
procedures and review with instructors. Finally, this paper summarizes initial feedback from active duty
instructors, and next steps for improving these tools.
THURSDAY, 3 DECEMBER, 2015 ROOM S320F
S-11 Critical Flight Decisions
1030
Toward Acquiring a Human
Behavior Model of Competition
vs Cooperation (15316)
1100
Embedded Simulation to Prevent
Tactical Surprise and Improve
Soldier Performance (15054)
1130
Distributed Soldier Representation:
Improving M&S Representation of
the Soldier (15123)
NOTES
96
TOWARD ACQUIRING A HUMAN BEHAVIOR MODEL OF COMPETITION VS. COOPERATION
David V. Pynadath, Ning Wang, and Chirag Merchant
Institute for Creative Technologies
University of Southern California
Los Angeles, CA
One of the challenges in modeling human behavior is accurately capturing the conditions under which
people will behave selfishly or selflessly. Researchers have been unable to craft purely cooperative (or
competitive) scenarios without significant numbers of subjects displaying unintended selfish (or selfless)
behavior (e.g., Rapoport & Chammah, 1965). In this work, rather than try to further isolate competitive vs.
cooperative behavior, we instead construct an experimental setting that deliberately includes both, in a way
that fits within an operational simulation model. Using PsychSim, a multiagent social simulation
framework with both Theory of Mind and decision theory, we have implemented an online resource
allocation game called “Team of Rivals”, where four players seek to defeat a common enemy. The players
have individual pools of resources which they can allocate toward that common goal. In addition to their
progress toward this common goal, the players also receive individual feedback, in terms of the number of
resources they own and have won from the enemy. By giving the players both an explicit cooperative goal
and implicit feedback on potential competitive goals, we give them room to behave anywhere on the
spectrum between these two extremes. Furthermore, by moving away from the more common two-player
laboratory settings (e.g., Prisoner’s Dilemma), we can observe differential behavior across the richer space
of possible interpersonal relationships. We discuss the design of the game that allows us to observe and
analyze these relationships from human behavior data acquired through this game. We then describe
decision-theoretic agents that can simulate hypothesized variations on human behavior. Finally, we present
results of a preliminary playtest of the testbed and discuss the gathered data.
EMBEDDED SIMULATION TO PREVENT TACTICAL SURPRISE AND IMPROVE SOLDIER
PERFORMANCE
Dr. Jonathan Stevens
Florida (UCF)
Orlando, FL
Ms. Latika Eifert
Army Research
Laboratory (ARL)
Orlando, FL
Timothy Baldwin
EOIR Technologies
Aberdeen Proving
Grounds, MD
Oleg Umanskiy
Dr. Boris Stilman
STILMAN Advanced Strategies
University of Colorado Denver
Denver, CO
The U.S. Army Science and Technology (S&T) Advisory Group created the Technology Enabled Capability
Demonstration (TECD) concept in order to demonstrate and measure progress towards meeting the Army's top
ten science and technology challenges. One of the designated challenges is the prevention of tactical surprise at
the small unit level. Operating under the premise that soldiers at the squad level lack sufficient situational
awareness to prevent tactical surprise, the TECD 3 effort was created to increase small unit situational
awareness through the fusion of various planning and intelligence systems into a small unit framework. The
Linguistic Geometry Real-time Adversarial Intelligence & Decision-making (LG-RAID) simulation is a
lightweight course of action (COA) planning tool that employs innovative algorithms to predict enemy activity
in a highly reliable and efficient manner. As such, LG-RAID was selected as a participating application in the
TECD 3 federation and was embedded on both individual soldiers and tactical vehicles in a lightweight mission
command system. In this paper, we discuss how the LG-RAID simulation improved soldier effectiveness,
situational awareness and facilitated the prevention of tactical surprise during the execution of four tactical
situational training exercises (STXs) held at Fort Dix, NJ and executed by the Army's Experiment Force
(EXFOR). Furthermore, we discuss the integration of LG-RAID into the TECD 3 framework and technical
challenges that were overcome. Results of this integration and exercise, presented in this paper, highlight the
potential value of embedded simulation at the tactical level.
97
DISTRIBUTED SOLDIER REPRESENTATION: IMPROVING M&S REPRESENTATION OF THE
SOLDIER
Manuel Diego, Clayton W. Burford
US Army RDECOM Army
Research Laboratory
Orlando, Florida
Joseph S. McDonnell, Ph.D., Bert
Davis, Gary Smith
Dynamic Animation Systems, Inc.
Fairfax, Viginia
Derrick Franceschini
StackFrame, LLC
Sanford, Florida
The Army has developed a breadth of Modeling and Simulation (M&S) capabilities representing platforms such
as fixed and rotary-winged aircraft, tracked and wheeled vehicles, and weapons systems for various uses and of
various fidelities. The Army has represented humans – soldiers, civilians, and threats – in its M&S as well.
These representations provide physical model characteristics for mobility, delivery accuracy, lethality, and
sensing, as well as behavioral representation to support tactical movement, clearing a building, obtaining Human
Intelligence (HUMINT), and treating simulated wounded. These models rarely represent the soldier as a
complex system, representing factors such as stress, human physiology, leadership, unit cohesion, and morale, to
name a few. Instead, the actions of the simulated soldier are often based on a deterministic model of human
behavior. When nondeterministic representations are used, they are often stochastic where random numbers
provide variability across iterations, but the variability comes from the random number seed, not from the
model. This provides unsatisfactory simulation results for those stakeholders interested in analyzing the effects
of the soldier representation, as the simulated soldiers appear robotic or even superhuman. This paper describes
the Distributed Soldier Representation (DSR) research and development effort that has been underway at the
Army Research Laboratory, Human Research and Engineering Directorate, Simulation and Training Technology
Center (ARL HRED STTC) for the past two years. In this paper, we describe our research that has identified
eleven areas of interest for improving soldier representation. We further describe the development of an
innovative Service Oriented Architecture (SOA) that provides a modern web services-based approach to
integrate disparate models to address these identified representation gaps. We describe the challenges and
benefits achieved by taking a web-services approach, as well as the lessons learned from the web-services
integration of the Effects of Stress with One Semi-Automated Forces (OneSAF). Finally, we discuss ongoing
development work.
S-12 The Business of Simulations
1330
A MBSE Approach in Modeling
Systems Using Hybrid
Simulation Techniques (15368)
1400
Simulator Architecture Upgrade
Utilizing Virtual Machines
(VMs) (15219)
1430
Improved Process for Bridging the
Technology Transition
Valley of Death (15103)
NOTES
98
A MBSE APPROACH IN MODELING SYSTEMS USING HYBRID SIMULATION TECHNIQUES
Asli Soyler Akbas
Orlando, Florida
Waldemar Karwowski
Industrial Engineering and Management Systems
Orlando, Florida
Modern simulation efforts, vastly improved with recent computing and visual power advancements,
provide solutions to various fields such as engineering, defense management and training. The current
body of knowledge on simulation from different domains displays this field’s wideness as well as the
amount of variation and interaction among its components. Such intricacy and scale drives the need to
adapt a holistic and robust approach aiming to support modelers with three core challenges. First, the
increasing complexity of modeled problems complicates the ground-up approach. Second, the involvement
of stakeholders from various fields and backgrounds introduce additional needs and expectations varying
from one to another, each facing unavoidable changes due to shifts in environmental conditions affecting
the modeling efforts as well as the model, itself. Finally, the need to maintain the coherency and efficiency
of validated models through structural change requests that arise from emerging variables, constraints or
states.
This paper proposes a methodology for modeling and maintaining hybrid platforms using Model-Based
Systems Engineering (MBSE) approach. The approach is demonstrated by building the architectural model
of a hybrid simulation platform using Systems Modeling Language (SysML). The finalized architecture,
which consists of an agent-based and systems dynamics parts, is then built and simulated in a C++ based
environment. Finally the findings are analyzed to evaluate the benefits and shortfalls of the approach in
aiding developers to overcome the challenges through their modeling efforts.
SIMULATOR ARCHITECTURE UPGRADE UTILIZING VIRTUAL MACHINES (VMS)
Thomas Bridgman
National TechnologiesAssociates
Wright Patterson AFB, Ohio
Elizabeth Gaugler
AFLCMC/WNSE
Wright Patterson AFB, Ohio
The Simulators Division of the Air Force Life Cycle Management Center (AFLCMC) located at WrightPatterson Air Force Base (WPAFB) Ohio is responsible for many Air Force simulator assets. Currently,
the state of some of the simulators consists of multiple legacy computer system architectures with nonsupported hardware and operating systems (OS). For example, some of the OSs includes MSDOS 6.22,
Windows NT, Windows XP, IBM AIX, and multiple versions of Linux. Furthermore, the simulators
consist of several racks of Personal Computer (PC) hardware running a single instance of an OS with
single software application. In addition, many of these PC systems are not fully utilizing their resources
including CPU, memory, storage, input/output, network, etc. A majority of a simulator’s PCs are internally
networked and synchronized together over an internal and external network via multiple switches and
routers. With the current system configurations usually consisting of outdated equipment, unsupported
hardware, software and OSs, an alternative architecture of Virtual Machines (VMs) may assist the
technical obstacles that arise with the inefficiencies of legacy simulator systems. Upgrading legacy
simulators with a single or a combination of bare-metal and native hypervisor architectures could provide
the means to maintain the legacy OS and software load while simultaneously storing the current OS and
application in multiple VMs within the same server. Along with hosting a simulators legacy OS, VMs may
also contribute to reduced system administration functions, lower physical system footprint, operational
redundancy, reduced power consumption and costs, and provide a more effective security boundary. This
paper will investigate some alternatives to simulator hardware, software, and security upgrades with VMs.
99
IMPROVED PROCESS FOR BRIDGING THE TECHNOLOGY TRANSITION VALLEY OF DEATH
Henry Marshall
Human Research and
Engineering Directorate
Technology Center
Orlando Florida
Robert Wells
Dynamic
Animations
Systems
Orlando,
Florida
Bob Burch
Dignitas
Technologies,
LLC
Orlando,
Florida
Paul Dumanoir
Army, Program Executive
Office for Simulation,
Training and
Instrumentation,
Project Manager, Integrated
Training Environment
Orlando, Florida
Jeff Truong
Effective
Applications
Corporation
Orlando,
Florida
One of the biggest obstacles facing research organizations is how to effectively develop innovative technologies that
transition into Programs of Records (PoRs). On the other hand, Project Managers (PMs) are interested in technology
developments efforts that mitigate the technology risks of their PoRs with minimal risks to their existing Engineering,
Manufacturing, & Development or Production acquisition phases. Solving this technology transition “valley of death”
has long been elusive as technologies have been developed that are not transitioned, and PoRs continue to have their
technology gaps that are not addressed. To attempt to solve the quagmire created by this mismatch, the Army Research
Laboratory, Human Research and Engineering Directorate, Simulation and Training Technology Center (ARL HRED
STTC) partnered with the Program Executive Office for Simulation, Training, and Instrumentation (PEO STRI)
Project Manager for Constructive Simulation (PM ConSim) to develop a new process to bridge the technology
transition chasm. The effort created a program called the Risk Reduction Test Bed (RRTB) with a defined process for
risk mitigation and streamlined the technology insertion from Research and Development (R&D) programs. This
program has quickly become a model for technology maturation and transition between these two organizations. The
initial phase of the process requires a capability and technology gap analysis that extends into the project’s users and
long-term life cycle. A concurrent activity within this phase includes building representative test beds to develop and
test the technologies. The process involves a gap analysis that determines which are feasible and provides the greatest
return on investment to the PoRs. The process then develops projects that look at possible solutions. This paper details
examples of how this process moves from gaps to solutions to transition to the PoRs providing a model example for
any organization seeking to improve their processes in this area.
TUESDAY, 1 DECEMBER, 2015 ROOM S320A
T-1 Designing for Training Effectiveness
1400
Measuring Training Effectiveness of
Lightweight Gamebased
Constructive Simulation (15007)
1430
Validating Scenario-based Training
Sequencing: The Scenario
Complexity Tool (15102)
1500
Adaptive Instructor Operating
Stations: Design to Decrease
Instructor Workload and Increase
Effectiveness (15336)
NOTES
100
MEASURING TRAINING EFFECTIVENESS OF LIGHTWEIGHT GAME-BASED CONSTRUCTIVE
SIMULATION
Dr. Jonathan Stevens
Ms. Latika Eifert
Dr. Boris Stilman
Dr. Oleg Umanskiy
Dr. Stephen R. Serge
Army Research
University of Colorado
STILMAN Advanced
Mr. Dean Reed
Laboratory (ARL)
Denver
Strategies
Orlando, FL
STILMAN Advanced
Denver, CO
Florida (UCF)
Strategies
Orlando, FL
Denver, CO
The U.S. Army continues to employ constructive and game-based simulation for training. While both classes of
simulation have been found to lower the cost of training, it is still unknown whether or not these classes are
actually effective training mechanisms. The Linguistic Geometry Real-time Adversarial Intelligence and
Decision-making (LG-RAID) simulation is a lightweight, game-based, constructive simulation that exploits
novel game theory to create intelligent, predictive and tactically-correct Courses of Action (COAs) for exercise
participants at the company echelon and below. The primary goal of this study was to examine the training
effectiveness and usability of the U.S. Army's LG-RAID simulation in an operationally relevant environment. A
secondary objective of this study was to assess both the usability and functionality of the simulation in order to
improve the technology through future design recommendations. Qualified Soldiers were randomly assigned to
one of two training treatments (LG-RAID or a traditional planning method) and tasked to develop, plan and
brief a tactically sound operational mission in order to empirically assess the training effectiveness of LG-RAID.
The independent variable was training treatment. Dependent variables included performance and individual
survey responses. Experimentation was conducted at Fort Benning, GA and performance was evaluated by
accredited Army instructors. Results of this study indicate that LG-RAID shows promise as an effective training
simulation tool when compared to the baseline condition.
VALIDATING SCENARIO-BASED TRAINING SEQUENCING: THE SCENARIO COMPLEXITY TOOL
Robb Dunne, PhD
Innovative Reasoning LLC
Orlando, FL
Stephen A. Sivo, PhD
Orlando, FL
Nathan Jones
MCSC PM TRASYS
Orlando, FL
Effective and efficient Scenario-Based Training (SBT) is sequenced using well-grounded instructional strategies
and learning theory. The primary instructional strategy employed by the Military requires that SBT is sequenced
in a “crawl-walk-run” trajectory. For software to sequence scenarios effectively and efficiently in this manner,
SBT needs objective, computational values of a scenario’s complexity, but designers, software engineers and
trainers operate without the necessary tools to objectively calculate Scenario Complexity (SC). This results in
subjectively sequenced SBT that may be ineffective, inefficient, or designed without attention to sound
instructional practices.
To address this issue, research in education, task complexity, task framework and cognitive resource principles
was integrated and an innovative SC tool (patent pending) comprised of an algorithm and supporting process,
was developed to objectively and computationally define SC. This paper presents findings from the use of the
SC tool to validate a training matrix embedded in the United States Marine Corps’ M1A1 Advanced Gunnery
Training System.
To establish that the SC tool is accurate and effective, it was first necessary to determine how consistent the
Subject Matter Expert (SME) evaluations of the scenario’s characteristics were. Then, using the results of their
input to the SC algorithm, determine how well the SME sequencing matched that of the training matrix. The
objective was to use the SC tool to verify and validate the “crawl-walk-run” sequencing of the training matrix
and identify any areas in need of adjustment.
After employing the SC tool, quantitative analyses showed that the SMEs were very consistent in their
formulations. Importantly, the SC tool revealed that the training matrix deviates alarmingly from “crawl-walkrun” sequencing. This paper also presents the study’s methodology and algorithm, lessons learned and the future
impact that this innovative SC tool may have upon design, development and evaluation of SBT and automated,
adaptive training.
101
ADAPTIVE INSTRUCTOR OPERATING STATIONS: DESIGN TO DECREASE INSTRUCTOR
WORKLOAD AND INCREASE EFFECTIVENESS
James A. Pharmer, Laura M. Milham
Naval Air Warfare Center Training Systems Division
Orlando, FL
John A. Valaitis, John Winters
Basic Commerce and Industries, Inc. (BCI)
Clifton Park, NY
Dahlgren, VA
Design of human computer interfaces that support instructors in conducting training events can be a
challenge. In some cases, instructor operator stations (IOSs) are not designed around the task of
instruction. Without a human factors approach to design, resultant systems can increase the workload of
the instructors, possibly decreasing the training value of the event. The objective of the Office of Naval
Research (ONR) sponsored Adaptive Training for Combat Information Center (ATCIC) effort is to utilize
adaptive training research to drive the design of IOS concepts that support instructor effectiveness and
increase efficiency through the reduction of workload for monitoring and debriefing scenario based
exercises. To achieve this objective, the team utilized the McCracken & Aldrich (1984) method of
predicting the Visual, Auditory, Cognitive, Psychomotor (VACP) task demands. As a measure of potential
improvement in temporal demands on instructors, a time-based predictive workload analysis identified
potential impacts of design improvements on time spent on each task within a scenario based training
exercise. This guided the development and evaluation of an instructor interface to support observation of
student performance, assessment, event driven performance checklists, and remediations. Results indicated
a 26% reduction in total estimated instructor task time required to conduct a scenario based training event
when supported by adaptive training tools. This approach complemented empirical user evaluations of the
IOS in a field setting (Milham, Pharmer, & Fok, 2015). This paper will discuss the approach, findings, and
how this method can be integrated into an iterative design approach to address workload issues with
operators.
TUESDAY, 1 DECEMBER, 2015 ROOM S320A
T-2 VR, AR, and AV for Training “Reality”
1600
Training Effectiveness Evaluation of
Augmented Virtuality
for Call for Fire Training: Insights
from a Novice Population (15014)
102
1630
Empirically Derived
Recommendations for Training
Novices Using Virtual Worlds
(15038)
NOTES
1700
Using Augmented Reality to Tutor
Military Tasks in the Wild
(15050)
METRICS ASSESSMENT TOWARD A TRAINING EFFECTIVENESS EVALUATION OF AUGMENTED
VIRTUALITY FOR CALL FOR FIRE TRAINING: INSIGHTS FROM A NOVICE POPULATION
Julie N. Salcedo, Stephanie J.
Lackey & Roberto Champney
Orlando, FL
Stephen R. Serge & Jonathan Hurter
Institute for Simulation & Training
Orlando, FL
Gino Fragomeni
Army Research Lab Simulation and
Training Technology Center
Orlando, FL
Call for Fire (CFF) is a highly complex and dynamic task to train. Existing CFF training systems offer
immersive training experiences, yet high set-up and implementation costs and limited system portability
inhibit fulfillment of throughput requirements. Augmented Virtuality (AV) may be a viable solution to
reduce costs associated with CFF Simulation-Based Training, improve system portability, increase
throughput, and enhance the immersive experience. AV involves the blending of live and virtual training
elements to create a highly immersive experience with greater task fidelity. This experiment represents an
initial metrics and experimental protocol assessment in a series of training effectiveness evaluation
experiments investigating the performance and learner perception tradeoffs of AV technologies applied to
the CFF task domain. Results reveal trends toward increased learner self-efficacy, positive perceptions of
system fidelity and usability, and high ratings for immersion, engagement, and presence. These findings
confirm the validity of the selected performance metrics and subjective measures for the assessment of AV
technologies for CFF training and also inform the empirical recommendations to improve the quality of
follow-on training effectiveness evaluations.
EMPIRICALLY DERIVED RECOMMENDATIONS FOR TRAINING NOVICES USING VIRTUAL
WORLDS
Crystal S. Maraj,
Karla A. Badillo-Urquiola,
& Sherry L. Ogreten
Institute for Simulation and Training
Orlando, FL
Stephanie J. Lackey
Orlando, FL
Douglas B. Maxwell
Human Research and Engineering
Human Simulation and
Training Technology
Orlando, FL
The U.S. Army Training and Doctrine Command’s Army Learning Concept 2015 and Army Training Concept
2025 discuss the requirements for adaptive soldier learning models with flexible training delivery methods.
Current Game-Based Virtual Environments (GBVEs) have the ability to provide Military Operations in Urban
Terrain (MOUT) training based on the Army’s requirements, but only for small unit operations. Existing
GBVEs lack the capability to support large numbers of users in the same environment at one time or allow the
users to engage in critical thinking. Virtual World (VW) technology offers viable solutions to flexibility and
scalability challenges found in traditional simulation-based MOUT training such as room clearing tasks, as well
as demonstrated the ability to impart valuable training for such tasks. Previous research indicates less
experienced Soldiers benefit from VW training (Lackey, Salcedo, Matthews, & Maxwell, 2014). The evidence
suggests a need to empirically explore the impact of VW training for operationally relevant tasks on
inexperienced populations. This paper presents the results from the second study in a multi-year series of VW
Training Effectiveness Evaluations (TEE). The present experiment investigates performance outcomes and user
perceptions of 64 novice Soldiers (e.g., ROTC Cadets) using traditional and VW training methods for a room
clearing task. Results indicate significant Pearson’s product-movement correlation coefficients between the
stress-state survey DSSQ and the workload survey NASA- TLX for each training condition and combined
training. The survey results offer insight into performance outcomes for the room clearing task. Furthermore, the
results reported herein contribute empirically-derived recommendations for the design, development, and
implementation of VW training.
103
USING AUGMENTED REALITY TO TUTOR MILITARY TASKS IN THE WILD
Dr. Joseph J. LaViola Jr. , Brian M. Williamson,
Conner Brooks, Sergiu Veazanchin
Orlando, FL
Dr. Robert Sottilare, Pat Garrity
U.S. Army Simulation & Training Technology Center
Orlando, FL
Intelligent Tutoring Systems (ITSs) have been shown to be effective in training a variety of military tasks.
However, these systems are often limited to laboratory settings on standard PCs and laptops which focus on
exercising cognitive skills (e.g., decision-making and problem solving) and may potentially limit the learning
and retention of the dismounted Soldiers and Marines training to master physical tasks. Augmented reality
presents the possibility of combining intelligent tutoring with hands-on applications in realistic physical
environments. In this paper, we examine the use of an augment-reality based adaptive tutoring system for
instruction in the wild, locations where no formal training infrastructure is present, and identify the challenges
that arise when developing such a system. We began the transition from desktop tutoring to the wild by
exploring an existing real life mockup of a market scene along with low cost commercial-off-the-shelf devices
(e.g., HMDs coupled with depth cameras) and a 3D model of the environment. The goal of our scanning
approach is to use “human in the loop” 3D scene acquisition via augmented reality so that the scene can be
scanned efficiently and with complete coverage. Using this 3D model, intelligent tutoring systems can
adaptively manage instruction while being aware of the physical and augmented objects in the scenario.
Furthermore, with this awareness of the physical environment, we hope to provide augmented effects and
objects (e.g., virtual humans) that register to the physical environment and respond realistically to interactions
with the trainee. We also explored developing a training scenario for evaluation of our system that is made to
work with emerging low-cost commercial augmented reality devices (e.g., Epson Moverio). Our approach
examines the merging of intelligent tutoring with augmented reality to be used for hands-on immersive training
of psychomotor tasks in a setting beyond the typical desktop tutoring session.
T-3 Physician Know Thyself! Enhancing Surgical Team Skills through Gaming and
Simulation
0830
Developing Game-Based Leadership
Training for Robotic
Surgeons (15198)
0915
The Use of Hyper-Realistic Surgical
Simulation (15244)
NOTES
104
DEVELOPING GAME-BASED LEADERSHIP TRAINING FOR ROBOTIC SURGEONS
Roger Smith & Alyssa Tanaka
Steve McIlwain & Brad Willson
Florida Hospital Nicholson Center
ARA/Virtual Heroes
Celebration, FL
Raleigh, NC
All surgeons must simultaneously perform as skilled practitioners and effective team leaders in the operating room.
This is further complicated in robotic surgery because the surgeon is removed a short distance from the operating table
and works from within a specialized cockpit. This separation creates a unique hurdle when a crisis arises that requires
the surgeon to disengage from the immediate steps of the surgery to provide leadership and guidance with issues
involving the team, the equipment, the room, or the patient.
To develop and test these skills we initially created a series of scenario-based videos with quizzes to evaluate surgeon
understanding of these leadership responsibilities. Using these as a guide, we developed a game-based virtual
environment containing the same information as the videos but in a 3D interactive space which is accessible through a
web browser. This environment presents accessible and engaging scenarios that include a scoring mechanism which
can assess the time to react to events, the actions that occur before and after a decision, and the correctness of the
decision made. The tool can also present alternative or repetitive scenarios when the student does not take the correct
action. This paper describes the development process and the interactions with the surgeons and operating room teams
which drove the design and content of the virtual environment. The paper also describes the longer term plans to
validate the content and introduction of the game to multiple surgical training sites around the country. Though the
virtual environment uses a more interactive method for presenting leadership and team decision making information,
we are interested in whether it is more effective than traditional didactic lectures, textual instructions, videos, and live
role playing.
THE USE OF HYPER-REALISTIC SURGICAL SIMULATION
Anthony J. LaPorta,
M.D., FACS,
Professor of Surgery
and Military Medicine,
Rocky Vista
University School of
Medicine
Parker, CO
Charles
Hutchinson, DO,
United States
Army
Mark Lea, M.D.,
FACS, Associate
Professor of Anatomy
and Surgery Rocky
Vista University
Douglas
Robinson, MS4,
Senior medical
student at Rocky
Vista University
Roy Alson, M.D.,
Michael Czekajlo,
Alan Moloff, D.O.,
PhD, key for
M.D., PhD. Director of
MPH. United States
Prehospital Care,
Simulation, Hunter
Army Retired Special
Wake Forest
Maguire Veterans
Operations and
University. Medical
Administration
Aerospace Medicine
Director of Disaster
Hospital, Virginia
Services, NC
Commonwealth
Winston-Salem, NC
University
Reginald
Lawrence Gaul,
Douglas Granger,
Tuan Hoang,
Franciose, M.D.,
M.D., FACS,
PhD, Director of
M.D., FACS,
FACS, Chairman
Chief Medical
Salivary
Commander,
of Trauma
Officer United
Biometrics at
United States
Surgery at Vail
States Ski Team,
Arizona State
Navy.
Valley Medical
Vail Valley
University and
San Diego, CA
Center
Medical Center
Salimetrics, Inc.
Traditional didactic educational approaches, though necessary to acquire basic biomedical knowledge, often fail to bring
medical students to a full comprehension of medical issues. This awareness often develops only after firsthand experiences in
the clinic. In order to enhance student competence and confidence as well as increase students’ ability to apply knowledge
and technical skills prior to commencing third-year clinical clerkships, Rocky Vista University College of Osteopathic
Medicine (RVUCOM) in Parker, Colorado, has developed an Intensive Surgical Skills Course (ISSC). This immersive short
course mimics a General Surgery rotation, and utilizes the Human-Worn, Partial-Task Simulator (also known as the "CutSuit") (Mueller, Moloff, Wedmore, Schoeff, LaPorta, 2012), (Hunt, B; Wall, V; LaPorta, A.J.; Rush, R; Moloff, A; Schoeff,
JE; Tieman, M; Lea, M; 2012), (Mueller G.; Hunt B.; Wall, V.; Rush, R.; Moloff, A.; Schoeff J.; Wedmore, I.; Schmid, J.;
LaPorta, A.J.; 2012). The balance between beneficial stress (induction of enhanced memory and recall performance) and
detrimental stress (impairment of memory and learning) may represent a fine line. Measuring the students' reaction to
stressful events before, during, and after realistic education scenarios creates a biomarker profile of the educational event,
which then is correlated with the training tasks. This profile, along with other qualitative or quantitative measures such as
technical tasks and cognitive knowledge, can be modeled in a simulation that increases the complexity of training without
overstressing the individual. This simulation model allows a professor to develop a course that maximizes the habituation and
thus retention of knowledge. This model is particularly applicable to physicians in training (medical students) as they seek to
achieve advanced skills and knowledge in the fields of triage, emergency medicine, and surgery in preparation for entering
their third year rotations. The central hypothesis of the present study is that medical students can advance their proficiencies,
and reduce performance anxiety, through intensive immersion training. This innovative medical simulation research may
provide a model for future medical education. It also suggests components of a possible model for determining the return on
investment (ROI) of simulation training dollars. With diminishing training dollars, more effective ways to determine ROI
have been advanced within the Department of Defense. (Oswalt, et. all, 2011)
105
T-4 Team Building: All for One and One for All
1600
Effects of Virtual Emergency
Training on Mine Rescue Team
Dynamics (15119)
1630
Alternative Front End Analysis for
Automated Complex
Systems (15121)
1700
The Team Multiple Errands Test: A
Platform to Evaluate
Distributed Teams (15264)
NOTES
ASSESSING THE EFFECTS OF VIRTUAL EMERGENCY TRAINING ON MINE RESCUE TEAM
EFFICACY
Hoebbel C, Bauerle T, Macdonald B, Mallett L
The National Institute for Occupational Safety and Health
Pittsburgh, PA
To reach trapped miners, underground mine rescue teams might be required to perform a variety of non-routine tasks
(e.g., fight fire, pump water, support unsafe roof) as they encounter hazardous and rapidly changing conditions. Due to
risks associated with such conditions, underground mine rescue team training has traditionally been performed using
live exercises in the form of competitive drills in above-ground facilities or open fields. Oftentimes these contests
utilize printed paper placards to represent environmental conditions and have strict rules which were developed for
comparative assessment purposes. Although widely used, these contests have undetermined ‘real world’ application,
are low fidelity, and have limited documented evidence for effectiveness. Both miners and subject matter experts have
emphasized the need for more realistic and engaging training environments to enhance the learning experience of all
miners and emergency responders. To this end, the National Institute for Occupational Safety and Health (NIOSH) has
developed a fully immersive dynamic virtual training environment for mine emergency responders. During training
scenarios, rescue teams approach the problem and perform as they would in “real life” utilizing virtual representative
equipment, with success dependent on effective communication and group decision-making. This study represents the
first documented empirical attempt at evaluating the effectiveness of virtual reality training for mine rescue teams in
the United States, and this paper discusses the utility of such environments for not only delivering realistic and
engaging training, but for conducting behavioral research activities. Associations among psychosocial factors such as
training climate, team familiarity, and team efficacy are examined and, in general, the study results support findings
and recommendations found in the emergency teamwork literature. The results of this effort will add to the research
base on mine emergency response training and assessment as well as provide insights into emergency response team
behavior.
106
ALTERNATIVE FRONT END ANALYSIS FOR AUTOMATED COMPLEX SYSTEMS
Natalie Drzymala, Tim
Buehner
Natim Research
Tallahassee, Florida
M. Glenn Cobb
U.S. Army Research
Institute
Fort Benning, Georgia
John Nelson
Engility Corporation
Leavenworth, Kansas
Linda Brent
The ASTA Group, LLC
Pensacola, Florida
A growing body of literature reports that task-based analyses alone are not sufficient for determining training
requirements for highly automated, complex systems that rely upon multilevel command and control integration.
This has spurred concerns among Army leaders that the traditional Systems Approach to Training (SAT) Front
End Analysis (FEA) strategy may not sufficiently identify training requirements for some emerging systems,
and provided impetus for our research effort to develop an alternative FEA strategy better suited for these types
of systems. The first phase of our effort focused on the research and design of potential alternative FEA
strategies. The second phase provided a use case application of an alternative FEA to existing air and missile
defense system training to validate and refine the strategy. During the third phase of our effort, we applied the
alternative FEA to an emerging integrated air and missile defense architecture. The refined alternative FEA
strategy supplements traditional SAT analyses with team-based and expertise-based analyses and was used to
successfully identify requirements beyond those found through traditional SAT methods alone.
EVALUATING DISTRIBUTED TEAMS WITH THE TEAM MULTIPLE ERRANDS TEST
Jamiahus Walton, Stephen Gilbert, PhD., Eliot Winer, Ph.D.,
Michael Dorneich, Ph.D., Desmond Bonner
Ames, IA
Modern day teams, whether in the military or civilian workplace, have the ability to achieve goals that are
otherwise unobtainable by individuals. The timing and characteristics of feedback that teams receive during
training are critical. Though there is a solid foundation of research on optimal feedback, there is limited
exploration of what constitutes ideal team feedback including addressing the individual team member versus the
whole team and whether that feedback is public (visible to the entire team) or private (visible only to one
member of the team).
Previous research that studied the effect of feedback on team performance has yielded slightly different
conclusions. For example, research focused on the privacy of feedback suggests that public feedback can have a
motivational effect that improves performance. The aim of this work is to discover the most effective
combination of the target and privacy of feedback.
To accomplish this goal a modified version of the Multiple Errands Test (MET) was developed to evaluate the
performance of three-member teams, the Team MET (TMET). The MET, normally used for evaluating
cognitive processing, requires that specific rules be followed while completing multiple tasks within a time
constraint. Participants performed the TMET while coordinating purchases in a virtual mall. In each of four
timed shopping sessions, participants received feedback on their performance as an individual and team.
Feedback was given in one of four conditions: individual private, team private, individual public, and team
public. Task performance and rule errors were measured as dependent variables. Results did not yield a broadly
significant effect of feedback condition on team or individual performance. However, the study did demonstrate
the validity of the TMET as a platform for assessing a team's ability to perform under heavy cognitive load.
107
T-5 Where’s Who Go? – Operational Considerations for LVC Training
0830
Exercise Management
Considerations for Live, Virtual, and
Constructive (LVC) Training
(15281)
0900
Integrating Warship Bridge,
Combat, and Deck Teams in LVC
Environment (15191)
0930
Capability Assessment of Test and
Live Training Systems
for Real Time Casualty Assessment
(15364)
NOTES
EXERCISE MANAGEMENT CONSIDERATIONS FOR LIVE, VIRTUAL, AND CONSTRUCTIVE (LVC)
TRAINING
Katherine P. Kaste, Kelly Neville & Melissa Walwanis
Naval Air Warfare Center Training Systems Division (NAWCTSD)
Orlando, FL
Amy Bolton
Office of Naval Research (ONR)
Arlington, VA
As integrated training events evolve to include Live, Virtual, and Constructive (LVC) entities, multiple
safety considerations, inside and outside of the cockpit, need to be considered. The current Navy Aviation
Simulation Master Plan (NASMP) states, “Safety must be an integral part of planning and execution. Live
blue and red platform displays, training mode functionality and training rules must help mitigate safety
risks associated with an LVC environment” (NASMP Policy IV). The Office of Naval Research’s (ONR)
Science and Technology (S&T) effort, Virtual and Constructive Representations on Live Avionics
Displays (VCR LAD), has identified multiple areas through interviews and thematic analysis where LVC
has the potential to impact integrated training. In this paper, we focus on implications for training exercise
management, including Range Training Officer (RTO) and Range Safety Officer (RSO) activities. There
are various areas of concern within training exercise management, including delegation of workload
amongst personnel (e.g., RTOs, RSOs) and whether existing interface designs and work support tools can
adequately support the conduct and oversight of LVC training. Additionally, new positions may need to be
introduced and integrated into the exercise management team. Further, new exercise management practices
may need to be adopted. This paper presents initial findings and associated recommendations for exercise
management technologies, roles, responsibilities, and practices, as well as, future research needed to
determine and evaluate specific solutions to those recommendations.
108
INTEGRATING WARSHIP BRIDGE, COMBAT, AND DECK TEAMS IN LVC ENVIRONMENT
Eric Phipps
Engility, Incorporated
San Diego, CA
Richard Gaughen
Camber, Incorporated
San Diego, CA
This paper investigates means to integrate USN surface warship bridge, deck and combat watch teams in a
Live, Virtual, Constructive (LVC) training environment. Issue: Currently the US Navy’s LVC distributed
training environment (called Fleet Synthetic Training - FST) only incorporates ship’s combat watch teams
and squadron staffs, participating from their ships in serious games (i.e., scenarios), but do not include
integral decisions and actions made by bridge and deck watch teams. Yet bridge and deck watch teams are
an integral part of the combat effectiveness of a ship, especially in the visual environment in which many
actions take place. Some examples of required total watch team integration are defense against fast, small
boat threats, critical infrastructure/critical asset protection, and vessel Visit, Board, Search and Seizure
(VBSS) operations. But in FST the bridge and deck watch teams are role-played by the white cell, muting
the team-building required to create a cohesive, effective combat crew, and missing the chain-of-command
interactions from gun crews to other ships to the squadron commodore. Means: This paper looks at two
possibilities for incorporating bridge and deck teams into scenarios. The first is to use a dedicated bridge
training facility integrated into FST. The second is to use virtual reality or augmented reality headsets,
worn by watchstanders aboard ship, and integrated into FST. Applications of both approaches are closely
examined, including the Office of Naval Research current effort to connect key personnel from the bridge,
deck crews and Combat Information Center aboard ship with helicopter crews in a common simulated
scenario. Optimal use of both virtual and augmented reality is examined.
CAPABILITY ASSESSMENT OF TEST AND LIVE TRAINING SYSTEMS FOR REAL-TIME CASUALTY
ASSESSMENT
Joan H. Johnston
U.S. Army Research Laboratory (ARL)
Human Research and Engineering Directorate
(HRED)
Simulation and Training Technology Center
(STTC),
Orlando, FL
Margaret D. Nolan
Training Analysis, Design & Evaluation
Division
Naval Air Warfare Center, Training
Systems
Division (NAWCTSD)
Orlando, FL
Jake Caldwell
(UCF)
Orlando, FL
The Army needs the ability to characterize the effectiveness of test and live training systems to improve realistic
training and real-time arbitration of casualties. For years, the Army testing and live training communities have
strived to develop a robust Real Time Casualty Assessment (RTCA) capability. Currently, both communities are
working closely to develop the ability to characterize the degree to which specific, identified capability upgrades
to test and live training systems could improve RTCA. This project leveraged a capabilities based assessment
method developed for the USMC Squad Immersive Training Environment (previously published as an I/ITSEC
paper- Johnston, Dunfee, et al., 2012). This paper describes the methods and findings of the Systematic Team
Assessment of Readiness Training (START) method as it was applied to a use-case of the Bradley Fighting
Vehicle (BFV) crew training. A baseline capability assessment of the live training environment for the BFV is
described in terms of its ability to support effective live Force-on-Force (FoF) training and RTCA. We describe
how the START method was employed to establish environmental attribute (e.g., battlefield effects) capability
gaps that are used to prioritize investments in test and live training systems based on the degree to which the
investments could improve training and RTCA effectiveness. Using the methodology, quantitative and
qualitative data on current testing and training capability was collected. The results of this data collection will be
presented in this paper. This is a collaborative effort among the Program Executive Office for Simulation,
Training, and Instrumentation (PEO STRI), the Army Test and Evaluation T&E Command, the Naval Air
Warfare Center Training Systems Division (NAWCTSD), and the US Army Research Laboratory Human
Research and Engineering Directorate (ARL-HRED).
109
T-6 ISR, CRM, TDM, Oh My! – Different Domains, Universal Strategies
1030
Improving Military Crew Resource
Management Using a
Commercial Strategy Game (15097)
1100
Simulation and Training Challenges
for Intelligence,
Surveillance and Reconnaissance
Analysts (15175)
1130
Effectiveness of Process Level
Feedback at Training Tactical
Decision Making (15201)
NOTES
IMPROVING MILITARY CREW RESOURCE MANAGEMENT USING A COMMERCIAL STRATEGY
GAME
Christopher Roos, Jelke van der Pal,
Ghanshaam Sewnath & Johan Meijer
National Aerospace Laboratory NLR
Amsterdam, The Netherlands
Lt. Col. Michel de Rivecourt
Centre for Man in Aviation CML,
Royal Netherlands Air Force
Crew Resource Management (CRM) training has been among the staple diet when addressing Human Factors
within military aviation, both in the initial and recurrent training phases. In recurrent training, CRM modules
traditionally focus on creating awareness of the role of non-technical skills such as decision making in safe
operations. While awareness can be successfully raised by CRM courses, actual change of non–technical
behavior is still a challenge. Military aviators often claim it is difficult to implement the theoretical aspects of
CRM training in practice. Therefore, a solution is needed where non-technical skills can be applied in a
relatively rich way, including team aspects and time pressure, to create situations where human factors issues
may be experienced and trained. At the same time, the solution is subject to many constraints: it needs to be
short (to fit into a regular CRM training day), cost effective and easy to use.
To address this challenge, the potential of commercial video games was evaluated empirically. Based on the
CRM training module, a ‘suitability analysis’ was performed on a range of different commercial video games,
decomposing the different game elements of each game. Resulting from the analysis, a training solution using
the game “XCOM – Enemy Within” was developed. This training solution was subsequently applied in a CRM
training course. Using video recordings, team and individual behavior have been analyzed thoroughly to identify
the game’s potential for producing CRM behavior as well as effect on dynamics of group behavior and decision
making.
Key training effectiveness evaluation points included depth and variety of challenges posed by the game, trainee
potential to implement CRM and instructor possibility to evaluate CRM behaviors. Results indicate the game is
effective in developing CRM behavior in trainees and leads to sufficient feedback input for instructors.
110
SIMULATION AND TRAINING CHALLENGES FOR INTELLIGENCE, SURVEILLANCE AND
RECONNAISSANCE ANALYSTS
Lisa Tripp, Elliot Humphrey, Christine Covas-Smith,
Jonathan Diemunsch
Air Force Research Laboratory
WPAFB, OH
Mike Garrity, Cullen Jackson, Mike Keeney, Sterling
Wiggins
Aptima
Washington, D.C.
Aircrews have leveraged simulation for several decades to immerse themselves in complex mission situations,
develop new concepts of operations (CONOPS) and tactics, techniques, and procedures (TTPs) for the next
fight, test new capabilities, and develop robust and adaptable mission performance. While the Air Force
endorses simulation-based training as a vital need for aircrews, currently there is not an analogous capability
available for intelligence analysts. This gap becomes more crucial as we prepare the next generation of analysts
for potentially drastically different operational environments where Air Supremacy is not guaranteed, denied
environments are the norm, and cyber warfare plays a frightening role. Simulation-based training is needed to
prepare analysts for these environments. Although, in general, simulation-based training is thriving across many
domains (e.g., flight simulation, driving simulation, shooting simulation), little work has focused on training for
Intelligence, Surveillance, and Reconnaissance (ISR) professionals. Requirements for developing a realistic,
simulation-based training environment for ISR tasks are distinctive from those required for Aircrews.
Simulation of the large variety of information sources is the key. The objective of the current effort was to
identify the requirements underlying development of a simulation-based training capability to maximize the
efficiency and effectiveness of training for ISR. To tackle this complex problem set, the Air Force Research
Laboratory (AFRL) Warfighter Readiness Research Division leveraged Mission Essential Competency analysis
in conjunction with cognitive task analysis to identify key requirements for high fidelity, simulation-based
training for the ISR. This paper will describe the challenges facing creation of simulation-based training for ISR,
work to develop a training capability for these critical warfighters, and our vision for future ISR simulationbased training.
EFFECTIVENESS OF PROCESS LEVEL FEEDBACK AT TRAINING TACTICAL DECISION MAKING
Meredith Carroll, Christina K. Padron, Stephanie
Quinn, Glenn Surpris, Brent Winslow
Orlando, FL
Erica Viklund
Pacific Science and Engineering
San Diego, CA
Decision making is a critical skill throughout all echelons of the military. From command and control to the
front line, Warfighters must be trained to quickly and effectively make decisions. Key to the development of
effective decision makers is the utilization of targeted learning strategies, designed to improve an individual’s
decision making process. One such strategy is process level feedback, which provides information regarding
how effectively an individual is utilizing task strategies or performing task sub steps necessary to achieve task
goals. Process level feedback can be employed to improve decision making skills by identifying and correcting
breakdowns in the decision making process. A process level feedback method to target decision making skills
was developed for use in simulation-based training. This feedback method incorporates outcome feedback with
process level feedback aimed at decomposing decision making performance into sub-processes using the OODA
loop as the theoretical model (Observe, Orient, Decide, Act; Boyd, 1987). Feedback is provided on errors/error
patterns across these sub-processes. This feedback strategy was evaluated in a series of experiments conducted
both in the laboratory and in the field with Marines. In the laboratory study, participants who were recruited
from the community performed tactical decision making tasks in a simulation testbed and either received the
process level feedback or a control condition of outcome feedback. A similar study was conducted with
experienced Marine Corps squad leaders. Marines at the School of Infantry East received training utilizing either
a simulation-based training approach which incorporated this process level feedback method or simulation
training with methods traditionally used in the Marine Corps. This paper will describe the process level feedback
method, present results of both experimental studies, and discuss implications and lessons learned for
implementing this method in a military training setting.
111
PDF FILES OF THE 2015 TUTORIAL PRESENTATIONS ARE INCLUDED ON THE PROCEEDINGS CD.
PLEASE SEE THE TUTORIALS SECTION OF THIS BOOK FOR SCHEDULE AND SYNOPSES DETAILS.
NOTES
112
TABLE OF AUTHORS
Abbott, Robert, 56
Browne, Michael, 83
Durlach, Paula, 41
Acharya, Girish, 58
Bryan, Derek, 11
Dykes, Carol Ann, 71
Akbas, Mustafa Ilhan, 15
Buehner, Tim, 107
Eady, Michael, 79
Alban, Angela, 49
Bullock, Charles, 83
Ebert, Mark, 94
Allbeck, Jan, 27
Bunch, Larry, 31
Eckman, Stephen, 86
Allen, Christine, 77
Burch, Bob, 100
Edwards, Harvey, 31
Allison, Dan, 75
Burford, Clayton, 98
Eifert, Latika, 97, 101
Alson, Roy, 105
Burke, Brian, 38
Enloc, Michael, 47
Amburn, Charles, 52
Burke, Shawn, 18, 77
Eslami, Mohammed, 28
Amico, William, 11
Caison, Chad, 57
Eum, Junho, 69
Amilde Lovlid, Rikke, 87
Caldwell, Jake, 109
Fabiano, Gregory, 54
Anastasopoulos, Panos, 54
Capriglione, Christin, 12
Faircloth, Mitchell, 79
Anderson, Benjamin, 56
Carabello, Helga, 38
Fautua, David, 19
Ariotti, Scott, 44
Carpenter, Jim, 58
Faxon, Christopher, 13
Armstrong, Stuart, 37
Carroll, Meredith, 111
Fefferman, Kevin, 78
Askvig, Jerrit, 82
Case, Scott, 81
Flanagan, Scott, 25
Atkinson, Beth, 87
Cerri, Anthony, 91
Fleener, Graham, 75
Ayaz, Hasan, 40
Cerritelli, Laura, 77
Fleming, Michael, 37
Badillo-Urquiola, Karla, 55, 103
Chalmers, Robert, 11
Forsythe, Chris, 56
Bair, Lisa Jean, 72
Champney, Roberto, 103
Fragomeni, Gino, 103
Baldwin, Timothy, 97
Chang, In-Chung, 66
Franceschini, Derrick, 90, 98
Balint, John, 27
Cintron, Luis, 76
Franciose, Reginald, 105
Bamberger, Robert, 11
Cobb, M. Glenn, 107
Franciosi, James, 51
Banko, Katherine, 39
Coolahan, James, 70
Frank, Mark, 54
Bannan, Brenda, 57
Cooley, Tim, 73
Franken, Igor, 74
Barge, Walter, 90
Cooney, Lisa, 49
Frascati, Alicia, 17
Barie, Sean, 68, 81
Coovert, Michael, 13
Frazier, Spencer, 16
Barnieu, Joanne, 26
Covas-Smith, Christine, 111
Freeman, Barbara, 16
Basavaraj, Prateek, 15
Cox, Brennan, 31
Freeman, Jared, 28, 64
Bauerle, T, 106
Craven, Patrick, 40
Freeman, Michael, 46
Beaubien, Jeffrey, 9
Creighton, Doug, 30
Friedman, Mark, 46
Bechtel, Bob, 15
Creighton, Tom, 42
Froytlog Hole, Siren Elise, 22
Beck, Dennis, 45
Crutchfield, James, 51
Fuller, Marvin, 17, 45
Bell, Morris, 37
Crutchfield, Richard, 81
Gallagher, Shane, 57
Benito, Juan, 16
Cullen, Sean, 29, 77
Gallant, Scott, 90
Bent, Margaret, 50
Czekajlo, Michael, 105
Gallimore, Jennie, 38
Benton, Nancy, 38
Davidson, Mary, 38
Garcia, David, 55
Bergenthal, Jeff, 68, 70
Davies, Amanda, 67
Garibay, Ivan, 15
Berking, Peter, 26
Davis, Bert, 98
Garibay, Ozlem, 15
Bickley, William, 23
De Rivecourt, Michel, 110
Garrity, Mike, 111
Biddle, Elizabeth, 71
Deakins, Richard, 73
Garrity, Pat, 33, 34, 104
Bink, Martin, 10, 20, 66
Dean, Courtney, 25, 64
Gaska, James, 83
Binsch, Olaf, 39
Deibler, Nina, 51
Gaughen, Richard, 109
Bitoun, Ariane, 92
DeJong, Steven, 35
Gaugler, Elizabeth, 99
Blake, Lea, 51
DeVrijer, Jur, 33
Gaul, Lawrence, 105
Blake-Plock, Shelly, 57
Diaz, Maria, 51
Gavin, Michael, 64
Bolton, Amy, 9, 31, 108
Diedrich, Frederick, 25
Georgiopoulos, Michael, 15
Boonekamp, Rudy, 33
Diego, Manuel, 98
Gilbert, Stephen, 107
Bosa, Romain, 92
Diemunsch, Jonathan, 111
Goldberg, Benjamin, 52
Boyce, Michael, 52
DiGiovanni, Frank, 24
Gordon, Steven, 73
Bradshaw, Jeffrey, 31
Dill, Kevin, 16
Grace, Paul, 50
Branzoi, Vlad, 29, 58, 77
DiPilla, Muchael, 59
Graddy, Courtney, 45
Brent, Linda, 107
Dominguez, Jose, 31
Graham, Cathy, 38
Bridgman, Thomas, 99
Dominguez, Manny, 38
Granger, Douglas, 105
Brimstin, Jay, 9, 13
Drake, David, 68
Graniela, Benito, 85
Brock, Timothy, 14
Drzymala, Natalie, 107
Grechkin, Timofey, 47
Brooks, Conner, 104
Dumanoir, Paul, 68, 81, 89, 100
Gregg, Brian, 89
Brookshire, Jonathan, 29
Dunne, Robb, 101
Grippin, Burt, 68, 81
113
I/ITSEC Website (www.iitsec.org) for ordering information. Limited numbers of single year copies of 1998-2009
are available. All papers from 1966 through 2000 are available in the I/ITSEC Compendium (order form at the
back of this book). Individual papers may also be ordered through the www.iitsec.org portal.
Grubb, Jefferson, 9
Guarino, Sean, 96
Gunter, Stephen, 53
Haag, Jason, 26
Hackett, Matthew, 78
Hadley, Steven, 83
Hallenbeck, Phil, 82
Hannachi, Tahar, 92
Hao, Jianping, 95
Harrison, Devin, 51
Harrs, Shawn, 71
Harvey, Edward, 17
Harvey, Robert, 41
Haynes, Jacqueline, 59
Hernandez, Gabriela, 40
Hester, Toumnakone, 13
Hieb, Michael, 27
Hill, Susan, 55
Hines, Karl, 89
Hoang, Tuan, 105
Hodges, Glenn, 24
Hoebbel, C, 106
Hoffman, Robert, 31
Hogan, Christopher, 22
Holden, Heather, 18
Holmes, Chris, 68
Holt, Lisa, 59
Holub, Joseph, 39
Holutiak, Jason, 33, 34
Hou, Ming, 87
Houston, Rebecca, 54
Hruska, Michael, 41
Hubal, Robert, 15
Hulme, Kevin, 54
Humm, Laura, 37
Humphrey, Elliot, 111
Hurter, Jonathan, 103
Hutchinson, Charles, 105
Hyland, Jessie, 26
Ianni, Steve, 95
Ingmundson, Paul, 38
Ingraham, Lorie, 51
Isaksen, Geir, 22
Ison, David, 47
Izzetoglu, Kurtulus, 40
Jackson, Cullen, 111
Jackson, James, 72
James, David, 10, 66
Jang, Jihyun, 91
Johnson, Mark, 86
Johnston, Joan, 18, 109
Johnstone, Michael, 30
Jones, Nathan, 12, 101
Jones, Randolph, 31
Jones, Tom, 24
Jorden, Neil, 37
Joseph, Jeremy, 85
Kapadia, Amit, 68
Karwowski, Waldemar, 99
Kaste, Katherine, 35, 108
Kawatsu, Chris, 15
114
Keeney, Michael, 9, 111
Keller-Glaze, Heidi, 23
Khan, Burhan, 30
Khimeche, Lionel, 92
Kilcullen, Tara, 41
Kilgore, Ryan, 22
Killilea, John, 87
Kim, Hee-Soo, 91
Kim, Jungyoon, 91
Kist, Richard, 74
Kleissas, Dean, 11
Knapp, Michael, 62
Knarr, Kenneth, 63, 65
Knott, Camilla, 25
Knox, Alan, 91
Ko, Li-Wei, 66
Kopyi, Antoni, 60
Koscielniak, Michael, 89
Kreutzer, Christine, 55
Kumar, Rakesh, 29, 58, 77
Lacerenza, Christina, 77
Lackey, Stephanie, 103
Lai, Peng-Wen, 66
LaPorta, Anthony, 105
LaViola, Joseph, 43, 104
Le, Vu, 30
Lea, Mark, 105
Leary, Mark, 93
Lee, Sangjin, 91
Leigeber, Daniel, 89
LeSueur, Kenneth, 94
Levesque, Jerome, 39
Lewis, Bridgette, 57
Lewis, Mark, 75
Li, Xingxin, 95
Lim, Teng Howe, 21
Lin, Chin-Teng, 66
Lin, Shih-Chuan, 66
Lipkin, Ilya, 76
Little, Anne, 49
Lloyd, Charles, 83
Long, Rodney, 26, 41
Luotsinen, Linus, 87
Lutz, Robert, 11
Macdonald, B, 106
Mallett, L, 106
Maraj, Crystal, 103
Marinier, Bob, 15
Marlow, Shannon, 77
Marshall, Henry, 89, 100
Maxwell, Douglas, 83, 103
Mayor, Marco, 75
Mazzeo, Mark, 77
McAlinden, Ryan, 47
McClain, Jonathan, 56
McCoy, Sean, 38
McDonnell, Joseph, 98
McEwen, Timothy, 22
McIlwain, Steve, 78, 105
McLaughlin, Robert, 53
McLaughlin, Ryan, 81
McLean, Angus, 60
McMillan, Mathew, 91
McNamara, Jennifer, 50, 57
McNeely, Danielle, 17
Medford, Ashley, 41
Megiveron, Michael, 48
Meijer, Johan, 110
Meinshausen, Donald, 79
Merchant, Chirag, 97
Metcalf, David, 77
Meyer, Christophe, 87
Michael, Tomer, 88
Milham, Laura, 102
Miller, Duncan, 8
Miller, John, 20
Millich, Sean, 94
Minkov, Yaniv, 88
Mizelle, John, 89
Moloff, Alan, 105
Monday, Paul, 80
Mondesire, Sean, 83
Moore, Ronald, 86
Morris, Karen, 54
Muller, Tijmen, 33
Munro, Jaeson, 86
Murphy, Jennifer, 41
Murray, Glenn, 58
Nauer, Kevin, 56
Nelson, John, 107
Nelson, Susan, 17
Neville, Kelly, 108
Newton, Carolyn, 41
Newton, Charles, 87
Nguyen, Vivian, 30
Nicholson, Denise, 31
Niehaus, James, 96
Nolan, Margaret, 109
Noordkamp, Wouter, 35
Norton, Heather, 17
Novak, Ana, 30
O’Connor, Michael, 94
O’Grady, Ryan, 31
Oden, Kevin, 40
Ogreten, Sherry, 103
Oh, Sangyoon, 69
Ohlman, James, 93
Olsen, Dale, 37
Oroszi, Terry, 38
Oskiper, Taragay, 29, 77
Padron, Christina, 111
Pappada, Scott, 28
Park, Samjoon, 91
Parkes, David, 79
Parsons, Doug, 73
Patel, Upesh, 53
Pearson, Jeff, 53
Pelaez, Juan, 85
Perez, Manuela, 45
Pettitt, Beth, 36
Pharmer, James, 102
Phillips, Jennifer, 63, 65
Phillips, Kelley, 40
Phipps, Eric, 109
Plew, William, 51
Pokorny, Robert, 59
Poltrack, Jonathan, 42
Poppinga, Gerald, 87
Powers, Johnny, 62
Priest, Heather, 9, 35
Priselac, Nancy, 17
Priselac, Stephen, 17
Puglisi, Matthew, 64
Punihaole, Tom, 94
Pynadath, David, 55, 97
Quinn, Stephanie, 111
Ratwani, Krista, 25
Raybourn, Elaine, 24
Read, Luke, 77
Rediger, Scott, 60
Reed, Dean, 101
Regan, Damon, 41
Reini, Seth, 31
Reitz, Emilie, 19, 61
Reynolds, James, 32
Riddle, Christian, 9
Riecken, Mark, 90
Rijken, Roel, 87
Roberts, Timothy, 33, 34
Robinson, Douglas, 105
Robson, Robby, 71
Rodriquez, Jose, 73
Roessingh, Jan Joris, 87
Roos, Christopher, 110
Rosick, Mary, 95
Ross, Karol, 63, 65
Roth, John, 73
Rutledge, John, 90
Sadagic, Amela, 73
Saffold, Jay, 33, 34
Salas, Eduardo, 18
Salcedo, Julie, 103
Samarasekera, Supun, 29, 58, 77
Sanchez, Jamie, 85
Sanders, Brian, 93
Schaffer, Richard, 29, 77
Schatz, Sae, 19
Scheidt, David, 11
Schnell, Tom, 60
Scott, Rosalyn, 38
Seavers, Greg, 12, 73
Sege, Stephen, 103
Seltzer, Robert, 17, 71
Serge, Stephen, 101
Service, Katherine, 31
Sessoms, Pinata, 31
Sewnath, Ghanshaam, 110
Shepherd, Jill, 19
Shih, Yi-Cheng, 66
Shoaf, Tovar, 33, 34
Shute, Valerie, 62
Siddiqui, Abdul, 17, 71
Siegfried, Robert, 28
Silva, Austin, 56
Sinatra, Anne, 18
Sivo, Stephen, 101
Sizintsev, Mikhail, 77
Smit, Selmar, 35
Smith, Craig, 32
Smith, Eileen, 17
Smith, Gary, 98
Smith, Matthew, 37
Smith, Michael, 62
Smith, Robert, 48
Smith, Roger, 45, 105
Smith, Todd, 47
Soh, Boon Kee, 21
Sotomayor, Teresita, 49
Sottilare, Robert, 18, 43, 52, 104
Soyler Akbas, Asli, 99
Sprinkle, Ronald, 62
Stacy, Webb, 9
Starsman, Scott, 43
Stensrud, Brian, 87
Stevens, Denise, 14
Stevens, Jonathan, 83, 97, 101
Stilman, Boris, 101
Stodd, Julian, 19
Suma, Evan, 47
Surpris, Glenn, 111
Tan, Kia Hong, 21
Tanaka, Alyssa, 45, 105
Terwilliger, Brent, 47, 93
Thiele, Luke, 111
Tosh, Andrew, 85
Toubman, Armon, 87
Tracey, Luke, 30
Tran, Hung, 84
Tripp, Lisa, 111
Truong, Jeff, 89, 100
Tucker, Christina, 93
Tucker, Jennifer, 25
Turcanik, Michal, 87
Uhl, Elizabeth, 10, 66
Ukwa, Jennifer, 19
Umanskiy, Oleg, 97, 101
Uphoff, Bruce, 89
Valaitis, John, 102
Van de Pal, Jelke, 110
Van den Berg, Tom, 28
Van der Poel, Nick, 35
VanDuyne, Christopher, 44
VanLent, Mike, 15
Vasquez, Sara, 58
Veazanchin, Sergiu, 104
Ventrone, Fred, 94
Viklund, Erica, 111
Vincenzi, Dennis, 47
Vitovitch, Nicholas, 58
Vogel-Walcutt, Jennifer, 65
Vogt, Brian, 48
Voshell, Martin, 22
Walton, Jamiahus, 107
Walwanis, Melissa, 9, 35, 108
Wang, Lubin, 62
Wang, Ning, 55, 97
Washburn, Nick, 41
Wedel, Douglas, 76
Wells, David, 11
Wells, Robert, 89, 100
Werk, Lloyd, 51
Werner, Anna, 58
Weyhrauch, Peter, 96
Wiggins, Sterling, 9, 111
Williams, Logan, 83
Willianson, Brian, 104
Willoughby, Mike, 81
Willson, Brad, 105
Wilson, Bradley, 78
Wilson, Elizabeth, 53
Wilson, Tom, 93
Winer, Eliot, 39, 107
Winner, Jennifer, 13
Winslow, Brent, 111
Winterbottom, Marc, 83
Winters, John, 102
Witcher, Ken, 93
Wittman, Robert, 68, 81
Wolverton, Michael, 58
Wray, Robert, 35
Wysocki, Tim, 51
Xu, James, 62
Yang, Meng-Shun, 66
Yang, Minsoo, 69
Yang, Xu, 95
Yarnall, Louise, 58
Yates, Floy, 73
Ye, Fei, 95
Young, David, 22
Zhao, Weinan, 62
Zheng, Weimin, 31
Zhu, Zhiwei, 58
Zou, Cliff, 75
Papers are available on the 2015 I/ITSEC CD ROM included in the Conference Attendee meeting bag, or visit the I/ITSEC 115
Website (www.iitsec.org) for ordering information. (Limited numbers of CDs from 1998-2014 are also available.) All papers from
1966 through 2000 are available in the I/ITSEC Compendium. Individual papers from 1966 through 2015 may also be ordered
through the www.iitsec.org portal.
Interservice/Industry Training, Simulation and Education Conference
The National Training and Simulation Association (NTSA)
An Affiliate of the National Defense Industrial Association (NDIA)
Suite 400, 2111 Wilson Boulevard • Arlington, VA 22201
http://www.iitsec.org • (703) 247-2569

2015IITSECProgramGuideLoRes

Transcription

Similar documents

iSIM iSIM

Power devices - Crosslight Software

VISUAL SIMULATION As viewed from Orono Fairgrounds South

When Colette Foisy-Doll arrived in Qatar last year

ASE2000 Version 2 - Applied Systems Engineering Inc.

sendai mediatheque - toyo ito

Army Looks to Training Center Upgrades

To a brochure about our facility, resources

Simulation Experience

2016_Supplement_Spon..