Short Resume

vaPV
(voice activated Procedure Viewer)
Type A Contract
SyberNet Ltd.
Inventor
Introduction
The original proposal described an on-board voiceactivated crew assistant that would enable the crew
of a space mission navigate detailed documents
using speech commands. The primary objective of
the proposal was to research the opportunity and
effectiveness of providing an alternative interface
for astronauts who needed to access detailed
information in “hands-busy” or “eyes-busy”
situations. With input from ESA the original
proposal was refined to focus on integrating a
speech command and control interface into an
existing procedure document viewer already
established and deployed within the space
community. This viewer is called the International
Procedure Viewer or (iPV) and is the result of a
joint collaboration between NASA and ESA.
The Innovation
The key innovation of the proposed solution was to
enhance an existing legacy application by adding
new modes of interaction to it. Developing multimodal desktop applications is a new and
challenging area for software developers, especially
in situations where new modes of interaction have
to be integrated into existing operational systems.
The IPV (International Procedure Viewer), used by
all astronauts on the International Space Station
when carrying out nominal and detailed
maintenance work, was an excellent candidate for
multi-modal use. Multi-modal access to this tool
offers the advantage to users of being able to
interact with the documentation system in situations
when they may not have easy access to the laptop
keyboard.
Currently, IPV provides visual-only output, via a
display, and uses the keyboard and mouse as its
input devices. The work carried out as part of this
activity added:
a) a voice command modality, in addition to
the standard keyboard input, and,
b) an optional Text-To-Speech audio output
that presents the text of the document to the
user aurally.
Results of the ITI activity
Two new modes of interaction were added to the
existing IPV tool, namely,
• voice commands for input, and,
• text rendered as speech, and presented to the
user as an aural output through the laptop
audio device
Because it was a legacy application, with a
widespread user community within ESA, there were
some constraints enforced on the project. These
were the following:
1. IPV was developed with Microsoft’s IE
explorer browser as the target end user
browser. No other browser would work so
any new technology had to be compatible
with IE Explorer 6.0+
2. IPV procedure content is stored as XML and
is transformed by XSL templates to be
presented as HTML in the browser. Any
speech enabling activities would have to
follow a similar process in order to preserve
the synchronisation required betweens
modes of interaction i.e. the active step’s
associated voice prompt had to be in synch
with the highlighted step displayed on the
screen and vice versa.
3. The XML structure and content for a
procedure could not be modified. The
structure is an internationally agreed
standard and the content is an operational
product under configuration control. Any
additional tagging required to support the
voice modality had to be introduced
dynamically.
The modified IPV tool also had a new visual
indicator to display the status of the voice subsystem. This is indicated in the following iPV
screen capture.
The following drawing gives an overall architecture
of the data flow within the IPV application and
indicates where the new voice-enabling component
has been added within the existing environment. All
key constraints were met. In particular, there was no
requirement to change the underlying XML format
for the documentation in order to add the voice
tagging needed.
XML Formatted
Procedure
(Unmodified)
Future applications
The concept, developed as part of this proof of
concept activity, has been proposed as an addition
to the future Columbus system laptop framework,
(LAPAP MkII), developed by Astrium. Skytek,
another space development company, have already
built a Java-based Checklist procedure viewer for
the LAPAP environment and a proposal to provide
a voice commanding capability for this viewer is
currently being prepared. The proposal, a
collaboration between Astrium, Skytek and
SyberNet, would provide a voice commanding
capability as a plug-in component in the LAPAP
MKII framework.
TOMCAT Container
vaPV
XSL Filter Rules for
Voice Recognition
Java Servlet
XML Filter
Existing XSL Filter
Rules for Visual
Presentation
DHTML document marked up
with Voice Enabling tags for
processing in the browser
Contacts
Keyboard
IE Explorer Browser
with Speech Add-In
SAPI
ASR Engine
SAPI
TTS Engine
Mouse
User
Mr. John Melody, SyberNet Ltd, Galway Business
Park, Dangan, Galway, Ireland
john@sybernet.ie
Mr. Mikael Wolff, ESA Project Officer, Noordwijk,
The Netherlands,
Mikael.Wolff@esa.int