Speech interaction system – how to increase its usability? Fang Chen

Speech interaction system – how to increase its usability?
Fang Chen
Department of Computing Science,
Chalmers University of Technology, SE-412 96 Göteborg, Sweden
fanch@cs.chalmers.se
Abstract
This paper discussed different issues related to the usability of
speech interaction system. It includes the usability concept,
different design approaches, design process and evaluation
questions for speech interaction system. Usability is a very
fuzzy concept, especially when it related to the speech
interaction system: it is hard to measure and it is very much
context dependent. The traditional user-centered design
approach may not be suitable for the speech interaction system
design since the users might not have enough knowledge to
see what the technology can do. Usage-centered design may
be the better method but there is not comprehensive theory
and methodology for the design process and evaluation.
1. Introduction
Speech technology has been significant progress. The
increased automatic speech recognition (ASR) rates (99 to
100% accuracy in laboratory condition) with large vocabulary
capacity and naturalness and robustness speech recognition,
the humanized synthetic voice, robustness dialogue system
design and language understand, together with the increased
computer speed and memory capacity, make the speech based
interaction system possible for the real-time application [1-3].
The results from Leduc’s performance benchmarking survey
[4] clearly indicated that speech recognition is becoming a
more mass-market option.
When it comes to the real-time application, usability of
the speech interaction system becomes important for the
acceptance of the user. There is new challenge for the speech
system interface design. The environment where the system is
going to be used can be dynamic; user’s vocal quality can be
unstable, and speech can be variance. Many research works
related to human factors on speech related interface design
were published in later 80’s and beginning of 90’ [5-7]. At
that time, the ASR technology was very poor; most of studies
were focus on feedback design and error corrections. The
results may be invalid for present system design applications.
The constant interests of employing speech interaction
design come from the common agreed advantage of speech as
“a natural and intuitive communication method” of human
beings. But such advantages are not obviously appearing in
any speech interaction system unless the designers understand
human cognitive behavior, human needs, and usability issues.
2. The usability concept
The usability requirements for the speech interaction system
are facing the new way of measuring the effectiveness,
efficiency and satisfaction, the three elements in the usability
evaluation. Among these three elements, effectiveness and
efficiency are close related to the functionality of the system
and directly affect the satisfaction, therefore is the essential
for the design. The effectiveness can be measured in terms of
the extent to which a goal or a task is achieved. The
efficiency means the amount of effort required to accomplish
a goal. Gibbon, et al [8] has described a large number of
different effectiveness and efficiency measurements.
The effectiveness can be understood as the acceptable
performance that should be achieved by a defined proportion
of the user population, over a specified range of tasks and in a
specified range of environments as the system design for.
The efficiency might be measured in terms of the time
taken to complete one task, or the errors that user makes
during the performance, as well as how much effort the users
have to invest on learning and understanding of how the
system is working and to be able to work on it. Acceptable
performance should be achieved within acceptable human
costs, in terms of fatigue, stress, frustration, and discomfort.
Sometime the usability evaluation by measuring these
three elements separately does not meet each other properly.
For example, two dialogue systems (MIMI and Tap&Talk)
for train timetable information were compared with usability
evaluations [9]. Effectiveness is measured by the number of
dialogues that were completed successfully, while efficiency
is measured by task completion time and user satisfaction is
measured subjectively. The results showed that: MIMI was
slightly better than Tap&Talk by effectiveness measurement
because solving recognition error was easier with the MIMI
interface; Tap&Talk is significantly better than MIMI by
efficiency measurement, because there were spoken prompts
in MIMI interface, but not in Tap&Talk, so the later one
worked faster. The satisfaction is measured by giving users
many questions, even most of the statements in this study
were judged bout equal for both interface, sometimes it was
in favor of the Tap&Talk interface. Even though, the overall
user satisfaction was not significantly higher for the
Tap&Talk interface, but most people are prefer to use this
system anyway.
Hone and Graham [10] made some systematic study on
user satisfaction towards speech input/output systems. There
are six factors in user attitude: perceived system response
accuracy, likeability, cognitive demand, annoyance,
habitability and speed. Under each label of the attitude, one
can design a set of questions to ask.
Satisfaction can be understood in different levels. It is the
human nature that never satisfies with what they have. The
needs to fulfill the functional requirement, to be able to solve
the problems are fundamental. The basic level is the
comfortable and confidence feeling when using the interface.
Learnability and flexibility of the system may affect the
comfortable feeling of the users [11]. As soon as this need is
fulfilled, people will look for the higher level of satisfaction,
such as the pleasant, exciting, fulfillment and happiness.
Figure 1 shows the pyramid of the human needs in different
levels.
Self-actualization needs:
Self-fulfilment
Psychological needs:
Esteen
Belonging and love
Basic needs:
Comfort
Confident
How to do?
Encouragement
Freedom
Suggestion
Feeling of being part of user community
A sense of mastery and competence
User feel comfortable in learning
Logical structure of the information presentation
Accurate, reliable, and predictable
Figure 1, The hierarchy of user’s needs I. Suggested
by Coe [12]
Usability is indeed a fuzzy concept. It can only be
meaningful within a specific context. One particular system
placed in one context will probably display different usability
characteristics when placed in a second context [13].
Usability is a property of the interaction among a product or a
system, a user and the task, or set of tasks and the
organization, society, environment the system is in use.
3. Usability in design process
To conceptualizing usability in the design process, Don
Norman [14], Ravden and Johnson [15] have pointed out
some design principles:
x Visibility: information presented should be clear, well
organized, unambiguous and easy to understand.
x Feedback: users should be given clear, informative
feedback on where they are in the system, what actions
they have taken, whether these actions have been
successful and what actions should be taken next.
x Consistency and compatibility: The way the system
looks and works should be consistent at all times, and
compatible with user expectations.
x Explicitness: The way the system works and is
structured should be clear to the user so user will easily
know how to use it. It shall show the relationship
between actions and their effects.
x Flexibility and constraints: the structure and the
information presentation should be sufficiently flexible
in terms of what the user can do, to suit different user
needs and allow them to feel in control of the system. At
the same time, the system shall also restrict certain kind
of user interaction that can take place at a given
moments.
x Error prevention and correction: The possibility user
error should be minimized, automatically detected and
easy to handle those which do occur.
x User guidance and support: easy-to-read and understand,
relevant guidance and support should be provided to
help the user understand the system.
There are very few studies on usability issues in design
speech interaction system. Dybkjaer and Bernsen [16]
discussed different criteria for spoken language dialogue
systems design. Their results matched well with above
principles:
x Learnability: The design shall always clear about the
user’s experience/knowledge to the system and how
quick they can learn about the interaction.
x Visibility: The system’s output language should be
naturalness and to guide users’ input language so that the
input language becomes manageable for the system.
x Explicitness: The system shall express its understanding
of the user’s intention and provide the information to the
user in a clear, unambiguous, correct, accurate and using
the language that the familiar to the user.
x Flexibility: Multimodal interaction is always more
preferable, but be careful to select an appropriate
modality for interaction on the specific task domain.
x Feedback: The user shall be informed on what is going
on in the system. The output voice quality shall has
natural intonation and prosody, with an appropriate
speaking rate.
x Error prevention and correction: Error handling is
always important for speech interaction system, as the
error may come from the system mis-recognize what the
user said, or even users’ mad the error.
x User guidance and support: Interaction guidance is
necessary for the users to feel in control during
interaction. A long and complicated “user manual”
provided to the first-time user is not suitable.
Leduc and Dougherty et al [4] has specially pointed out
the important of consistency in the design. The consistency
has two aspects; consistent with previous usage and internally
consistent. The task handling by speech system shall match
the users’ pre-experiences of handling the task by other
systems. The similar tasks will be fulfilled in a similar
manner using identical terms throughout the speech
application.
It is necessary to specify the usability design principles
into different speech interactive systems. The detail of these
design principles for in-vehicle information system design
shall be in somehow different compare with a spoken
dialogue system placed inside the house, as in a mobile
environment, user’s attention shall be kept on the road. The
designer will not only consider the information system itself,
but also the safety drive and effect from stress.
4. Problems with user-centred approach to the
design
The user-centered approach to the design (UCD) can
enhance the usability of the products. In this approach, the
potential users are involved in the entire design process from
the early beginning. The design process is more or less driven
by the user. It specializes of user experiences. The typical
design process is shown in figure 2.
In practical, there are very few organizations managing to
implement the UCD process. The problems associated with
UCD come from the possible issues like [13]: User issues;
organizational commitment; developer skills; and resource
constraints. User experience and knowledge, user’s
expectation, user contribution and agreement and user
diversity are the factors make the user involve difficulties.
Users may expect the new system to be simply an improved
version of the old one; users may not be able to step back
from their daily practices to see how technology can change
the way they work, and/or they might not be familiar with the
design methods used or the technology, and may simply feel
over-awed by the design process (which leads to them feeling
unqualified to comment). It is hard to have the user to
contribute to the quality of the design solution. At the same
time, it is not easy to collect all the information about the user
[17].
6. Context of evaluation
Usability evaluation
Meets all the requirements
1. Plan the human centrd process
Design philosophy
Identify design team and users
Time-line
Success criteria
2. Specify the context of use
Understand the characteristics of
User, tasks, organization,
Environment
Task analysis
5. Evaluate designs against user
requirements
Getting feedback for design
Assess the achievement of user and
Organizational objectives
3. Specify the user and
organizational requirements
Allocation of tasks among users
Functional requirements
Performance criteria
Usability criteria
4. Produce design solutions
Collect knowledges for design
Concrete design solution
Prototypes and user tests
Iterating above process untill satisfy
Figure 2. The interdependence of user centred design
activities (developed from ISO-13407)
A speech interaction system is not simply replacing
keyboard input with speech input, which has been
unfortunately the case as in the history of speech technology
application. There are many studies on comparing the
keyboard input with speech input in different context and the
results show sometime positive and sometimes negative of
using speech as one of the input modality.
The speech interaction system should take the advantages
of naturalness and the intuitive of human speech
communication, so the interaction between human and system
shall be totally different compared with the traditional
human-computer interactive systems as using manual input
and visual output. Even the speech communication is
happened to people’s daily life, it does not mean that users
would know what the speech technology can do and how the
interface shall be looked like, as their past experiences with
the human computer interaction system may not apply to the
speech interaction system.
The UCD process might not be the best choice for speech
interaction system design. Instead of focus on the user,
Rakers [18] suggests to focusing on the roles, goals and
responsibilities people have. This idea leads to the usagecentred approach of the design. The usage-centred approach
focuses on the use of the interaction between humans and the
system, its environment and social-organization. Human
behavior is goal orientated and event that happens in the
living and working environment have its meaning to the user.
The notion of meaning, constraints from technology and
environment and goal are closely related to its specific
context task that the user performs [19]. In this approach, the
potential users are not necessary be involved into the design
process. The possibility of what the technology can do shall
be externally exploded by the technical experts, the usability
of the system shall be the essential for the design, while the
user’s high level fulfillment (as shown in figure 1) shall be
satisfied.
The usage-centred approach to the speech interaction
system design is a new concept and lack of the
comprehensive design theories and methodologies on design
process, problem analysis and evaluation.
5. Problems with user evaluation
In the usage-centered approach, some user tests shall be
carried out in the design process. The purpose to carry out
such test is not for getting user’s opinion of the design, but to
understand better the user and the interaction between the
user and the products, thus to take the most benefit from the
developed technology and to increase the usability.
There is no applicable usability evaluation theory for the
speech interaction system. In general, any evaluation theory
should be able to handle at least the following problems [20]:
x What are the characteristics of overall system
performance? How to measure it? At what level should
they be taken?
x How should we choose the test persons? Should they be
naïve people, or experts?
x How much training is required to arrive at stable
performance where we can be sure that we are
evaluating the properties of the interface and not just the
learning and adaptive behavior of the operators?
x How detail and fidelity the scenarios should be? What
should be their properties? Which properties of the
interface should be used? How can context be specified?
x If general principles are being evaluated, what is the
minimal set of applications which are needed in the
evaluation?
The above questions are more or less related to the
evaluation of the basic requirements to the usability. How to
measure the high level fulfillment of the needs and
satisfaction? Any evaluation is context driving, is it possible
to build a general theory to guide the usability evaluation for
different speech interaction system? External studies need to
be carried out in this area.
6. Problem with design guidelines
Usability design requires integration of multiple disciplines,
such as speech technology, computer technology, the
knowledge in the application domain, human cognition,
human
factors
and
ergonomics,
and
even
social/organizational knowledge. It is impossible for one
designer to have all the knowledge. Therefore many design
“guidelines” are issued in different user interface design
books, or journal papers [21-23]. It was believed that design
“guidelines” should be the helpful and useful for the
engineers to use in their design. Many (if not all) guidelines
vary in the extent to which they are derived from their
specific research findings. Their scope is rarely made explicit
and it remains for the designer to judge the applicability of a
guideline to a particular user interface and to apply it
accordingly. The body of the guidelines is incomplete – many
design issues are just not covered by guidelines [24].
Guidelines from different resources may differ in detail, may
not tell clearly of the application conditions; and may even be
contradictory. People try to abstract the guidelines with
abstracted statement to show its “external validity”, or try to
pretend as having a high “concept level”, while any design is
context depending. How to make sure that the engineer used
the “right” guideline? For example, the guidelines for speech
interface design provided by Baber [23]:
x Match the type of work the recognizer is intended to
perform with the characteristics of the available
technology
There are many problems with such “guidelines”: how to
match the technology and the type of the work? What are the
criteria? How to measure the match?
Life [24] has suggested that if the guideline could be
presented as “IF (condition) THEN (system performance
consequence) BECAUSE (interaction model constraint),
HENCE (guideline, expressed as a system design
prescription)”, then it can be very handy. This is almost an
impossible dream, because nobody can cover all the possible
application conditions that a speech interaction system may
be applied. Many of the application conditions are
unpredictable in present condition, and application context
may change due to the development of the technology and
people’s needs. At the same time, there are almost unlimited
consequences, and constraints one can identify according to
the application context. The matrix of the three entities with
enormous amount of variability in each can come out with
millions of detail guidelines, which may be difficult for the
designers to find out the proper guidelines, while it may still
be danger of not covering the situation which the designer is
working for.
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
7. Discussion and conclusion
Usability is very much context dependent. The traditional
user-centered design approach may not be suitable for the
speech interaction system design due to the limited
knowledge and understanding of what the technology can do.
Usage-centered design focus on the roles, goals and
responsibilities people has. To fulfill the high level needs is
the final goal of the design, while the usability of the system
is the fundamental requirement. Usage-centered design is a
new concept. There is not comprehensive theory and
methodology for the design process and evaluation.
Design “guidelines” is not a good solution to help the
designer to increase the usability of the system. The
integration of multiple disciplines knowledge is important for
the design and it is hard to find the short cuts.
8. References
[1]
Steeneken, H. J. M., "Potentials of speech and language
technology systems for military use: an application and
technology oriented survey," NATO, Defence Research Group
AC/243(Panel 3)TR/21, 1996.
[17]
[18]
[19]
[20]
[21]
[22]
[23]
[24]
Weinstein, C. J., "Opportunities for advanced speech processing
in military computer-based systems," Proceedings of the IEEE,
vol. 79, 1991, pp. 1626-1641.
Weinstein, C. J., "Military and government applications of
human-machine communication by voice," Proc. Natl. Acad.
Sci. USA, vol. 92, 1995, pp. 10011-10016.
Leduc, N., Dougherty, M., Ankaitis, V., "Measuring the
performance of speech applications: a user-centered approach,"
in Universal Access in HCI: Towards an information society for
all, C. Stephanidis, Ed., Lawrence Erlbaum Associates,
publishers, 2001, pp. 372-376.
Jones, D., Hapeshi, K., Frankish, C., " Human factors and the
problems of evaluation in the design of speech systems
interfaces," People and Computers III: proceedings of the third
conference of the British Computer society; Human-computer
interaction specialist group, 1987, pp. 41-49.
Jones, D. M., "Automatic speech recognition in practice,"
Behav. & lnf Tech, vol. 11, 1992, pp. 109-122.
Baber, C., Hone, K.S., "Modelling error recovery and repair in
automatic speech recognition.," International Journal of ManMachine Studies, vol. 39, 1993, pp. 495-515.
Gibbon, D., "Handbook of multimodal and spoken dialogue
systems: resources, terminology and product evaluation," in The
Kluwer international series in engineering and computer
science: Kluwer Academic, 2000.
Sturm, J., Bakx, I., Cranen, B., Terken, J., "Comparing the
usability of a user driven and a mixed initiative multimodal
dalogue system for train timetable information," EuroSpeech
2003, 2003, pp. 2245.
Hone, K. S., Graham, R., "Subjective assessment of speechsystem interface usability," Eurospeech 2001, 2001.
Stanton, N., Human Factors in Consumer Products, Taylor &
Francis, 1998.
Coe, M., Human Factors for Technical Communicators, John
Wiley & Sons, 1996.
Smith, A., Human-Computer Factors: A study of Users and
Information Systems, The McGraw-Hill Companies, 1997.
Norman, D., The Design of Everyday Things. New York, Basic
Books, 1988.
Ravden, S. J., Johnson, G.I., Evaluating Usability of HumanComputer Interfaces: A practical method. Chichester, Ellis
Horwood, 1989.
Dybkjaer, L., Bernsen, N.O., "Usability evaluation in spoken
language dialogue system," Proceedings of the Workshop on
Evaluation for Language and Dialogue Systems, Association
for Computational Linguistics 39th Annual Meeting and 10th
Conference of the European Chapter (ACL/EACL) 2001, 2001,
pp. 9-18.
Eason, K. D., "User-centred design: for users or by users?,"
Ergonomics, vol. 38, 1995, pp. 1667-1673.
Rakers, G., "Interation design process," in User Interface
Design for Electronic Appliances, B. T. K. Baumann, Ed.
London and New York, Taylor & Francis, 2001, pp. 7-47.
Flach, J. M., Tanabe, F., Monta, K., Vicente, K.J., Rasmussen,
J., "An ecological approach to interface design," Proceedings of
Human Factors and Ergonomics Society 42nd Annual Meeting,
1998, pp. 295-299.
Moray, N., "Advanced displays can be hazardous: the problem
of evaluation," pp. 59-62.
Jones, D., Hapeshi, K., & Frankish, C., "Design guidelines for
speech recognition interfaces," Applied Ergonomics, vol. 20,
1989, pp. 47-52.
Baber, C., "Automatic speech recognition in adverse
environments," Human factors, vol. 38, 1996, pp. 142-155.
Baber, C., Noyes, J., "Speech control," in User Interface Design
for Electronic Appliances, B. T. K. Baumann, Ed. London and
New York, Taylor & Francis, 2001, pp. 190-208.
Life, M. A., Long, J.B., "Providing human factors knowledge to
non-specialists: a structured method for the evaluation of future
speech interfaces," Ergonomics, vol. 37, 1994, pp. 1801-1842.