KITTY: Keyboard Independent Touch Typing in VR

KITTY: Keyboard Independent Touch Typing in VR
Carsten Mehring
Dept. of Mechanical and Aerospace Eng.
University of California, Irvine, CA 92697
{cmehring}@uci.edu
Falko Kuester, Kunal Deep Singh and Michelle Chen
Dept. of Electrical Eng. and Computer Science
University of California, Irvine, CA 92697
{fkuester,singhk,mchen}@uci.edu
Abstract
A new hand- or finger-mounted data input device is presented, using traditional touch-typing skills as method of (alphabetic) character input thus providing an ultra-portable
solution for “quiet” data input into portable computer systems. The presented keyboard independent touch-typing device (KITTY) offers high data input rates and a virtually zero
learning curve for new users.
1
Motivation
With recent advances in portable computing and in particular the design of pocket PCs [13] and eye-glass displays
[14], the development of new augmented reality (AR) environments has been made possible. However, one of the current shortcomings in these AR systems is the lack of intuitive
input devices that provide users with complete control over
their workspace. In particular in environments where voice
input is undesirable or infeasible, touch-typing capabilities
have to be available to allow for intuitive access to possibly
complex data.
2
Introduction
The size, layout and weight of traditional keyboards make
them poor candidates for use in mobile AR applications.
In particular, if integration with mobile computing devices,
such as personal digital assistants (PDAs) or pocket personal
computers (PocketPCs) is desired, a new generation of natural handheld input devices is required.
Today’s mobile computing devices frequently include a
physical or digital miniature keyboard, that may be integrated with the device, worn on the user’s wrist or simply
be part of the display area. Unfortunately, these keyboards
generally require the use of both hands, one holding the device and the other for finger-based or stylus-based data input.
The selection of individual keys on a hand held display-based
keyboard, for example, is a rather inefficient process and requires the user to look at the keyboard devise. Handwriting
Figure 1. Prototype glove system illustrating initial contact
placement.
recognition can increase input rates after extensive training
but is still very limiting. This performance gap is problematic if the portable computing devices are to be operated at
the level suitable for today’s computational platforms.
Other devices for data input into portable computing
units, include chording devices, such as a hand-mounted keypads that require the user to press different key combinations
to generate the characters found on a standard keyboard. Further alternatives include microphones in combination with
voice recognition software. The former input devices typically do not meet the data input rates desired for full-scale
interaction and typically require the user to learn a chording/coding language particular to the specific device. On
the other hand, privacy concerns or ambient noise frequently
limit the use of voice recognition devices. Possible scenarios
include input of confidential or sensitive data via voice recognition in conference settings, meetings, or crowded public
places, such as airports.
3
Related Work
With a demand for decreased keyboard size and increased
portability, while maintaining the traditional keyboard layout, new device concepts are required. A first attempt was
the development of foldable keyboards that allowed users to
carry familiar input devices with reduced physical footprint.
While these devices are certainly more compact than traditional full-size keyboards during transport, a folding keyboard is still relatively large in the unfolded state in which
its surface requirements match that of a traditional full-size
device.
Other devices have tried to reduce the number of keys
rather than reducing the overall size of the keys in order to
achieve size reductions. For example, one-handed keyboards
use only half of a traditional keyboard by mapping two characters on each key. This requires the addition of a switch
(keyboard identification) key, enabling the user to toggle between two sets of characters, each representing one-side of a
full-size keyboard. Even though “half-keyboards” may have
a very small footprint, they commonly still require a support
surface.
The “body-wearable keyboard” and similar the so-called
“Gesture-Pad”, which is under development by Sony Inc.,
place “keyboard-keys” on the users clothing or a skeletal
structure worn on the users body. Even though this increases
portability, it greatly restricts the users arm-movement and
flexibility for data input.
Many of the existing keyboard- or keypad-less data input devices for portable computing applications, use finger
or hand mounted sensors in order to generate the signals that
would normally be generated by pressing a keyboard key.
The ”finger-mounted computer input device” [11], the ”fingerless glove for interacting with a data processing system”
[15] and the ”Virtual Keyboard” by Senseboard Technologies [12] use pressure sensors on the users fingers or sensors
(mounted on gloves) measuring flexure of the fingers in order to determine data input via finger positions with respect
to a physical or virtual reference surface. Generally, these
devices assume that the reference surface functions as virtual touch-typing environment. As for folding keyboards, the
need of a physical support surface greatly limits the mobile
computing capabilities of these devices.
In response to this limitation, hand- or finger-mounted
input devices have been proposed, such as the ”Chording
Glove” [16] and ”Twiddler” [1]. The Chording Glove has
contacts at various positions on the fingers and thumbs and
a combination of contacts (chord) is pressed simultaneously
to generate a specific keyboard event. Although chording
devices remove the need for a support surface, their use requires substantial training as the user has to learn a new coding language.
Vaughan R. Pratt describes a device-independent digital
sign langue (Thumbcode), using keys on the users fingers
that are activated by the thumbs of the same hand [10]. In order to generate different input signals with the same key, the
user has to pair neighboring fingers according to a prescribed
coding language. As with chording, thumbcoding requires a
user to essentially learn a new coding language in order to
enter data into the computing device.
The functionality of the so-called Lightglove by Howard
[4, 3], is based on the generation of a ”light-matrix” below
the users hands. Penetration and location of penetration of
this matrix by the users fingertips is detected and electronically mapped to an overlay assigning each position a character according to the layout of a traditional keyboard. This
operation of the device involves simultaneous scanning of the
matrix with various light emitting diodes or lasers, evaluation
of the scattered electromagnetic waveforms by various light
sensitive detectors and signal filtering in a bandpass filter in
order to reject non-correlated ambient signals. Generation of
the optical matrix and detection of finger position in this matrix is fairly complicated. Furthermore, the system has to be
calibrated (possibly in real-time) for specific ambient conditions, such as strong ambient illumination or electromagnetic interference. The lack of tactile feedback (providing
confirmation of data input as is the case for touch-typing on
a keyboard) may also result in lower data input rates. The
use of an optical reflection matrix to detect signal input by
predicting finger position with respect to the optical matrix
below the user’s hands does not allow for free finger motion
without the risk of data input. Instead, the user has to hold
the hands and fingers in a more-or-less stretched out position
in order to omit interference with the optical matrix. The involuntary generation of signals is a common problem with
these devices and has to be addressed. Finally, many of the
devices described in this section are large, counter-intuitive,
and too cumbersome for fast data input. Most users when
confronted with having to learn an entirely new coding language in order to enter information will likely decide against
using such a device. This is particularly true for good VR
or AR applications that should require only very limited user
training.
For VR/AR and mobile computing applications, intuitive,
flexible, and efficient, keybaordless input devices are urgently needed. KITTY, a device prototype that addresses
these objectives is presented in this paper (Figure 2).
4
KITTY Interface
The presented KITTY interface overcomes the limitations
described in the previous sections by providing intuitive access to common touch-typing skills. To accomplish this, the
device provides multiple contacts on each finger that are only
activated when the appropriate finger and thumb contacts are
combined. The layout of these contacts follows the traditional QWERTY keyboard layout, allowing anyone with previous touch-typing experience to immediately use the device.
A significant advantage of this device is that arbitrary finger
motion is allowed without data input, as long as finger contacts and thumb contacts on a single hand do not meet. In
other words, KITTY allows the user complete freedom for
!
1
@
2
Q
Esc
Caps
Lock
#
3
W
A
$
4
E
S
Shift
^
6
R
D
Z
Ctrl
%
5
X
T
F
&
7
Y
G
C
V
*
8
U
H
B
(
)
9
I
J
O
K
N
<
,
M
:
;
D E
F
D0E0
F0
A
C B
1 23
A0
C0 B0
4
Back
}
]
"
’
>
?
.
/
Alt
+
=
{
[
P
L
~
‘
Alt
_
−
0
Ins
|
\
Enter
Shift
u
l
d
Del
r
678
5
A
3F /
./!
,# (!
055 % P'
"
bottom
top
row
(dashed)
and
row (dash-dot)
characters.
@
#
$
%
^
&
*
(
)
_
+
Back
2
3
4
5
6 7
9 −
=
!137"
02
is referred
to
as
the
top
row
and 8the
row0 below
the
base
row
{
}
|
Q W E R"5"
T
Y 9"95"
U I %"%5
O P Esc
[
]
\ referred
(ZXCV ...) is annotated with a short dash: line
and
"
" Caps A
"F Figure
K illustrates
S D
G H
L
Enter
;
’ the
Lock
to as
the bottom
row.
3J also
enumer
$"$5"
8"85"
<A"A5
>
? u
Shiftleft-hand
Shift (5-8)
Z X fingers
C V (1-4),
B N right-hand
M ,
ation of
fingers
.
/
Ctrl
"
(A,A0),
"
r and
~
l
d
Alt
Alt
Ins
Del
thumbs
with
thumb
contacts
(B,B0),
(C,C0),
(D,D0),
‘
Figure 3. Standard US-keyboard (HP Omnibook 500 C) with
division of alphabetical characters into ‘base’ row (solid),
Figure 2. KITTY (left) and Twiddler (right) in comparison.
hand movements while only generating input when fingerand thumb-contacts are combined.
(E,E0), (F,F0) employed
of the wired protoD key-map
A for the
E
C B
F
1 23
678
(3)
(6)
type as discussed
in
Section
4.2.
A0
D0
E0 F0
5
4
C0 B0
(2)
Ctrl
_−
4.1
Finger-Thumb Touch-Typing
(7)
Del
A
3F /
Key-Map
Wired ./!
Prototype ,# (!
055 % P' " Figure
4 illustrates
the location
of electrical
contacts
on
the user’s
hands
as employed
within
present
wired
37"
02the
prototype and following
key layout
in Figure 3. Both
the
"5"
9"95"shown
%"%5
Left Hand
(TOP) " are
Hand
(TOP)
fingerand
thumb-contacts
An
electric
employed.
Right
"
cir
$"$5"
8"85"
A"A5
cuit is closed and a signal is generated upon closing of one
with
one
"thumb-contact
as
" below
finger-contact
discussed
A
1F G
32 in
detail.
" 5 $5 P
' P' "’
Enter
4.2
Caps Lock
(1)
+=
|\
Two different key-maps were proposed for the fingermounted KITTY interface. Section 4.2 discusses the alphabetic key-mapping as employed in the wired prototype
shown in Figure 8. The mapping of alphabetic keys is described in the framework of a comprehensive mapping procedure for the entire set of keys/commands found on a standard keyboard. Note that, the wired prototype shown in the
photograph employs a commercially available keyboard by
GrandTecT M and provides alphabetic data-input functionality as well as punctuation marks and a limited number of essential commands such as “Delete”, “Space”, “Return”, and
“Shift”. Section 4.3 describes a slightly modified key-map
procedure for alphanumeric character input. This mapping is
proposed for the new wireless KITTY interface currently in
development.
Figure 3 illustrates a standard QWERTY keyboard, which
was used as the reference for the design of the KITTY keyboard mapping. While the QWERTY layout was chosen for
the initial implementation other key mappings such as Dvorak or the half-keyboard can be easily supported.
Using the standard QWERTY keyboard (Figure 3) via
touch-typing, a user’s fingers are normally in a home position that includes placing the left pinky on the A-key, the
ring finger on the S-key, the middle finger on the D-key and
the left index finger on the F-key. The home position for the
right hand includes placement of the right index finger on
the J-key, the right middle finger on the K-key, the index finger on the L-key and the right pinky on the ;-key. This row
(ASDF ...), which is annotated in Figure 3 with a solid line,
may be referred to as the base row. The row above the base
row (QWERTY ...) is annotated with a long dashed line and
(5)
(4)
Alt
(8)
Esc
:;
{[
Tab
l
Shift
Shift
~‘
Ins
Back
}]
(A0)
Space
u
r
(D0)
A
d
D
(B0)
(B)
(E0)
(E)
(C)
(F)
(C0)
(F0)
(3)
(6)
(2)
Ctrl
_−
(5)
(4)
(7)
Del
"’
Enter
Caps Lock
(1)
+=
|\
Alt
(8)
3
2
Esc8
Shift
Shift
:;
{[
~‘
4
(A0)
Space
(D0)
Ins
u
r
7
5A
d
D
(B)
Tab
9
6
Back
}]
1
l
(B0)
(E0)
10
A
7F F
F %!
(E)
(C)
(F)
Left Hand (TOP)
(C0)
Right Hand (TOP)
(F0)
A
D
A
1F G 32
" 5 $5 P
' ;F
P'
A
3Y35 *
5Y6
shown
in
P/'
lines,
P'
As
Figure
4 by dashed
finger
contacts 1-8
$ are located
on the
palm-sideof
the user’s
hand3Y0
near
the tops
There
435 of the
fingers
or the
fingertips.
is one
finger
contactA
7F F
pinky,
9" on the
ring, middle and index finger of each hand (1-F %!
FigureLeft4.Hand
Wired
con(TOP) prototype: Location of a) fingertip
Right Hand (TOP)
tacts (1-8) (dashed lines indicate location on the inside of
each hand), b) thumb contacts (A(0) - D(0)) and c) ‘sidefinger’ contacts for ‘special’ character input.
2
3
8
9
4
1
5
6
7
10
D
8). Also, there are six Acontacts A-F, A0-F0
on each thumb;
three contacts on each inner thumb and three contacts on
Left Hand (TOP)
Right Hand (TOP)
each outer
thumb. As described in more detail
below, signals for letters A-Z are generated by contacting one of the
A
;F 3Y35 *
5Y6
P/' P' $ 3Y0 435 9" A
3F /
./!
,# (!
055 % P' " 37" 02 "5" 9"95" %"%5 six
A-F,
finger
contacts
1-8
"
with
one
of the
thumb
"
contacts
$"$5" 8"85" A"A5 A0-F0
on the
same hand.
" " As
shown in Figure 4, there are supplemental finger con-
Tables 2(a) through 2(f) illustrate finger-thumb contact
combinations employed for the wired prototype in order to
generate the signals that would be generated by pressing the
keys on a standard QWERTY keyboard. The middle thumb
tacts located along
the sides of the user’s fingers nearest to
(3)
(6)
contacts B, E are used for characters in the base row. The top
the user’s(2) _−thumbs.
Any
given finger
may have (7)three sup(5)
Ctrl
(4)
thumb contacts A, D are used for characters in the top row
Del
"’ second and
plemental contacts located proximate
to the first,
Enter
and the bottom thumb contacts C, F are used for characters in
Caps Lock
(1)
+=
third
segment
ofAltthe user’s finger. These supplemental(8)con|\
Esc
the bottom row. For example, on a QWERTY keyboard the
Tab
Shift
tacts are
used
to~‘ generate
signals Shift
representing lspecial char:;
{[
character “a” is the character that would be typed using the
u
Ins
Back
r located near
acters. For
example,Space
the supplemental
contacts
(A0)
(D0)
}]
fifth finger of the left hand in the home (base row) position.
d
D
the base and middle of(B)Athe (B0)
ring (E0)
and pinky
fingers of the right
(E)
With the present method, the signal representing the charac(C)
(F)
hand are used to generate
the same signals
that would be genLeft Hand (TOP)
Right Hand (TOP)
(C0)
ter “a” is generated when the finger contact 1 on the pinky
(F0)
erated by the arrow (left, right, up and down) keys. A signal
finger of the left hand (see Figure 4) is closed with the thumb
is generated by contacting a particular supplemental contact
contact B (representing the base row). Since the left pinky
with A
the thumb
D located
the inside
tip of the
1F contact
GA or on
32
finger is used to press the “q” character located on the upper
“Thumb-coding”
is
thumb
of the same
hand. In
other words,
row of a keyboard, the signal representing the character “q”
5
P use of
used "
for these
special
characters
due $5
to the
infrequent
is generated by closing the contact between the pinky finger
'
P'
these characters.
contact 1 and the top thumb contact A (see Figure 4). As can
be seen in Figure 3 and Table 2(b), the combination of finger
contact 1 and thumb contact A is used to generate the signal
representing the character “q”. Since the index finger is used
two characters
on a given
A
7Fto reach
F =
row,
9an additional thumb
contact
for
each
row
is
located
on
the
back or outside of the
F %! !
A
D
user’s thumbs. See contacts A0-F0 in Figure 4. For example, to generate the signal representing the character “f”, the
Left Hand (TOP)
Right Hand (TOP)
finger-thumb-contact combination 4-B is used. The contact
combination used to generate the character “g” is 4-B0 which
implies contacting the left index finger 4 with the center conA
;F
3Y35
Figure 5. Wired Prototype: Additional contacts 1–10 lotact on the outside of the user’s left thumb B0 (see Figure 4).
*
cated on the fingernails and/or neighboring finger seg
5Y6
Note that, due to ergonomic reasons, finger-thumb contact 1ments of each hand for input of numbers 0–9 and (upon
P/'
P'
C is not used to generate “z” but rather 8-E. Accordingly, “;”
using ‘Shift’) several ‘special’ characters. Data input is
$ 3Y0 and “ :” are generated by one of the supplemental contacts
achieved
by closing
contacts
with
thumb
1–5435
contact
A
located on the sides of the fingers as shown in Figure 4.
and
6-10 with thumb contact B, respectively.
contacts
9" The previously described method to generate numeric
data input (using contacts 1-10 in Figure 5), avoids the need
As illustrated in Figure 5, additional contacts (denoted by
of placing four thumb contacts on the inside of each thumb
numbers 1-10) may be located on the fingernails (for the inin order to mimic a total of four rows on a standard keydex fingers also on the neighboring finger segment) for input
board. The latter is problematic considering the finite size
of digits 0-9. Upon using the “Shift” contacts, several special
of the electrical contacts and the limited area on the inside
characters can be generated using these additional contacts.
thumb which is easily accessible by the finger contacts on
Data input for digits 1-5 is achieved by closing finger conthe palm-side tips of the fingers.
tacts 1-5 in Figure 3 with thumb contact A and data input
for digits 6-9 and 0 is achieved by Closing finger contacts
4.3 Key-Map Wireless Interface
6-10 in Figure 3 with thumb contact D. Using contacts 110 (of Figure 5) for input of digits 0-9 still uses fingering
Table 1 illustrates an alternate keyboard mapping based
or finger movement analogous to the fingering on a standard
on four thumb contacts (T1-T4) and six finger contacts (LP,
keyboard; however, now signal input is not achieved by comLR, LM, LI1-LI3) placed on the pinky, ring, middle and inbining a contact on the inside tip of the fingers with a thumb
dex fingers as shown in Figure 6. Contact T1 is placed on
contact but rather by combining a contact on the outside or
the thumb nail, T2 on the thumb pad, T3 on the center of the
nail-side of the finger tips with the thumb contact located on
thumb pad and T4 on the bottom of the thumb pad (just below
the palm-side tip of the thumb of the same hand. In other
the knuckle). As for the prototype key-map described in Secwords, when using contacts 1-10, the thumb contacts A, D
tion 4.2, finger-contacts represent rows and thumb-contacts
represent the upper-most or fourth row of keys on a standard
represent columns on a standard keyboard. Here, a new lakeyboard carrying the number keys.
belling is employed to allow the reader to easily distinguish
2
3
8
9
4
1
5
6
7
10
LP
1
Q
A
Z
T1
T2
T3
T4
LR
2
W
S
X
LM
3
E
D
C
LI1
4
R
F
V
LI2
5
T
G
B
LI3
RI3
Shift
Space
Shift
Space
RI2
6
Y
H
N
RI1
7
U
J
M
RM
8
I
K
<,
RR
9
O
L
>.
RP
0
P
”0
?/
Table 1. The KITTY keyboard to hand mapping based on four contacts per thumb (T1-T4) and six additional finger contacts for
the left (L) and right hand (R).
between both mapping schemes.
LM
LI1
LR
RI1
LI2
RI2
LI3
RI3
RM
RR
LP
RP
T1
T1
T2
T3
T4
T2
T3
T4
Right Hand
Left Hand
Figure 6. Finger contacts for wireless KITTY implementa-
the top knuckle, LI2 or RI2, corresponds to the second column of keys (”t”, ”g”, and ”b” for the left index and ”y”,
”h”, and ”n” for the right index) that the index finger presses
on the keyboard during touch-typing. The location of this
contact follows the sequential feel of moving from finger
to finger and column to column. LI3 and RI3 are used for
the ”Shift” and ”Space” keys, which have no finger contacts
that match the row-column arrangement. Keeping in mind,
however, that the ”Space” key sits below the ”Shift” key, the
third index contact is paired with T4 for ”Shift” and T3 for
”Space”. The T2 contact is reserved for a missing number
key as discussed below.
tion.
TI-2
T2-W
T3-S
T4-X
TI-3
T2-E
T3-D
T4-C
TI-1
T2-Q
T3-A
T4-Z
TI-7
T2-U
T3-J
T4-M
TI-4
T2-R
T3-F
T4-V
T2-Y
T3-H
T4-N
T2-T
T3-G
T4-B
T1
Left Hand
TI-9
T2-O
T3-L
T4-> .
T2-6
T3-Shift
T4-Space
T2-5
T3-Shift
T4-Space
T2
T3
T4
TI-8
T2-I
T3-K
T4-< ,
TI-0
T2-P
T3-" '
T4-? /
T1
T2
T3
T4
Right Hand
Figure 7. Keyboard to glove mapping of wireless KITTY
implementation.
According to the alphanumeric mapping summarized in
Table 1 and illustrated in Figure 7 each finger contact is
linked to the column of characters it would press on a keyboard using touch-typing. For example, the middle finger of
the left hand, LM, is linked to ”e”, ”d”, and ”c”, the same
keys this finger would normally press on a keyboard using
touch-typing. The three thumb contacts match the three letter rows on a normal keyboard. The contact at the tip of
the thumb, T2, corresponds to the top row of the keyboard,
T3 to the base row, and T4 to the bottom row. Using the
left middle finger again as an example, LM would yield ”e”
when touched to T2, ”d” when touched to T3, and ”c” when
touched to T4.
In this modified key-map, each index finger has two more
contacts on the side of the finger, LI2, LI3, RI2, and RI3,
spaced above and below the top knuckle. The contact above
Numeric input is provided by simply placing another contact, T1, on the back of the thumb on the nail. The finger contacts are touched to T1 consecutively in keeping with the traditional keyboard assignments. This configuration for numbers works also well for non-touch-typists since it follows
a traditional method of finger counting. Starting at the left
pinky, LP touches T1 to generate the number 1, and then LR,
LM, and LI1 follow for numbers 2 to 4. The number 5 is the
missing number that uses the LT3 to T2 combination. Although an odd combination, it still follows the flow of the
series as the tip of the thumb is used for number 5 like the
finger pads for the other numbers. The right hand picks up
number 6 in a mirror series to the left hand, with 6 to 0 moving through RT3, RT1, RM, RR, and RP in that order.
As with the key-map discussed in the previous section, a
drawback to the described key-to-glove mapping is the bottom row key on the pinky. Touching LP or RP to T4 has
an “awkward feel” to it. A momentary slight strain is felt
in stretching the pinky towards the lower pad of the thumb.
Like learning an awkward chord on a guitar, it is possible
that the strain will lessen with prolonged use as the hand adjusts towards this unaccustomed position. One could argue
that the keys ”Z” and ”? /” are used so infrequently that there
is no concern regarding continuous straining of the tendons.
If it is found, however, that the awkwardness and discomfort
of the particular finger combination becomes an issue, an additional contact could be placed on the sides of the middle
fingers to offer an alternative and a more comfortable combination.
5
Hardware Implementation
Two different implementation strategies were investigated
supporting a wired and wireless configuration of the KITTY
device.
5.1
Wired Prototype
Initially, the KITTY interface was implemented with minimal effort using the “Virtually Indestructible” foldable keyboard by GrandTecT M (see Figure 8). The mentioned keyboard was chosen due to the 100 % correspondence between
the circuit layout on the two membranes of the keyboard
and the key-mapping described in Section 4.2, with fingercontacts representing keyboard columns and thumb-contacts
representing keyboard rows. Wires were attached to the corresponding locations on the keyboard membrane circuitry using self-adhesive copper tape. Wirewrap was used to keep the
design sleek. The wire ends were then collected and attached
to two golf gloves according to the layout shown in Figure 4.
Figure 8. First prototype system utilizing traditional roll-up
keyboard as the keyboard encoder.
Clearly, the wiring or printed circuitry on the two
membranes of the ”Virtually Indestructible Keyboard” by
GrandTec perfectly matches (without modifications) the
mapping proposed in Section 4.2. The GrandTecT M keyboard uses two membranes, membrane A onto which the
wires for the finger contacts are attached and membrane B
onto which the wires for the thumb contacts are attached.
The contacts for the letters on membrane A are connected in
a column format and the contacts on membrane B are connected in a row format.
5.2
Wireless Interface
Once a finger-thumb contact has been established according to the mapping schemes discussed earlier, various methods can be used to generate the signals. For example, wires
from the contacts may be connected directly to a keyboard
encoder. The keyboard encoder may be located on top of the
user’s hand. For example, the keyboard encoder may be attached to a glove, such as the one shown in Figure 9. The
signal that is generated can then be transmitted to the computing device via wire or wireless, e.g., infrared (IR) or the
like. The computing device can then display the character
represented by the signal.
Keyboard Encoder – A 8x12 keyboard encoder would suffice for the keys mapped according to Section 4.3. However,
a 8x16 matrix is preferred; the larger matrix allows future
functionality for additional keys such as macro keys. The
smallest possible platform and low power consumption are
other factors in deciding which keyboard encoder to use. A
small platform is necessary to keep the wearable device unobtrusive and comfortable to wear for the user. Since power
would be supplied by batteries, it is important that power
consumption be minimized for longer battery life. For the
above reasons, the Motion Encoder ME1001A keyboard encoder was selected for the wireless KITTY interface.
Thumb contacts are wired to the row pins of the encoder,
while the finger contacts connect to the column pins. The
encoder transmits a single keystroke as a TTL serial signal
at 2400 bps with no parity, 8 data bits and 2 stop bits. A
hexadecimal number from 0x00 to 0x7f (decimal 0 to 127)
indicates which key on the matrix was selected. This number
is transmitted as part of a byte and occupies bit positions 0
to 6. The seventh bit indicates the key state of key down
or up. The recommended 6MHz resonator is attached to the
encoder. Other additional components that must normally be
added to other encoders, such as pull up resistors, are internal
to this encoder and helps keep the size of the device as small
as possible.
Wireless Signal Transmission – The KITTY interface was
created with portability and convenience in mind. Accordingly, it was our goal to add wireless signal transmission capabilities to a second generation KITTY interface providing
data-input via gloves which are physically disconnected of
the computing device itself. From the wide range of wireless
technologies available, BluetoothT M technology has been
used. BluetoothT M wireless technology is being used extensively in wireless and portable devices like cell phones
and PDA’s, and has become a wireless standard. It provides
plug and play type connectivity between any two or more
BluetoothT M devices. One of the goals behind the ongoing development of KITTY is to provide an input device that
could work with PC’s, PDA’s, Pocket PC’s and any other device that requires alphanumerical input. Hence the use of
BluetoothT M wireless technology was appropriate for this
application.
Within the new wireless KITTY data-entry system, the
serial output of the encoder is fed into a BluetoothT M
RS232/UART DCE Terminal that is configured to communicate serially with the encoder. For this prototype the Wire-
less Futures BlueWAVE RS232 PCB DCE Terminal is used
that connects to a generic BluetoothT M USB dongle/adapter
that supports the BluetoothT M serial port profile. This DCE
terminal was chosen for sake of programmability, low power
consumption and its size. The prototype has been developed
for a Windows 9x and Windows NT platform. The adapter
used in this case is the D-Link DBT-120 BluetoothT M USB
adapter. It does not require any external power supply, but
receives all necessary power from the USB port. The DCE
terminal requires a supply voltage that is between 3.3V to
9V DC, which is accomplished by using four AA Ni-MH
rechargeable batteries that provide 1900mAh each and a total potential difference of 5.2V. The terminal’s average current consumption is 1.8mA when it is in the idle mode and
60.9mA while it is transferring data.
The DCE terminal measures about 40mm x 30mm x
5mm, and the antenna extends about an inch over that. The
voltage is applied across pin 1 and pin 12 of the DCE terminal with pin 1 connected to the positive terminal and pin 12
connected to the negative terminal of the 5.2V battery pack.
The serial data is connected to the TxD pin, which is pin 6 of
the DCE terminal. Pins 3 and 4 are connected to each other
in order to disable any hardware flow control. The serial data
signal and the voltage supply are connected to the DCE terminal using a 0.05” (1.27mm) pitch ribbon cable.
Both the adapter and the DCE terminal conform to the
BluetoothT M v1.1 standard. When power is switched on
for the DCE terminal it will become “discoverable” by other
BluetoothT M devices. Since it is a DCE terminal it can only
be used as a slave module. The BluetoothT M serial port profile creates a virtual COM port on the PC, which is used for
connecting the two devices. The virtual COM port is similar to a physical COM port, but instead of having a wire for
transferring the data, the data is transferred wirelessly. The
serial data from the encoder is received by the terminal and
is transmitted to the adapter. The virtual COM port is also
configured to the same setting as the encoder and the terminal. Then a software driver decodes the data received and
produces the appropriate keystroke. The information is transmitted at 2400MHz to 2483.5MHz frequency, as specified by
the BluetoothT M standard. The driver is responsible for configuring the virtual Com port, waiting for the data, decoding
the data and producing the appropriate character using the
keybd event() system call.
5.3
Design Concepts
Various designs have been considered for implementing
the described KITTY interface. The employed prototype design shown in Figure 1 and Figure 9 uses contacts attached
to gloves (only one glove is shown in the figures carrying
only contacts for alphabetic character input). The contacts
are connected by wires to the encoder and (for the wireless
implementation) transfer electronics. Preferably, the gloves
are fairly thin and lightweight, thereby allowing the user to
easily manipulate his fingers.
An alternative configuration also shown in Figure 9 has
contacts mounted on clips or half-rings. The clips are attached to a skeletal structure. The keyboard encoder and
transfer electronics (e.g., wireless transmission electronics)
are also mounted to the skeletal structure. Figure 9 shows an
exemplary skeletal structure for the left hand. The skeletal
structure rests against the user’s palms (or alternatively on
the back of the user’s hand) and the contacts are positioned
on the user’s fingers by placing the clips (on which the contacts are mounted) around the user’s fingers and thumbs. The
clips can be attached to the skeletal structure in a way that
allows adjustment of the positions of the clips in order to
accommodate different finger and thumb lengths. The skeletal structure can also be adjustable in order to accommodate
hands of varying sizes. The main purpose of the skeletal
structure is to preserve the integrity of the overall system.
Figure 9. Top: Glove-based prototype design. Bottom:
“Click-on” skeletal-structure design.
The designs discussed here can also be configured to
be used on one of the user’s hands to simulate a “HalfKeyboard”. The location of contacts on the hand being used
(either right or left) would be the same as in the configuration of contacts when using finger/thumb based touch-typing
in the two-handed configuration described above.
6
Results
With recent advances in portable computing and in particular the design of pocket PCs and eye-glass displays, efforts aimed at the design of portable input devices and techniques have increased. While voice recognition is becoming
a strong contender for next generation input technology, privacy concerns and ambient noise problems are still a challenge and alternative approaches are required. The presented
KITTY interface consists of a finger-mounted device mimicking finger movements similar to traditional touch-typing
in order to provide alphanumerical data input. This low cost
device guarantees a rapid learning curve for everyone experienced in touch-typing while offering significant flexibility.
Two prototypes were build one using a PSII and one using a
USB version of the GranTec keyboard. Priced at around $50
US for parts the KITTY device proved to be a very cost effective solution. The proof-of-concept implementation of the
KITTY device was tested with wearable devices and desktop
systems and demonstrated its effectiveness.
References
[1] Handykey Corporation. www.handykey.com, 2003.
[2] M. E. Horn. Single-hand mounted and operated keyboard.
U.S. Patent 5,552,782, September 1996.
[3] R. B. Howard. Wrist-pendent wireless optical keybaord. U.S.
Patent 6,097,374, August 2000.
[4] Lightglove. www.lightglove.com, 2003.
[5] MachineBrain.com. www.machinebrain.com, 2003.
[6] E. Matias, I. S. MacKenzie, and W. Buxton. Half-QWERTY:
Typing with one hand using your two-handed skills. In Proceedings of ACM CHI’94 Conference on Human Factors in
Computing Systems, volume 2 of INTERACTIVE EXPERIENCE, pages 51–52, 1994.
[7] E. Matias, I. S. MacKenzie, and W. Buxton. A wearable computer for use in microgravity space and other non-desktop environments. In Proceedings of ACM CHI 96 Conference on
Human Factors in Computing Systems, volume 2 of INTERACTIVE POSTERS: CHI in Space, pages 69–70, 1996.
[8] Matias Corporation. www.halfkeyboard.com, 2003.
[9] R. P. McCarty.
Wearables central: Softwear.
wearables.blu.org, 2003.
[10] V. R. Pratt. Thumb-code: A device-independent digital sign
language, July 1998.
[11] K. R. Prince. Finger-mounted computer input device. U.S.
Patent 5,581,484, December 1996.
[12] Senseboard Technologies. www.senseboard.com, 2003.
[13] Stanford Wearables. wearables.stanford.edu, 2001.
[14] The MicroOptical Corporation. www.microopticalcorp.com,
2003.
[15] S. Vance, L. Migachyov, W. Fu, and I. Hajjar. Fingerless
glove for interacting with a data processing system. U.S.
Patent 6,304,840, October 2001.
[16] A.
Wozniak.
Pic-key
chorded
keyboard.
http://mudlist.eorbit.net/ adam/pickey/, 1999.
Finger-Thumb Contact
1-B
2-B
3-B
4-B
4-B0
Character
A
S
D
F
G
(a) Left-hand base-line characters.
Finger-Thumb Contact
1-A
2-A
3-A
4-A
4-A0
Character
Q
W
E
R
T
(b) Left-hand top-line characters.
Finger-Thumb Contact
1-C
2-C
3-C
4-C
4-C0
Character
Z
X
C
V
B
(c) Left-hand bottom-line characters.
Finger-Thumb Contact
5-E0
5-E
6-E
7-E
8-E
Character
H
J
K
L
Z
(d) Right-hand base-line characters.
Finger-Thumb Contact
5-D0
5-D
6-D
7-D
8-D
Character
Y
U
I
O
P
(e) Right-hand top-line characters.
Finger-Thumb Contact
5-F0
5-F
6-F
7-F
8-F
Character
N
M
<,
>.
?/
(f) right-hand bottom-line characters.
Table 2. Prototype Key-Map for KITTY device.