An Advanced Real-Time Binocular Eye Tracking System Using a

An Advanced Real-Time Binocular Eye Tracking System Using a High Frame-Rate Digital Camera!
高速撮影カメラを用いたリアルタイム両眼眼球運動計測システム
Keiji Matsuda1,Kenji Kawano2!
松田圭司1、河野憲二2
Human Technology Research Institute, AIST, Tsukuba, Japan, 2. Dept.Integ Brain Sci. Grad.Sch.Med. Kyoto University, Kyoto, Japan!
1. 産総研ヒューマンライフ 2. 京都大院医認知行動脳科学
Summary!
!
We developed a new binocular eye tracking system by adopting a USB-3.0 digital camera that provides high sensitivity, resolution and frame rate. The system is non-invasive and inexpensive and can be used for monkeys and humans. Infrared light
illuminates two eyes and the reflected image of the iris and the black image of the pupil of each eye are captured by the camera. The center of the pupil is calculated and tracked over time. The system was originally developed to measure movements of one
eye. In the present study, we have improved it for binocular eye movements by adopting a new wide-field, hi-resolution, and hi-frame-rate camera. Since the camera has a 2048x2048 pixels resolution, we can capture the images of both eyes
simultaneously and calculate positions of the two eyes at each frame. The eye position data can be read out on line via computer network. The adoption of the WINDOWS 7/8/8.1 x 64 as the operation system makes this binocular eye tracking system userfriendly. Because of the high frame rate of the digital camera, the sampling rate of the system can be as high as 700Hz. By using this system, we succeeded in characterizing vergence eye movements of humans when ocular fixation shifts between two
targets placed at different distances in the 3-D space. !
Supported by KAKENHI (24650105).!
Windows 7, 8, 8.1 x64
model!
Company!
Option/Memo!
Grasshopper3 GS3-U3-41C6NIR-C!
Point Grey Research, Inc.! USB 3.0 Cable!
Ai AF Micro-Nikkor 60mm f/2.8D!
Nikon!
Camera!
Left Horizontal angle plot
Lens!
C-Mount ADAPTER for Nikon!
USB3.0
Kenko!
Gaze angles and posi*on calculated by the data measured from the synthe*c eyes. LeV and right horizontal angle mean errors are -­‐0.037, -­‐0.084 [deg]. Their standard devia*ons are 0.036,0.030 [deg]. Maximum error was 15.1mm at depth 687.1mm, aVer applied a 10 points (20ms) moving average filter, maximum error decreased.
for Nikon Micro-Nikkor 60mm!
actual measurement= 0.996 theoretical value+(−0.022)
R^2= 1.000
10
−1
for Nikon Micro-Nikkor 60mm!
Infrared filter!
AVC-1004!
akibasecurity.com!
Infrared illumination!
Infrared illumination
PC
Stimulus display
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Windows7, 8, 8.1 x64
−2
●
●
●
●
●
1500
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
7
6
●
●
●
●
●
●
5
●
●
●
●
4
●
●
●
●
●
●
●
●
3
−3
actual mesurements [deg]
Kenko!
actual mesurements [deg]
62S PRO1D R-72!
Camera
Depth [Z position]
actual measurement= 0.966 theoretical value+(11.438)
R^2= 0.999
●
●
●
●
●
●
8
eye
0
●
●
●
●
9
Lens mount converter!
Right Horizontal angle plot
actual measurement= 0.999 theoretical value+(−0.083)
R^2= 1.000
●
●
●
●
●
●
●
●
●
●
●
●
●
actual mesurements [mm]
Output
DA Converter TCP/IP File
theore*cal depth[mm] mean error[mm] maximum error[mm] maximum err [mm] 1718.7 52.2 131.3 75.1 1145.7 29.9 60.5 39.2 859.1 12.9 28.9 19.6 687.1 1.7 15.1 6.3 572.4 8.0 16.5 11.5 490.5 1.4 6.9 3.2 429.0 1.3 5.7 3.1 342.9 0.4 2.8 1.3 285.4 2.0 4.2 2.8 213.5 0.2 1.4 0.5 170.1 0.3 1.0 0.5 Result 1!
System outline!
●
●
●
●
●
●
●
−4
●
●
●
●
●
−5
●
●
●
●
●
−6
●
●
●
●
●
●
−7
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
1000
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
2
−8
●
●
●
500
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
1
●
−9
●
●
●
●
●
●
●
●
●
−10
0
Methods!
0
2
3
4
5
6
7
8
9
10
−10
−9
−8
−7
theoretical values [deg]
⇡
a=
HL
2
⇡
b = + HR
2
c=⇡ a b
A
B
C
=
=
sin a sin b sin c
c
+
+
A
B
HL
HR
a
C b
（0,0)
（0,-­‐C/2)
Horizontal X
（C/2,0)
a, b, c: the three angle of the triangle. A, B, C: the three sides of the triangle. 4
4
3
3
2
2
1
●
0
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
−1
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
−2
●
●
●●
●● ●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
0
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
−1
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
−3
−2
−1
0
500
1000
1500
theoretical values [mm]
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●●
●●
●
−2
●●
●
●
●●
●
●●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●●
●●●
●●●
●●●
−3
●
−4
●
●●
●●
●
●
●
●
● ●●
●● ●
●●●
●●
●
●
●●
●
●●
● ●●
●●
●
●
●●
● ● ●
●
● ●● ●
●
●
●
●●●●●
● ●● ●
●
●
●
● ●●
●
●
●●●●●●
●●●
●●
●
●
●●
● ●●●●
●
●
●●
●●
●
●
●
●
●●●●
●
●
●
●
●●
●
●●
●●●●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●●
●
●
●
●● ●
●●●
●●●●●●●●
●●
●●
●
●
●●●●
●
●
●
●●
●
●
●
●
●●
●●
●●
● ●
●
● ●●●
●
●
●
●●●●●●
●
●
●
●
●●
●
●●
●●
●
● ●
●
●
●●●●
●
●● ●●
●
●
●●
●●
●
●●●
●
●
●
●●
●
●
●
● ●●●
●●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●● ●
●●
●
●
● ●●●
●
● ●●●
●●
●
●
●
●
●
● ●●
●
●
●
● ●
●
●●●
●●● ●
●
●
●●●●●● ●
●●●●
●
●●
●● ●
●●
●
●●
●●
●
●
● ●
●
●●●
●
●●
●
● ●●●●●
●
●● ●
●
● ● ●
● ●
●
●
● ●●
●●●●●
●
●
●
●●●●
●●
●
●●●
● ●
●
●●
● ● ●●
●●● ●
● ● ●
●
●
●
● ●
●●●
●●
●● ● ●
●● ●
●●
● ● ●●
●●●
●
● ●
−4
●
●● ●
●●
●
−5
−5
0
500
1000
1500
0
500
1000
Depth [Z position] [mm]
C
(x, z) = (B cos a
, B sin a)
2
cos a sin b C
sin a sin b
= (C
,C
)
sin c
2
sin c
4
2
2
0
-2
-4
-4
1000
500
tal
tal
VL+VR
y = z tan(
)
2
1500
on
on
2
-4
-2
0
riz
riz
-4
-2
0
Ho
Ho
ver*cal Y
0
-2
4
1500
Depth [Z position] [mm]
4
0
y
●
●
● ●
●
●
●
● ●
●
●
●
● ●
●●
● ●●●●
●●
●● ●
●
●
● ●●●
●
●●
● ●
●
●● ●
●
●●●●
●
●●●
●●
●
●
●
●
●
●
●
●●● ●
●
●
●●
●
●●●●
●●
●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●● ●
●
●● ●
●
●●
●●●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●●
●●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●●●
●
●●●
●
●
●
●●
●●
●●
●
● ●●
●
●
●
●
●●●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
● ●●●
●●
●
●●●
●●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●●
● ● ●
●
●●
●
●
●
●
●
●
●
●●●●●●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●●●
●
●
●●●
●
●
●●
●
●
●●
●●●●
●
●
●
●
●●●●
● ●
●●● ●●●●●●●● ●●
●●●●●
●
●●●●●
●
●●
●●
●● ● ●
●
●
●●●
●●
● ● ●
●●●
●●
● ●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●● ●●
●
●●
●
● ●●
●
●
●● ●
●●
●
●
●
●
●●
●
−3
Then we calculated the ver*cal posi*on. The ver*cal gaze angle of the leV eye (VL) and that of the right eye (VR) were calculated by the our soVware. We used average of them. gaze posi*on
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
−4
1
Vertical
-­‐
5
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
−5
YZ plot
5
Vertical
depth Z
gaze posi*on
XZ plot
Horizontal [X position] [mm]
−6
theoretical values [deg]
Vertical [Y position] [mm]
We calculated the horizontal/ver*cal gaze angle of each eye by processing its video-­‐image. The origin was set on the center of eyes. The horizontal gaze angle of the leV eye (HL) and that of the right eye (HR) were calculated by the our soVware. “HL” is posi*ve and “HR” is nega*ve in the figure. C corresponds to the inter-­‐ocular distance, ~60mm, though differences can be detected among individuals. According to the law of sines, the target posi*on of the gaze in X-­‐Z plane can be described as follows. -­‐
1
●
●
●
●
●
●
●
Depth
2
4
500
0
original 3-­‐D Space plot
1500
1000
Depth
applied a moving average filter 3-­‐D Space plot
Result 2!
Gaze posi*ons of a subject applied a 10 points (20ms) moving average filter.
B
VL+VR
2
（0,0)
Eye movements according to Yarbus (1957)
z
+
-­‐
100
depth Z
80
60
Vertical
40
Experiment １!
20
We evaluated system accuracy by using the synthe*c eyes. We set leV and right eyes horizontal angle {1, -­‐1}, {1.5, -­‐1.5}, {2.0, -­‐2.0}, {2.5, -­‐2.5}, {3.0, -­‐3.0}, {3.5, -­‐3.5}, {4.0, -­‐4.0}, {5.0, -­‐5.0}, {6.0, -­‐6.0}, {8.0, -­‐8.0}, {10.0, 10.0} [deg], ver*cal angle 0 [deg]. We measured leV and right eyes’ gaze angles and posi*ons for 1 second at each sejng.
0
left eye
-150
synthe*c eyes
600
-50
500
0
400
50
Horizontal
150
200
Depth
700
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
100mm
50
●
600
●
500
center to up−left
center to up−right
up−left to center
up−right to center
● ●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
0
100mm
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●●
●●
●●
●●
●
●
●●
● ●●●●
●
●●
●
●
●
●
●
●
●
●
● ●
● ●
●
●●
●
● ●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●●
●● ●
●
●
●
●●
●
●●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●●●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
● ●
●●
●
●● ●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●● ●
●
●
●● ●●
●
●
●●
●
●
●
●● ●
●●
●
●
●
●
●
●
●
●
●
● ●●
● ●
● ●
● ●
● ●●
center to up
up to center
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
●●●
●
●
●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
● ●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●●●
●
●●
● ●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
● ●
●
●
●
●
●
●●
●●
●
●
●●
●
●●●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
● ●●
●●
●
●
●
●
●
●
●
● ●●●●●
●●
●
●
●
●
●
● ●
●● ●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●●
● ●
●
●
●
●●
●
●
●
●
● ●● ● ● ●
●●●
●
●
●
●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●● ●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●●● ●
●
●
●
●
●
●●
●
●
●
●
●
●● ●
●
●
●
●
●
● ●●●●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●
● ●●
● ●
●
●
●
●●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●●●
●
●
● ●
●
●
●
●●● ●
●
●●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●● ●
●
●
●
●
● ●●
●
● ●●
●
●
●
● ● ●
● ● ●
●
●
●
● ●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
● ● ●●
●●
●
●
●
●
●
●
●●
● ●
●
●
● ●
●●●
●
●
●
●
●
● ● ● ●●
●
●●
●
●●
● ●
●
●●
●
●
● ●
●●
● ●● ●●
●●
● ●
● ●● ●●
●
●●
● ●●
● ● ●●
●
●●
● ●
●
●
● ● ●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
● ● ●● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●● ● ●
●
●
●
●
●●
● ●●
●●
●
●
●
●
●● ● ●●
●
●
●
● ●●
●
●● ●
●●
●●
●
●
●
●●
●
●
● ●
● ●●●
●
●●
●
●
●
●●
●
●● ● ●
●
●●
●●
● ●
●
● ●
●
●
●●
●●
●
● ●●
● ●●●
● ●●
●
●
●
●
●
●●
●● ●
●
●●
●●
●●
●
● ●●
●
●●
●
●
●●
●
●
●●
●●
●●
●●
●
●
●
●●
●
●
●
●●
●●
●●
●
●
●
●
●●●
●
●●●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●●●
●
●
●●
●●
●
●
●●
● ●
●
●
●●
●
●
●
●
●
●
●●
●●●
●●
●
●
●
●
●
●
●●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Depth [mm]
100
Vertical [mm]
100mm
300
100
Experiment 2
We measured gaze posi*ons in 3-­‐D space, sampling frequency was 500Hz. The subject moved his gaze 5 ways. 1.  The center to leV, leV to center 2.  The center to up-­‐leV, up-­‐leV to center 3.  The center to up, and up to center. 4.  The center to up-­‐right, up-­‐right to center 5.  The center to right, right to center right eye
Reference: Enright JT, Changes in vergence mediated by saccades. J. Physiol. 350: 9-­‐31, 1984 Yarbus AL, Eye movements during changes of the sta*onary points of fixa*on. Biophysics 2: 679-­‐683, 1957 700
-100
left eye
right eye
400
300
●
●
center to up−left
●
center to up−right
●
up−left to center
●
up−right to center
●
center to left
●
left to center
●
center to right
●
right to center
600mm
200
200
300
400
500
Depth [mm]
600
700
−150
−100
−50
0
50
100
150
Horizontal [mm]
300mm
Conclusion!
The target posi*ons from the view point of the human subject.
The posi*ons of the visual objects.
By using the data measured from the synthetic eyes, we found that the system can measure the eye movement within 0.2 [deg]
accuracy (3sDs). The range of the accuracy is less than 15.1mm at 687mm from the subject and less than 2.8mm at 343mm.
By using this system, we succeeded in characterizing vergence eye movements of humans when ocular fixation shifts between
two targets placed at different distances in 3-D space.