view as pdf

Machine Learning for
Sensorimotor Procesing
Daniel D. Lee
Biological inspiration
u
Biological motivation for the Wright brothers in designing
the airplane
Robot dogs
Custom built version
u
Sony Aibos
Platforms for testing sensorimotor machine
learning algorithms
Robot hardware
u
Wide variety of sensors and actuators.
Perception and motor control
u
Wide variety of sensors and actuators are readily available
Legged league
u
u
u
u
u
u
Each team consists of 4 Sony Aibo
robot dogs (one is a designated
goalie), with WiFi communications.
Field is 3 by 5 meters, with orange
ball and specially colored markers.
Game played in two halves, each 10
minutes in duration. Teams change
uniform color at half-time.
Human referees govern kick-off
formations, holding, penalty area
violations, goalie charging, etc.
Penalty kick shootout in case of ties
in elimination round.
Recently implemented larger field and wireless
communications among robots.
Upennalizers in action
u
GOOAAALLL! 2nd place in 2003.
Robot software architecture
Perception
(Sensors)
u
Sense-Plan-Act cycle.
Cognition
(Plan)
Action
(Actuators)
Perception
View from Penn’s
omnidirectional camera
Robot vision
Color segmentation: estimate P(Y,Cb,Cr | ORANGE) from training images
Region formation: run length encoding, union find algorithm
Distance calibration: bounding box size and elevation angle
Camera geometry: transformation from camera to body centered coordinates
u
Tracking objects in structured environment at 25 fps
Image reduction
Deterministic position:
L
È(53,65,27) (52,67,35)
˘
Í(48,68,31)
˙
O
Í
˙
M
(250,213,196)˙˚
ÍÎ
u
(xball , yball )
Probabilistic model (Kalman):
P(xball , yball )
144 ¥176 ¥3 RGB image to 2 position coordinates
Image manifolds
È0.0˘
Í0.5˙
Í ˙
Í0.7˙
Í1.0 ˙
Í ˙
Í M ˙
Í0.8˙
Í ˙
Í0.2˙
Í0.0˙
Î ˚
Pixel vector
u
Variation in pose and illumination give rise to low
dimensional manifold structure
Matrix factorization
Data matrix:
ÈÊ M ˆ
r ˜
Í
Á
X = x1
ÍÁ ˜
ÍÎÁË M ˜¯
ªWV
˘
ÊM ˆ
Á xr ˜ L˙
˙
ÁÁ 2 ˜˜
˙˚
ËM ¯
v1
vM
W11
x1
WNM
x2
x3
x4
xN
M
xi = Â Wik vk
k =1
u
u
Learning and inference of hidden units
is equivalent to matrix factorization.
Prior probability distribution of hidden variables.
Linear factorization (PCA)
ÈÊ M ˆ
r
X = ÍÁ x1 ˜
ÍÁ ˜
ÍÎÁË M ˜¯
˘
ÊM ˆ
Á xr ˜ L˙
˙
ÁÁ 2 ˜˜
˙˚
ËM ¯
u
W ,V
subject to constraints:
M
W TW = I
k=1
VV T diagonal
Xij ª Â WikVkj
u
min X - WV
Equivalent to singular value decomposition.
Degeneracy eliminated by constraining W
orthonormal, rows of V orthogonal.
PCA representation
W: 49 hidden units
V
¥
X
=
Original:
u
Linear model: representation uses positive and negative
combinations to reconstruct original image.
Vector quantization
M
Xij ª Â WikVkj
k=1
min X - WV
W ,V
Winner take all constraint:
ÈÊ 0 ˆ
ÍÁ 0 ˜
V = ÍÁ ˜
ÍÁ 0 ˜
ÍÁ 1 ˜
ÎË ¯
˘
Ê0ˆ
˙
Á1˜
Á ˜ L˙
˙
Á0˜
˙
Á0˜
Ë ¯
˚
Unary vectors
u
u
Columns of V constrained to be unary.
Winner take all competition between hidden
variables in modeling data.
VQ representation
W: 49 hidden units
V
¥
X
=
Original:
u
Weights cluster data into representative prototypes.
Constraints and representations
Model
Constraints
Representation
VQ
vk = d kk'
very sparse
r r
v ⋅ v¢ = 0
distributed
for some k’
PCA
u
u
Hard constraints give rise to very sparse representations, with bases
identifiable as prototypes.
Soft constraints lead to a very distributed representation but with
features difficult to interpret.
Biological motivation
u
u
u
Firing rates of neurons are nonnegative
Synapses are either excitatory or inhibitory
What are the functional implications of these
nonlinear constraints for learning?
Nonnegative constraints
Hidden Prior:
M hidden variables
P (v) > 0 iff v ≥ 0
v1
Excitatory Weights:
Wij ≥ 0
Visible Variables:
M
xi ª ÂWik vk
k =1
u
vM
W11
WNM
Noise
s1
x1
s2
x2
s3
x3
s4
N visible variables
Nonnegative hidden variables and weights.
x4
sN
xN
Nonnegative factorization algorithm
min
W ≥0,V ≥0
F = Â X ij - (WV )ij
ij
Derivatives:
∂F
= (W T WV )ij - (W T V )ij
∂Vij
∂F
= (WVV T )ij - ( XV T )ij
∂Wij
u
2
Inference:
Vij ¨ Vij
(W T X )ij
(W T WV )ij
Learning:
Wij ¨ Wij
( XV T )ij
(WVV T )ij
Multiplicative algorithm can be interpreted as diagonally
rescaled gradient descent.
Convergence proof
G ( v , vn )
Upper bound:
F ( v ) £ G ( v , vn )
F (v)
vnvn +1vmin
F ( vn ) = G ( vn , vn )
v
Update rule: vn +1 = arg min G (v, vn )
v
F (vn +1 ) £ G (vn +1, vn ) £ G (vn , vn ) = F (vn )
u
Minimizing auxiliary function G is guaranteed to
monotonically converge to local minimum of F.
NMF learning
NMF representation
W: 49 hidden units
V
¥
X
=
Original:
u
u
Learned weights decompose the images into their
constituent parts.
Nonnegative constraint allows only additive combinations.
Learning nonlinear manifolds
Kernel PCA, Isomap, LLE, Laplacian Eigenmaps, etc.
u
Many recent algorithms for nonlinear manifolds.
Locally linear embedding
r
r
E(W ) = Â Xi - Â Wij X j
i
j
r
r
F(Y ) = Â Yi - Â WijY j
i
u
2
2
j
LLE solves two quadratic optimizations using eigenvector
methods (Roweis & Saul).
Translational invariance
u
Application of LLE for translational invariance.
Action
M. Raibert’s hopping
robot (1983)
Inverse kinematics
l1
q2 l2 q 3 l3
q1
H = R(q1 ) oT (l1 ) o R(q2 ) oT (l2 ) o R(q 3 ) oT (l3 )
u
Degenerate solutions with many articulators.
Eye movements
Eye muscles
(Yarbus, 1967)
u
Fast eye movements to scan visual environment
Neural integrator
Pastor et al., PNAS 91, 807 (1994)
Control of eye position
Line attractor
u
Low dimensional dynamics for motor control
Gaits
u
4-legged animal gaits
Walking
Inverse kinematics to calculate
joint angles in shoulder and knee
u
Parameters tuned by optimization techniques
Behavior
SRI’s Shakey (1970)
Finite state machine
Search for
ball
See
ball
Goto ball
position
Kick ball
u
Close
to ball
Event driven state machine.
Sensor-actuator mapping
Sensor space
u
Actuator space
Construct low dimensional representations for reasoning
about sensor stimuli to motor responses
Image correspondences
Object 1
u
Object 2
Correspondences between images of objects at same pose
Data from the web...
u
http://www.bushorchimp.com
Learning from examples
Given Data (X1,X2):
N1 examples
n labeled
of object 1
correspondences
(D1 dimensions)
N2 examples
of object 2
(D2 dimensions)
X1
D1¥n
D1¥N1
?
X2
D2¥n
?
D2¥N2
u
Matrix formulation (n << N1, N2 )
Supervised learning
D1¥n
D1¥N1
?
D2¥n
?
D2¥N2
Training
Data
u
Fill in the blanks:
(D1 +D2 ) ¥ n labeled data
D1 ¥ D2 parameters
Problem overfitting with small amount of labeled data
Supervised backprop network
Original:
Reconstruction:
Original:
Reconstruction:
u
15 hidden units, tanh nonlinearity
Missing data
D1¥n
D1¥N1
?
D2¥n
?
D2¥N2
D1+D2
EM algorithm:
Iteratively fills in missing data statistics, reestimates
parameters for PCA, factor analysis
u
Treat as missing data problem using EM algorithm
EM algorithm
X1
min min  Xi - WiYi
Yic =Y jc Wi
Yi ¨ RYi
2
i
X
Wi ¨ Wi R -1
so that Y T Y = I
X2
Y1
Y
Y2
u
Alternating minimization of least squares objective
function.
PCA with correspondences
Original:
Original:
u
15 dimensional subspace, 200 images of each object, 10 in
correspondence
Common embedding space
Two input spaces
Common low dimensional space
LLE with correspondences
r1
1
1 r1
E(W ) = Â Xi - Â Wij X j
i
j
r1
1 r1
F(Y ,Y ) = Â Yi - Â WijY j
2
r2
2 r2
+Â Yi - Â Wij Y j
2
1
j
r2
2
2 r2
E(W ) = Â Xi - Â Wij X j
i
2
2
i
2
i
Correspondences:
u
j
j
r1 r 2
i ΠSc : Yi = Yi
Quadratic optimization with constraints is solved with
spectral decomposition
LLE with correspondences
Original:
Original:
u
8 nearest neighbors, 2 dimensional nonlinear manifold
Summary
u
u
u
u
Adaptation and learning in biological systems for
sensorimotor processing.
Many sensory and motor activations are described by an
underlying manifold structure.
Development of learning algorithms that can incorporate
this low dimensional manifold structure.
Still much room for improvement…