VO Neurale Netzwerke und Biorobotik

Transcription

Helmut A. Mayer
Department of Organismic Biology
Department of Computer Sciences
University of Salzburg
WS 13/14
Outline
Introduction
Biological Neural Networks
Artifical Neural Networks
Mobile Autonomous Robots
Introduction
Topics
I
Artificial Neural Networks (ANNs)
I
I
I
I
Basic model of Biological Neural Networks (BNNs)
“Bits instead of chemicals“
ANNs enable computers to learn
Evolutionary Robotics
I
I
I
Evolving robot behavior
Different controllers (robot brains)
Neurocontrollers (ANN is robotic brain)
Introduction
Overview ANN
I
Artificial Neural Networks
I
I
I
I
I
I
ANN history: Hodgkin–Huxley, McCulloch–Pitts, Perceptron,
Adaline
Multi–Layer Perceptrons
ANN Training, Back–propagation
Spiking Neural Networks
Recurrent Networks, Hopfield
Introduction
ANN Application Example
I
Application: Weather Forecast
I
Goal: weather forecast of next day based on todays weather
I
Input: e.g., temperature, pressure, wind
I
Output: sunny, overcast, rain
I
Training data: historic weather data
I
Test data: historic weather data NOT presented in training
I
Test results are the only quality measure (!)
I
ANN does not need explicit domain knowledge (!)
Introduction
Hopfield Image Memory I
Introduction
Hopfield Image Memory II
Biological Neurons
I
Nervous System: Control by Communication
I
Massive Parallelism, Redundancy, Stochastic “Devices”
I
Humans: 1010 neurons, 1014 connections (conservative
estimation), 101,000,000 possible networks(!)
I
Neurons: cell body (soma), axon, synapses, dendrites
I
Membrane Potential of −70mV
sodium (Na+ ) and potassium (Ka+ ) ions (chloride Cl − ions)
I
Sodium pump constantly expells Na+ ions
I
Complex interaction of membrane, concentration, and
electrical potential
Schematic Neuron
Dendrites
Soma
Nucleus
Synapses
Axon
Electrical Signals
I
Potassium is in equilibrium, sodium NOT
I
Sodium conductance is a function of the membrane potential
I
Above threshold sodium conductance increases, ion channels,
depolarization, polarization, refractory period = action
potential
I
Frequency coding, is (all) information encoded in action
potentials?
I
Hodgkin/Huxley–Model (of voltage across membrane)
Hodgkin-Huxley Equation
I
Hodgkin and Huxley (1952), Nobel Prize in 1963
4
3
C dV
dt = Iext − gK n (V − VK ) − gNa m h(V − VNa ) − gL (V − VL )
I
I
I
I
I
I
I
I
I
I
I
I
V . . . Membrane Voltage [mV ]
µF
C . . . Membrane Capacity [ cm
2]
µA
Iext . . . Excitation Current [ cm2 ]
S
gK . . . Potassium Conductance [ cm
2]
VK . . . Potassium Equilibrium Voltage [mV ]
n. . . Potassium Gate Activation [1]
S
gNa . . . Sodium Conductance [ cm
2]
VNa . . . Sodium Equilibrium Voltage [mV ]
m. . . Sodium Gate Activation [1]
h. . . Sodium Gate Inactivation [1]
S
gL . . . Leak Conductance [ cm
2]
VL . . . Leak Equilibrium Voltage [mV ]
Electrical Membrane Circuit
I ext
OUT
U Na
UM
UK
UL
CM
G Na
GK
IN
GL
Real and model spike
Measured Data
Evolved Model
40
V [mV]
20
0
-20
-40
-60
-80
0
2
4
6
8
t [ms]
10
Synapses
I
Electrochemical processes, neurotransmitters
I
Connect axon–dendrites (but also axon–axon,
dendrites–dendrites, synapses–synapses)
I
Spatio–temporal integration of action potentials
I
Excitatory and inhibitory potentials
postsynaptic duration ∼ 5 ms
I
Neurotransmitters influence threshold (permanent changes =
learning = closing/opening of ion channels)
Neuron Models
I
Level of Simplification?
I
McCulloch and Pitts (1943), Logic Model
Binary signals, no weights, simple (nonbinary) threshold
Excitatory and inhibitory connections (absolute, relative)
Addition of weights
I
Rosenblatt (1958), Perceptron
Real–valued weights and threshold
Generic Neuron Model
I
Generic Connectionist Neuron
Nonlinear activation (transfer) function
Widely used in todays ANNs
oi
net i
oj
I
w i,j
oi
Next generation: spiking neurons, hardware neurons, biological
hardware
Networks
I
Mysticism of Neural Networks
I
Functional Model: f : Rn → Rm
node structure, connectivity, learn algorithm
I
“Black Box Syndrom”, unexplicable ANN decisions
I
Basic Structures: Feed–Forward (MLPs, Kohonen),
f = f (g (x))
Recurrent (Recurrent MLPs, Hopfield),
f = f (xt , f (xt−1 ), f (xt−2 ), . . .)
ANN Training
I
ANN Training (Learning, Teaching): Adjustment of network
parameters
I
General Training Methods
I
I
I
Supervised Learning (Teacher, I/O–Patterns)
Reinforcement Learning (Teacher, Learn Signal)
Unsupervised Learning (No Teacher, Self–Organization)
ANN Application Domains
I
Constraint Satisfaction (Scheduling, n–Queens)
I
Content Addressable Memory (Image Retrieval)
I
Control (Machines, “ANN Driver”)
I
Data Compression (Autoencoder)
I
Diagnostics (Medicine, Production)
I
Forecasting (Financial Markets)
I
General Mapping (Function Approximation)
I
Multi Sensor Data Fusion (Remote Sensing)
I
Optimization
I
Pattern Recognition (Voice, Image)
I
Risk Assessment (Credit Card)
Perceptron Learning
I
I
Perceptron Learn Algorithm
Two classes P and N, linear separation
Start: Random w~0 , t := 0
Test: Random ~
x ∈P∪N
If ~
x
If ~
x
If ~
x
If ~
x
∈
∈
∈
∈
P
N
P
N
~t ~
and w
x
~t ~
and w
x
~t ~
and w
x
~t ~
and w
x
> 0 ⇒ Test
< 0 ⇒ Test
≤ 0 ⇒ Add
≥ 0 ⇒ Sub
~ := w
~t + ~
Add: wt+1
x
~ := w
~t − ~
Sub: wt+1
x
t := t + 1
Exit if no weight update ∀~
x
I
I
I
Perceptron Convergence Theorem
“Attack” on perceptrons: M. Minsky and S. Papert
Perceptrons (1969)
The XOR–function of two boolean variables x1 , x2 cannot be
computed with a single perceptron. (Connectedness)
Earlier Models
I
I
I
I
Hebbian Learning, Donald Hebb (1949)
“When an axon of cell A is near enough to excite a cell B and
repeatedly or persistently takes part in firing it, some growth
process or metabolic change takes place in one or both cells
such that A’s efficiency as one of the cells firing B is
increased.”
Generalized Hebb Rule
∆wi,j = η ai bj
Linear Associator, input vector ~a, output vector ~b
W = η ~b ~aT
ADALINE (Adaptive Linear Element), Widrow and Hoff
(1960)
Threshold, error signal, error function has single minimum
learning rule is special case of backpropagation
Widrow–Hoff Learning Rule
yj
w k,j
Ik
zk
tk
I
I
I
I
∂E
Gradient Descent, ∆wk,j = −η ∂w
k,j
PP
Pm
(p)
(p)
(p)
(p)
Error E = p=1 E , E = k=1 (tk − Ik )2
p training patterns, m output neurons, t . . . target value
Pm
Ph
(p)
(p) 2
∂E (p)
∂
j=1 wk,j yj ) ) =
∂wk,j = ∂wk,j ( k=1 (tk −
(p)
(p)
(p)
= −2(tk − Ik )yj
I
Omitting pattern index p
∆wk,j = η(tk − Ik )yj = ηδk yj
Multi–Layer Perceptron
I
Learning as minimization (of network error)
I
Error is a function of network parameters
I
Gradient descent methods reduce error
I
Problem with perceptrons with hidden layers
I
Backpropagation = Iterative Local Gradient Descent
Werbos (1974), Rumelhart, Hinton, Williams (1986)
I
Error–Backpropagation, output error is transmitted backwards
as weighted error, network weights are updated locally
I
Weight update ∆wj,i = ηδj ai
Generalized error term δ
I
Common transfer functions: differentiable, nonlinear,
monotonous, easily computable differentiation
Error–Backpropagation I
x1
I1
v11
H1
y1
w 11
v21
v12
x2
I
I
w 21
H2
I2
y2
v22
I
z1
w 12
w 22
P
P
Hj = ni=1 vj,i xi Ik = hj=1 wk,j yj
yj = f (Hj ), zk = f (Ik )
P
(p)
(p) 2
Error E (p) = 12 m
k=1 (tk − zk )
∂E
Output Layer: ∆wk,j = −η ∂w
k,j
∆wk,j = ηδk yj mit δk = (tk − zk )f 0 (Ik )
z2
Error–Backpropagation II
I
∂E
Hidden Layer: ∆vj,i = −η ∂v
j,i P
∆vj,i = ηδj xi mit δj = f 0 (Hj ) m
k=1 δk wk,j
I
Local update rules propagating error from output to input
I
Present all p patterns of the training set = 1 Epoch (complete
training e.g., 1,000 epochs)
I
Batch Learning (Off–line): accumulate weight changes for all
patterns, then update weights
I
On–line Learning: update weights after each pattern
Self Organizing Maps
I
Cerebral Cortex: topologically ordered maps (sensory inputs)
I
Kohonen (1982)
Topological map, vector quantization, competitive learning
hj,i
wj,i
I
x
xi
Abbildung: A two–dimensional self organizing map.
SOM Formation
I
SOM Learning, unsupervised, no targets
I
Competition: input triggers winner–takes–all
I
Cooperation: identify topological neighborhood
I
Synaptic Adaptation: enhance response of winner and
neighbors
SOM Properties
I
Approximation of input space, topological ordering
Abbildung: SOM formation of unit square topology (from
www.learnartificialneuralnetworks.com)
.
I
Robots as an abstraction of living beings
I
Sensors, e.g., light, infra–red, camera
I
Actors, e.g., wheels, chains, legs
I
Controllers, e.g., program, fuzzy rules, ANNs
I
Small, cheap robots → robot swarms
I
Real and simulated robots
EMMA2
Robotic Learning
I
Embodiment of computers (interaction with environment)
I
Controllers (Heuristics, Fuzzy Rules, Artificial Neural
Networks)
I
Phylogenetic learning (evolution) and ontogenetic learning
(reinforcement)
I
Robot should learn to cope with unknown problems
I
Robot simulator allows fast experiments
NeuroController
Amplifier
M
Actors
Sensors
M
Modulator
Videos
I
Evolved Virtual Creatures, 1994(!)
I
Big Dog, Military Robot, 2010
I
RoboCup NAO League, 2011
I
ROBOTICA Mid Size League Final, 2011
SIMMA – Simulator for EMMA
I
Java framework simulating and visualizing robotic experiments
I
Tests evolved neurocontrollers
The Peg Pushing Task
I
Robot should move peg to light source
I
Fitness is final distance peg-light (10 epsiodes, 30 seconds
each)
I
Robot has 6 infra-red and 3 light sensors
I
Basic ANN structure 9-4-2
I
Evolution of weights, neurons, links (1,000 generations,
population size 50, 20 runs)
The Robot
6 cm
Infra−Red Sensor
Wheel
Light Sensor
Sample of Best Evolved Networks
(a)
13/13/19/-5.69/0.9071
(b)
14/12/15/-1.37/0.9980
(d)
16/11/11/+9.37/0.9954
(e)
17/12/16/+7.73/0.8849
(c)
15/11/15/+8.67/0.9749
(f)
18/13/24/X/0.9333

VO Neurale Netzwerke und Biorobotik

Transcription

Similar documents

Masterarbeit - Institut für Informatik

KDL-46V3000 - Audio Lab of Ga