Procedura di identificazione dei parametri cinematici di un veicolo e

IMEKO 2010
London, UK, 1-3 September 2010
Calibration of a vision-based system for
displacement measurement in planetary
exploration space missions
Marco Pertile1, Marina Magnabosco2,
Stefano Debei1
1
Dept. of Mechanical Engineering, University of Padova,
Via Venezia 1, 35131 Padova (PD), Italy.
2 Cranfield University, UK.
PURPOSE
To perform a detailed uncertainty analysis and calibration of a
measurement system of displacement based on a stereo camera
and applied to a simulated planetary scene.
METHOD
Experimental tests
using a reference
displacement
instrument, to obtain
a calibration curve
Probabilistic
propagation to
evaluate the
uncertainty of an
indirect measurement
performed by a
mathematical model
IMEKO 2010
Experimental tests to
evaluate the uncertainty
of input quantities used
to calculate the output
quantity (displacement)
London, UK, 1-3 September 2010
INTRODUCTION
In planetary exploration space missions, the position of a vehicle on the planet surface
can not be measured in a easy way:
1. An odometry system (e.g. optical encoders), that measures the rotation of the
wheels, has wide uncertainty due to slippage of wheels on a natural, often sandy or
slippery, surface.
2. GPS-like positioning systems are accurate but are not yet available on
extraterrestrial planets.
3. Inertial navigation sensors exhibit unacceptable drifts.
Need of a reliable and accurate displacement instrument is particularly relevant.
In this presentation a measurement system based on a stereo camera is described
IMEKO 2010
London, UK, 1-3 September 2010
MEASUREMENT ALGORITHM
Camera model
A
pin-hole
camera
model
is
assumed:
-X, Y, Z defines the 3D position of
landmarks (3D points);
-x, y are the coordinates of the
projection of a landmark using an
ideal camera, aligned like the real
camera but with a focal length
equal to 1 (in length units).
• Intrinsic parameters: define the functional relationship between projection x, y,
expressed in length units, and projection x’, y’, expressed in pixels; they comprise: the
pixel densities along the two axes, the focal length, the position in the image of the
principal point (intersection of the optical axis with the sensor), distortion parameters.
• Extrinsic parameters: the relative position and rotation between the two cameras.
IMEKO 2010
London, UK, 1-3 September 2010
MEASUREMENT ALGORITHM
The measurand is the displacement of a calibrated stereo system and the measurement
is performed using the images acquired in an initial position and in a second one.
Indirect measurement
Output:
a numerical evaluation of the displacement
Input :
1. Positions in pixels of the projections on the image plane of 3D points (landmarks);
2. Intrinsic and extrinsic parameters of each camera
IMEKO 2010
London, UK, 1-3 September 2010
MEASUREMENT ALGORITHM
1. Positions of 2D keypoints on the image plane
a) A detector locally analyses the images and finds out regions that are projections of
landmarks and can be used as features. The Hessian-Affine detector is selected.
b) A descriptor provides representations of the detected regions. Thus, the descriptor
allows to search corresponding features (regions that are projections of the same
landmark) in the acquired images and to perform their matching:
• between the two cameras;
• between images acquired by the same camera but in two different positions.
This matching phase is required
to measure the 3D position of
features and then the position of
the stereo system.
The SIFT (Scale Invariant
Feature Transform) descriptor is
selected.
Image 1
IMEKO 2010
London, UK, 1-3 September 2010
Image 2
MEASUREMENT ALGORITHM
2. Computation of 3D position of landmarks
Once the features are identified in images and matched between the two cameras, the
3D positions of landmarks are computed by the middle point triangulation method.
3. Stereo system displacement
Once the 3D landmarks are evaluated for two positions of the stereo system, the
displacement is computed as the rigid translation that makes the corresponding 3D
points overlap.
IMEKO 2010
London, UK, 1-3 September 2010
CALIBRATION
Determination of intrinsic and extrinsic parameters of the stereo system
The first step to calibrate the whole measurement system is the determination of intrinsic
and extrinsic parameters of the stereo system.
The known Zhang method is selected:
1. It uses a plurality of images of a chessboard acquired in different positions and
orientations;
2. It comprises:
a) a first analytical evaluation of parameters for both cameras;
b) a nonlinear optimization technique based on the maximum likelihood criterion
(Levenberg-Marquardt algorithm); lens distortions, especially radial distortion,
are taken into account the for each camera.
IMEKO 2010
London, UK, 1-3 September 2010
CALIBRATION
Calibration of the whole measurement system
The stereo system is mounted on a mechanical slide having 2 degrees of freedom and
is aimed at a simulated planetary scene obtained with crumpled brown paper.
The whole system is calibrated
separately along two orthogonal
directions:
a
first
substantially aligned
one
with the
optical axes of the cameras (axial
displacement) and the second
one substantially orthogonal to
the
optical
axes
(transverse
displacement).
For each direction the stereo system is moved from an initial position to a final one with
11 steps, while the displacement is measured by the laser interferometer and by the
stereo system in order to build a calibration curve for each direction of translation.
IMEKO 2010
London, UK, 1-3 September 2010
UNCERTAINTY ANALYSIS
The uncertainty associated with the following quantities are analyzed and evaluated:
1. intrinsic and extrinsic parameters of the stereo system, whose uncertainties are
evaluated during the stereo calibration by applying a Monte Carlo simulation to the
Zhang method; in this sub-problem the input quantities are the chessboard
dimensions, and the positions of the chessboard points on the image plane;
2. FEATURE POSITIONS in the image plane, which have two main contributions:
1. the sensor and reading electronics uncertainty;
2. the lighting angular variation of the scene.
IMEKO 2010
London, UK, 1-3 September 2010
UNCERTAINTY ANALYSIS
2D position of features on the image plane
A. uncertainty (commonly referred to
as image noise in computer vision)
associated with the image sensor
and its electronics
B. Variation of lighting conditions
A.
Sensor and reading electronics
uncertainty depends on the acquired
A simulated rocky scene
is employed
scene
Sensor and reading electronics
uncertainty is assumed of random
type
IMEKO 2010
200 images of the same scene were
acquired with both cameras in fixed
positions and in the same shooting
conditions
London, UK, 1-3 September 2010
UNCERTAINTY ANALYSIS
2D position of features on the image plane
B. Variation of lighting conditions
On a planet surface, the sun angular position may change between two acquisition
positions.
Time tm required for the stereo system to move from an initial position to a final one
is evaluated assuming a moving velocity of a rover on a planetary surface
Time tm allows to estimate a maximum angular variation (between two consecutive
image acquisitions) of the lighting if the planet (e.g. Mars) rotation and orbit is known
IMEKO 2010
London, UK, 1-3 September 2010
UNCERTAINTY ANALYSIS
Dedicated experimental tests
lamp
camera
slide
Simulated
scene
PC
IMEKO 2010
London, UK, 1-3 September 2010
UNCERTAINTY ANALYSIS
2D position of features on the image plane
B. Variation of lighting conditions
The angular variation of lighting
between two images of the same
scene make the shadows move and
yields a translation of features on the
image plane
IMEKO 2010
20 images of the same scene are
acquired varying the lamp position
to simulate the maximum angular
variation of sun on the planet
surface.
London, UK, 1-3 September 2010
RESULTS
Measured AXIAL displacement: stereo system vs. laser interferometer.
IMEKO 2010
London, UK, 1-3 September 2010
RESULTS
Measured TRANSVERSE displacement: stereo system vs. laser interferometer.
IMEKO 2010
London, UK, 1-3 September 2010
RESULTS
Difference between measurements of
the
stereo
system
and
the
interferometer
Axial direction
Transverse direction
IMEKO 2010
London, UK, 1-3 September 2010
RESULTS
Observations:
1. The uncertainty evaluated for axial displacements is larger than that obtained for
transverse displacements.
Explanations:
a) the uncertainty of 3D points acquired by the stereo system is much wider along
the axial direction than along a transverse direction, due to the small distance
between the two cameras.
b) This disadvantage along axial direction is partially compensated by the fact
that the number of image features correctly matched in case of axial
displacement is generally greater than the number of matched features in case of
transverse displacement.
2. Along both directions, the evaluated uncertainty increases with the measured
displacement.
Explanation: the larger the displacement and the fewer the features correctly matched
among images (the averaging effect associated with a large number of matched
features decreases along the displacement direction).
IMEKO 2010
London, UK, 1-3 September 2010
CONCLUSIONS
1. A vision-based displacement instrument was described and calibrated
using a simulated planetary rocky scene.
2. Dedicated experimental tests were performed to evaluate the most
significant uncertainty sources using the simulated scene. Particular
attention was dedicated to the uncertainty contributions of the feature
detector and of lighting conditions.
3. Two different motion directions were analyzed and the evaluated
uncertainty were compared.
IMEKO 2010
London, UK, 1-3 September 2010
APPENDIXES
IMEKO 2010
London, UK, 1-3 September 2010
MEASUREMENT ALGORITHM
Computation of 3D position of landmarks
The 3D position of landmarks is computed by the middle point triangulation method:
For the two projections of the same 3D landmark, the algorithm finds the 3D points with
the minimum distance, belonging respectively to the preimage lines of cameras 1 and 2.
This points define a segment orthogonal to the two skew preimage lines. The middle
point of this segment is selected as the measured 3D point of the landmark (feature).
Stereo system displacement
To compute the displacement, the following steps are performed:
1. The 3D position is calculated for all features (landmarks) detected by both cameras
when the vision system is in an initial position P1.
2. The vision system is moved (cameras are rigidly connected) from the initial position
P1 to a second position P2, the same procedure is used to compute the 3D positions
of the features detected by both cameras in the second position P2.
3. Common features detected in both positions P1 and P2 are identified.
4. The displacement is computed as the rigid translation that makes the corresponding
features overlap.
IMEKO 2010
London, UK, 1-3 September 2010
UNCERTAINTY ANALYSIS
2D position of features on the image plane
A. Contribution of the sensor and reading electronics uncertainty
The scene acquired for uncertainty evaluation is not uniform
For each pixel: the mean value calculated using all 200 images is subtracted from the
same pixel of all images (normalization)
The probability density function (PDF) of the uncertainty contribution is evaluated by
the frequency histogram of the normalized grey levels of all pixels of all images.
PDF of the normalized grey levels
grey levels
IMEKO 2010
London, UK, 1-3 September 2010
UNCERTAINTY ANALYSIS
2D position of features on the image plane
A.
detector
and
descriptor
Corresponding
features
The
sensor
and
reading electronics
uncertainty
contribution is added
to all pixels
Distance between the features
found in both images
Monte Carlo simulation
IMEKO 2010
London, UK, 1-3 September 2010
UNCERTAINTY ANALYSIS
2D position of features on the image plane
A. Contribution of the sensor and reading electronics uncertainty
All the distances obtained from all iterations are used to build a frequency histogram
along the x and y axes
This histogram is used to evaluate the PDF of the 1D displacements of image features
due to the sensor and reading electronics uncertainty
PDF of feature displacements along x axis [pixels]
The evaluated PDF
are
substantially
equal along the two x
and y axes
IMEKO 2010
London, UK, 1-3 September 2010
UNCERTAINTY ANALYSIS
2D position of features on the image plane
B. Variation of lighting conditions
Comparing two images at a time,
displacements of all common
features are calculated.
PDF of feature displacements along x axis [pixels]
Along x
IMEKO 2010
The PDFs of feature displacements
are evaluated by the frequency
histograms along x and y axes.
PDF of feature displacements along y axis [pixels]
Along y
London, UK, 1-3 September 2010