Real-time digital holographic microscopy Ventseslav Sainov and Elena Stoykova

OPTIMESS2009
Buytaert
1
Real-time digital holographic microscopy
2
3
4
Ventseslav Sainova and Elena Stoykovab
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
Abstract
A multi-camera implementation of the in-line phase-shifting digital holographic
microscopy is a straightforward solution for real-time measurement. The work presents
a portable phase-stepping holographic microscope for measurement in a real time
operation mode. The interferometric system is based on a parallel arrangement of four
Mach-Zehnder interferometers with equalized optical paths. A conventional
microobjective-projective optical system is used to form the image in infinity. A set of
beam splitters and adjustable polarization filters on the way of the object and reference
wavefronts ensure the same contrast of fringe patterns in the plane of CCD targets.
High precision of the phase shift in every interferometer is achieved by incorporation of
highly accurate optical retarders. The registration module consists of four monochrome
CCD cameras and a color camera for simultaneous recording of four phase shifted
monochrome images together with a color image in white light. The laser source is CW
or pulse generating DPSS laser at 532 nm wavelength which is suitable for investigation
of leaving cells by 3D imaging and fluorescence analysis. Taking in view the pointwise
nature of the phase-shifting technique, a modified algorithm for phase retrieval is
derived. The algorithm requires preliminary calibration of the system by a phasestepping interferometric measurement in each channel. The calibration allows for
compensation of errors induced by the misalignments between the object and reference
wavefronts. The modeling of the system performance and error analysis confirm its
reliable behavior without imposing drastic restrictions on system alignment. Some
experimental verification is also provided.
Contact information
a
vsainov@optics.bas.bg
Central Laboratory of Optical Storage and Processing of Information,
Bulgarian Academy of Sciences
Acad. G.Bonchev Str., Bl.101, 1113 Sofia, Bulgaria
b
elena@optics.bas.bg
Central Laboratory of Optical Storage and Processing of Information,
Bulgarian Academy of Sciences
Acad. G.Bonchev Str., Bl.101, 1113 Sofia, Bulgaria
1
OPTIMESS2009
Buytaert
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
Introduction
76
77
78
79
80
81
82
83
84
85
86
87
88
A multi-camera implementation of the in-line digital holographic microscopy is another potential
solution for the real-time operation mode. In [19] we proposed a system with four Mach Zhender
interferometers which, combined with four 2D sensors, yield four phase-shifted at π/2 fringe
patterns. To prove the reliability of the measurement and to evaluate tolerable variations of the
erroneous factors which influence the accuracy of object reconstruction, we modeled its operation
for the case of zero order interference patterns on the photo sensors apertures. We showed that
by a proper calibration it is possible to compensate for the errors induced by the misalignments
between the object and reference wave fronts. In the present work we continue the analysis of
the system for the more realistic case when the recorded patterns in the separate channels contain
a certain number of fringes and provide some experimental verification of the proposed
calibration procedure.
Digital holography which implements numerical reconstruction of the holographic image from
digitally recorded interfering object and reference beams has witnessed a real progress as a result
of recent advances in laser sources, two dimensional photo sensors (CCD or CMOS cameras) and
digital signal processing techniques [1-3]. Digital holography exhibits some exceptional features
which makes it especially suitable for microscopy [4] as quantitative phase-contrast imaging of
transparent biological samples without need of multiple exposures or mechanical scanning [5,6],
numerical focusing at different planes within the space occupied by a bulky object and, hence, its
3D reconstruction and localization along the optical axis [7], numerical manipulation of the
propagating optical field for correction of aberrations [8] or multi-wavelength interferometry [9].
Digital holographic microscopy is especially promising for marker-free observation of moving
objects like particles, plankton, living bacteria and cells due to its property to trace volumetrically
movement directions, velocities and trajectories of these objects [10]. It can be used for invariant
feature recognition in space and frequency domains [11].
Using of in-line set-ups in digital holography has become possible after introduction of phaseshifting in the reference beam [12,13]. For the purpose, at least three phase-shifted interference
patterns should be recorded. This is a serious obstacle for realization of a real-time observation
mode. Different solutions have been recently proposed to solve the problem with an
instantaneous phase stepping measurement. A highly stable low-noise phase retrieval at rate up to
8 Hz is described in [14] by Fourier decomposition of a low-coherence optical image field into
two spatial components and introduction of controllable phase-shifts in one of them. The
approach is further developed by application of Hilbert transform in [15]. A parallel phaseshifting digital holography was proposed and realized in [16] by spatial segmentation of the
reference wavefront using an array of cells consisting of 3 or 4 different phase retarders in front
of the image sensor. A two-step simultaneous phase-shifting detection has been also introduced
as a further improvement of this method by utilization of a 2x1 cell configuration array of
polarizers [17] or retarders [18]. The high rate of recording in the proposed solutions is achieved
at the expense of some restrictions which should be set on the intensities of the reference/object
waves, introduction of a carrier frequency or utilization of a decreased number of pixels which
contribute to image reconstruction etc.
2
OPTIMESS2009
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
Buytaert
Multi-camera interferometric system for real time operation mode
Experimental set-up
To realize a real time operation mode through simultaneous recording of four phase-shifted
interference patterns, we built the set-up, shown in Figure 1. The presented system is based on
parallel image plane holographic recording by four CCD cameras using four identical Mach
Zhender interferometers with phase-shifted at π/2 reference beams in one of their arms. The
system in Figure 1 can be used both for digital holographic recording and image-plane
interferometric phase determination. To minimize the influence of errors, precise selection and
adjustment of optical elements and CCD cameras is required to equalize the optical path changes
and to set correct phase-steps in the four Mach Zhender interferometers. The optical paths in the
interferometers are controlled by the incorporated in them beam-splitters, optical compensators
and reflectors. The required phase steps are introduced by phase retarders. The challenge in the
technical realization of such a multicamera system is the requirement for all photo sensors to
register images which correspond to the same spatial phase distribution caused by the object.
Utilization of four separate optical channels for simultaneous pattern acquisition inevitably
increases the number of error sources. One should take into account the non-linear response of
the CCD cameras, distortions due to slight differences in the point spread functions of the optical
channels, non equal background intensity and contrast of the recorded interference patterns, small
misalignments between the fronts of the interfering waves etc.
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
Figure 1: Multi-camera system for parallel image plane holographic recording
3
OPTIMESS2009
132
133
134
135
136
137
138
139
140
141
142
Buytaert
Phase retrieval
The CCD cameras in Figure 1 record the interference patterns in the image plane:
2
I l (x, y; φ l ) = U O (x, y ) + U Rl (x, y ) = U Rl
2
+ UO
2
+ U Rl U O∗ + U Rl *U O ,
l = 1,2,3,4 (1.1)
where U O ( x, y ) = AO ( x, y )exp[iϕ0 (x, y )] and U Rl (x, y ) = ARl (x, y ) exp[iφ l ] are the complex amplitudes
of the object and reference beams in a channel “l” respectively, φ l is the constant phase-step
introduced between both beams, and ∗ denotes complex conjugate. As it should be expected,
registration of four phase-shifted at π / 2 interference patterns I l ( x, y ) = I [x, y; (l − 1)π 2] , l =
1,2,3,4 makes possible calculation of the object complex amplitude U 0 ( x, y )
1
{I1 (x, y ) − I 2 (x, y ) + i[I1 (x, y ) − I 3 (x, y )]}
U O ( x, y ) =
4U R∗
or the phase
⎡ I 4 (x, y ) − I 2 ( x, y ) ⎤ 2π
⎥=
⎣ I1 ( x, y ) − I 3 ( x, y ) ⎦ λ
ϕ0 ( x, y ) = arctan ⎢
(1.2)
h(x, y )
∫ [n (x, y, z ) − n ]dz
s
m
(1.3)
0
143
144
145
146
147
148
149
150
151
152
provided the complex amplitude of the reference beam is the same in all channels. In (1.3) λ is
the wavelength of the illuminating beam, ns is the refraction index of the sample, nm - that of the
medium, and h( x , y ) is the height of the object. The intensity and the phase distribution of the
reconstructed real image are determined by Fresnel-Kirchoff integral from the recorded hologram
multiplied by the reference wavefield in the hologram plane. In particular, calculation of (1.3) at
assumption of equal background and contrast of the four recorded interference patterns permits to
retrieve information about the optical thickness of microscopic phase objects. The fringe patterns
which are registered by the four CCD cameras in Figure 1 are given by
I l ( x, y ) = I l0 ( x, y ) + γ l ( x, y )cos Ψl ( x, y ) + N l ( x, y ) , l = 1,2 ,3,4 ,
(1.4)
153
where the slow varying functions I l0 ( x , y ) and γ l ( x , y ) give the background and the contrast of
154
155
156
the fringe pattern in a channel “l”, the phase Ψl ( x , y ) contains the relevant information and
N l ( x, y ) is the additive noise term. The phase Ψl ( x , y ) can be written as:
Ψl (x , y ) = ϕ0 ( x , y ) + ψ l ( x , y ) +
π
(l − 1) + δl ,
2
(1.5)
157
158
159
160
161
162
The information about the phase object is encoded in the phase ϕ 0 (x, y ) = 2πλ−1 (n s − n m )h(x, y )
where, without the lack of generality, we assume that the refraction index is constant throughout
the sample. The error terms, δl and ψ l ( x , y ) , describe the phase step error and the error induced
by the misalignment of the wave fronts of the object and reference waves for the channel “l”
respectively. These errors may be different in the four channels thus creating a serious obstacle
for correct object reconstruction.
163
164
165
166
As we have shown in [19], the misalignment problem can be solved by preliminary calibration of
the system. The phase errors ψ l ( x , y ) are systematic and can be measured in each of the channels
by successive acquisition of four phased shifted at π/2 fringe patterns recorded without the object.
Then, if we assume that the backgrounds and contrasts of fringes (1.4) are adjusted to be very
4
OPTIMESS2009
167
168
169
Buytaert
close in the four channels and the phase terms ψ l ( x , y ) are known, we straightforwardly obtain
the following corrected formula for the object phase:
tgϕ0 =
170
(I 4 − I 2 )(a1 + a3 ) − (I1 − I3 )(b2 + b4 )
(I1 − I 3 )(a2 + a4 ) + (I 4 − I 2 )(b1 + b3 )
(1.6)
171
172
173
where we have
174
175
Simulation and experimental verification
176
177
178
179
180
181
182
The reliable behavior of the algorithm was proved in [19] by modeling the system performance
when the cameras in Figure 1 register practically zero order fringe patterns at misalignments
between the plane object and reference wavefronts up to half a minute. In the present work we
evaluate applicability of (1.6) for the case when the phase error ψ l ( x , y ) represents some more
complicated surface which leads to a large number of fringes across the aperture of each CCD.
Let’s assume for the current analysis that ψ l ( x , y ) is formed as a result of interference of a plane
reference wave U l0 (x, y ) = Al0 exp[− j (koxl x + koyl )], where (koxl , koyl ) is the wave-vector in the “l” channel,
183
184
185
186
187
188
189
190
al ( x , y ) = cos ψ l ( x , y ) , bl (x , y ) = sin ψ l ( x , y )
[
(1.7)
]
and an object spherical wave U lR (r ) = A lR exp{− jk (x − x l )2 + ( y − y l )2 / 2 R} in paraxial approximation,
r
where R is the radius of the spherical wavefront and (x l , y l ) are the coordinates of the wavefront
apex with respect to the coordinate system attached to each camera; k = 2π / λ . We could accept
this simplification due to the fact that the specific forms of ψ l ( x , y ) , l = 1,2,3,4, are not important
for further considerations; the fact that matters is that they i) do not coincide in the separate
channels,r and ii) introduce a large phase variation in the recorded patterns. If the plane wave
vectors k 0l subtend angles 90°- ηlx ,y ,z with X,Y and Z axes the phase term is given by
ψ l ( x, y ) =
(
)
[
]
2π ⎧
1
l
l
(x − x l )2 + ( y − y l )2 ⎫⎬
⎨ x sin η x + y sin η y −
2R
λ ⎩
⎭
(1.8)
191
192
193
194
Figure 2 depicts the grey scale maps of four phase-shifted at π/2 fringe patterns recorded by the
four cameras in Figure 1 at misalignment of 5' and 15' between the plane reference beams and
the planes of the CCD apertures. The patterns are simulated as arrays of 8-bit encoded intensities
for N x × N y = 512×512 pixels at equal pitch along X and Y axes Δ x = Δ y = Δ = 0.2 μm and λ =
195
196
0.532 μm with equal constant backgrounds and contrasts without phase step errors and noise. The
object is a small flat top cylinder with height h = 10 μm and ns − nm = 0.2. As it can be seen, the
197
increase in the angles ηlx ,y ,z leads to different values of the distance between the apex of the
198
199
spherical front and the center of the cylinder. The algorithm (1.6) restores the object without
error from the fringe patterns shown in Figure 2.
5
OPTIMESS2009
Buytaert
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
Figure 2: Grey scale 8-bit maps of four phase-shifted at π/2 fringe patterns recorded in the four optical channels of
the system in Figure1 (simulation): top) the misalignment between the plane reference beam and the plane of the
CCD aperture is about 5' in each channel; bottom) the misalignment is about 15' in each channel.
Figure 3: Spatial distribution of the error in reconstruction of a cylindrical object with height of 10 μm when the 8bit values of background intensities in the four channels correspond to (128,127,126,125); left) without noise and
phase step errors; middle) with a phase step error λ/80; right) with a phase step error λ/80 and noise with standard
deviation 2.
Figure 4: Left: reconstruction of a cylindrical object with height of 10 μm at misalignment between plane reference
beams and the CCD planes of 2'; right: maximum and minimum values of the reconstruction error at increasing
difference between the background intensities in neighbouring channels (hollow symbols – with a phase step error
λ/80 , solid symbols – without a phase step error).
6
OPTIMESS2009
237
238
239
240
241
242
243
244
245
246
247
248
249
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
Buytaert
However, phase step errors, noise, as well as different backgrounds and contrasts in the channels
strongly affect the accuracy of restoration and set limitation on acceptible misalignments. To
evaluate the influence of these errors we modeled the phase step error in each channel as a
N (0,σ ph ) number and added a zero-mean normally distributed additive noise with standard
deviation σ N . Figures 3 and 4 show the reconstruction error ε (x, y ) in micrometers that is
calculated as a difference between the reconstructed surface, hˆ( x, y ) , and the real one, h( x, y ) , for
a misalignment of the order of 2' at increasing difference n between the of 8-bit encoded
uniformly distributed backgrounds, I 0l , in the neigbouring channels. We see that such a
difference introduces a substantial regular error (Figure 3,left) in the reconstruction of flat
surfaces even at n equal to 1 or 2. Presence of noise or a phase step error masks this regular
structure with random fluctuations (Figure 3). The influence of error sources is more tolerable in
the case of sloping surfaces, e.g. a part of a dome as is shown in Figure 5. We see that
satisfactory 3D reconstruction is possible even at large misalignments and values of n.
250
251
252
253
254
255
256
257
258
259
Figure 5: Wrapped phase maps obtained from a simultaneous measurement of four phase-shifted at π/2 fringe
patterns for a slope of a dome (a microlens) with radius 800 μm (simulation): left) misalignment - 2', background
intensities – (120,128,124,132); middle) misalignment - 4', background intensities – (120,128,136,112); right)
misalignment - 8', background intensities – (120,128,124,132). In all maps the standard deviations of the phase error
and the additive noise are λ/80 and 2 respectively.
For experimental verification we chose a similar object – a section of a polycarbonate positive
spherical microlens – mainly to handle possible large misalignments. The object was observed
through a microobjective (Plan Apochromat 25x/0/∞), followed by a projective (8x) and a
collimating lens with a focal distance 79 mm and a diameter of the aperture 20 mm. Each channel
was calibrated by a phase-shifting measurement without the object with a four step algorithm. To
ensure that each camera captures the same object filed, we made additional calibration with a
1951 USAF resolution test chart. The result from the simultaneous phase-shifting measurement
using the four cameras is compared to the results obtained in the single channels in Figure 6 and
Figure 7. The recorded fringe patterns were filtered using a James-Stein filter for the
multiplicative noise with size of window 5x5. As it could be seen, we achieved good qualitative
agreement between the 3D reconstructions from the simultaneous phase-shifting measurement
and traditional implementation of the four-stepping technique. Precision of the measurement can
be improved by finer adjustment and additional preprocessing of the fringe patterns as applying
of more suitable denoising and normalization of the recorded fringe patterns.
7
OPTIMESS2009
Buytaert
282
283
284
285
286
287
288
289
290
Figure 6: 3D reconstruction of a section of a positive spherical microlens obtained from a phase-shifting
measurement in each of the four separate channels of the holographic microscope in Figure 1 (Camera 1-4) and
from a simultaneous measurement of four phase-shifted at π/2 fringe patterns using all cameras.
291
292
293
294
Figure 7: 3D reconstruction of a section of a positive spherical microlens obtained from a phase-shifting
measurement in each of the four separate channels of the holographic microscope in Figure 1 (Camera 1-4) and
from a simultaneous measurement of four phase-shifted at π/2 fringe patterns using all cameras.
8
OPTIMESS2009
295
296
297
Acknowledgements
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
References
Buytaert
This work is supported by COST Action MP0604.
[1] SCHNARS, O., JUPTNER, W., Direct recording of holograms by a CCD target and numerical
reconstruction, Appl. Opt., 33(2): 179-181, 1994.
[2] CUCHE, E., BEVILACQUA, F., DEPEURSINGE, C., Digital holography for quantitative phasecontrast imaging, Opt. Lett. 24 (5): 291-293, 1999.
[3] FERRARO, P., DE NICOLA, S.,COPPOLA, G., FINIZIO, ALFIERI, D. PIERATTINI, G.,
Controlling image size as a function of distance and wavelength in Fresnel-transform reconstruction of
digital holograms, Opt. Lett. 29 (8): 854-856, 2004.
[4] MARTÍNEZ-LEÓN, L. PEDRINI, G., OSTEN, W., Applications of short-coherence digital
holography in microscopy, Appl. Opt. 44 (19): 3977-3984, 2005.
[5] MANN, C. ,YU, L. LO, C.-M., KIM, M., High-resolution quantitative phase-contrast microscopy by
digital holography, Opt. Express 13: 8693-8698, 2005
[6] CARL, D., KEMPER, B., WERNICKE, G., VON BALLY, G., Parameter-optimized digital
holographic microscope for high-resolution living-cell analysis, Appl. Opt. 43 (35): 6536-6544, 2004
[7] DUBOIS, F., SCHOCKAERT, C., CALLENS, N., YOURASSOWSKY, C., Focus plane detection
criteria in digital holography microscopy by amplitude analysis, Opt. Express 13: 5895-5908, 2006
[8] COLOMB, T., CUCHE, E., CHARRIÈRE, F., KÜHN, J., ASPERT, N., MONFORT, F., MARQUET
P., DEPEURSINGE, C., Automatic procedure for aberration compensation in digital holographic
microscopy and applications to specimen shape compensation, Appl. Opt. 45 (5): 851-863, 2006
[9] GASS, J., DAKOFF, A., KIM, M.K., Phase imaging without 2π ambiguity by multiple-wavelength
digital holography, Opt. Lett. 28 (13): 1141-3, 2003
[10] KEMPER, B., CARL, D., SCHNEKENBURGER, J., BREDEBUSCH, I., SCHÄFER, M.,
DOMSCHKE, W., VON BALLY, G., Investigations on living pancreas tumor cells by digital holographic
Microscopy, J. Biomed. Opt. 11: 034005, 2006
[11] JAVIDI, B., TAJAHUERCE, E., Three-dimensional object recognition by use of digital holography,
Opt.Lett. 25 (9): 610-612, 2000
[12] YAMAGUCHI, I. ZHANG, T., Phase-shifting digital holography, Opt. Lett. 22 (16): 1268-1270,
1997
[13] YAMAGUCHI, I., KATO, J. , OHTA, S., MIZUNO, J., Image formation in phase-shifting digital
holography and applications to microscopy, Appl. Opt. 40 (34): 6177-6186, 2001
[14] POPESCU, G., DELFLORES, L., VAUGHAN, J. , BADIZADEGAN, K. IWAI, H., DASARI, R.,
FELD, M., Fourier phase microscopy for investigation of biological structures and dynamics, Opt. Lett. 29
(21): 2503-5 (2004).
[15] IKEDA, T., POPESCU, G., DASARI, R., FELD, M. , Hilbert phase microscopy for investigating
fast dynamics in transparent systems, Opt. Lett. 30 (10): 1165-1167, 2005.
[16] AWATSUJI, Y., SASADA, M., AND KUBOTA,T., Parallel quasi-phaseshifting digital holography,
Appl. Phys. Lett. 85: 1069–1071, 2004.
[17] NOMURA, T. ,MURATA, S., NITANAI, E. , NUMATA, T., Phaseshifting digital holography with a
phase difference between orthogonal polarizations, Appl. Opt. 45 (20): 4873–4877, 2006.
[18] AWATSUJI, Y. TAHARA, T. KANEKO, A., KOYAMA, T. , NISHIO, K. , URA, S. , KUBOTA, T.
MATOBA, O. , Parallel two-step phase-shifting digital holography, Appl.Opt. 47 (2): D183-D189, 2008
[19] SAINOV, V., STOYKOVA, E., Multifunctional phase-stepping interferometer for measurement in
real time, In: International symposium to commemorate the 60th anniversary of the invention of
holography, 27-29 October, 2008, Springfield, MA, USA, R.J. Priputniewicz, ed., Society for
Experimental Mechanics, 2008, 53-60.
9