Available here

Transcription

Available here

Multimodal imaging towards
individualized radiotherapy
treatments
Laurent Massoptier
Yu Song
Editors
This book is the result of the 3rd summer-school organized by the SUMMER
Marie Curie Research Training Network, which has received funding from the
European Union Seventh Framework Programme (FP7-PEOPLE-2011-ITN)
under grant agreement PITN-GA-2011-290148.
The information and views set out in this publication are those of the authors
and do not necessarily reflect the official opinion of the European Union.
Neither the European Union institutions and bodies nor any person acting on
their behalf may be held responsible for the use which may be made of the
information contained therein.
In addition, the authors and publishers have used their best efforts in preparing this book. They
make no representation or warranties with respect to the accuracy or completeness of the content
of this book. Neither the authors nor publishers shall not be liable for any damages, including but
not limited to commercial, incidental or consequential damages.
Requests for ordering or permission to make copies of any part of the work should be emailed to
the coordinator of SUMMER project: laurent.massoptier@aquilab.com or addressed by postal
mail to Laurent Massoptier at AQUILAB, Parc Eurasanté - Biocentre Fleming, 250 rue Salvador
Allende, 59120 Loos Les Lille, France.
Copyright © 2014
All rights reserved.
ISBN 978-94-6186-309-6
EDITORS-IN-CHIEF & SCIENTIFIC PROGRAMME
Laurent Massoptier
AQUILAB, Lille, France
Yu Song
Delft University of Technology, The Netherlands
Hortense Kirisli
BOARD OF THE SUMMER PROJECT
David Gibon
Wolfgang Birkfellner
Center for Medical Physics and Biomedical Engineering, Medical University Vienna, Austria
Ursula Nestle
University Medical Center Freiburg, Germany
Anne Laprie
Institut Claudius Regaud, Toulouse, France
Umberto Sabatini
Santa Lucia Foundation,
Scientific Institute for Research, Hospitalization and Health Care, Rome, Italy
Katja Bühler
VRVis Zentrum für Virtual Reality und Visualisierung Forschungs-GmbH, Vienna, Austria
Yu Song
Delft University of Technology, The Netherlands
CONTENTS
Design challenges in incorporating segmentation methods into radiotherapy software ................ 5
Aselmaa A, Song Y, Goossens R
Diffusion registration of Lung CT ................................................................................................ 12
Jurisic M, Hauler F, Furtado H, Birkfellner W
Automated evaluation of multi-modal image rigid registration .................................................... 17
Hauler F, Jurisic M, Furtado H, Nestle U, Birkfellner W
Subcortical structures segmentation on MRI using support vector machines .............................. 24
Dolz J, Kirisli HA, Vermandel M, Massoptier L
Evaluation of 4D PET tumour segmentation algorithm with dynamic experimental phantom
measurements ............................................................................................................................... 32
Carles M, Fechter T, Christ U, Chirindel A, Schaefer A, Mix M, Nestle U
A threshold and region-growing based algorithm for 18FDG-PET 4D GTV delineation ............. 42
Fechter T, Carles M, Chirindel A, Christ U, Nestle U
fMRI: resting-state networks and task-evoked activations in the presence of brain tumours ...... 49
Tuovinen N, de Pasquale F, Sabatini U
Defining new regions at risk: fiber tractography for planning radiotherapy of brain tumours ..... 54
Hamamci A, Tuovinen N, de Pasquale F, Sabatini U
Exploiting MRSI data properties to improve quantification ......................................................... 63
Laruelo A, Chaari L, Batatia H, Rowland B, Ken S, Ferrand R, Tourneret JY, Laprie A
Human computer interaction in segmenting organs at risk for radiotherapy: a pilot study .......... 69
Ramkumar A, Dolz J, Kirisli HA, Schimek-Jasch T, Adebahr S, Nestle U, Massoptier L, Varga
E, Stappers PJ, Niessen WJ, Song Y
Nanoparticle technology: future opportunities in cancer treatment .............................................. 80
Jain S, Butterworth KT
The ART of translation: from research to clinical application ..................................................... 86
Verheij M, Sonke JJ
Enabling fast analysis and fusion of MR spectroscopy imaging ................................................. 90
Nunes M, Rowland B, Schlachter M, Ken S, Matkovic K, Laprie A, Bühler K
4D PET/CT visualization in radiotherapy planning ...................................................................... 96
Schlachter M, Fechter T, Nestle U, Bühler K
A. Aselmaa et al.
5
Design challenges in incorporating
segmentation methods into
radiotherapy software
Anet Aselmaa1, Yu Song1, and Richard Goossens1
1
*
Faculty of Industrial Design Engineering, Delft University of Technology, The Netherlands
A.Aselmaa@tudelft.nl
Abstract: Radiotherapy treatment planning is a complex multi-participant process. In a
technology-driven context such as radiotherapy, a good software design balances between
automation and user interactions. In this paper, we discuss the design challenges for
incorporating segmentation methods into the radiotherapy treatment planning software, more
specifically for the contouring task. Using object-oriented modelling, we identify main design
challenges in the categories of general usability, navigation, workflow, and flexibility of
interactions. We also highlight that a multidisciplinary approach to the design process is needed
to be able to incorporate medical, technical and usability knowledge.
Index Terms — Radiotherapy, Automation, Contouring, Design.
INTRODUCTION
Designing software for professionals is a challenge on its own, but designing software in a
technology-driven context such as radiotherapy poses even more challenges. On one hand, use of
information technology can help decreasing human errors [1]. On the other hand, poor usability
can severely hinder the effectiveness of clinician’s work [2]. As such, it is necessary for the
software designer to become familiar with the medical needs, working environment as well as
with the technological advancements. Once there is a good understanding, it is possible to
propose an initial design concept that could be further improved through co-design sessions.
Radiotherapy is a complex, multi-participant process [3]. The full treatment planning process
involves multiple clinicians and can take from hours to days to be completed. Contouring, one of
the sub-processes of the treatment planning where the contours of all important regions of interest
(ROIs) are created, has been identified as the weakest link in the treatment planning [4].
The contouring process begins with defining the list of ROIs to be contoured. This is then
followed by contouring each of these ROIs (as depicted on Figure 1). Most ROIs are independent
of each other and can be contoured in any order. However, some ROIs (e.g. gross tumour volume
and clinical target volume) are dependent on each other and need to be contoured sequentially. In
addition, in clinical practice, the contours are often created by a resident, and therefore a more
senior oncologist needs to validate (and adapt if needed) the contours.
The process of contouring a ROI depends on the type of ROI and the software used. The type of a
ROI defines which image modality should be used. For instance, skull can be well defined on a
CT scan. The software used, however, defines which type of segmentation methods is available
(as shown on Figure 2).
SUMMER Project (PITN-GA-2011-290148) is funded by the 7th Framework Programme of the European Commission
(FP7-PEOPLE-2011-ITN).
6
Design challenges in incorporating segmentation methods into radiotherapy software
Figure 1. Simplified activity diagram of a contouring process
A segmentation method is a specific tool or an algorithm that enables the user to segment
(contour) a ROI. Within this paper, we classify segmentation methods into three categories: fully
automatic, semi-automatic and fully manual.
 A fully automatic segmentation method requires no input or interaction from the user for
creating contours (besides starting it).
 A semi-automatic segmentation method is combining algorithms with user interactions for
creating contours. The algorithmic support can vary from seamless to the user (e.g. 3D
“Smart Brush” [5]) to almost fully automatic (e.g. user input is only required for
initialization).
 A fully manual segmentation method assumes no extra algorithmic support from the
software (e.g. the line is drawn exactly how the mouse cursor moved).
Figure 2 Simplified contouring process of a ROI for three types of segmentation methods: fully automatic,
semi-automatic and fully manual.
Automation has a lot of potential for many tasks in radiotherapy treatment planning. For instance,
the development of (fully or semi-) automated image segmentation methods is one of the key
topic of research (e.g. the Brain Tumour Segmentation Challenges (BraTS) at MICCAI (Medical
Image Computing and Computer Assisted Intervention) conferences in 2012 – 2014). Current
methods of automated segmentation are usable in certain situations; however, it would be
necessary to define which methods are usable for which ROIs, and on which types of datasets [6].
As in any software interface design, general usability principles need to be taken into account.
For example, [7] defines the main quality components of usability as learnability, efficiency,
memorability, errors, satisfaction, and utility. With the increasing number of segmentation
methods available, designing software interface that is balanced between automation and user
interaction while having high usability will be a challenge.
Multimodal imaging towards individualized radiotherapy treatments
Summer-school of SUMMER project (July 2014)
A. Aselmaa et al.
7
The aim of this paper is to discuss possible scenarios of using segmentation methods for
contouring regions of interest, and to highlight different design challenges posed by those use
scenarios. For identifying these scenarios, an object-oriented approach is taken. And then based
on the identified use scenarios, the design challenges are summarized.
OBJECT-ORIENTED VIEW ON CONTOURING
Object-oriented modelling approach allows describing relevant objects and the relations among
them in a compact way. It allows identifying different use scenarios, which then can be used as a
basis for the interface design process. In this research, the UML object diagram is used for
modelling objects and relations involved in the contouring process. Typically, UML diagrams are
used in software engineering. However, the use of UML diagrams is not restricted to this area
and there is increased interest in using UML diagrams for describing other higher level (e.g.
business [8]) processes.
In a high level view of contouring process, the main objects involved are ‘tumour’, ‘patient’,
‘ROI’, ‘image dataset’, ‘segmentation method’, and ‘user interaction’ ( Figure 3). The main
relations between any pair of these objects can be summarized as follows:
 The list of ROIs depends on the tumour and the patient.
 A ROI is identifiable in one or more image datasets.
 A ROI has one or more segmentation methods suitable for segmenting it.
 A segmentation method uses one or more image datasets.
 A segmentation method can have no user interaction or numerous user interactions.
 A segmentation method can be able to segment one ROI or multiple ROIs.
Figure 3 Simplified object diagram representing all potential relations between main objects within ROI
contouring process
For example, oedema might be identified as one of the ROIs in a brain tumour case. Oedema can
be identified well on MRI T2-weighted images or MRI FLAIR images [9]. At the same time, for
example, a fully automatic brain tumour segmentation method called ABTS, is claiming high
success rate in segmenting oedema present for glioblastoma multiform cases by using MRI T2
and MRI FLAIR image datasets [10]. In addition, their segmentation method is also able to
segment another ROI - the Gross Tumour Volume (GTV).
For most ROIs, the best suitable image dataset for contouring is known from clinical practice and
medical research. At the same time, there is a growing knowledge on which segmentation method
performs well for which ROI(s). Therefore, it is feasible to find an optimal segmentation
method(s) for specific ROIs within a software solution.
Designing such a software system that is incorporating multiple segmentation methods for
multiple ROIs is not trivial as there are various realistic scenarios of creating ROIs by using
8
different segmentation methods. Each of these scenarios gives additional design consideration.
One-to-one relations (e.g. a ROI is identifiable only on one image dataset) on their own do not
pose design challenges compared to one-to-many scenarios. However, enabling all different
scenarios within one software design poses usability and interaction design challenges. Table 1
summarizes the design challenges posed by one-to-many use scenarios.
Table 1 Design challenges posed by one-to-many use scenarios
Main one-to-many use scenarios
Design challenges
There are multiple
ROIs for the tumour
of one patient
- Navigation between ROIs
- Managing segmentations of
dependent ROIs
A ROI is identifiable
on more than one
image dataset
- Intuitive navigation
between image datasets
A segmentation
method segments
multiple ROIs
- Intuitive use within
workflow
One ROI has more
than one suitable
segmentation
methods
- Balance between user
freedom and cognitive load
- Navigation between
different segmentation
results of a ROI
- Creation of a composite
contour based on multiple
segmentations
… and the
segmentation
methods require
different types of user
interaction
- Consistent user interactions
- Intuitive use within
workflow
A segmentation
method requires
substantial user
involvement
- Clear user interactions
- Minimized amount of
interactions
- Balanced interactions
… and uses more
than one image
datasets
- Intuitive navigation
between image datasets
DESIGN CHALLENGES
Different clearly defined scenarios allow designing a solution more fitting for the clinical needs.
However, the priorities of these scenarios will depend on the number and types of segmentation
A. Aselmaa et al.
9
methods incorporated into the software. Implementing too many segmentation methods can
become costly without bringing significant benefits. At the same time, not having enough
segmentation methods will hinder the usability (e.g. fully manual segmentation methods require
too much time from the users and thus is not supporting efficiency).
A starting point for such a software design is to review available segmentation methods (fully
automatic, semi-automatic and fully manual) and for each of them, to specify the image dataset
types suitable as input, their success rates for different ROIs, and also the required user
interactions. For example, [11] investigated user interactions for three semi-automatic
segmentation methods (parametric active contours, geometric active contours and graphical
models) and proposed optimal user interactions for them. However, their work is not giving
overview of the success rate of segmentation methods for segmenting specific ROIs based on
specific image datasets.
Once there is a sufficient knowledge base available to incorporate segmentation methods,
detailed graphical user interface design work can begin. In this design phase, the design
challenges we have identified need to be tackled. We have summarized the design challenges
identified in previous section (Table 1) into four categories (Table 2): general usability, navigation,
workflow, and flexibility of interactions.
Table 2 Major design challenges to be addressed within the design of software incorporating numerous
segmentation methods
Category
Design challenge
General usability
Minimized amount of interactions
Clear user interactions
Consistent user interactions
Intuitive navigation between image datasets
Navigation
Workflow
Flexibility of
interactions
Intuitive navigation between ROIs
Navigation between different segmentation results of a ROI
Managing segmentations of dependent ROIs
Intuitive use of a segmentation method within the workflow
Creation of a composite contour based on multiple segmentations
Balanced interactions
Balance between user freedom and cognitive load
DISCUSSION
We have highlighted several design challenges based on different envisioned scenarios.
However, the real challenge will remain in reaching a user interface design solution that solves
all of these in a usable way. Poor communication among different stakeholders has been
highlighted as one of the main reasons for the failure of software projects [12]. Even though we
have used object-oriented approach for identified different use scenarios of segmentation
methods, we see that it is necessary the interface design process itself follows user-centred design
approach [13].
The ISO standard 9241 (part 210) highlights the needs for a multidisciplinary design team and
iterative approach. For the design of a software solution incorporating numerous segmentation
methods, tight collaboration between developers and users is a prerequisite for the success of the
development.
10
CONCLUSION
In this paper, based on object-oriented modelling, we have highlighted different use scenarios
with segmentation methods and discussed the design challenges in incorporating numerous
segmentation methods into single software. Those various design challenges are categorized into
four categories: general usability, navigation, workflow and flexibility of interactions. To tackle
those challenges, a multidisciplinary design team, which is able to incorporate medical, technical
and usability knowledge, is often needed.
The next step of this research is to generate possible interface design prototypes to tackle these
challenges. Ideally these prototypes will be improved iteratively in collaboration with clinicians
and developers of segmentation algorithms. The feasibility of this concept will be also evaluated.
ACKNOWLEDGMENT
This project has received funding from the European Union’s Seventh Framework Programme
for research, technological development and demonstration under grant agreement no PITN-GA2011-290148.
REFERENCES
[1] Pham JC, Aswani MS, Rosen M, Lee H, Huddle M, Weeks K, Pronovost PJ. Reducing
medical errors and adverse events. Annual review of medicine 2012; 63:447-463.
[2] Viitanen J, Hyppönen H, Lääveri T, Vänskä J, Reponen J, Winblad I. National
questionnaire study on clinical ICT systems proofs: physicians suffer from poor usability.
Int J Med Inform 2011; 80(10):708-725.
[3] Aselmaa A, Goossens RHM, Laprie A, Ramkumar A, Ken S, Freudenthal A. External
radiotherapy treatment planning – situation today and perspectives for tomorrow. Innovative
imaging to improve radiotherapy treatments, Lulu Enterprises Inc Ed 2013 (ISBN: 978-1291-60417-7), 1:77-84.
[4] Njeh CF. Tumour delineation: The weakest link in the search for accuracy in radiotherapy. J
Med Phys 2008; 33(4):136-140.
[5] Parascandolo P, Cesario L, Vosilla L, Pitikakis M, Viano G. Smart Brush: a real time
segmentation tool for 3D medical images. IEEE Int Symp Image Sign Process Anal 2013;
689-694.
[6] Whitfield GA, Price P, Price GJ, Moore CJ. Automated delineation of radiotherapy
volumes: are we going in the right direction? Br J Radiol 2013; 86(1021):20110718.
[7] Nielsen J. Usability 101: Introduction to Usability. Jakob Nielsen's Alertbox, 08 June 2014,
http://www.nngroup.com/articles/usability-101-introduction-to-usability/
[8] Eriksson HE, Penker M. Business Modeling with Uml, Wiley Chichester, 2000.
[9] Aselmaa A, Goossens RHM, Rowland B, Laprie A, Song Y, Freudenthal A. Medical factors
of brain tumour delineation in radiotherapy for software design. ACCEPTED for
publication.
[10] Diaz I, Boulanger P, Greiner R, Hoehn B, Rowe L, Murtha A. An automatic brain tumour
segmentation tool. Int Conf Eng Med Biol Soc 2013; 3339-3342.
[11] Zhu Y. Towards more desirable segmentation via user interactions. 2013.
[12] Charette RN. Why software fails [software failure]. IEEE Spectrum 2005; 42(9), 42-49.
A. Aselmaa et al.
11
[13] Dis I. 9241-210: 2010. Ergonomics of Human System Interaction-Part 210: Human-Centred
Design for Interactive Systems. International Standardization Organization (ISO).
Anet Aselmaa Born in Estonia 30th October 1986. Received B.Sc. (2007) and
M.Sc. (2010) in Business Information Technology from Tallinn University of
Technology, Estonia.
She has worked in the field of web-based software solutions’ design and
development in Estonia, Hungary and Sweden. Currently she is a PhD candidate
in the Technical University of Delft, faculty of Industrial Design Engineering.
Her research is about “Designing for Sensemaking” in the context of
radiotherapy treatment planning.
12
Diffusion registration of Lung CT
Miro Jurisic1*, Frida Hauler1, Hugo Furtado1,2 and Wolfgang Birkfellner1,2
1
Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Austria
² Christian Doppler Laboratory for Medical Radiation Research for Radiation Oncology, Medical
University of Vienna, Austria
*
miro.jurisic@meduniwien.ac.at
Abstract: Deformable registration is an image processing method that maps a homologous point
set from different images. It is used in a clinical environment for image fusion, treatment
assessment, patient positioning, dose recalculation or patient follow-up. We present the
calculation of variation approach for joint diffusion regularization and image segmentation. The
main task of segmentation in our method is to assist the deformable registration by including
segmentation masks as an additional channel of the original image. We used 4D lung CT of 4
patients to test our method. Results show that the segmentation assisted diffusion registration
preforms similar as plain diffusion registration, but with different more physical deformation
fields.
Index Terms — Calculation of variations, deformable registration, image segmentation.
INTRODUCTION
Non-rigid image registration is an important problem in various clinical specialties. This task is
to systematically place separate images in a common frame of reference. In this way the
information contained in images can be optimally integrated or compared [1,2]. Both deformable
registration and segmentation can be stated as an energy optimization method for an appropriate
functional E(f)(1). Once the energy model is set, we calculate the function that minimizes the
energy. One of the methods for calculating the function to be minimized is the Euler-Lagrange
method (2).
min E(f )=∫ L( ⃗x , ∇ f ( ⃗x ))
f
Ω
dE ∂L
d ∂L
=
− ∑
=0
d f ∂ f i=x , y, z d i ∂ f i '
f i '=
(1)
∂f
∂i
;
(2)
One of the main problems with deformable registration is the ill-posed structure of the problem.
In fact, more unknowns than equations exist. In order to deal with this shortcoming, a
regularization term is introduced. Regularization can be placed explicitly in the energy
minimization approach or it can be implicitly included into the algorithm as a deformationsmoothing field [3]. An overview of different regularization terms is presented in [2].
MATERIALS AND METHODS
Diffusion registration
The first step to deformable registration by variation calculus is to define an appropriate energy
functional (3). The most common functional separate data or image terms
(4), which depend
M. Jurisic et al.
13
only on spatial locations of the moving and fixed image, and a regularization term ( ⃗ )(5),
which is defined as a function of smoothing the deformation field to support only “physical”
deformation [2]. Diffusion registration has a regularization term in the form of the sum of
gradient magnitudes for components of the deformation field (5) [3], [4], [5].
Ereg (⃗u )=S data +α R(⃗
u)
1
S data = ∫ (F(⃗x )−M (⃗x , ⃗u ))2 d ⃗x
2Ω
1
R(⃗u )= ∫ ∑ (∇ ui)2 d ⃗x
2 Ω i= x, y , z
(3)
(4)
(5)
Now that the energy functional is set, we can start writing the Euler-Lagrange equations for a
diffusion registration. Equation (6) shows the iterative approach for updating the x-component of
the deformation field. It this equation the time is a subsidiary variable to represent iterations. It is
an interesting fact that all three components of the deformation field are independent of each
other.
∂u x −d Ereg
d
=
=−(F−M) ∇ u M + ∑
α ∇ ux
∂t
d ux
i=x , y , z d i
x
(6)
Piece-wise constant Mumford-Shah segmentation
Similar to the diffusion registration, the segmentation problem can be also stated as an energy
minimization problem, but of course with a different functional. Since our main focus is a
registration, we used a simple energy functional known as piece-wise Mumford-Shah model (7)
[6]. The main idea behind this model is to divide image into foreground and background with the
constant values. Looking at the form of (7), we observe the similarity with diffusion functional.
The data term – the squared difference between model image I(c,Φ) and original image I_0- is
similar to the data term of the diffusion registration. The regularization term has also a similar
form, but in contrast to the gradient magnitude of the deformation field, we have a gradient
magnitude of the interface area between two regions. The constants that represent foreground and
background are calculated as in (8). According to (8), constants that represent the regions are just
an average value of the region Φ_k.
E seg=
1
1
2
2
(I (c ,Φ )−I 0) d ⃗x + β∫∣∇ Φ∣ d ⃗x
∫
2Ω
2 Ω
c k (Φ)=
∫Ω I 0 d ⃗x
(7)
i
∫ d ⃗x
Ω
c=(c 1, c 2,. .. , c n) ;
Φ k =k ∈Ωk , k=1,2,... , n
k
(8)
(9)
The regions
are represented by the integer value (9) and the numerical iterative update term
for regions given by the Euler-Lagrange equation (10).
∂Φ =−d E seg =−(I (c , Φ)−I )∇ I (c , Φ)+
∑ ddi β ∇ Φ
Φ
0
∂t
dΦ
i=x , y , z
(10)
14
Fig. 1: This figure show original image, in upper left corner, and 3 segmentation mask for different type of
tissue. Upper right corner image shows separation between air and tissue, lower left separates tissue by
density and finally the lower right image show bone separation.
Fig. 1 shows an example of segmentation masks used for the registration. We used three different
segmentations pair for registration. The first pair separates air from the body. The second pair of
masks is used to separate soft tissue by density, where bones are included as a dense tissue. The
third pair of masks separates a solid from the soft tissue and air. The different masks were created
by the same iterative method (10), but with different pre-processing steps (different thresholding)
and different coefficients. If you take closer look at (8) and (9) you can notice that our approach
also leaves possibility to have a multiple regions segmented. Fig. 2 shows an example for the
three regions segmentation. We found out that three is a limitation for the number of regions for
our approach for the segmentation of the lung CT.
Fig. 2: Left image shows original CT slice, while the right image shows segmentation of the original image
to three regions.
Segmentation assisted registration
Since our goal is a segmentation assisted registration, we have to introduce another energy term,
the one that connects registration and segmentation (11). Our approach consists in segmenting
both a moving M, a fixed image F and of changing the data term
to
.
Ecoupled =R(⃗
u)+E seg (F)+E seg (M )+S average
(11)
The average term (12) takes into account squared differences between all pairs, the intensity
image as well as the masks. Since this term only depends on the deformation field and not on its
M. Jurisic et al.
15
derivatives, we need to calculate only the derivation of an average data term
respect to the deformation field (13).
S average =
1
∫
2(n+1) Ω
∂ Saverage
1
=
∫
∂u x
(n+1) Ω
[∑
n
k=1
[∑
Aorta
Heart
k=1
]
(I fix , k −I mov ,k )2+(F (⃗x )−M (⃗x ))2 d ⃗x
(12)
]
(I fix , k −I mov ,k )∇ u I mov ,k +(F(⃗x)−M (⃗x ))∇ u M d ⃗x
x
Patient 1
Lung
n
with
Patient 2
x
Patient 3
(13)
Patient 4
Normal
Assisted
Normal
Assisted
Normal
Assisted
Normal
Assisted
0.986±
0.97±
0.969±
0.95±
0.983±
0.978±
0.982±
0.980±
0.003
0.02
0.004
0.01
0.003
0.008
0.007
0.004
0.867±
0.87±
0.88±
0.87±
0.85±
0.84±
0.85±
0.86±
0.02
0.02
0.02
0.02
0.08
0.08
0.03
0.03
0.94±
0.94±
0.942±
0.008
0.94±
0.937±
0.931±
0.937±
0.936±
0.01
0.01
0.02
0.007
0.007
0.007
0.007
Table 1. Evaluation of Registration: Organ Overlaps (%)
RESULTS
The mapping performance of the proposed algorithm against simple diffusion registration is
evaluated quantitatively. For comparison, the both algorithms are performed on the same sets of 4
real patient 4D lung CT each with 10 time steps. Organ overlaps or dice coefficients, of the semimanual created mask in the reference image (time step 0) and the transformed organs from source
images (other time steps) are used as metrics to assess the quality of the registration (Table 1).
From Table 1, we can conclude that both of these algorithms lead to excellent registration results
for all three organs. The difference between methods is up to 1%. The highest accuracy is for the
lung in both cases with more that 96%.The aorta is an organ with lower contrast relative to its
surrounding. In this case, registration preforms with accuracy of more than 84%.
Fig. 3: Difference between moving and fixed image, a) before registration, b) after registration without
segmentation masks, c) segmentation assisted registration and d) only segmentation mask are used for
average image.
16
CONCLUSION
Segmentation assisted registration introduces prior knowledge of the data which helps to achieve
a more natural image registration. The diffusion registration is driven by the image gradients and
the segmentation mask; this helps to amplify these gradients at the locations of the edges of the
masks. The process helps the data term to overcome a diffusion smoothing term. Further research
is needed to quantitatively assess deformable registration trough landmarks distance. Our future
work will continue to bring these two methods together and we will also explore how registration
can assist segmentation. Furthermore we will explore coupling of segmentation with different
regularization term.
ACKNOWLEDGMENT
I thank Tobias Fechter for helpful discussion during my secondment visit to University Medical
Center in Freiburg.
REFERENCES
[1] Hajnal JV, Hill DLG. Medical image registration. CRC Press, 2001.
[2] Sotiras A, Davatzikos C, Paragios N. Deformable medical image registration: a survey.
IEEE Trans Med Imag 2013; 32(7):1153-1190.
[3] Modersitzki J. Numerical methods for image registration. Eur J Nucl Med Mol Imaging
2009; 36(S1):S44-S55.
[4] Thirion JP. Image matching as a diffusion process: an analogy with Maxwell's demons.
Medical Image Analysis1998; 243260.
[5] Pennec X, Cachier P, Ayache N. Understanding the "Demon's Algorithm": 3D non-rigid
registration by gradient descent. Med Image Comput Comput Assist Interv1999; 597606.
[6] Mumford D, Shah J. Optimal approximations by piece-wise smooth functions and
associated variational problems. Communic Pure Appl Math 1989; 42(5):577-685.
[7] Chan TF, Vese LA. Active contours without edges. IEEE Trans Imag Process 2001;
10(2):266-277.
[8] Guyader C, Vese LA. A combined segmentation and registration framework with a
nonlinear elasticity smoother. Scale Space and Variational Methods in Computer Vision,
2009; 600-611.
Miro Jurisic is born in Croatia in 1986. He got a Master degree in
Computational Physics from University of Split, Faculty of Natural Sciences in
2011. During his studies, he conducted research in many different field of
physics like superconductivity Ising model using GPUs.
Since February 2012, he is part of European FP7 project called SUMMER. His
task in the project is multimodal deformable image registration of medical
images. One of sub-problems in this project is the use of GPUs to accelerate
registration algorithms. His host institution is a Medical University of Vienna,
where he is currently enrolled in PhD program N094 for Medical Physics, under
supervision of Prof. Dr. Wolfgang Birkfellner.
F. Hauler et al.
17
Automated evaluation of multi-modal
image rigid registration
Frida Hauler1*, Miro Jurisic1, Hugo Furtado2, Ursula Nestle3, Wolfgang Birkfellner1,2
1
*
Center for Medical Physics and Biomedical Engineering, Medical University Vienna, Austria
2
Christian Doppler Laboratory for Medical Radiation Research for Radiation Oncology,
Medical University Vienna, Austria
3
University Klinikum Freiburg, Germany
frida.hauler@meduniwien.ac.at
Abstract: In radiotherapy (RT) complementary image information from multi-modal data is
used. More precisely, multi-modal image registration aligns images from different modalities like
computed tomography (CT) and cone beam CT or magnetic resonance imaging (MRI) into the
same frame of reference. For high precision dose planning, the accuracy of this registration
process of volume data is crucial; therefore a reliable and robust evaluation method for registered
images is needed in clinical practice.
The gold standard validation methods are visual inspection by radiation oncology experts and
evaluation based on fiducial markers. However, visual inspection is a qualitative measure with a
range of 2-6 mm inaccuracy, and it is time consuming and prone to errors. On the other hand, the
fiducial based evaluation is an invasive method when markers are fixated to bone or organs. In
clinical practice, a robust non-invasive automated method is needed to validate registration of
multi-modal images.
The aim of this study is to introduce and validate an automatic landmark-based accuracy measure
for multi-modal rigid registration using feature descriptors. For validation of the method, a
porcine dataset with fixed fiducial markers was used to compare our accuracy measure with the
fiducial registration error (FRE).
In addition the method robustness was tested on 10 lung clinical cases. After registration,
landmarks are automatically found by feature descriptors and a comparison of those intrinsic
landmarks yields target registration error (TRE) of point pair’s landmarks with manually
annotated landmarks TRE was carried out.
Index Terms — rigid registration evaluation, accuracy measure based on feature descriptors,
automatic landmark based evaluation
INTRODUCTION
In cancer treatment, radiotherapy (RT) is one of the main therapeutic measures next to surgery
and chemotherapy. The goal of RT is to give a high dose of ionizing radiation to tumour regions
called target volumes while sparing the surrounding healthy tissue. To successfully destroy the
tumour cells a dose of 50 to 90 Gy is necessary, delivered in a cycle of up to 30 daily fractions.
To spare the surrounding healthy tissues and organs at risk (OAR), utmost precision is necessary
to define the tumour structures (called clinical target volume - CTV) and exact beam control is
needed to deliver the high dose to the planned target volume (PTV).
In radiotherapy (RT) different image modalities help the diagnosis and target volume definition
18
Automated evaluation of multi-modal image rigid registration
providing different diagnostic information. These can be anatomical images as CT, MRI acquired
with different sequences, x-rays and ultrasound (US) or functional images, such as Positron
Emission Tomography (PET), functional MRI (fMRI) and single-photon emission computerized
tomography (SPECT). However, no single modality can contain all the diagnostic information for
reliable determination and delineation of malignant tissues.
To obtain better tumour targeting during RT treatment, complementary information from multimodal images needs to align into the same coordinate system using 3D-3D registration
algorithms. Registration is the determination of an optimal geometrical transformation which
aligns points in one dataset (moving image) with corresponding points in other dataset (fixed
image) taken at various points in time or by different scanners. Registration is a wide field with
an arsenal of proven algorithms, but still there is a gap in defining the accuracy of the registration
which for tumour delineation is crucial.
The current gold standard validation methods of registration are visual inspection by a radiation
oncology experts and fiducial-based evaluation. The visual inspection is a time consuming,
qualitative measure, depending on inter-observer variability between the experts and lacks a
standardized quantitative measure of registration accuracy. However, registration will be used by
experts and their opinion has crucial importance for validation. A more quantitative validation is
based on fiducial markers applied on surface or inside of the body. Fixed fiducial markers require
invasive intervention while fiducial markers attached to the skin can move. In any case the
fiducial-based validation is considered a gold standard evaluation for a quantitative measure of
registration error, calculating the FRE – usually the root-mean-square distance between
corresponding fiducial points [1] on different image modalities after the registration. Maurer et
al. [2] suggested two more useful measures of error analysing the accuracy of registration, the
fiducial localization error (FLE) - the error that stems from measuring the fiducial position - and
the target registration error (TRE) which is the distance between corresponding points other than
fiducial after registration.
In this paper we propose a reliable, automatic and non-invasive method for measuring the
accuracy of the registration outcome. For validating the new accuracy method, a porcine dataset
with fixed fiducial markers is used comparing our method with the target registration error of
fiducial.
Datasets
For testing the accuracy of the evaluation method, a multi-modality dataset of a porcine specimen
was used with seven fixed fiducial markers and known registration gold standard [3]. The dataset
is publicly available on http://midas3.kitware.com/midas/community/3. The pig skull was
supplied by the Department of Biomedical Research, Medical University of Vienna.
The reference dataset includes a CT, CBCT scans and MR-T1, T2 and PD weighted images.
Fiducial markers were fixed to the bony skull of the specimen, namely on the os rostale, os
nasale, one of the foramen supraorbital, foramen infraorbital and os lacrimale. The fiducial
markers of 10 mm diameter were made of steel (for kV x-ray imaging and megavoltage
electronic portal imaging), aluminium (for CBCT images) and polytetrafluoroethylene (for CT
images). Plastic hollow sphere were filled with olive oil for MR-compatible markers.
The CT volume were scanned by a 64-slice Spiral Philips CT scanner, consisting of 825 slices
with 0.8 slice thickness, each slice containing 512x512 pixel size with intra-slice resolution
0.63x0.63 mm2. The kV, MV images and CBCT images were acquired by Elekta Synergy linear
F. Hauler et al.
19
accelerator, equipped with electronic portal imaging device (EPID) and a CBCT system. In case
of CBCT imaging two images with different field of view (FoV) were acquired, one with 276
mm and the other with 425 mm. Imaging parameters for the small FoV were 540x540x520
voxels of 0.5 mm3 size and for the larger FoV 410x410x264 voxels of 1.0 mm3 size.
The 20 lung clinical datasets consist of CT scans acquired with a Helical GE Medical Systems
scanner. Image parameters were 512 x 512 x 95 voxels resolution, with 2.5 mm slice thickness
and 0.97 x 0.97 x 2.5 mm3 voxel size. The kV CBCT images have been acquired during
treatment with Elekta Linac, 2 mm slice thickness, 410 x 410 x 132 voxels of 1 x 1 x 2 mm3 size.
As the scanning time is in the range of 2 minutes no breath-holding was asked from the patients.
Rigid registration and image pre-processing
For rigid registration we used a commercial software package Analyze 11.0 (AnalyzeDirect Inc.,
Visualization and Analysis Software). Due to different slice thickness of multi-modal images,
rigid registration is sensitive to uncertainties. To avoid these uncertainties, all images were preprocessed by re-sampling to an isotropic voxel size of 1mm3 using cubic spline interpolations.
For validation of registration accuracy by fiducial it is important that the target point of interest is
not part of the set of registration points, so in porcine phantom data set fiducial markers are
masked out before registration. During registration the CT volume is considered the fixed image
and the CBCT the moving images. For registration, Analyze 11.0 uses Mutual Information
metric, so the fiducial markers are used only for evaluation of the registration accuracy.
Evaluation of rigid registration
To validate the accuracy of registration the gold standard way is to calculate the target
registration error (TRE), the distance between corresponding points other than fiducial points
from the two images after registration. In order to define the corresponding points of the target
points, anatomical landmarks have been determined manually or automatically using feature
descriptors.
In case of porcine phantom data, the manually annotated points were cervical C2 vertebra, the
brain, the maxilla and mandible in CT, CBCT and MR images using Analyze 11.0. The reason to
choose these anatomical landmarks was that they are not deform with the soft tissue and are
visible on both image modalities. These reference slices also contains the fiducial markers to
calculate the fiducial registration error (FRE) as the root-mean-square distance between
corresponding fiducial points. On same reference slices, the feature detection algorithm was
applied from all three views to calculate the target registration error (TRE) after registration as
distance between the corresponding points detected by feature descriptor and matched by
correlation (Fig.1).
Fig. 1: Finding automated llandmarks using SURF descriptor on porcine phantom dataset
The robustness of evaluation method was tested on 20 lung cases comparing the TRE calculated
between corresponding points obtained by manual landmarks against automatic landmarks based
on feature descriptors.
20
Fig. 2: Coronal view of Patient05, manually annotated landmarks of anatomical features on a.) CT, b.)
CBCT images. c.) Presumptive matches of pair-landmarks after correlation match. After eliminating outliers
using RANSAC on d.) CT and e.) CBCT slices, the inlier matches f.) on half transparent overlapped CT and
CBCT slices.
Manually annotated features have been chosen according to Grig et al. [4]: anatomical landmarks
such as the apex of the lung, aortic arch, heart, spine, sternum, carina (bronchus bifurcation) and
the tumour have been manually annotated by expert and checked by two radiologists.
In automatic case, after registration of CT and CBCT images, features on both fixed and moving
images are located using the SURF algorithm from the MatLab OpenSURF Computer Vision
Library [5].
Fig. 3: Anatomical landmarks found by SURF descriptors (a,b). Presumptive matches are shown in (c),
outlier elimination by RANSAC is shown in (d) for CT and CBCT (e). Matches on overlaid CT and CBCT
are shown in (f).
The interest points (distinctive locations like corners, blobs, T-junctions) are detected by a
Hessian detector (Fig. 2, 3: a, b). The neighbourhood of every interest point is represented by a
feature vector. The calculation time is directly proportional to the dimension of the descriptor, so
SURF detector relies on integral images [6] and only 64 dimensions are used to reduce the
computational time. The size of the filters is set by the octave parameter. Higher octaves use
larger filters and the image data to find larger size blobs. Increasing the number of scale levels to
compute per octave detect more blobs at finer scale.
Indifferent of the landmark's defining automatically or manually, the feature vector elements are
F. Hauler et al.
21
matched between both images by building a correlation matrix by pair of points which correlate
in both directions inside of a maximum search radius. From resulting presumptive match the
outliers have been eliminated (Fig.2 ,3: d,e) by RANdom SAmple Consensus (RANSAC) [7].
Validation of evaluation method
To validate the accuracy measure obtained by our approach the FRE of pig dataset is compared
with accuracy measure obtained as TRE between the corresponding points of CT and CBCT
features found by SURF descriptor and matched by correlation. According to Pawiro et al.
validation of the gold standard registration of the porcine data we calculated the fiducial
registration error (FRE) as the root-mean-square distance between corresponding fiducial points
after registration. Moreover, the fiducial localization error (FLE) for N=7 fiducial markers can be
calculated conform Eq. (1)
FLE 2 =
N
FRE 2 (1)
N 2
RESULTS
Phantom data set
The gold standard accuracy measure is considered to be the fiducial registration error (FRE)
between the corresponding fiducial points on CT fixed image, and CBCT moving image. This
value is providing a ground truth for validation of the accuracy measure based on feature
descriptors. For six fiducial measured from three views, the FRE and standard deviation
measured after registration of CBCT to CT volume was 1.8 ± 0.7 mm. The FLE using the Eq.1 is
2.1 mm.
TRE calculated between automatic landmark pair-points determined by SURF descriptor for all
three views was also 1.8 ± 0.7mm, on sagittal view 1.1 ± 0.5 mm, coronal view, 1.8 ± 0.7 mm
and on axial view, 2.0 ± 0.7.
Patient data
In absence of a gold standard, we compare the target registration error calculated between
manually annotated landmarks and the TRE of our method, between the matched pair-points
from landmarks found by SURF descriptor represented in Table 1.
DISCUSSION
Next to internal error due to organ motion, each diagnostic and treatment preparation step leads
to the accumulation of uncertainties including setup errors which finally affect the accuracy of
the defined PTV. Multi-modal image rigid registration, as a basic step before the deformable
registration and tumour delineation has an important role to determine the accuracy of PTV and
this role will get more importance especially in particle therapy which more precisely localizes
the radiation dosage of proton beam.
An evaluation method for rigid registration accuracy needs to accomplish several criteria to be
viable in clinical practice as to be accurate, robust, fast, automatic and non-invasive. Our method
is automatic, non-invasive and fast. We need to prove the accuracy and the robustness of the
method, validating with existent accuracy evaluation methods.
The mean of the TRE based on six fiducial markers from the phantom porcine head and the mean
of TRE, the distance between the inlier pair-points obtained by our method was equal. We need
to double check this result, calculating the FRE instead of TRE based on fiducial markers.
In patients case, due to breathing motion and the different histogram content of the CT and
22
CBCT images the accuracy measure on lung cases has higher range of inaccuracy than on the pig
data. We found that, automated accuracy measure is 0.5 mm inaccurate than the manual
annotation which is acceptable taking in consideration the saved time. Inaccuracies in patients’
data set incorporate both the registration error and the inaccuracy of our method.
automatic
Patients A
C
S
manual
stdev(A) stdev(C) stdev(S) A
0.9
C
S
FP001
2.8 1.9 2.6
0.7
1.1
FP002
2.3 2.9 2.2
0.9
0.9
FP003
2.3 2.8 2.4
0.8
0.9
1.1
FP004
2 2.5 2.7
0.9
0.9
0.6 2.2 1.7
FP005
2.1 2.7 2.6
1.2
0.9
1.1
AP001
3 2.2 2.8
0.7
AP002
2.6 2.3 2.9
AP003
stdev(A) stdev(C) stdev(S)
2 1.4 2.2
0.1
0.2
4.1
2
0.6
0.8
0.1
2 1.7 1.7
0.4
0.8
0.5
2
2.1
0.7
0.7
2 2.2
0.7
0.7
1.4
0.7
1 2.1 2.2 2.1
2.1
1.1
1.1
1
0.9
0.9 2.1 2.2 2.1
0.9
0.8
1.1
3.1 2.7 2.4
1
1
0.9 2.2 1.7 2.1
1.2
1
0.6
AP004
2.8 2.5 2.8
0.9
1
1.1
0.7
0.8
AP005
2.1 3.1 2.4
1
0.8
0.9
2 1.9
1.1
0.9
Total
2.5 2.6 2.6
0.9
0.9
0.9
2 1.8 2.1
1
0.8
1 1.6 1.4
2
1 1.4
2 1.9
0.8
Table 1: TRE for corresponding landmarks detected by SURF descriptor and manually annotated
landmarks.
CONCLUSION
Based on the results obtained, defining an automatic accuracy measure using the SURF feature
detection algorithm can be considered a promising method. In the future we aim to integrate the
validation into an open-source framework and make publicly available, to provide an automatic
non-invasive accuracy measure for registration algorithms. As a future perspective for the
validation method based on 2D feature detection could also be elaborated for 3D features.
ACKNOWLEDGMENT
H. Furtado is supported by Christian Doppler Laboratory for Medical Radiation Research,
Medical University Vienna. We also gratefully thank for the patient data provided by the
F. Hauler et al.
23
Christian Doppler Laboratory for Medical Radiation Research for Radiation Oncology
Department, Medical University Vienna.
We would like to express our gratitude for the visual inspection for lung patients to Ursula Nestle
from University Klinikum Freiburg and to Claudio Spick and Stephan Polanec radiologists from
Department of Radiology, Medical University Vienna for analysing the correctness of manual
annotation.
REFERENCES
[1] Bay H, Tuytelaars T, Van Gool L. SURF: Speeded up robust features. Lecture Notes in
Computer Science 2006; 3951:404-417.
[2] Fitzpatrick JM, Hill DL, Shyr Y, West J, Studholme C, Maurer CR Jr. Visual assessment of
the accuracy of retrospective registration of MR and CT images of the brain. IEEE Trans
Med Imag 1998; 17(4):571-585.
[3] Pawiro S, Markelj P, Pernuš F, Gendrin C, Figl M, Weber C, Kainberger F, NöbauerHuhmann I, Bergmeister H, Stock M, Georg D, Bergmann H, Birkfellner W. Validation for
2D/3D registration I: a new gold standard data set. Med Phys 2011; 38:1481-1490.
[4] Grgic A, Nestle U, Schaefer-Schuler A, Kremp S, Kirsch CM, Hellwig D. FDG-PET-based
radiotherapy planning in lung cancer: optimum breathing protocol and patient positioningan intra-individual comparison. Int J Rad Oncol Biol Phys 2009; 73(1):103-111.
[5] Evans C. OpenSURF - open source SURF feature extraction library. 2009, Notes on the
OpenSURF Library.
[6] Lowe DG. Object recognition from local scale-invariant features. Proc IEEE Int Conf
Computer Vision 1999; 2:1150-1157.
[7] Zuliani M, Kenney CS, Manjunath BS. The multiransac algorithm and its application to
detect planar homographies. Proc Int Conf Imag Process 2005; 3:153-156.
Frida Hauler received her Mag. degree in Mathematical-Physics and Computer
Science at Faculty of Mathematics and Computer Science of Transilvania
University of Brasov, Romania. She graduated as Biomedical Engineer Msc. at
Budapest University of Technology and Economics, Budapest, Hungary.
She worked as PACS interface developer at Innomed Medical Zrt., as Test
Engineer at Knorr Bremse and QA Engineer at Cognex, Budapest. Currently, she
is a PhD student at Center for Medical Physics and Biomedical Engineering at
Medical University Vienna, Austria. Her research subject is “Multi-modal
volume registration techniques ready for clinical use in radiotherapy”
24
Segmentation of subcortical structures on MRI by using machine learning techniques
Subcortical structures segmentation on
MRI using support vector machines
Jose Dolz1, 2*, Hortense A. Kirisli1, Maximilien Vermandel2 and Laurent Massoptier1
1
2
*
AQUILAB, Loos Les Lille, France
Inserm U703, Université Lille 2, CHRU Lille, Loos Les Lille, France
jose.dolz@aquilab.com
Abstract: Medical imaging has evolved during the last years to become a fundamental tool for
diagnosis, treatment and follow-up of patient diseases. Particularly, in oncology, medical imaging
plays a key role in the diagnosis, treatment and follow-up of brain tumours. Magnetic resonance
imaging (MRI) is often the medical imaging method of choice when soft tissue delineation is
necessary. However, in clinical practice, organs at risk (OARs) delineation is often still
performed manually by experts, or with very few machine assistance. As a consequence, the
current delineation process has two major drawbacks: it is time consuming, and achieves low
reproducibility. Although several methods to (semi-) automatically segment subcortical structures
on MRI have been proposed to overcome these limitations, segmentation still remains
challenging, with no general and unique solution. Among these methods, machine learning
techniques, and more specifically support vector machines (SVM), have proved to outperform
most of the proposed methods. Hence, SVM can be considered as state of the art in regards to the
segmentation of subcortical structures.
Index Terms — Support Vector Machines, MRI, subcortical structures, segmentation.
INTRODUCTION
Medical imaging, which was initially used for basic visualization and inspection of anatomical
structures, has evolved during the last years to become an essential tool for virtually all major
medical conditions and diseases. Particularly, in oncology, advanced medical imaging techniques
are used for tumour resection surgery and for subsequent radiotherapy treatment planning (RTP).
Medical imaging plays a key role in the diagnosis, treatment and follow-up of brain tumours,
which are nowadays the second most common cause of cancer death in men ages 20 to 39 and the
fifth most common cause of cancer among women age 20 to 39 [1]. In daily clinical practice,
magnetic resonance imaging (MRI) is often the medical imaging method of choice when soft
tissue delineation is necessary.
During RTP, the tumour to irradiate, i.e. clinical target volume (CTV), as well as healthy
structures to be spared, i.e. the organs at risk (OARs), must be precisely delineated. These
segmentations are crucial inputs for the RTP, in order to compute the parameters for the
accelerators, and to verify the dose constraints. However, in clinical practice, OARs delineation
on medical images is still performed manually by experts, or with very few machine assistance
[2]. As a consequence, the current delineation process has two major drawbacks: it is time
consuming, and achieves poor reproducibility. To overcome these major issues, various
computer-aided systems to (semi-) automatically segment anatomical structures in medical
images have been developed and published in recent years. However, (semi-) automatic
J. Dolz et al.
25
segmentation of subcortical brain structures still remains challenging, with no general and unique
solution.
Initial approaches of brain segmentation on MRI focused on the classification of the brain into
three main classes: white matter (WM), grey matter (GM) and cerebral-spinal fluid (CSF) [3].
More recent methods include tumours and adjacent regions, such as necrotic areas, in addition to
the primary cerebrum tissues [4]. Those methods are only based on image intensity. Because of
the weak visible boundaries and similar intensity values between different subcortical structures
(i.e. OARs), segmentation of subcortical structures can hardly be achieved based solely on signal
intensity. Consequently, additional information, such as prior shape, appearance and expected
location, is therefore required to perform the segmentation.
The terminology “subcortical structures” as used in this chapter refers to subcortical GM
structures within the brain that are not included as part of the cortex and are present in the depth
side of the brain. In addition, the hippocampus, which is often considered a cortical structure, is
included in our definition of subcortical structures.
Several methods to segment subcortical structures on MRI have been proposed [5-8]. Atlas-based
segmentation methods are among the most used techniques to perform the segmentation of such
structures. These models rely on comparing the images under study with a pre-computed
anatomical atlas of the brain. In [5], an extended review of the use of atlas-based segmentation
methods to segment subcortical structures on MRI is presented. In addition to atlas-based
methods, which only use a priori shape information, statistical models of shape and texture have
been also employed [6-7]. In these approaches, correspondences across a training dataset are
established, and the statistics of shape and intensity variation are learned and parameterized in
terms of mean and eigenvectors, often by using principal component analysis (PCA). New
instances are therefore constrained to a subspace of allowable shapes and textures, which are
defined by the eigenvectors and their modes of variation. As a consequence, statistical models
may be over-constrained, not generalizing well to un-sampled population, particularly for small
amounts of training data relative to the dimensionality. Contrary to statistical models, deformable
models provide flexibility and do not require explicit training. Deformable models are defined as
curves or surfaces, which are deformed under the influence of internal and external forces. While
internal forces are related to curve features and try to keep the model smooth during the
deformation process, external forces are the responsible of attracting the model toward object
boundaries, and are related to image features of regions adjacent to the curve. Nevertheless, they
are sensitive to initialization, noise and complex topologies. This makes deformable based
segmentation methods being used in combination with other approaches, like in [8], where the
evolution of the deformation is constrained by using a statistical model.
Machine Learning techniques have been extensively used in the MRI analysis domain almost
since its creation. Among all the existing machine learning techniques, support vector machines
(SVM) represents one of the latest and most successful statistical pattern classifiers, and it has
received a lot of attention from the machine learning and pattern recognition community.
Although SVM approaches have been mainly employed for brain tumour recognition [9] in the
field of medical image classification, recent works have also used them for tissue classification
[10] and segmentation of anatomical human brain structures [11-13]. By introducing machine
learning methods, algorithms developed for medical image processing often become more
intelligent than conventional techniques. Powell et al. [11] showed the improvements in the
resulting relative overlaps when using machine learning methods (artificial neural networks
(ANN) and SVM) to segment subcortical structures [11]. In this work, four methods were
26
compared: template based, probabilistic atlas, ANN and SVM. It was showed that machine
learning algorithms outperformed the template and probabilistic-based methods when comparing
relative overlap between results obtained.
Because of their incapability to generate new un-sampled shapes, which considerably differs
from the shapes in the training set, most of the techniques previously presented (except machine
learning methods) might fail in the presence of brain lesions, such as tumours. This makes
machine learning methods, and particularly SVM approach, more suitable techniques to perform
the segmentation of subcortical structures, especially in such situations. Hence, SVM and its
application to the segmentation of subcortical structures will be the focus of this chapter. In the
next section, details of the basis of SVM and its use as optimizer for the segmentation problem
applied to subcortical structures are presented. In addition, the experimental work carried out is
also described in that section. In Results, some outcomes of the experimental work using the
proposed approach are presented. The paper concludes with some outlines plans for future work.
Support vector machines: the basics.
Support vector machines and their variants and extensions, often called kernel-based methods,
have been studied extensively and applied to various pattern classification and function
approximation problems. Basically, the main idea behind SVM is to find the largest margin
hyper-plane that separates two classes. The minimal distance from the separating hyper-plane to
the closest training example is called margin. Thus, the optimal hyper-plane is the one showing
the maximal margin, which represents the largest separation between the classes (Fig.1.b). The
training samples that lie on the margin are referred as support vectors, and conceptually are the
most difficult data points to classify. Therefore, support vectors define the location of the
separating hyper-plane, being located at the boundary of their respective classes.
b)
c)
a)
Solution
Mapping
Kernel
Function
Figure 1. Process of mapping input samples to a higher dimensionality space to make the data linearly
separable.
The growing interest on SVM for classification problems lies in its good generalization ability
and its capability to successfully classify non-linearly separable data. First, SVM attempts to
maximize the separation margin –i.e., hyper-plane- between classes, so the generalization
performance does not drop significantly even when the training data are limited. Second, by
employing kernel transformations to map the objects from their original space into a higher
dimensional feature space [14], SVM can separate objects which are not linearly separable (Fig
1). Moreover, they can accurately combine many features to find the optimal hyper-plane.
Therefore, SVM globally and explicitly maximize the margin while minimizing the number of
wrongly classified examples, using any desired linear or non-linear hyper-surface.
J. Dolz et al.
27
Use of SVM to segment subcortical structures
SVM approach has been successfully applied to the segmentation of subcortical structures.
Powell et al. [11] compared the performance of ANN and SVM when segmenting subcortical
(caudate, putamen, thalamus and hippocampus) and cerebellar brain structures. In their study the
same input vector was used in both machine learning approaches, which was composed by the
following features: probability information, spherical coordinates, area iris values, and signal
intensity along the image gradient. Although obtained results were very similar, ANN based
segmentation approach slightly outperformed SVM. However, they employed a reduced number
of brains to test (only 5 brains), and 25 manually selected features, which means that
generalization to other datasets was not guarantee.
In machine learning, during the training of classifiers, if the number of image features is large, it
can lead to ill-posing and over fitting, and reduce the generalization of classifiers. One way to
overcome this problem is to reduce feature dimensionality. For this purpose, PCA was used in
[12], followed by a SVM classification to identify statistical differences in hippocampus.
However, selection of the proper discriminative features is not a trivial task, which has already
been explored in the SVM domain. To overcome this problem, AdaBoost algorithm was
combined with a SVM formulation [13]. In a first step, AdaBoost was used to select the features
that most accurately span the classification problem. Then, SVM fused the selected features
together to create the final classification. Furthermore, four automated methods for hippocampal
segmentation using different machine learning algorithms were compared: hierarchical
AdaBoost, SVM with manual feature selection, hierarchical SVM with automated feature
selection (Ada-SVM), and a publicly available brain segmentation package (FreeSurfer). In their
proposed study, the benefits of combining AdaBoost and SVM approaches were evaluated
sequentially.
Experimental set-up
Input SVM
Vector Element
V1
V2
V3
V4
V5
V6 – V9
V10-V21
Explanation
Intensity value of the pixel under examination
Angle between pixel and centre point with respect to the horizontal line
Distance from the pixel under examination to the centre point
Probability Map value
Geodesic Map distance
4 Gradient image values across the largest gradient
12 signal intensity values along each of the two axis (i.e. 3 pixels for each side)
Table 1. Features that are used in the SVM input vector.
To present robustness and efficacy of the use of SVM to optimize segmentation problem, corpus
callosum was segmented in a set of sagittal MRI images. A set of 16 sagittal images containing
the corpus callosum and 16 manual labelled masks were used. Each input vector for the SVM
classifier consisted of 21 elements, and it was formed by the elements shown in Table 1.
Regarding the kernel selection to map the training samples, a Radial Basis kernel was used for
the purpose of this chapter. SVM segmentation method was divided into two steps: training and
classification. While for the training step 7 cases were selected, for the classification step the 16
available cases were used.
The first step was to create a binary mask with the manual labelled masks. This mask was
computed by applying an “or” operation to all input labels in the training set. To make sure that
corpus callosum was inside the mask in all input images, a security margin was provided to the
created mask. This mask was applied to all the images both in the training and in the
classification steps in order to prune the image pixels. Features selected as components of the
input SVM vectors are therefore extracted from the inner mask region (Fig 2).
28
The probability map was derived from the manual label associated to each image. The skeleton
was extracted from this label, and a Gaussian distribution from the resulted skeleton was applied
to obtain a simulated probability map. Instead of using the whole skeleton, only its centre point
was used as mask to compute the geodesic distance [15] in the training step. For the classification
step, however, the used mask is the computed skeleton of the input label (Fig 3).
2D
Features
MR Images
Manual Labels
Common Mask
Training Set
Common Mask applied
to all the images
Extract
Features
SVM
Training
Model
Training
Map pixel pruning using the common mask
Figure 2. SVM Training Process.
Input Data
Manual Approximated
Label
Skeleton from the
manual label
Probability Map from
skeleton
Geodesic Map from the
center of the skeleton
Figure 3. Process of obtaining the SVM input vector features of an input image.
To test the reliability of the segmentation algorithm, two segmentation results were compared to
manually defined regions. First, the output of the classification proposed approach with no post
processing was used. Second, a post processing step was applied to the classifier output. In this
process, isolated small blobs were removed from the segmentation result. The results reported in
this chapter were provided by computing the Dice similarity coefficient (DSC). The DSC(X,Y) is
defined as the ratio of twice the intersection over the sum of the two segmented results, X and Y.
According to this, DSC > 0.8 represents high agreement, 0.6 < DSC ≤ 0.8 indicates substantial
agreement, and 0.4 < DSC ≤ 0.6 moderate agreement.
RESULTS
Experiments demonstrated that the classification approach proposed in this chapter performed
well when segmenting the corpus callosum. From Fig 4.a, it can be observed that 13 out of 16
cases reported DSC values higher than 0.8 for both with and without post-processing. Mean DSC
values obtained by the proposed approach were 0.85 – with a standard deviation value of 0.07 –
for the non-post processing cases, and 0.89 – with a standard deviation value of 0.05 – for the
cases where the post processing was applied. Regarding the influence of the post processing step,
it increased the DSC values of the non-processed results around 3-4% as average.
An important aspect to take into account when working with learning algorithms is the time
required for the search, as well as for the training. In the proposed experiment, MATLAB was the
language chosen, and it run over an Intel Xeon processor at 3.06 GHz. The time required to
extract all the features in all the images used as training set was around 24 seconds in total. With
this set of input features, the SVM training took nearly 18 seconds. In the other hand, the
proposed approach segmented each of the input images in a time close to 3.5 seconds, where the
classification process represented around 25% of this time, and the rest was the feature
extraction.
J. Dolz et al.
(a)
29
(b)
Overlapping SVM Classification (Corpus Callosum)
1
0,9
0,8
Dice's Coefficient
0,7
0,6
Dice Score
without
Post
Processing
0,5
0,4
(c)
0,3
0,2
0,1
Brain 16
Brain 15
Brain 14
Brain 13
Brain 12
Brain 11
Brain 9
Brain 10
Brain 8
Brain 7
Brain 6
Brain 5
Brain3
Brain 4
Brain2
Brain1
0
Figure 4. (a) DSC obtained by the proposed approach for all the brain cases used in this work. Result
segmentation example of the proposed approach without (b) and with (c) post processing.
DISCUSSION
MRI is widely used to identify subcortical brain structures for diagnosis, treatment and follow-up
in brain tumours cases. During RTP, these subcortical structures (i.e. OARs) must be precisely
delineated, which is currently done manually by experts, or with very few machine assistance.
OARs delineation is therefore a time consuming process with poor reproducibility in clinical
practice. The automatic segmentation method presented in this chapter is motivated by these
limitations. By introducing machine learning methods, algorithms developed for medical image
processing often become more intelligent than conventional techniques. Machine learning
techniques - and SVM in particular - outperform classical segmentation methods, leading to
improvements in the resulting relative overlaps as reported in the work of Powell et al [11].
Particularly, segmentation of the corpus callosum has been proposed and evaluated in this
chapter. The use of SVM - as trained in the proposed experiment - to automatically segment the
corpus callosum evidences a high agreement between automatic segmentation result provided
and manual labels. Experiments also showed that in some cases, overlapping between automatic
segmentation and manual labels is considerably lower than in other cases (Fig 4.a). These cases
showed to have an intensity distribution of the corpus callosum different from the samples in the
training set. During the training phase, these intensity values were not sampled as a part of the
input vector belonging to the corpus callosum. As a consequence, during the search process,
input samples containing these intensity values were not properly classified. The introduction in
the dataset of a range of samples that can represent a wide variability would improve the
segmentation in such situations. In addition, although a post processing step of the output does
not highly increase the DSC, it removes small labels that do not belong to the object to segment
(Fig 4.b and 4.c).
The time required by the proposed approach (both for training and search) was not considerably
high. However, it has to be noted that only 2D images were used. Since the inclusion of more
features and the use of volumes instead of images may be required to improve the segmentation
result, the time required might dramatically increase, becoming an impractical approach. One
solution to overcome this issue is to reduce the dimension of the input features, removing
redundant features from the input vectors. As in the work of [12], PCA can be successfully
30
applied to solve this.
In the presented experiment, the probability map used as feature of the input vector was extracted
from the manually labelled mask. Ideally, this probability map would be obtained from the
registration step, as in [11], where the input image is registered with an atlas and the labels are
propagated. This step makes the segmentation challenging, particularly in those subcortical
structures which are close to brain lesions. If deformation caused by the lesion is not accordingly
interpreted, the probability map, and consequently the segmentation result might fail.
CONCLUSION
An automatic approach to segment the subcortical structures on MRI has been presented.
Although it has been only evaluated in one subcortical structure, there are some recent works that
have proved the efficiency of machine learning methods when segmenting subcortical structures.
The purpose of this chapter is to give an idea of how SVM works and some applications that have
already used it for the segmentation.
The main direction for future research is to examine the extension of this method to a set of
subcortical structures which are involved in external radiotherapy and radio-surgery. Since a fully
automatic approach is highly demanded in clinical practice, the use of the propagated labels to
create the probability map is inside of our scope for this research. Additionally, we aim to extend
this approach to its use in 3D images. However, as explained before, some considerations have to
be taken into account in this case. As a consequence, the inspection of the dimensionally
reduction of the features used as input vector is also required.
ACKNOWLEDGMENT
REFERENCES
[1] American Cancer Society. Cancer Facts & Figures 2014. American Cancer Society; 2014.
[2] Whitfield GA, Price P, Price GJ, Moore CJ. Automated delineation of radiotherapy
volumes: are we going in the right direction? The British journal of radiology 2013;
86(1021):20110718-20110718.
[3] Xuan J, Adali T, Wang Y. Segmentation of magnetic resonance brain image: integrating
region growing and edge detection. Proc Int Conf Imag Process 1995; 3:544-547.
[4] Ahirwar A. Study of techniques used for medical image segmentation and computation of
statistical test for region classification of brain MRI. Int J Inf Tech Comput Sc 2013, 5(5).
[5] Cabezas M, Oliver A, Lladó X, Freixenet J, Bach Cuadra M. A review of atlas-based
segmentation for magnetic resonance brain images. Comput Meth Prog Biomed 2011;
104(3):e158-e177.
[6] Babalola KO, Cootes TF, Twining CJ, Petrovic V, Taylor C. 3D brain segmentation using
active appearance models and local regressors. Med Image Comput Comput Assist Interv
2008; 401-408.
[7] Rao A, Aljabar P, Rueckert D. Hierarchical statistical shape analysis and prediction of subcortical brain structures. Med Imag Anal 2008; 12(1):55-68.
J. Dolz et al.
31
[8] McIntosh C, Hamarneh G. Medial-based deformable models in nonconvex shape-spaces for
medical image segmentation. IEEE Trans Med Imag 2012; 31(1):33-50.
[9] Zhou J, Chan KL, Chong VF, Krishnan SM. Extraction of brain tumour from MR images
using one-class support vector machine. Conf Proc IEEE Eng Med Biol Soc 2005; 6:64116414.
[10] Akselrod-Ballin A, Galun M, Gomori MJ, Basri R, Brandt A. Atlas guided identification of
brain structures by combining 3D segmentation and SVM classification. Med Image
Comput Comput Assist Interv 2006; 9(Pt2):209-216.
[11] Powell S, Magnotta VA, Johnson H, Jammalamadaka VK, Pierson R, Andreasen NC.
Registration and machine learning-based automated segmentation of subcortical and
cerebellar brain structures. Neuroimage 2008; 39(1):238-247.
[12] Golland P, Grimson WE, Shenton ME, Kikinis R. Detection and analysis of statistical
differences in anatomical shape. Med Image Anal 2005; 9(1):69-86.
[13] Morra JH, Tu Z, Apostolova LG, Green AE, Toga AW, Thompson PM. Comparison of
AdaBoost and support vector machines for detecting Alzheimer's disease through automated
hippocampal segmentation. IEEE Trans Med Imaging 2010; 29(1):30-43.
[14] Burges CCJC. A tutorial on support vector machines for pattern recognition. Data mining
and knowledge discovery 2.2 (1998): 121-167.
[15] Criminisi A, Sharp T, Blake A. Geos: Geodesic image segmentation. Computer Vision–
ECCV 2008; 99-112.
Jose Dolz attended the Polytechnics University of Valencia (Spain) as an
undergraduate, where he received his MSc degree in telecommunications and
electrical engineering in 2010. After earning his MSc degree, he has worked at
the university and in the private industry as computer vision researcher. He is
currently an Early Stage Researcher at Aquilab, Lille, France. In addition, he is
also enrolled as PhD candidate at the Ecole Doctorale Biologie et Santé of Lille 2
University. His research lies primarily within the ﬁelds of image processing and
computer vision, where his work and research interests within these fields are
image segmentation, feature extraction, image tracking and augmented reality.
His work in image segmentation has been motivated and directed toward
problems in medical imaging, especially in radiology treatment plans and radiosurgery.
Hortense Kirişli comes from Chamonix-Mont-Blanc, France. In 2008, she
obtained her Engineering degree in Electronics from the ENSEEIHT (Toulouse,
France). Then, her research focused in cardiovascular image analysis at the
Biomedical Imaging Group Rotterdam, the Netherlands, where she obtained her
Doctorate degree in June 2013.Since, Hortense is working as a R&D Engineer at
Aquilab, Lille, France. She is developing software prototypes that make use of
multi-modality imaging techniques for improved personalized external
radiotherapy treatment planning, as part of the European 'SUMMER' project. She
is leading the ‘technical research integration and quality assurance’ workpackage, and contributes to two other work-packages, dealing with the design of
evaluation protocols, the clinical database, the design of user-interfaces, as well
as user-testing studies. Hortense's main expertise is medical imaging technology
research and its translation for clinical end users.
32
Evaluation of 4D PET segmentation algorithm
Evaluation of 4D PET tumour
segmentation algorithm with dynamic
experimental phantom measurements
Montserrat Carles1*, Tobias Fechter1, Ursula Christ2, Alin Chirindel1, Andrea Schaefer3,
Michael Mix2 and Ursula Nestle 1
1
Department of Radiation Oncology, University Medical Center Freiburg, Germany
Department of Nuclear Medicine, University Medical Center Freiburg, Germany
3
Department of Nuclear Medicine, University Medical Center Homburg, Germany
2
*
Montserrat.carles@uniklinik-freiburg.de
Abstract: Retrospectively gated 4D PET/CT is a valuable clinical tool to improve target
definition and to assess lesion motion for radiotherapy planning. An approach to reduce the
variability of delineation consists in relying on automatic or semi-automatic segmentation
methods. Validation of accuracy (fidelity to the truth) and robustness (reproducibility) are crucial
steps for clinical application of any computer algorithm. However, it requires the identification
of a gold standard. In this work, a semi-automated contrast oriented algorithm for tumour
delineation adapted to 4D PET images is validated with dynamic experimental phantom
measurements. Phantom images have the main advantage that object properties can be easily
measured and modified. Phantoms employed have been chosen based on their design properties,
with the aim of reproducing main degrading factors in lung tumour contour: size, shape and
movement of the target as well as target to background ratio (TBR). Results show that apart from
target diameter (ф) lower than three times system spatial resolution (FWHM), no other of these
parameters compromises the algorithm response. For target volumes with ф > 3FWHM, an
average accuracy in activity estimation (Ameasured/Atrue) of 0.9 ± 0.5 has been obtained.
Furthermore, for all the measurements, diameter and maximum excursion difference between
measured and true values are lower than system spatial resolution. From the results, it could be
concluded that the semi-automated contrast oriented algorithm adapted to 4D PET is applicable
to a broad range of cases with an acceptable accuracy. From our point of view, this evaluation
proves that it is reasonable to consider the feasibility of this algorithm for clinical use.
Index Terms — 4D, evaluation, phantom, segmentation, PET/CT.
INTRODUCTION
Quantitative analysis of PET/CT images has become an established method for diagnosis, staging
and evaluation of therapy tumour response [1], [2]. Accurate volume definition is significant
important in radiotherapy because it represents the targeted volume by the radiation beam. The
functional information conveyed by PET has been proved to be useful in the definition of target
volumes for various pathological entities [3]. However, several challenges with PET image
segmentation are recognized mostly related to the low spatial resolution and high noise
characteristics of PET images. In addition to these, respiratory organ motion has been identified
M. Carles et al.
33
as a potential major source of error in tumour localization for PET/CT imaging [4]. On modern
PET/CT scanners it is now possible to perform retrospectively respiratory gated imaging [5], [6].
As a result of this data processing, an improvement of image quality and better accuracy in tracer
concentration estimation should be obtained by the compensation of motion effects.
Manual delineation of target volumes on PET images is very laborious, time consuming and
suffers from significant variability, even among experts. An approach to reduce the variability of
delineation consists in relying on automatic or semi-automatic segmentation methods [7]-[10].
Validation of accuracy (fidelity to the truth) and robustness (reproducibility) are crucial steps for
clinical application of any computer algorithm. Empirical evaluation of segmentation algorithm
consists on to judge the quality of the segmentation algorithms by applying them to test images
and measuring the quality of segmentation results. Several types of test images can be used in
validation. Phantom images have the main advantage that the experimenter can easily measure
and modify the true object properties and compare them to those obtained by the delineation
algorithm. Besides, in contrast to the simulated images, experimental measurements lead to
images that contain the same exact system degradation factors as the clinical images.
In this work we focus on the empirical validation of a 4D-PET segmentation algorithm with
dynamic experimental phantom measurements. Phantom features permit evaluation of algorithm
robustness when different parameters are varied. These parameters are chosen based on their
relevance in lung tumour delineation with PET images: target to background ratio (TBR) as well
as size, shape and movement of the target. By comparing true values of phantom properties with
results obtained in the delineation, the algorithm performance is easily and precisely evaluated.
Semi-automated segmentation algorithm
The semi-automated segmentation algorithm evaluated is based on the adaptive thresholding
algorithm (contrast oriented algorithm) developed and published by the Homburg group [11],
[12]. In the present approach, this algorithm is adapted for retrospectively gated PET images and
background is calculated from image histogram [13]. The semi-automated character of this
algorithm relies on that the user is required to indicate a tumour pixel in one of the imaging
frames. Subsequently, the target volume is automatically delineated in all frames of the PET
acquisition. This algorithm is described in detail by Tobias Fechter in section “A threshold and
region-growing algorithm for 18FDG-PET 4D GTV delineation” of this book.
a) Photo courtesy of the company
b)
Fig. 1: Medical QUASAR respiratory motion phantom (a). Cylindrical insert developed by Homburg with
the oval glass placed in off-centre position (b).
34
4D PET/CT
PET/CT scans were performed on the Medical Philips System GEMENI TF 16 Big Bore. This
system has a transverse (axial) spatial resolution of 4.4 (4.7) mm at 1 cm from the camera central
axis and 4.9 (5.1) mm at 10 cm. The 10 min/bed PET acquisition was retrospectively gated in 10
phases based on the breathing curve provided by a pressure sensor belt. Data was sorted, divided
and reconstructed taking into account 10 equal-time intervals between consecutive maximum
amplitudes of the breathing curve. The BLOB-OS-TOF image reconstruction led a pixel size of
0.117x0.117x0.2cm3 and 0.4x0.4x0.4 cm3 for CT and PET respectively.
TBR and Target Size in a sinusoidal movement along SI direction
The NEMA NU2 2001 Image quality phantom, consisting of a body phantom and an insert with
six hollow spheres of various sizes, was used. Contrast oriented algorithm for 3D PET image was
previously validated with phantom measurements (spheres ranged from 173 to 14 ml) with
different TBRs (values ranged from 20 to 3) and a target activity concentration of 43 kBq/ml
[12]. In the present evaluation of the 4D algorithm version, we focus on activity values within
the range observed in a previous lung cancer 4D image quantification study (APPENDIX). The
aim is to validate the feasibility of this 4D implementation for high noise characteristics of 4D
PET images in lung tumour. Sphere volumes of the NEMA phantom ranged from 0.5 to 25 ml
and three different measurements were carried out with TBRs 33, 28 and 10. Target activity
concentration ranged from 10 to 16 kBq/ml and the background concentration from 1 down to 0.5
kBq/ml. The NEMA phantom was placed on the QUASAR motion platform. For all the
measurements, same movement was applied to the NEMA phantom: respiratory motion was
simulated with the approach of a sinusoidal movement along superior-inferior (SI) direction
(A=20 mm, T=4.5 s) [14] [15].
3D Movement and Shape
Patient’s breathing leads to a complex tumour movement which is more significant in the SI
direction with a smaller displacement in anterior-posterior (AP) and lateral (LR) directions [16].
By this approach, the final pathway of the target is actually a 3D movement. In this set of
measurements, the Medical QUASAR respiratory motion phantom, 30x20x12 cm3 body shape,
Fig.1a, was employed. Manufactured cylindrical insert with a glass fillable volume in an offcentre position, Fig.1b, was placed on the body phantom. This insert was connected to a stepper
motor that induced rotation (10mm and 4.5s) and translation (20mm and 4.5s) motion respect to
the static body phantom. Three different target geometries were considered for the same
movement and TBR (12.8 target, 1.6 kBq/ml background): 13 and 25 ml spheres and a 7 ml oval.
Irregular 1D Movement
In order to consider realistic breathing patterns, 3 irregular waveforms provided by the QUASAR
motion platform were applied along the SI direction to the oval target, see Fig.2, according to the
set-up employed in the previous section, see Fig.1.
Data Analysis
For each measurement, the 4D segmentation algorithm was applied to 10 PET frames, which
were retrospectively reconstructed from PET/CT acquisition. Due to their relevance in
radiotherapy planning, the analysis of the algorithm response involved volume delineation and
position tracking. Position tracking refers to position of the centre of gravity of the segmented
volume along different frames. In addition, average activity concentration estimation for the
segmented volumes is also analysed. The parameters chosen for the evaluation of volume and
activity response are: diameter difference and activity recovery coefficient (фmeasured - фtrue
M. Carles et al.
35
and Ameasured/Atrue, respectively), both of them averaged over the 10 frames (µ ± σ). For the
analysis of tracking position response, maximum excursion of the target is evaluated (ME).
(a)
(b)
(c)
Fig. 2: Irregular motion patterns along the SI direction applied to the oval target within the QUASAR
phantom: P1 (a), P2 (b), P3 (c). The same pattern is repeated throughout imaging acquisition.
RESULTS
TBRs and Target Size in a sinusoidal movement along SI direction
Algorithm response for different TBRs and target sizes are studied in this section. Results
obtained for diameter difference and activity recovery coefficient are shown in Fig.3.
Target position tracking resulted from segmented volumes along frames reproduces the
sinusoidal movement. The average value of maximum excursion (ME) calculated over all NEMA
phantom spheres is 42.2 ± 1.4 mm (MEtrue: 40mm).
(a)
(b)
Fig. 3: Algorithm response for the NEMA phantom in a sinusoidal SI movement. Diameter difference (a)
and activity recovery coefficient (b).
3D Movement and Shape
In this section, we study the effects on the algorithm response when circular and translational
movements are applied simultaneously to the insert with respect to the body phantom, Fig.1.
Three different targets are considered within the insert: sphere 13ml (S2), sphere 25ml (S3) and
oval 7ml (O2). The translation movement applied in this measurement does not differ from the
one applied in the previous measurement, but now also sinusoidal movement is applied in both
AP and LR directions.
For S2, S3 and O2, tracking position along SI and LR directions is shown in Fig.4. Maximum
excursion along the SI direction has been obtained for each target: the ME average value is 42 ±
2 mm.
36
(a)
(b)
Fig. 4: Tracking position for different targets following a 3D movement within the QUASAR body
phantom, see phantom in Fig.1. Initial position is defined as the position of the centre of gravity for the
segmented volume in frame 0. Maximum excursion of the target is obtained for each direction. Graphics
show the displacement normalized to the maximum excursion along SI (a) and LR (b) directions.
Algorithm characteristics (see detailed description of seed position propagation in T. Fechter
work), requires to verify that 3D movement does not compromise algorithm performance. With
this aim, Table I shows the comparison of activity and diameter accuracy for 3D-movement of
the target (column QUASAR Phantom in Table I) and 1D-movement (column NEMA Phantom
in Table I).
QUASAR
Phantom
NEMA Phantom
TBR=8
O2
S2
S3
7 ml
12.7 ml
25.5 ml
фmeas – фtrue (mm)
0.6 ± 1.3
0.4 ± 0.9
Ameas/Atrue
0.84 ± 0.05
0.88 ± 0.02
TBR=10 V [6 - 25] ml
Average
Average
0.8 ± 1.0
0.6 ± 0.2
1.1 ± 0.8
1.19 ± 0.03
0.97 ± 0.19
0.99 ± 0.06
Table I shows the comparison diameter and activity accuracy for similar target volumes in QUASAR
Phantom (3D movement) with respect to the NEMA Phantom (1D movement). Volumes of the NEMA
spheres considered for the comparison are: 6 ml (ф=22mm), 11 ml (ф=28mm) and 25 ml (ф=37mm).
Three irregular motion patterns along the SI direction (P1, P2, P3 in Fig.2) are applied to the oval
target (O2), according to the set-up shown in Fig.1. Irregular breathing cycles translate in poorer
tracer spatial distribution response for the target volume. In this section, the objective is to study
the effects on the accuracy for the algorithm response when irregular cycles, that mimic the SI
contribution in real breathing cycles, are applied to the target.
For the motion patterns applied, Fig.5 shows average values of activity and diameter accuracy (a)
and position tracking along the SI direction (b). Additionally, in order to study tracking position
response, “theoretical” position of the target along frames is calculated according to the system
data processing, PX Teor (X: 1,2, and 3) in Fig.5. These “theoretical” positions represent the
maximum accuracy achievable defined by the system response. Their comparison with the
tracking position provided by the algorithm is shown in Fig.5b. This graphic shows that the
algorithm is able to follow the “theoretical” path of the tumour for all the irregular motion
patterns: P1, P2 and P3. In order to evaluate ME provided by the algorithm, for each of the
irregular motion patterns applied, average maximum-to-minimum distance (Daverage) and the
M. Carles et al.
37
maximum-to-minimum distance for the cycle with the largest amplitude (Dmax) have been
calculated. Table II shows that the ME provided by the algorithm, 24 mm, is larger than
Daverage for all the patterns applied and covers Dmax in two of them.
TBR=8
O2
Average
7 ml
( P1, P2, P3 )
Φmeas-Φtrue
-0.1 ± 0.5
Ameas/Atrue
0.88 ± 0.06
(a)
(b)
Fig. 5: Algorithm response for irregular movement of the target along the SI direction within the QUASAR
phantom. Diameter and activity response (a) and tracking position along the SI direction (b). PX Teor (X:
1,2, and 3) refers to the target position along frames calculated according to the system data processing
which was applied to the theoretical movement.
mm
P1
P2
P3
Daverage
17 ± 6
16 ± 6
17 ± 5
Dmax
27
22
23
MEalgorithm
24.0 ± 1.3
24.0 ± 1.3
24.0 ± 1.3
Table II: For each irregular motion pattern applied to the target, Fig.2, the average maximum-to-minimum
distance (Daverage) and the maximum-to-minimum distance for the cycle with the largest amplitude
(Dmax) are calculated. In this table their values are presented in comparison to the ME provided by the
algorithm.
DISCUSSION
Retrospectively gated 4D PET/CT is a valuable clinical tool to improve target definition and to
assess and measure lesion motion for radiotherapy planning. An approach to improve the
consistency and reproducibility of tumour delineation consists in relying on automatic or semiautomatic segmentation methods. In this work, a semi-automated contrast oriented algorithm for
tumour delineation adapted to 4D PET images is evaluated.
Algorithms evaluation requires identification of a gold standard. In this evaluation, the algorithm
is applied on images resulted from dynamic experimental phantom measurements. Phantoms
employed are chosen based on their design properties, with the aim of reproducing main
degrading factors in lung tumour contour. With this approach, the robustness and accuracy of
segmentation algorithm can be evaluated with respect to: size, shape and movement of the target
and TBR. Accuracy in volume delineation, position tracking and activity concentration
estimation is studied.
Results obtained for activity accuracy when the algorithm is applied to the first set of
measurements, Fig.3b, show a significant dependence on target volume. Moreover, the algorithm
38
is not able to segment the smallest sphere for the lowest TBR. Diameters of the spheres that
present poorer accuracy (10 and 13 mm) justify considering partial volume effect (PVE) as a
possible explanation. PVE is an activity underestimation which increases with the ratio of system
spatial resolution to diameter of the finite volume (FWHM/ф). It is known that this degradation
becomes significant for ф < 3FWHM [17]. Consequently, this effect is not expected to be
observed in large spheres. In concordance to this, an average the activity recovery coefficient
(Ameasured/Atrue) of 0.97 ± 0.07 is obtained for NEMA spheres with ф larger than 3FWHM.
For the smallest sphere in the lowest contrast, not only the PVE is a limiting factor. Furthermore,
the lower contrast, the target absolute activity concentration (9.3 kBq/ml) and the acquisition
time per frame (1 min) translate in a high noise image. As a result, the segmented volume extends
along the background and no spherical target can be segmented.
Diameter differences presented in Fig.3a show that, although larger diameter differences are
obtained for small spheres; these values are, in any case, lower than the system spatial resolution.
No statistically significant diameter differences are obtained for ф > 3FWHM.
Maximum excursion of the tumour during image acquisition plays an important role in lung
radiotherapy. Average values obtained for ME show an overestimation. However, this
overestimation is substantially smaller than system spatial resolution. Furthermore, an
overestimation rather than underestimation is preferable for ME clinical use.
Apart from the case of the smallest sphere, no statistically significant effect on the algorithm
response (activity, diameter and position tracking) is observed for the TBRs considered: 33, 28
and 10.
In realistic breathing cycles, apart from the main displacement contribution along SI direction,
displacements along LR and AP directions are also observed. In the adapted version of the
algorithm to 4D images, position of the seed placed by the user is extended to all the frames. In
this process (see detailed description in T. Fechter work) relative displacement of target along
frames could compromise the algorithm response. 1D movement of the target has been replaced
for 3D movement in order to ensure the feasibility of the method to track the target in realistic
movement conditions.
From results in Table.I, it is concluded that 3D movement of the target does not compromise the
accuracy of activity and diameter estimation, obtained when 1D movement was applied to the
target. Furthermore, independently of the volume and shape of the targets studied (S2, S3 and
O2), the algorithm is able to track the path of the target along both SI and LR direction, Fig.4. As
it happens for the activity and diameter estimation, no degradation of the accuracy in ME
estimation along the SI direction is obtained for a 3D target movement, ME=42 ± 2 mm, respect
to a 1D movement of the target, ME=42.2 ± 1.4.
For irregular movements along the SI direction, results of activity and diameter accuracy shown
in Fig.5 are compatible with the accuracy reported in Table I (O2) for a regular 3D movement of
the same target. In order to evaluate ME provided by the algorithm for these irregular patterns, its
comparison with the average maximum-to-minimum distance (Daverage) and the maximum-tominimum distance for the cycle with the largest amplitude (Dmax) is shown in Table III. ME
provided by the algorithm is larger than Daverage for all the patterns applied and covers Dmax in
two of them. This result is particularly relevant due to the ME use given in radiotherapy routine.
M. Carles et al.
39
CONCLUSION
In order to validate the algorithm response for target without inactive wall, additional
measurement with alginate spheres is planned to be included shortly in this evaluation. However,
from the results reported in this study, it could be concluded that the semi-automated contrast
oriented algorithm adapted to 4D PET is applicable to a broad range of cases with an acceptable
accuracy. From our point of view, this evaluation proves that it is reasonable to consider the
feasibility of this algorithm for clinical use. Because of this, our future work will focus on
algorithm evaluation with lung cancer patients.
APPENDIX
4D image quantification study for lung cancer patients from our centre has been carried out. The
aim of this study was to measure activity concentration for several structures in retrospectively
gated PET images and to derive the values required for experimental phantoms. Activity
calculation and volume definition were performed according to EANM procedure guideline
recommendations and applied in 4D PET images: mean activity concentration for 3D isocontours
at 50% of the maximum (mC50).
Average values of mC50 for different thoracic structures are reported in Table.A.I. As expected,
tumour activity shows high variability, in contrast to a more uniform FDG distribution within
normal tissues.
Average mC50 ( kBq/ml )
Tumour
Heart
17 ± 4
8.6 ± 2.2
Lung
1.9 ± 0.8
Table A.I
Liver
Torso
7.0 ±1.6
1.8 ± 1.1
Furthermore, the structure with highest activity varies among the patients. For patients with
highest activity in tumour, the quantification results in average values of 24.3, 6.3, 1.7, 6.2 and
2.2 kBq/ml for tumour, heart, lung, liver and torso, respectively. In patients with liver and heart
as most intense organs, average values for separate compartmental quantification were also
obtained.
In this 4D PET quantification we have derived from clinical data several FDG concentrations
required for anthropomorphic phantom measurements. Results show that in order to cover
different clinical scenarios it is recommended to employ activity distributions with different
highest activity structures (tumour, heart and liver) and the quantification results obtained could
be employed as reference values.
ACKNOWLEDGMENT
40
REFERENCES
[1] Weber WA. Assessing tumour response to therapy. Nuclear Med 2009; 50(1):S1-S10.
[2] Slotman BJ, Lagerwaard FJ, Senan S. 4D imaging for target definition in stereotactic
radiotherapy for lung cancer. Acta Oncol 2006; 45(7):966-972.
[3] Townsend DW. Positron emission tomography/ computed tomography. Nuclear Med 2008;
38(3):152-166.
[4] van Herk M. Errors and margins in radiotherapy. Seminars in Radiation Oncology 2004;
14(1):52-64.
[5] Nehme SA, Erdi YE. Respiratory motion in positron emission tomography/computed
tomography: a review. Semin Nucl Med 2008; 38(3):167-176.
[6] Nygaard DE, Aznar MC. Respiratory Motion Management in CT and PET/CT for Radiation
Therapy Planning. EANM: PET/CT Radiotherapy Planning 2006; 3(4.2):98-115.
[7] Zaidi H, Naqa IE. PET-guided delineation of radiation therapy treatment volumes: a survey
of image segmentation techniques. Eur J Nucl Med Mol Imaging 2010; 37(1):2165-2187.
[8] Zhang YJ. Evaluation and comparison of different segmentation algorithms. Pattern
Recognition Letters 1997; 18(10):963-997.
[9] Nestle U et al. Comparison of different methods for delineation of 18FFDG PET-positive
tissue for target volume definition in radiotherapy of patients with non-small cell lung
cancer. J Nucl Med 2005; 46(8):1342-1348.
[10] Zhang H, Fritts JE, Goldman SA. A survey on evaluation methods for image segmentation.
Patter Recognition 1996; 29(8):1335-1346.
[11] Schaefer A, Kremp S, Hellwiq D, Rübe C, Kirsch CM, Nestle U. A contrast-oriented
algorithm for FDG-PET-based delineation of tumour volumes for the radiotherapy of lung
cancer: derivation from phantom measurements and validation in patient data. Eur J Nucl
Med Mol Imaging 2008; 35(11):1989-1999.
[12] Schaefer A, Nestle U, Kremp S, Hellwiq D, Grgic A, Buchholz HG, Kirsch CM. Multicentre calibration of an adaptive thresholding method for PET-based delineation of tumour
volumes in radiotherapy planning of lung cancer. Nuklearmedizin 2012; 51(3):101-110.
[13] Christ U, Fechter T, Mix M, Hennig J, Nestle U. Automatic background determination for
contrast-based threshold segmentation in PET imaging based on histograms. EANM’13
(Lyon), Eur J Nucl Med Mol Imaging, 40(S2).
[14] Geramifar P, Zafarghandi MS, Ghafarian P, Rahmim A, Ay MR. Respiratory induced
errors in tumour quantification and delineation CT attenuation- corrected PET images:
Effects of tumour size, tumour location, and respiratory trace: A simulation study using 4D
XCAT phantom. Mol Imaging Biol 2013; 15(6):655-665.
[15] Park SJ, Ionascu , Killoran, Mamede , Gerbaudo, Chin and Ross Berbeco1. Evaluation of
the combined effects of target size, respiratory motion and background activity on 3D and
4D PET/CT images. Phys Med Biol 2008; 53(13):3661-3679.
[16] Seppenwoolde Y, Shirato H, Kitamura K, Shimizu MD, van Herk M, Lebesque JV,
Miyasaka K. Precise and real-time measurement of 3D tumour motion in lung due to
breathing and heartbeat, measured during radiotherapy. Radiat Oncol Biol Phys 2002;
53(4):822-834.
[17] Soret M, Bacharach SL, Buvat I. Partial-Volume Effect in PET Tumour Imaging. Nucl Med
M. Carles et al.
41
2007; 48:932-945.
contrast-based threshold segmentation in PET imaging based on histograms”. EANM’13
(Lyon), Eur J Nucl Med Mol Imaging, 40(S2), October 2013.
Montserrat Carles, Spain, June 1983. Bachelor’s Degree in Physics (20012006) and Medical Physics Master (2006-2008) offered by University of
Valencia, Spain. In 2012, Ph.D (Thesis: Image Quality Performance and
Optimization in Positron Emission Tomography) at the Corpuscular Physics
Institute (IFIC) in Valencia, Spain.
She developed a research work at the Optics Department of the University of
Valencia (2006), at the Oncology Institute of Valencia (2006-2007) and, since
September 2013, she joined UKL Freiburg as an Experienced Researcher for the
SUMMER project. Her main role is to evaluate the SUMMER system
demonstrator relating to the clinical use of PET images.
In 2010 Dr. Carles got the Valencia Idea Award in Biotechnology and
Biomedicine.
42
4D GTV delineation
A threshold and region-growing based
algorithm for 18FDG-PET 4D GTV
delineation
Tobias Fechter1*, Montserrat Carles1, Alin Chirindel1, Ursula Christ² and Ursula Nestle1
1
*
Department of Radiation Oncology, University Medical Center Freiburg
² Department of Nuclear Medicine, University Medical Center Freiburg
tobias.fechter@uniklinik-freiburg.de
Abstract: In the last years 4D 18FDG-PET has evolved to an important instrument in
radiotherapy treatment planning for NSCLC. In contrast to 3D PET acquisitions it has the
potential to better depict a moving target - the tumour during breathing cycle. However, the more
amount of information leads to a higher amount of work for contouring the tumour in all its
positions. Currently, there are several algorithms for contouring NSCLC on 18FDG-PET images
on 3D but hardly any algorithms for 4D. In this work we investigate the possibility to extend the
knowledge from an existing 3D threshold algorithm and to apply it in 4D data segmentation. The
algorithm was evaluated by contouring 3 different 4D phantom measurements. As a figure of
merit the volume of the final segmentation was compared to the real volume of the shape to be
segmented. The results suggest that the presented 4D segmentation algorithm is a proper tool for
segmenting 4D 18FDG-PET scans of NSCLC.
Index Terms — segmentation, phantom, 4D, PET, motion.
INTRODUCTION
Positron emission tomography (PET) is an imaging technique in the field of nuclear medicine. In
contrast to imaging like CT and anatomical MR, PET is capable to depict biochemical and
physiological processes in vivo. This is done by injecting the patient a radio tracer which
accumulates inside the body according to certain cellular metabolic pathways or receptor
interaction. The annihilation of the positrons emitted during decay of the radio tracer gives rise to
the emission of two simultaneous and collinear photons. These photons are registered by the
scanner and the information is used to reconstruct the spatial distribution of the radio tracer. It
has been shown that the 18f-fludeoxyglucose (18FDG) tracer is preferentially accumulated in
tumours, thus making it suitable for oncological imaging [1]. It has been established that 18FDG
is a very accurate diagnostic method for non-small-cell lung cancer (NSCLC) and therefore it
plays an important role in the delineation of the gross tumour volume (GTV) for radiotherapy
treatment planning [2,3]. However, as the tumour motion excursions due to breathing can be
more than 2 cm and the PET acquisition needs several minutes, conventional 3D PET scans show
a blurred tumour which can lead to imprecise GTV delineations [4,5]. Reasons for this are that
blurring makes the tumour look larger and causes an underestimation of the measured tracer
uptake. In response to these, four dimensional (4D) PET was introduced representing time
dependent imaging acquisition according to patient respiration, which is segmented in different
T. Fechter et al.
43
time bins. This method yields, depending on the number of time bins, multiple 3D images
showing the motion of the tumour during respiration. In clinical practice up to 10 time bins are
used, which is increasing the segmentation effort significantly.
For treatment planning the standard is to calculate the internal target volume (ITV) or the midposition of the tumour to define an appropriate target that covers the tumour while respiration [4].
This is usually done on CT but has its drawbacks for central lung tumours [6]. Here a 4D PET
GTV can add information for a more precise calculation.
Automatic and semi-automatic methods have been developed to ease the process of GTV
contouring [7]. However, none of them was designed for 4D segmentation and to our knowledge
there are no real 4D segmentation algorithms designed for thoracic PET scan contouring. In this
work we investigate the applicability and performance of an adapted version of a contrast
oriented algorithm [8] in delineating 4D GTVs which can be used for ITV or mod-position
calculation. This algorithm was chosen because of its computational simplicity and its promising
results in 3D PET [9]. We investigated how to apply the algorithm to 4D PET scans and the
quality of the results with 3 different phantom measurements.
4D PET
The PET/CT scans were performed on the Medical Philips System GEMENI TF 16 Big Bore
with a transverse (axial) spatial resolution of 4.4 (4.7) mm at 1 cm from the camera central axis
and 4.9 (5.1) mm at 10 cm. The 10 min/bed PET acquisition is retrospectively gated in 10 bins
based on the breathing curve provided by a pressure sensor belt. The BLOB-OS-TOF image
reconstruction leads a pixel size of 0.117 x 0.117 x 0.2 cm³ and 0.4 x 0.4 x 0.4 cm³ for CT and
PET respectively.
Phantom
Tumour motion excursions due to breathing are the highest in the superior-inferior direction and
lower in the anterior-posterior as well as the medial-lateral direction [10]. To simulate this kind
of motion the Medical QUASAR respiratory motion phantom was employed (fig. 1). The main
components of this type of phantom are:
 a cylindrical insert, with a fillable glass volume in an off-centre position
 the phantom body (30 x 20 x 12 cm³)
 a motor that induces a circular and translational movement of the insert with respect to
the static phantom body
a) (courtesy of the company)
(b)
Fig. 1: a) QUASAR Respiratory Motion Phantom with body-shaped thoracic phantom, interchangeable lung
component for target insertion, variable speed engine and adaptable motion platform. b) shows the oval
shaped volume with a volume of 7ml.
44
4D GTV delineation
Three different glass volumes were considered to simulate different tumour shapes. Two spheres
with a volume of 12 ml and 25 ml respectively and an oval shaped volume with 7 ml volume.
The target to background (BG) ratio was 12.8 kBq/ml foreground (FG) activity to 1.6 kBq/ml BG
activity. These values are within the clinically observed range in 4D 18FDG image quantification
for 10 patients with NSCLC.
Segmentation Algorithm
This section will first explain the 3D version of the used segmentation algorithm [8]. After that
different ways how the algorithm can be extended to the fourth dimension will be outlined.
Hereafter the part of the dataset on which the algorithm is working on is called the region of
interest (ROI).
1) 3D Algorithm
The segmentation algorithm in use is a region growing and threshold based algorithm. The
workflow of the algorithm was designed as follows:
 determine FG pixels
 determine the BG pixels
 calculate the threshold T
 based on T start a growing region
(1)
Determine FG Pixels
The first step of the algorithm is to find the highest pixel intensity (M) inside the ROI. Then a
threshold of 70 % of M is applied to the ROI. The mean intensity of the resulting pixel regions is
referred to as mSUV70. Within these regions the local maxima are located. If a user is only
interested in specific maxima the algorithm offers the possibility to mark them with seed points.
In this particular case M and mSUV70 are determined only around the seed points.
(2)
Determine the BG Pixels
All pixels inside the ROI that have a pixel value between 1 % and 15 % of M are assumed to be
BG values, as previously detailed [11]. mBG represents the mean pixel value of all pixels
regarded as BG.
(3)
Calculate T
T is calculated using the following equation:
(1)
The variables a and b are PET scanner dependent and used to adjust the influence of BG and
lesion pixel intensity on the threshold and have been determined from previous phantom
measurements work [8].
(4)
Start A Region Growing From The Maxima
In the last step of the algorithm a region growing method is used to determine the final
segmentation result. The region growing starts from every maximum that was found in step 1.
Then all pixels in the 6-neighborhood are examined. If the neighbouring pixels have a value
higher than T, they will be added to the segmentation. Then the neighbourhood of the added
pixels is examined. This repeats until there are no neighbours left with a pixel value higher
than T.
T. Fechter et al.
45
2) 4D Algorithm
The 3D version of the presented algorithm can be extended to 4D datasets in 3 different ways.
These 3 methods are:
 Calculate mSUV70, mBG and T for every time bin separately.
 Calculate mSUV70, mBG and T in one time bin and apply the resulting T to all other
time bins.
 Treat the whole 4D dataset as one volume and use a 4D neighbourhood for the region
growing instead of a 3D neighbourhood.
(1)
Treat Every Time Bin Separately
Every time bin of the dataset is treated as an independent 3D volume and segmented with the 3D
version of the segmentation algorithm. If a seed point is placed by the user in one time bin, it will
be automatically propagated to all other time bins, by searching for the maximum pixel value in
the neighbourhood in the next time bin. The neighbourhood is defined by the current coordinates
of the seed point transferred to the next time bin plus two pixels in positive and negative x-, yand z-direction.
(2)
Calculate T In One Time Bin And Apply It To All Others
With this method T is calculated in one time bin and then applied to all other time bins.
(3)
Treat The Whole 4D Dataset As One Volume
With this method the 4D dataset is treated as one volume. Every step explained in the 3D version
of the algorithm is extended to the fourth dimension.
The physically most accurate method is method 1 because it addresses all different timedependent tumour positions during respiration and calculates T for each different position.
Method 2 and 3 both are not as exact as method 1 as they take only the information of one
position or lose information by using the absolute 4D maximum for the calculations. Preliminary
analysis showed that method 3 was at least as computing intensive as method 1 but not as
accurate. Therefore it was discarded and not further investigated. This was the reason why only
method 1 and method 2 qualified for further investigation. The assumption was that if the error of
measurement for these methods were not statistically significant, then it would be more efficient
to calculate the threshold only in one time bin and apply it to all other time bins
For statistical testing the above described phantom with the 3 different volumes was used. In
every 4D dataset the shape to be segmented was contoured with method 1 and method 2. For the
contouring with method 2, every time bin was contoured 10 times (every time with a threshold
from a different time bin). From the resulting volumes the real volume was reduced which
yielded the error of the segmented volume. The data was analysed with a Kruskal-Wallis test.
This test was preferred to one factor ANOVA because the Levene test showed different variances
in the groups which is an exclusion criterion for the ANOVA analysis. The null hypothesis was
that there is no difference between the errors (alpha was set to 5 %) of the two methods and that
all samples belong to the same population.
As the Kruskal-Wallis test gives only an indication whether all groups belong to the same
population or not, a Mann-Whitney U test was applied afterwards to find out which groups differ.
The Mann-Whitney U test was done with a Bonferoni correction to reduce the Type I error (alpha
was set to 0.45). The null hypothesis for the Mann-Whitney U test was the same as for the
Kruskal-Wallis test but for 2 groups.
46
4D GTV delineation
RESULTS
This section shows the statistical comparison of the 4D GTVs (each consisting of 10 3D GTVs)
obtained with method 1 and method 2. To get the following results the variables a and b in (1)
were set to 0.44 and 0.24, respectively. The rank sums for the Kruskal-Wallis test can be seen in
table I. The size for every group was 30. The test resulted in a p-value of 0.00000395. As this is
much lower than alpha, we rejected the null hypothesis and can assume that the groups don’t
belong to the same population.
m1t0
4589
m2t0
4031
m2t1
3395
m2t2
5500
m2t3
5459
m2t4
5208
m2t5
5807
m2t6
3544
m2t7
6163
m2t8
6721
m2t9
4202
Table I: Rank sums of the Kruskal-Wallis test. The column m0t0 shows the rank sum for method 1. m1t0 to
m1t9 show the rank sums for method 2. All values were round to integer.
a) Time bin 0
b) Time bin 0
contoured
c) Time bin 5
contoured
d) Time bin 8
contoured
Fig. 2: Different time bins of the 4D PET dataset of the oval shaped volume. a) shows time bin 0. b) – d)
show the volume contoured in different time bins.
To find out which groups are different a Mann-Whitney U test was applied. Table II shows the pvalues for the comparison of the results of method 1 to all results of method 2. According to the
p-values we rejected the null hypothesis, that the samples origin from the same population for all
groups but for m2t0 and m2t9.
m2t0
0.0048
m2t1*
0.0020
m2t2*
0.0029
m2t3*
0.0032
m2t4*
0.0041
m2t5*
0.0019
m2t6*
0.0028
m2t7*
0.0009
m2t8*
0.0002
m2t9
0.0053
Table II: p-values for the comparison of the results of method 1 to all results of method 2. The asterix
indicates that the difference is significant.
Table III shows the mean in ml and the coefficient of variation of the segmented volume error.
The segmented volume error was calculated by subtracting the real volumes from the resulting
volumes of the algorithm.
T. Fechter et al.
µ
CV0
CV1
CV2
CV
m1t0
0.600
0.238
0.009
0.126
0.264
47
m2t0
0.543
0.311
0.006
0.127
0.313
m2t1
0.417
0.321
0.006
0.202
0.327
m2t2
0.787
0.211
0.006
0.168
0.245
m2t3
0.700
0.258
0.006
0.131
0.192
m2t4
0.850
0.184
0.007
0.114
0.296
m2t5
0.717
0.170
0.006
0.144
0.145
m2t6
0.433
0.252
0.006
0.186
0.253
m2t7
0.767
0.114
0.006
0.116
0.127
m2t8
0.927
0.118
0.007
0.143
0.154
m2t9
0.510
0.174
0.005
0.183
0.238
Table III: the mean (µ) in ml and the coefficient of variation of the segmented volume error. CV0 is the
coefficient of variation for the 7 ml volume, CV1 for the 12.7 ml volume, CV2 for the 25.5 ml volume and
CV for all volumes, respectively.
DISCUSSION
The statistical tests showed that method 2 yielded solely comparable results to method 1 with the
thresholds calculated in time bin 0 and time bin 9. In table III can be seen that in several cases,
method 2 returned a more précised measurement compared to method 1. The results of method 2
with a lower mean error of volume measurements than method 1 are m2t0, m2t1, m2t6 and m2t9.
The time bins 0, 1, 6 and 9 all showed the target in or close to its highest or lowest position in the
superior-inferior direction. This may give rise to the assumption that if the calculation of T was
done with a dataset where the tumour is at a certain position we would get results as good as or
better than with method 1. But this is like the chicken or the egg causality dilemma: You would
need to know the pathway of the tumour before contouring, which is unfortunately not possible in
clinical practice. Method 2 would have been acceptable only if it had showed no significant
difference to method 1 in all times bins.
The fact that method 1 showed a low error in volume segmentation (less than 10 pixels for an
average of 229 pixels per target) and is physically the most accurate method, points out that the
only proper way to calculate the necessary variables for the proposed 4D algorithm is to do the
calculations for every time bin separately. Although the final result is a time dependent result, the
algorithm itself takes no 4D information into account for its calculations and is, like the already
existing methods not a real 4D but more a 3D segmentation algorithm applied to every time bin
separately. Despite that the presented algorithm appears to be a fast and accurate algorithm for
contouring 3D GTVs in every time bin, which can be used for ITV and mid-position calculation.
CONCLUSION
In this work we analysed different ways to extend a threshold and region growing based 3D
18
FDG-PET segmentation algorithm to delineate NSCLC in 4D PET datasets. Furthermore we
were able to derivate from the results one way of contouring a 4D GTV that gives the user the
most appropriate results. In the light of these promising results on phantom measurements,
further testing of our proposed algorithm on patient data is needed to establish its value for
clinical use.
ACKNOWLEDGMENT
48
4D GTV delineation
REFERENCES
[1] Gambhir SS, Czernin J, Schwimmer J, Silverman DH, Coleman RE, Phelps ME. A
tabulated summary of the FDG PET literature. J Nucl Med 2001; 42(S):1S-93S.
[2] Baum RP, Hellwig D, Mezzetti M. Position of nuclear medicine modalities in the diagnostic
workup of cancer patients: lung cancer. Q J Nucl Med Mol Imaging 2004; 48(2):119-142.
[3] Mayor S. NICE issues guidance for diagnosis and treatment of lung cancer. BMJ 2005;
330(7489):439.
[4] Wolthaus JW, Sonke JJ, van Herk M, Belderbos JS, Rossi MM, Lebesque JV, Damen EM.
Comparison of different strategies to use four-dimensional computed tomography in
treatment planning for lung cancer patients. Int J Radiat Oncol Biol Phys 2008; 70(4):12291238.
[5] Nehmeh SA, Erdi YE, Rosenzweig KE, et al. Effect of respiratory gating on reducing lung
motion artifacts in PET imaging of lung cancer. Med Phys 2002; 29:336-371.
[6] Schrevens L, Lorent N, Dooms C, Vansteenkiste J. The role of PET scan in diagnosis,
staging, and management of non-small cell lung cancer. Oncologist 2004; 9(6):633-643.
[7] Zaidi H, El Naqa I. PET-guided delineation of radiation therapy treatment volumes: a
survey of image segmentation techniques. Eur J Nucl Med Mol Imaging 2010; 37(11):21652187.
[8] Schaefer A, Kremp S, Hellwig D, Rübe C, Kirsch CM, Nestle U. A contrast-oriented
algorithm for FDG-PET-based delineation of tumour volumes for the radiotherapy of lung
cancer: derivation from phantom measurements and validation in patient data. Eur J Nucl
Med Mol Imaging 2008; 35(11):1989-1999.
[9] Nestle U, Kremp S, Schaefer-Schuler A, Sebastian-Welsch C, Hellwig D, Rübe C, Kirsch
CM. Comparison of different methods for delineation of 18F-FDG PET-positive tissue for
target volume definition in radiotherapy of patients with non-Small cell lung cancer. J Nucl
Med 2005; 46(8):1342-1348.
[10] Seppenwoolde Y, Shirato H, Kitamura K, Shimizu S, Herk MV, Lebesque JV. Precise and
real-time measurement of 3D tumour motion in lung due to breathing and heartbeat,
measured during radiotherapy. Int J Radiat Oncol Biol Phys 2002; 53:822-834.
contrast-based threshold segmentation in PET imaging based on histograms. Eur J Nucl
Med Mol Imaging 2013, 40(S2).
Tobias Fechter is an Austrian engineer. He studied medical computer science at
the technical university of Vienna and received his M.Sc. in 2011. He wrote his
master thesis at the VRVIS on the topic “Deformation Based Manual
Segmentation in Three and Four Dimensions”. After the study he moved to
Aarau in Switzerland where he worked on the implementation of a HIS.
In June 2012 he started as a Marie Curie Early-Stage Researcher for the
SUMMER project and is currently working at the department for radiation
oncology at the university medical centre in Freiburg with a focus on PET image
segmentation.
N. Tuovinen et al.
49
fMRI: resting-state networks and
task-evoked activations
in the presence of brain tumours
Noora Tuovinen1*, Francesco de Pasquale1, Umberto Sabatini1
1
*
Santa Lucia Foundation, IRCSS, Rome, Italy
n.tuovinen@hsantalucia.it
Abstract: As part of the Software for the Use of Multi-Modality images in External Radiotherapy
(SUMMER) project, one of the specific aims is to study brain tumour patients with multimodal
magnetic resonance imaging and integrate this information into radiation therapy to improve
treatment planning. The focus of this research is to investigate the impact of tumours and
radiation treatment on brain functions thus giving a better understanding of the deficits occurring
in the patients. This paper shows initial results with tumour patients based on functional magnetic
resonance imaging and discusses the possible meaning of observed activations in the brain
comparing task-based and resting-state experiments.
Index Terms — task-based and resting-state fMRI, brain tumour, radiotherapy, plasticity, DefaultMode Network, Sensorimotor Network.
INTRODUCTION
This work investigates brain tumour patients with functional MRI (fMRI) to identify regions at
risk (RARs) for radiotherapy (RT) planning. Integration of fMRI into RT is not widely used and
the literature is limited [1]. Previous task based fMRI studies focused on identifying functional
areas for safe tumour resection. Here, additional information was collected on functional
connections in the brain through resting state fMRI. Tumour and treatment effects on functional
connectivity are still under investigation [2]. A multimodal approach allows investigating
alterations even far away from the lesions and this might characterize deficits occurring at larger
scales [3]. In this work, functional connectivity and task-fMRI activations were integrated to
elucidate mechanisms of brain recovery, compensation and plasticity after surgery and RT. In
particular, focus was given on Default Mode Network (DMN), which plays an important role in
cognitive and memory functions, and to the sensorimotor network (SMN) involved in the motor
function. For further understanding of the role of MR-imaging with brain tumours patients, the
reader is invited to study a previous SUMMER-school article on the topic [4].
fMRI data were acquired by 3T Philips Achieva with block design paradigm for motor task (EPI
sequence, TR/TE = 3.00s/30ms) and at rest (EPI sequence, TR/TE = 2.00s/30ms). The data were
processed using FSL-software [5]. FSL's FEAT (fMRI data analysis tool) was used for preprocessing (motion and slice time correction, brain extraction, 5mm smoothing). Statistical
50
Rest and task-based fMRI in the presence of brain tumors
analysis with cluster based thresholding was performed for both left and right finger tapping. To
assess functional networks, 3 five minute resting state runs were analysed with MELODIC ICA
(independent component analysis). The obtained activations and functional networks were then
registered to T1-weighted image using FSL's FLIRT tool (6DOF).
RESULTS
As a reference, in Fig.1, ICs revealed by the analysis corresponding to typical resting state
networks of DMN (left) and SMN (right) for two healthy subjects are reported and overlaid on
T1-weighted images.
Fig.1: DMN and SMN networks overlaid on T1-weighted images in two healthy subjects as revealed by
ICA.
In Fig.2, anatomical locations for surgical cavities after tumour resection are first reported (left).
In addition, ICs corresponding to DMN (middle) and SMN (right) obtained from six tumour
patients during rest are presented. It can be noted that patients A-C have changes in their DMN
topology. Interestingly, the map in B shows a wider extension of the areas involving the medial
prefrontal regions compared to the other patients. Changes in the topology of SMN can be noted
for patients C-F.
N. Tuovinen et al.
51
Fig.2:Tumour lesions (left), DMN (middle) and SMN (right) for six patients as revealed by ICA overlaid on
T1-weighted images. Patients A-C present disruptions in the topology of DMN and patients C-F in the
SMN.
Furthermore, task activations (Fig.3) in the same patients were obtained and compared with the
sensorimotor network information. Patients A and B showed altered activations for affected
motor area. Right finger tapping from patient A revealed activation on the ipsilateral hemisphere
while SMN was still obtained on both hemispheres. For patient B, while the activation in the
expected motor area (precentral gyrus) on the affected hemisphere was not revealed by the GLM
analyses, it was obtained as part of the SMN. All of the patients showed alterations in at least one
of the networks disrupting the affected hemisphere with the exception of patient C who had
disruptions in both of the networks. It is worth noting that the histology confirmed an
oligodendroglioma on this patient, while the other patients had glioblastoma multiforme.
52
Rest and task-based fMRI in the presence of brain tumors
Fig. 3: Left and right finger tapping activations on tumour patients. Right finger tapping on patient A
revealed ipsilateral activations suggesting network coupling and compensation. For patient B, left finger
tapping activation was not revealed by GLM. Patients C-F showed expected activation locations for finger
tapping.
DISCUSSION
This study shows that DMN and SMN are identifiable from resting fMRI data with ICA robustly
in healthy subjects and brain tumours patients. It also points to a possible functional reorganization of DMN in which the loss of the right angular gyrus node induces a stronger
coupling of medial prefrontal areas. Furthermore, comparisons of task activations and functional
connectivity structures showed that sometimes the activations in the affected hemisphere were
altered while the SMN topology seemed intact. This might suggest that the underlying functional
connectivity is still preserved near the lesion. Notably, since patients were able to perform the
task, our results indicate a potential compensation mechanism achieved through network
connections, i.e. although seriously damaged the area contralateral to the task might enrol the
corresponding ipsilateral one. One could speculate that functional connectivity and activations
are integrated so that the role played by one region could be performed by a distant one as long as
it is part of the same functional circuit. This seems to suggest that RT planning should take into
consideration not only task specific RARs but also the corresponding networks. Future work
extends this study for wider patient population taking into account longitudinal changes in task
activations and resting state networks after radiotherapy treatment..
ACKNOWLEDGMENT
N. Tuovinen et al.
53
REFERENCES
[1] Kovács A, Tóth L, Glavák C, Liposits G, Hadjiev J, Antal G, Emri M, Vandulek C, Repa I.
Integrating functional MRI information into conventional 3D radiotherapy planning of CNS
tumours. Is it worth it? Journal of Neuro-Oncology 2011; 105(3):629-637.
[2] Esposito R, Mattei PA, Briganti C, Romani GL, Tartaro A, Caulo M. Modifications of
Default-Mode Network Connectivity in Patients with Cerebral Glioma. PLoS One 2012;
7(7):E40231.
[3] Lee MH, Smyser CD, Shimony JS. Resting-State fMRI: A Review of Methods and
Applications. AJNR American Journal of Neuroradiology 2013; 34(10):1866-1872.
[4] Tuovinen N. Role of MR-Imaging for brain tumours. Innovative imaging to improve
radiotherapy treatments, 2013; 1:55-62.
[5] Jenkinson M, Beckmann CF, Behrens TE, Woolrich MW, Smith SM. FSL. NeuroImage,
2012; 62(2):782-790.
Noora Tuovinen is a Marie Curie Early-Stage Researcher in the SUMMER
project currently situated at Santa Lucia Foundation (IRCCS) in Rome, Italy. She
is conducting her PhD at the University of Chieti-Pescara in Italy with studies on
neuroscience and imaging. Her research interest is in functional MRI studying
the changes of functional regions and resting-state networks in brain tumour
patients.
Previously, she has graduated as a biomedical electronics engineer (M.Sc.) from
Tampere University of Technology in Finland doing her Master Thesis on
superconductivity and quench (CERN, Geneva, Switzerland).
54
Fiber tractography in radiotherapy
Defining new regions at risk: fiber
tractography for planning radiotherapy
of brain tumours
Andac Hamamci1,*, Noora Touvinen1, Francesco de Pasquale1 and Umberto Sabatini1
1
*
Santa Lucia Foundation, I.R.C.C.S, Rome, Italy
a.hamamci@hsantalucia.it
Abstract: Diffusion MR fibre tractography, which allows non-invasive mapping of the white
matter fibre structures in vivo, offers new possibilities in radiation oncology such as protecting
major tracts from high dose irradiation. However, application of the methods, developed for
investigating white matter structure and connectivity in healthy subjects, to the tumour patients
causes additional methodological issues. In this work, snake method of the computer vision
literature, is applied to the diffusion MR fibre tractography problem, to develop an interactive
method suitable for the clinical workflow of our target application, radiotherapy planning.
Additionally, an efficient way for calculating and minimizing the internal energy of the snake by
closed-form expressions for membrane and thin-plate energy integrals are derived and presented.
Validations on synthetic diffusion phantom, on a healthy subject and on a glioma patient reveal
the potential of the method to be accepted in the clinical practice.
Index Terms — Diffusion, MRI, Tractography, Snake.
INTRODUCTION
Since the beginning of the 21st century, diffusion MR fibre tractography, which allows noninvasive mapping of fibre structures in vivo, has become increasingly popular in neuroscience
community to investigate the white matter architecture and connectivity in the central nervous
system. The first step in processing diffusion MR data is to characterize the local diffusion
characteristic for each voxel. This is usually performed by fitting a tensor model such as in
Diffusion Tensor Imaging (DTI) or High Angular Resolution Diffusion Imaging (HARDI)
techniques [1] such as Diffusion Spectrum Imaging (DSI) [2] or Q-Ball Imaging (QBI) [3]. In the
second step, calculated local diffusion properties are analysed to obtain fibres or connectivity of
the regions, by streamline techniques, such as FACT [4], probabilistic tractography [5] or global
tractography techniques [6].
On the other hand, diffusion MR tractography is receiving more attention also in clinical practice.
This technique is applied to the non-invasive preplanning for cerebral surgery, multimodal
navigation [7] and in radiation oncology to protect major tracts from high dose irradiation [8].
Common tractography methods are applicable to the tumour patients to an extend [9]. In clinical
applications, usually, the question is to provide the "location" of the known track in the best way
by using the available data and knowledge. However, common tractography methods suffer from
being too much local or blind to the normal human anatomy, search for the "existence" of the
A.Hamamci et al.
55
track. Moreover, the method should satisfy some requirements specific to the application. For
example it should be robust to the noise due to the limited examination time; be robust to the
signal abnormalities due to the lesion; require low post processing time; allow interaction in an
intuitive way.
In one of the similar non-parametric method, a simulated annealing approach is employed to find
a non-rigid transformation to map a fibre bundle from a fibre atlas to the patient's diffusion tensor
data, minimizing a specific energy functional [10]. In general, deformable registration of
diffusion MR images serves a solution for the problem by providing the mapping between the
atlas fibres and the patient data [11].
Our main motivation in this paper is to develop an interactive method suitable for the clinical
workflow of our target application, radiotherapy planning. For this purpose, we propose to
employ the snake method of the computer vision literature [12], which minimizes an energy
functional associated with a parametric curve to find the desired solution, to the diffusion MR
fibre tractography problem in “Materials and Methods” section. Additionally, an efficient way
for calculating and minimizing the internal energy of the snake by closed-form expressions for
membrane and thin-plate energy integrals are derived and presented. Validations were carried out
on synthetic diffusion phantom, on a healthy subject and on a glioma patient and presented in
“Results and Discussion” section.
The method is basically starting with a parametric curve model and searching for the "best"
location looking at the data and given constraints.
In this work, a cubic b-spline is used for its higher order of continuity. The coordinate in
dimension x{1,2,3} of the kth segment of a cubic spline, vkx (s), can be represented by the
matrix equation as:
(1)
where s[0,1] is the parameter,
is the coordinate of the i control point in dimension
x{1,2,3} and M is the spline matrix determining the type of the spline.
Pxi
th
A spline is completely defined by its control points having 3 N CONTROL POINTS parameters. So, the
energy associated with the spline can be interpreted as a mapping E:R3 NCP→R and can be
minimized in control point coordinate space, i.e.:
(2)
Two binary seed regions, S{0,1}
are defined on image domain, R
. The snake is
initialized as a straight line between the centre of the mass of the initial and final seed regions and
the local minimum energy solution is obtained by the conjugate gradients method as described in
[13]. First and last end points of the snake are repeated 3 times to handle the discontinuity at the
curve ends. The total energy of the spline is defined as the sum of the internal, image and seed
energies as:
XxYxZ
XxYxZ
56
(3)
Internal Energy
The internal spline energy can be written as in [12]
(4)
The spline energy is composed of a first-order term controlled by  which makes the snake act
like a membrane (minimizing the length of the curve) and a second-order term controlled by 
which makes it act like a thin plate (smoothing the curve). For simplicity,  and  parameters are
assumed to be constant through the curve, such that (s)= and (s)=.
Membrane Energy
Membrane energy can be written as a sum over the segments as
(5)
Integrals for different dimensions are separable
(6)
Considering the spline definition in matrix form in (1) follows
(7)
which can be integrated to obtain a closed form expression for the energy of the k th segment:
(8)
The total membrane energy is calculated by summing over segments (k) and dimensions (x):
(9)
Let's evaluate the derivative wrt x coordinate of the ith control point
(10)
Substituting the spline definition in matrix form results
A.Hamamci et al.
57
(11)
where ij is the Kronecker delta function. Finally, summation over segments results the following
convolution operation on control points for the derivative:
(12)
Thin-Plate Energy
Similar to the membrane energy, total thin plate energy is the summation over the segments
(13)
which can be calculated by the following expression
(14)
The derivative wrt x coordinate of the ith control point is
(15)
Image Energy
For simplicity, the path that maximizes the integral of the fractional anisotropy FA[0,1] of the
diffusion tensor, which is a measure of the anisotropy, is searched by the following external
energy term:
(16)
where the convolution with the gaussian function is
(17)
Note that the integration is defined over the arc-length instead of the internal parameter of the
spline resulting in a geometric form independent of the parameterization. Writing in terms of the
parameter of the curve, normalizing with the total length of the spline to prevent shortening and
summing over the segments results
(18)
Each integral over the parameter, s, are evaluated numerically and summed over the segments.
Derivatives are also calculated numerically by central difference formula. Note that for each
58
segment is defined locally, it is sufficient to sum over the segments that are affected by the
change of the point location.
Seed Energy
Let Si and Sf are the sets of seed points for initial and final ends of the fibre. In order to penalize
the distance of the end points of the fibre to the seed regions, the following energy term is
utilized:
(19)
where d is the distance function defined as:
(20)
First and last control points, which are repeated 3 times, are also guaranteed to be the end points
of the curve. The gradients with respect to the first and last control points are
(21)
and zero for the others.
RESULTS AND DISCUSSION
Validations on the Fibercup Phantom
Validations were carried out on synthetic diffusion phantom to evaluate the accuracy of the
method and to compare with the other algorithms. The synthetic phantom, introduced in [14]
which is used in FiberCup tractography challenge in MICCAI conference in London in 2009 and
made publicly available on the webpage (http://www.lnao.fr/spip.php?rubrique79) of LNAO lab.
of Neurospin in France is used [15]. Seed points and ground truth for 16 fibres are available with
the data. We should note here that, because our method requires seeds for initial and final ends of
the fibre, the initial and final points of the ground truth are set as the seed points instead of the
single seed per fibre, provided; which makes a direct comparison with the other techniques,
participating in the contest, unfair. The fibres are initialized as a straight line between the seed
points and evolved by using the proposed method on the fractional anisotropy (FA) maps,
generated by fitting a diffusion tensor model in Diffusion Toolkit software. The fibre results
obtained with the ground truth are presented qualitatively in Fig. 1. Measures of the average
distance, tangent and curvature deviations from the ground truth are calculated by the provided
software and given in Table 1. One observation is that, parallel fibres tend to follow the same
trajectory which maximizes the FA, instead of following parallel trajectories. In 2 of the cases,
the blue and red fibres at the bottom in Fig.1, the algorithm get stuck into the nearest local
minima. Better initialization with atlas priors, user guidance or a better optimization algorithm
can help to overcome this problem in practice.
A.Hamamci et al.
59
Figure 1. Fibbers obtained by the proposed method (left) on the FiberCup phantom and ground truth (right),
represented by the same colour coding, overlaid on the b0 image.
Table 1. Comparison of the obtained fibres with the ground-truth for FiberCup phantom. Average of the
distance, tangent and curvature of the deviation from the ground truth is calculated by using the method
presented in [15].
Fiber ID
Fiber 1
Fiber 2
Fiber 3
Fiber 4
Fiber 5
Fiber 6
Fiber 7
Fiber 8
Fiber 9
Fiber 10
Fiber 11
Fiber 12
Fiber 13
Fiber 14
Fiber 15
Fiber 16
Distance (mm)
1.6
3.9
4.6
0.8
3.1
4.3
5.8
20.7
1.4
22.2
4.9
1.2
9.8
4.5
1.9
4.1
Tangent (degs)
11
11
8
2
7
8
11
77
4
66
9
4
77
81
6
7
Curvature (mm-1)
0.024
0.027
0.014
0.009
0.013
0.008
0.019
0.109
0.008
29.063
0.014
0.018
124.808
164.280
0.011
0.012
Validations on the Clinical Dataset
Tractography of the corticospinal tract of a healthy subject and a glioma patient are presented to
demonstrate the application of the method on a clinical setting. The data is acquired using the 3T
Philips MRI scanner with dual 32 channel coil in our hospital using 64 non-collinear diffusion
directions with a b-factor of 1000, 2x2x2mm cubic voxel size and TE/TR = 76ms/6731ms
parameters.
For manual tracking, fibres are generated using the tensor fit and FACT tracking algorithms in
60
Diffusion Toolkit software. Corticospinal tract is generated by the neuroscience expert using
Trackvis software according to the guidelines provided by the protocol in [16]. The fractional
anisotropy (FA) maps are generated by fitting a diffusion tensor model in Diffusion Toolkit
software. To use with the proposed method, brain stem segmentation, which is available during
routine radiotherapy planning, and finger area of the motor cortex, which is often mapped by
functional MRI, are chosen as the seed regions. Brain stem of the subjects are labelled manually
and the motor cortical areas are mapped by functional MRI experiment using a finger tapping
task and analysed in FSL software package with a standard pipeline. The results obtained are
presented in Fig. 2 for the healthy subject for both hemispheres and for the glioma patient in Fig.
3 with the colour encoded FA map. For the healthy subject, there is a high overlap between the
obtained fibre and the expert segmentation, except the location of their extension to the motor
cortex, which depends on our usage of the finger tapping task to map the motor cortex. However,
for the glioma patient the expert failed to generate any fibre of the corticospinal tract due to the
presence of the tumour whereas the proposed method resulted in a fibre location comparable with
the colour FA.
Figure 2. Corticospinal tract obtained in 3D by the proposed method (blue) for the left (on left) and right (on
right) hemispheres of a healthy subject, projected on a sample 2D coronal FA slice. Seed regions used are
represented by red contours, whereas the expert segmentation projected on 2D is in yellow colour.
Figure 3. Corticospinal tract obtained in 3D by the proposed method (blue) (on left) for the left hemisphere,
A.Hamamci et al.
61
projected on a sample 2D coronal FA slice. The expert segmentation projected on 2D is represented by
yellow contour. Colour coded FA map for the same slice (on right) where red colour represents left-right,
blue represents superior-inferior and green represents anterior-posterior principle diffusion direction.
CONCLUSION
In this paper, application of the energy minimization framework using a snake model to the
diffusion MR fibre tractography problem was presented. Although, the proposed framework
should not be thought as an alternative to the neuroscience tools investigating the white matter
connectivity, it has the potential to be accepted in the routine practice due to its advantages in
locating a known tract. Firstly, it is robust to the local variations and doesn't fail on interruptions,
i.e. fibre crossings or signal abnormalities due to the noise or lesions. It allows adjusting the level
of confidence for the provided seed regions and the degree of smoothness of the fibre.
In general, considering the vision literature on active contours, proposed framework allows to
impose global priors, i.e. shape priors. Moreover, it is suitable for the development of interactive
techniques based on well-established spline editing algorithms.
Our future work includes: the usage of the principle diffusion directions or ODF's in image term;
handling bundle of fibres; handling the tumour segmentations for a specific case (i.e. pass
through or prevent to intersect with the tumour); better initialization strategies i.e. using
population atlases; developing user interaction techniques; utilizing the fibre atlas as shape prior;
and new target applications, i.e. arcuate fasciculus.
ACKNOWLEDGMENT
REFERENCES
[1] Tuch DS, Reese TG, Wiegell MR, Makris N, Belliveau JW, Wedeen VJ. High angular
resolution diffusion imaging reveals intravoxel white matter fiber heterogeneity. Magnet
Res Med 2002; 48(4):577-582.
[2] Wiegell MR, Larsson HBW, Wedeen VJ. Fiber crossing in human brain depicted with
diffusion tensor MR imaging. Radiology 2000; 217(3):897-903.
[3] Tuch DS. Q-ball imaging. Magnet Res Med 2004; 52(6):1358-1372.
[4] Mori S, Crain BJ, Chacko VP, van Zijl PC. Three-dimensional tracking of axonal
projections in the brain by magnetic resonance imaging. Annals of Neurology 1999;
45(2):265-269.
[5] Behrens T et.al. Characterization and propagation of uncertainty in diffusion-weighted mr
imaging. Magnet Res Med 2003; 50(5):1077-1088.
[6] Mangin JF, Fillard P, Cointepas Y, Bihan DL, Frouin V, Poupon C. Toward global
tractography. NeuroImage 2013; 80(0):290-296.
[7] Duffau H. The dangers of magnetic resonance imaging di_usion tensor tractography in brain
surgery. World Neurosurgery 2014; 81(1):56-58.
62
[8] Koga T, et.al. Outcomes of diffusion tensor tractography integrated stereotactic
radiosurgery. Int J Rad Oncol Biol Phys 2012; 82(2):799-802.
[9] Nguyen-Thanh T, et.al. Global tracking in human gliomas: A comparison with established
tracking methods. Clinical Neuroradiology 2013; 23(4):263-275.
[10] Barbieri S, Klein J, Bauer M, Nimsky C, Hahn H. Atlas-based fiber reconstruction from
diffusion tensor mri data. Int J Comput Assist Radiol Surg 2012; 7(6):959-967.
[11] Yeo B, Vercauteren T, Fillard P, Peyrat J, Pennec X, Golland P, Ayache N, Clatz O. Dtrefind: Diffusion tensor registration with exact finite-strain differential. IEEE Trans Med
Imag 2009; 28(12):1914-1928.
[12] Kass M, Witkin A, Terzopoulos D. Int J Comput Vision 1988; 321-331.
[13] Nocedal J, Wright S. Numerical optimization. Springer 2nd Ed. (2006).
[14] Poupon C, et.al. New diffusion phantoms dedicated to the study and validation of highangular-resolution diffusion imaging (hardi) models. Magnet Res Med 2008; 60(6):12761283.
[15] Fillard P, et.al. Quantitative evaluation of 10 tractography algorithms on a realistic diffusion
MR phantom. NeuroImage 2011; 56(1):220-234.
[16] Wakana S, et.al. Reproducibility of quantitative tractography methods applied to cerebral
white matter. Neuroimage 2007; 36:630-644.
A. Laruelo et al.
63
Exploiting MRSI data properties to
improve quantification
Andrea Laruelo1*, Lotfi Chaari2, Hadj Batatia2, Ben Rowland1, Soléakhéna Ken1, Regis
Ferrand1, Jean-Yves Tourneret2 and Anne Laprie1
1
University of Toulouse, IRIT - INP-ENSEEIHT, Toulouse, France
2
*
Laruelo.Andrea@claudiusregaud.fr
Abstract: Magnetic resonance spectroscopic imaging (MRSI) is a non-invasive technique able to
provide the spatial distribution of relevant biochemical compounds commonly used as
biomarkers of disease. The low signal-to-noise ratio (SNR) of the MRSI data makes the
quantification of MRSI signals a challenging problem. The incorporation of prior knowledge has
been proved to be an efficient approach to increase the robustness of the quantification. We
describe in this paper the most recent advances in this field and we propose an original
quantification method that exploits an interesting property present in MRSI data: sparsity. This is
a well know property in the signal processing community that has rarely been explored before in
the context of MRSI signals quantification. Experiments on synthetic MRSI data demonstrate
that the accuracy and robustness of the quantification are improved with the proposed scheme. In
addition, sparsity is exploited both along the spatial and spectral dimensions of the data. This
method is particularly interesting for high-resolution MRSI studies where SNR is a major
limitation.
Index Terms — MRSI data, signal processing, sparsity, metabolite quantification.
INTRODUCTION
MRSI is a non-invasive technique that has become a valuable tool to characterize metabolic
processes and neurological disorders [1]. MRSI has been proved to provide relevant information
on tumour characteristics, progression and response to treatment not available from conventional
morphological MRI imagining [2]. During the last years, MRSI has gained ground against singlevoxel spectroscopy due to its ability to provide the distribution of the metabolites over a large
volume. However, despite numerous publications on the subject [3-8], the difficulty to obtain
accurate estimates of metabolite concentrations from MRSI data is slowing down the
incorporation of this technique in clinical routines. MRSI signals present lower quality than
single-voxel measurements due to the trade-off between spectral and spatial resolutions for a
reduced scan time for each voxel. A common approach to improve the robustness of the
quantification of MRSI signals is the use of a suitable prior knowledge.
First methods incorporating prior knowledge
One of the first methods incorporating prior knowledge was AMARES [3]. It allows considering
various forms of constraints on the spectral parameters. This additional information may be the
specification of upper and lower bounds, for frequency, damping or phase. It also allows
imposing any linear relation, like differences or ratios, between individual parameters. The
64
Exploiting MRSI data properties to improve quantification
decrease of the Cramér-Rao lower bounds (CRLB) [10] confirmed in simulation experiments and
on in vivo data [3] encouraged the development of quantification methods benefiting from the
incorporation of prior knowledge. Prior knowledge on the spectral parameters can be also
incorporated by making use of experimentally measured (in vitro) or simulated metabolite
profiles [11], as implemented in [5,6].
Spatial prior knowledge
More recently, some quantification methods have proposed to exploit not only the spectral, but
also the spatial neighbourhood of MRSI signals by incorporating spatial priors into the
quantification model. One of the early approaches to exploit spatial prior knowledge was
introduced with FITT [4] which proposes an iterative MRSI fitting methodology that includes a
spatial smoothing step. After parameter estimation, selected parameters (line width, frequency
shifts, phase) are only accepted if consistent with a local neighbourhood. LCmodel [5] also
provides a spatial fitting mode for MRSI. It first analyses a central voxel and then proceeds
outwards using the results from previously fitted voxels for initialization and as a soft constraint
for new fits. More recently, Kelm et al [7] proposed Bayesian smoothness prior to improve the
fitting of MRSI data. This method assumes that some selected spectral parameters (frequency,
damping and phase) have spatially smooth variations. AQSES-MRSI [8] combines different
methods to incorporate spatial information. They propose a dynamic approach, in which the
starting parameter values are adjusted for each voxel at each iteration, the bounds on the relevant
parameter values are iteratively adapted and spatially smooth parameter maps are imposed (for
frequency shifts and damping corrections). In all these approaches, spatial prior knowledge is
directly imposed on a set of selected spectral parameters. We propose a novel quantification
scheme which exploits the sparsity (few non-zero coefficients) on the wavelet domain of the
MRSI data (Fig.1) with the aim of increasing the SNR (Signal to Noise Ratio) of the signals [9].
This method may be applied in addition to any other method incorporating other types of prior
knowledge (as the ones mentioned above). A quantification solution is formulated for the whole
MRSI grid but, with difference to previous approaches, the presented method is more flexible and
less restrictive so that sharp spatial features are preserved. In order to simultaneously fit all
signals in the MRSI grid and to introduce spectral-spatial information, a fast proximal
optimization algorithm is proposed to recover the optimal solution.
Quantification model
Let S be the observed MRSI signal corresponding to a 2D slice involving R spatial positions. Let
also U be the matrix containing the contribution of the metabolites to the observed signal at each
spatial position r. Assuming that the water signal and the macromolecular contribution have been
previously suppressed, we propose to estimate the metabolite amplitudes from the following
inverse problem:
(1)
Prior knowledge
Based on the observation that MRSI signals are sparse in the wavelet domain both in the spectral
and spatial dimensions (Fig.1) we propose to solve the inverse problem (1) by incorporating the
prior knowledge as regularization terms:
(2)
A. Laruelo et al.
65
2 is the data fidelity term,
 is the covariance matrix of the
 1
noise, F (T) is a 1D (2D) orthonormal wavelet decomposition operator and  and  are
where
D(S, HU)= S  HU
regularization parameters that balance the compromise between the spatial and the spectral
dimensions.
The relative concentration of the metabolites can be then estimated as the minimizer Û of such
criterion:
(3)
Since J is strictly convex (D is strictly convex and both regularization terms are convex), the
uniqueness of the target solution is guaranteed. However, J is not differentiable and so standard
gradient-based algorithms for minimization cannot be used. An appropriate method to solve this
optimization problem is the Simultaneous Direction Method of Multipliers algorithm described in
[12]. Key advantages of this algorithm are the guaranteed convergence to the global minimum
and the efficiency since computations can be parallelized.
Data
Simulated signals were obtained as a linear combination of the four largest cerebral metabolite
profiles detectable at long echo time, Choline (Cho), Creatine (Cr), N-acetyl-aspartate (NAA) and
Lactate (Lac). Metabolite profiles were obtained from quantum mechanical simulations of a spinecho MR experiment. Different levels of additive white Gaussian noise were added to the ground
truth signal.
Simulation experiments
In order to evaluate the accuracy and the robustness of the proposed method, a Monte Carlo study
on N=20 synthetic MRSI data sets of size 6 x 6 at five different levels of noise was performed.
The SNR of each signal s, has been computed in the frequency domain as:
(4)
where
s ref
denotes the noiseless signal .
Each data set was quantified with the proposed Spectral-Spatial Regularization method (SSR)
and two well-known methods in the field: the voxel-by-voxel approach AQSES [6] and AQSESMRSI [8]. The unbiased standard deviation of the obtained amplitudes for each metabolite k has
been calculated at each voxel as:
(5)
a k , aˆ k
where
are the true and estimated amplitudes respectively. At each level of noise, the
standard deviations of the estimated amplitudes and the corresponding Cramér-Rao lower bounds
are compared. This gave us an indication of the gain in the accuracy that can be achieved using
the proposed method.
A second experiment was designed in order to check that the proposed method SSR preserves
spatial features. A MRSI dataset of size 10 x 10 containing a region with healthy appearing
signals and a region with tumour-like signals delimited by a sharp edge was generated. The
66
results obtained with SSR are compared with the results from the voxel-by-voxel approach
AQSES.
(a)
(b)
Fig. 1: MR spectra (a) and metabolite maps (b) have a few dominating coefficients in the wavelet domain
RESULTS
Monte Carlo Experiments
In Fig.2, the mean standard deviation (std) of the metabolite concentrations estimated with each
method are compared with the CRLB obtained from the voxel-by-voxel approach. The proposed
method, SSR, outperforms all the other methods for all the levels of noise. It reduces the std by a
mean of 41%, from 24% to 54% depending on the level of noise. This shows how the inclusion
of the proposed prior knowledge prevents the quantification algorithm from moving away from
the true solution by narrowing down the search space. As a result, the method becomes, not only
more accurate, but also more robust being able to cope with signals containing high levels of
noise.
Sharp Edges
Fig.3 presents the results on a data set where the border between the region of spectra
representing healthy and abnormal tissue is a sharp edge. The first row shows the ground truth
amplitude values for NAA, Cr, Cho and Lac. The following rows show the differences from the
ground truth and the results obtained using a voxel-by-voxel method (AQSES) and the proposed
method. Compared with AQSES, the proposed method visibly improves the estimates of the
metabolites and therefore provides metabolite distribution maps closer to the ground truth.
CONCLUSIONS
Based on the observation that MRSI data are sparse in the wavelet domain both in the spectral
and spatial dimensions, a novel method incorporating sparsity promoting priors has been
presented. Results on synthetic data confirm that the accuracy and robustness of the
quantification of MRSI signals can be improved by using this method. A more detailed
description of the method and results on in vivo data will be presented in a future work.
Quantification methods able to provide accurate metabolite distribution lead the way to
individualized biologically tailored radiotherapy treatments.
A. Laruelo et al.
67
Fig.2: Mean standard deviation (std) of the estimated metabolite amplitudes at five different levels of noise
(SNR: -0.5, 2, 4.5, 7, 10). Black: CRLBs, green: AQSES; blue: AQSES-MRSI, red: proposed method
(SSR).
(a)
(b)
(c)
Fig.3: a) Ground truth (True metabolite concentrations); b) Difference between the ground truth and the
concentrations estimated with AQSES; c) Difference between the ground truth and the concentrations
estimated with SSR
ACKNOWLEDGMENT
REFERENCES
[1] Laprie A, Catalaa I, Cassol E, McKnight TR, Berchery D et al. Proton magnetic resonance
spectroscopic imaging in newly diagnosed glioblastoma: predictive value for the site of
postradiotherapy relapse in a prospective longitudinal study. Int J Radiat Oncol Biol Phys
2008; 70(3):773-781.
[2] Deviers A, Ken S, Filleron T, Rowland B, Laruelo A. et al. Evaluation of lactate/N-acetylaspartate ratio defined with MR spectroscopic imaging before radiotherapy as a new
predictive marker of the site of relapse in patients with glioblastoma multiforme, accepted
for publication in IJRBOP.
[3] Vanhamme L, van den Boogaart A, Van Huffel S. Improved method for accurate and
efficient quantification of MRS data with use of prior knowledge. J Magn Reson 1997;
129:35-43.
[4] Soher BJ, Young K, Govindaraju V, Maudsley AA. Automated spectral analysis iii:
application to in vivo proton MR spectroscopy imaging. Magn Reson Med 1998; 40:822SUMMER Project (PITN-GA-2011-290148) is funded by the 7th Framework Programme of the European Commission
68
831.
[5] Provencher S. Automatic quantitation of localized in vivo 1H spectra with lcmodel. NMR
Biomed 2001; 14:260-264.
[6] Poullet JB, Sima DM, Simonetti AW, De Neuter B, Vanhamme L et al. An automated
quantitation of short echo time MRS spectra in an open source software environment:
AQSES. NMR Biomed 2007; 20(5):493-504.
[7] Kelm BM, Kaster FO, Henning A, Weber MA, Bachert P et al. Using spatial prior
knowledge in the spectral fitting of MRS images. NMR Biomed 2012; 25(1):1-13.
[8] Sava AC, Sima DM, Poullet JB, Wright AJ, Heerschap A et al. Exploiting spatial
information to estimate metabolite levels in 2D MRSI of heterogeneous brain lesions. NMR
Biomed 2011; 24:824-835.
[9] Laruelo A, Chaari L, Batatia H, Ken S, Rowland B et al. Hybrid sparse regularization for
Magnetic Resonance Spectroscopy. Conf Proc IEEE Eng Med Biol Soc 2013; 6768-6771.
[10] Cavassila S, Deval S, Huegen C, van Ormondt D, Graveron-Demilly D. Cramér-rao bounds:
an evaluation tool for quantification. NMR Biomed 2001; 14:278-283.
[11] Graveron-Demilly D, Diop A, Briguet A, Fenet B. Product-Operator Algebra for Strongly
Coupled Spin Systems. J Magn Reson 1993; 101:233-239.
[12] Combettes PL, Pesquet JC. A proximal decomposition method for solving convex
variational inverse problems. Inverse Problems 2008; 24(6):065014.
Andrea Laruelo attended the Universidad Complutense of Madrid (Spain) as
an undergraduate, where she received her MSc degree in Mathematics. After
earning her MSc degree, she has worked at the European Space Astronomy
Centre (ESAC, Madrid) as a Software Engineer. She is currently an Early Stage
Researcher at Institut Claudius Regaud, Toulouse (France). Her research lies
primarily within the fields of magnetic resonance spectroscopic data processing,
where her work and research interests within these fields are data denoising,
spectroscopy data pre-processing and quantification.
A.Ramkumar et al.
69
Human computer interaction in
segmenting organs at risk for
radiotherapy: a pilot study
Anjana Ramkumar1*, Jose Dolz2, Hortense A Kirisli2, Tanja Schimek-Jasch3, Sonja
Adebahr3, Ursula Nestle3, Laurent Massoptier2, Edit Varga1, Pieter Jan Stappers1, Wiro J
Niessen1,4 , Yu Song1
1
Delft University of Technology, Delft, The Netherlands
2
AQUILAB, Loos-les-Lille, France
3
4 Erasmus MC - University Medical Center, Rotterdam, The Netherlands
*
A.Ramkumar@tudelft.nl
Abstract: An accurate segmentation of organs at risk in CT images is a prerequisite in
radiotherapy treatment planning. Although there are a number of automatic segmentation
methods, most of them require user interactions during the pre- and post-processing stages. Those
interactions directly influence the effectiveness of the segmentation results and the efficiency of
the process. In this paper, we explored the effects of user interactions in using a semi-automatic
segmentation method, which is based on an algorithm combining watershed and graph-cut
methods. The aims of this study are 1) to identify if the users can are comfortable with those
interactions, 2) to evaluate the quality of results with respect to manual segmentations, and 3) to
explore relations of human-computer interactions and the quality of the results in the use of the
method. Based on pre-defined protocols, two physicians contoured lung, heart and spinal cord in
several cases using the proposed method. The contouring process was video-taped and the
segmentation results were analysed. Comparing the results to manual segmentation, an average
Dice similarity coefficient (DSC) of 0.95, 0.7 and 0.8 was obtained by both clinicians for lungs,
heart and spinal cord respectively. In the qualitative evaluation, despite a lot of post processing
actions, the users were satisfied with the proposed method, as they were able to control the
system to produce sound results in an efficient manner.
Index Terms — segmentation, interaction, graph-cut algorithm, radiotherapy.
INTRODUCTION
About 50% of cancer patients receive radiotherapy at some point during the course of their
disease [1]. In radiotherapy planning, the primary aim is to maximize the delivery of radiation
dose to the tumour while sparing normal tissues. In order to deliver precise treatment, accurate
segmentation of tumour and normal tissues is required [2]. Many segmentation methods were
developed for tumour delineation. They can be categorised to 1) manual segmentation methods,
2) automatic segmentation methods and 3) semi-automatic segmentation (SAS) methods. Manual
segmentation is the process in which a physician delineate the tumour or the organs manually
using his/her own clinical knowledge and other clinical reports, e.g., radiology report,
70
HCI in radiotherapy segmentation
histopathology report, etc. With the increasing amount of imaging data, this process is considered
to be time consuming [2]. During automatic segmentation, computer algorithms control the
delineation process and generate segmentation results, despite the prior-knowledge of physicians.
They usually require few user interactions [3]. However, these methods can only be applied
successfully within pre-defined conditions and rich post processing is often needed. SAS
methods are partially supervised interactive methods. Often they are the combination of the
manual and automatic methods and in many cases, these methods are considered to be a robust
method [4]. In the use of a SAS method, the user gives an initial input and the algorithm
generates the results. If the results are not satisfactory, the user can then adjusts the initial input
or manually correct the results. This process is often repeated until a satisfactory result is
obtained.
In the development of SAS methods, research efforts have been paid on the computational [5], as
well as the Human-Computer Interaction (HCI) part [6,7]. HCI is an important part in SAS
method and typical SAS methods are, e.g., region growing, split-and-merge and threshold [8],
usually have different patterns of HCIs. The effectiveness and efficiency of a SAS method
depend on the combination of the expertise of a user and the power of the computational method
to achieve the desired segmentation result [9].
Figure 4: Typical information flow in SAS methods
Figure 1 presents the information flow with a typical iterative cycle in the SAS method. To use
the method, the user needs to give their inputs at the initialization stage, the post processing stage
or both. Here the initialization stage refers to the pre-processing steps that take place before
running the computational algorithm. In the figure, the expert perceives the output of the
computer via the interface and process the information to perform some actions. The term action
in a SAS method refers to the initialization of the algorithm. While user performs those actions,
the results of those actions will be displayed on the interface as an input from user for
verification. Then the computational algorithm processes the data and the result is seen as the
output. This cycle continues until the user is satisfied. In different SAS methods, the types of
A.Ramkumar et al.
71
inputs for the user to initialize the algorithm might be different. For instance: in some cases the
users may need to set different parameters at the initialization stage. In other cases users are
requested to draw a point or a line on images, where the inputs are positions in the image [6]. In
those cases, the location of those inputs is important since they could affect the outcomes of the
segmentation algorithm.
In summary, manual segmentation is time consuming and the automatic segmentation is faster
but always require a rich manual post-processing (e.g. case studies in Brainlab [10], ABAS [11]
software). Hence SAS is usually required. However, the SAS method also needs initialization and
post manual correction. Thus an optimal HCI is often needed in order to generate reliable,
repeatable and satisfactory result with efficient interactions. Using a developed SAS method, we
conducted a pilot study 1) to identify if the users can accept the new interactions, 2) to evaluate
the quality of results referring to the manual segmentation, and 3) to explore relations of HCI and
the quality of the results in the use of the method. The paper is organized as follows. The SAS
method is introduced in Section 2. Section 3 describes our approach in the study. Section 4
presents the experimental setup. Experimental results are analysed and illustrated in Section 5,
and discussed in Section 6.Conclusion of the study are drawn in Section 7.
SAS METHOD
Figure 2: User interface of the SAS plugin in MITK with foreground (red) and background (blue) seeds
The SAS prototype used in this study was developed as a plugin on the MITK platform. The
MITK platform is a medical imaging and interaction toolkit. We developed our prototype based
on the version MITK 2013.09.0 [12]. The accuracy of this prototype has been already
investigated in [13]. Figure 2 shows a screen shot of the user interface of Organ At Risk OAR
prototype. On the right side of the interface are tools which are used for drawing and manual
modifications. The left window has the data manager, which allows the users to select data and
set them visible/invisible. There are also several scroll bars at the bottom of the left windows
which are called as image navigators and are used by the user to scroll through the images. The
main rendering window is presented at the centre with 4 quadrants, 3 of them show different
orthogonal views. The bottom right quadrant shows the result as a 3D image. The top left
quadrant (the axial view), shows some background and foreground seeds which were drawn by
the user.
As an SAS method, the user interactions are engaged in the segmentation process, including
initialization of the algorithm and post-processing of the result. Figure 3 shows the flowchart of
72
the SAS method used in this study. The first step to do is to choose the dataset one wants to
segment. The physicians then had to choose the organ that they want to contour. After selecting
the organ, the physician initializes the algorithm by drawing the foreground and background
seeds. Here, the term foreground refers to the region they want to include in the OARs and rest of
the regions needs to be considered as background. For instance, in a task of contouring the lung
as Figure 2, the red lines are foreground seeds and the blues lines are background seeds,
respectively. Once the algorithm has given the output the user need to inspect the results if it is
good or not. If the user is satisfied with the result then they can end their task. If not then the user
has two ways to correct the result: 1) If it is a minor correction, the user can manually modify the
segmentation; 2) If it is a major correction, the user can re-draw the foreground and/or
background seeds on any of the orthogonal planes and run the segmentation again till the user is
satisfied.
Figure 3: Process of segmentation of the proposed SAS method
OUR APPROACH
Usability is an important issue of the SAS methods as the user will frequently interacts with the
system. ISO 9241 part 11 defines usability as “the extent to which a product can be used by
specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a
specified context of use” [14]. Here effectiveness refers to how completely and accurately the
work/goal is reached. Efficient refers to how much effort, time, and/or costs users paid to finish a
task. Satisfaction denotes how much users are satisfied with the process of completing the given
task. If the usability inspection, or testing, is first carried out at the end of the design cycle,
changes to the interface can be costly and difficult to implement, which in turn leads to usability
recommendations [15].
In order to identify the usability problems in the proposed SAS method and offer advices for
A.Ramkumar et al.
73
further improvements, in this study, we used the user examine method [16]. This study started
with the usability inspection where the first author performed usability inspection and reported it
to the developers. Once the issues were fixed, we then conducted a pilot study before going for
the main user testing. The pilot test has two purposes: 1) identify major problems regarding the
usability of the algorithms; 2) testing and verifying usability test method and protocols. After the
pilot testing, the software was further developed based on those findings and advices.
As the users’ suggestions and preference will directly influence the usability design [16], during
the usability testing, the user reactions are observed and the segmentation process is video
recorded. After the experiment, the method of interview, and questionnaire were used to
qualitatively evaluate the process and the results. Finally, the segmentation results are analysed
quantitatively referring to a manual contouring result.
EXPERIMENTAL SETUP AND PROTOCOL
a) Imaging data set
Dataset of seven patients who underwent planning CT (pCT) for lung cancer treatment planning
were selected. Five out of seven pCT images were acquired on a Philips Gemini TF Big Bore
PET/CT scanner and remaining of the pCT images were acquired on Siemens SOMATOM
emotion CT scanner. Each scan was taken based on the lung protocol followed in the Clinical
University Hospital, Freiburg, Germany.
b) Participants
Two resident physicians from Clinical University Hospital, Freiburg, Germany, joined the study.
Physician 1 had 4 years of experience in the field of contouring and physician 2 had 7 years of
experience.
c) Experimental task
In this pilot study, the physicians had to contour the lungs, the heart and the spinal cord, as these
organs are a subset of the mandatory OAR to be contoured for lung cancer treatment planning.
Both the physicians contoured the organs in the above mentioned order.
d) Test setup & protocol
This study was conducted at the Department of Radiotherapy in Clinical University Hospital,
Freiburg, Germany. Before the beginning of the study, it was explained to the physicians that this
prototype will be a SAS system and they were asked if they have any previous experience with
semi-automatic segmentation. The physicians were then explained about the designed user
interactions in this prototype, in particular the terms foreground and background since it is a new
term for the physicians. They were also explained that in this prototype, they only need to
initialize the system by specifying the foreground and the background of OARs. Using this
information, the algorithm can carry out the segmentation at each iteration. They were also given
a Radiation Therapy Oncology Group (RTOG) [17] atlas, in case they require some clarification
regarding the anatomical extension of the organs. As the user interface was new for the
physicians compared to their daily work, they were given a flow chart which instructed them the
process.
e) Evaluation measure
More recently in healthcare research, there has been an upsurge of interest in the combined use of
qualitative and quantitative methods [18]. In this study, we computed both qualitative and
quantitative measurements and correlated them. A supervised quantitative evaluation technique
74
has been used in this study: the segmentation results from the physicians were compared to the
standard 2D manual segmentation done by an expert physician using the Oncentra treatment
planning from Nucletron [19]. The Dice similarity coefficient (DSC) [20] was computed for each
result referring the manual segmentation result. DSC is denoted as 2c/(a+b), Where a is volume
of segmentation result using the proposed SAS method, b is the volume of the manual
segmentation result and c is intersection of a and b. If the DSC equals to 1, it symbolises a prefect
overlap between the 2 volumes. If it is near 0, it means a least overlap volume. As explained in
[21, 22], the DSC is sensitive to variations in shape, size and position and a value of S >0.7
indicates a strong agreement. To compute the DSC we used a program developed based on the
Mevislab [23]. Video recording was used for detailed analysis of the study. For instance, the
number of interactions required at the initialization phase was compared. Qualitatively, a post
study usability questionnaire and semi-structured interviews were conducted at the end of the
testing to find their experience about the interface and also about the prototype.
RESULTS
The OAR delineation results generated in the experiments have been compared against the
manual segmentation done for the RT planning. Figure 4-6 shows the result of DSC of the lung,
heart and Spinal Cord. The x-axis on the graph shows the patients and the y-axis shows the DSC.
The blue bar indicates the DSC of physician 1 and the red bar indicates the DSC of the physician
2.
1
DSC
0,9
P1
P2
0,8
0,7
Pt 1
pt 2
pt 3
pt 4
pt 5
pt 6
pt 7
Figure 4: Dice Similarity Coefficient of Lung
1
DSC
0,9
P1
P2
0,8
0,7
Pt 1
pt 2
pt 3
pt 4
pt 5
pt 6
pt 7
Figure 5: Dice similarity coefficient of the heart
A.Ramkumar et al.
75
1
DSC
0,9
P1
P2
0,8
0,7
Pt 1
pt 2
pt 3
pt 4
pt 5
pt 6
pt 7
Figure 6: Dice similarity coefficient of the Spinal Cord
5
16
Number of initial interactions
14
4
12
3
10
P1
8
P2
6
P1
2
P2
1
4
0
2
System
Information
usefulness
0
Lung
Heart
Interface
SC
Figure 7: Average number of interactions
Figure 8:Results of questionnaires
Figure 7 shows the number of interactions required in the initialization phase for each organ. This
number is considered as the total number of slices the physicians has to give their input. Even if
the physicians had to re-run the segmentation by drawing the seeds, it is included as the initial
interaction. Figure 8 shows the result of the questionnaire which was divided into 3 parts as
usefulness of the system, information on the system and the interface of the system. In this
context system refers to the SAS prototype. The system usefulness had questions regarding the
efficiency and effectiveness of the prototype against the task performed by the users. Information
contained questions regarding the information/ tools available for the users to perform the task.
Interface had questions regarding the general interface design and about user preference towards
the interface.
Table 3: Semi-structured interview
Organs easier to contour
Organs they liked to contour
Physician 1
Lung & Spinal Cord
Lung & Spinal Cord
Physician 2
Spinal Cord
Lung & Spinal Cord
DISCUSSION
The user interaction considered in this study is very unique from the one which the users use in
their daily work. The users just have to draw few strokes, to indicate the system which is a
foreground and a background. This is less time consuming when compared to their normal way
of interactions as they do not have to segment the whole organ. The user can select a few
76
representative slices and enter foreground and background seeds in these slices. Since all voxels
are connected in a 3D graph the information on what parts of the 3D volume are of interest and
what parts should be considered background will propagate appropriately. Moreover, upon the
initial segmentation the user can scan through the segmented slices and enter correcting seeds in
some of the slices where the results are not satisfactory.
Lung
From the qualitative results mentioned in Table 1, it was apparent that it is the easiest organ to
contour. Both physicians felt happy and satisfied after contouring the lung since they did least
manual corrections. The quantitative analysis also revealed that, DSC of lung is much higher for
both physicians compared to other structures. The number of initial interactions was the same for
both. The interesting thing to note here is that even though physician 2 did a lot of manual
correction than physician 1, but the DSC of physician 1 was slightly higher than physician 2 in
almost all cases. Majority of the manual segmentation required was for deleting the proximal
bronchial tree and sometimes trachea. From the video analysis it was also found that the number
of foreground seeds used was much lesser than the number of background seeds.
Heart
Segmentation of the heart in medical images is a challenging and important task for many
applications [22].The upper border of the heart was considered at the level of the inferior aspect
of the pulmonary artery passing the midline and extend inferiorly to the apex of the heart
[17].The physicians also considered it is a difficult organ to segment as they found it hard to
identify the borders. However the DSC values were more than 0.7 for all the patients and hence
the result is considered to be satisfactory. The variations in DSC could be due to using different
starting and ending slices of the organ, as both the physicians always had a variation of one or
two slices. On the other hand it was found from the video analysis that a lot of post manual
corrections were done by both the physicians. As the algorithm did not give a perfect boundary of
the organ, the physicians had to manually correct the boundary in all the datasets. A lot of manual
corrections could also lead to a better result, which is reflected in the value of DSC (>0.7). From
the interview, it was found that heart was cognitively demanding to contour as they have to think
a lot in drawing the background seeds because of many structures surrounding it.
Spinal cord
The number of initial interactions required in spinal cord was higher compared to other organs,
especially for physician 2 (Figure 7). The result is a bit controversial comparing to the literature
[24], where it was mentioned that high number of interactions may reach better results than
medium number of interactions. In this study, results with medium number of interactions
produces better results than higher number of interactions. The reason for this is that spinal cord
was the lengthiest organ and hence the physician considered putting lot of seeds in axial slices.
However, the reason for the physician to have less initial interaction is that the physician
considered the other planes (e.g. sagittal or coronal planes) as well to draw her seeds. From the
interview it was proved that spinal cord was easier to contour as they physicians understood
where to draw their background seeds. So cognitively it was less challenging to segment. Similar
to Mendili et al. [25], this study also had a DSC of more than 0.8 for all the patients except one.
The patient 6 has comparatively a low DSC for the heart and spinal cord. The reason could be
that it is acquired from a different machine compared to other datasets.
The platform used in this study was the MITK workbench and the plugin was specially
developed for the proposed SAS method. As an experimental interface, many conventional tools
for manual segmentation, for instance, interpolation options, 3D ball tool, pearl tool, etc., were
A.Ramkumar et al.
77
missing. This leads to a lower grade the interface of the prototype (Figure 8). However, the
physicians agreed that it was possible to complete the task without the availability of the tools
they mentioned. Also we expected the physicians to run the algorithm at least 3-4 times and but
in most cases, the physicians run the algorithm once and then conducted a manual modification.
That means the physician were not confident that a good result can be generated by running the
algorithm. In addition, most of the interactions were conducted in the axial plane. This could be
improved by using the other orthogonal planes to contour.
Limitations
Firstly, the main limitation of this study was that it was not possible to measure the time taken as
the physicians did a lot of manual contouring. Secondly, the prototype sometimes crashed due to
memory leakages. This influences the interaction process. Thirdly, the datasets were obtained
from two different machines and hence the intensity of the datasets was different.
CONCLUSION AND FUTURE WORK
In this paper, a pilot study of the usability of a SAS method for contouring OARs in radiotherapy
is presented. Based on the outcomes of several experiments, it is found that the users are willing
to accept our method and the interaction style, as they feel that they can control the system to
produce sound results in a more efficient manner. Besides, it is also concluded that in the
proposed SAS method, there is no correlations between the number of interactions and the
quality of result. Thus even with very limited user interactions, it is possible to produce high
quality results, provided that the users understands the method and is familiar with the interface.
In the pilot, it is observed that a lot of post manual corrections are engaged in the post processing,
which is not optimal. Our future study will focus on getting a better understanding of the process
in order to guide users to achieve the results using minimal user interactions. In addition, the
designed foreground / background interactions will be compared to the methods used in clinical
practices to further identify the advantages and limitations of the proposed SAS method.
ACKNOWLEDGMENT
for research, technological development and demonstration under grant agreement no PITN-GA2011-290148. The authors would like to express their appreciations to other members of the
SUMMER consortium for their valuable advices regarding the proposed research.
REFERENCES
[1] Radiotherapy- http://www.iirrt.ie/about-us/information-for-patients, accessed on 14th June
2014.
[2] Whitfield GA, Price P, Price G J, Moore CJ. Automated delineation of radiotherapy
volumes: are we going in the right direction? Brit J Radiol 2013, 86(1021):20110718.
[3] Zhao F, Xie X. An Overview of Interactive Medical Image Segmentation. 2013, (7), 1–22.
[4] Harders M, Member S, Székely G, Member A. Enhancing Human – Computer Interaction
in Medical Segmentation. Proc IEEE 2003; 91(9):1430-1442.
[5] Blake A, Rother C, Brown M, Perez P, Torr P. Interactive image segmentation using an
adaptive gmmrf model. Europ Conf Comput Vision (ECCV) 2004; 428-441.
78
[6] Olabarriaga S, Smeulders A. Interaction in the segmentation of medical images: A survey.
Med Imag Anal 2001; 5(2):127–142.
[7] Zhu Y. Towards More Desirable Segmentation via User Interactions, Thesis and
Dissertation, 2013.
[8] McGuinness K, O’Connor NE. A comparative evaluation of interactive segmentation
algorithms. Pattern Recognition 2010; 43(2):434-444.
[9] Karray F, Alemzadeh M, Saleh JA, Arab MN. Human-Computer Interaction : Overview on
State of the Art. 2008, 1(1), 137–159.
[10] Brainlab-https://www.brainlab.com/radiosurgery-products/iplan-rt-treatment-planningsoftware/ , accessed on 1st may 2014.
[11] ABAS-http://www.elekta.com/healthcare-professionals/products/elekta software/treatmentplanning-software/contouring-software/atlas-based
autsegmentation.html?utm_source=abas&utm_medium=redirect&utm_campaign=redirects ,
accessed on 16th may 2014.
[12] MITK -http://www.mitk.org/ accessed on 16th may 2014.
[13] Dolz J, Kirisli HA, Viard R, Massoptier L. Combining watershed and graph cuts methods to
segment organs at risk in radiotherapy, Proc. SPIE 9034,Medical Imaging 2014: Image
Processing, 90343Z (March 21, 2014); doi:10.1117/12ISO 9241-143.
[14] Ergonomics of human-system interaction -2012, Part 11.
[15] Holzinger A. Usability engineering methods for software developers communications of the
ACM 2005; 48(1):71-74.
[16] Gong C. Human-computer interaction: The usability test methods and design principles in
the human-computer interface design. 2nd IEEE Int Conf Comput Sc Info Tech 2009; 283–
285.
[17] Guidelines for organs at risk delineation in thoracic radiation therapy. Based on RTOG
guidelines. http://www.rtog.org/CoreLab/ContouringAtlases/LungAtlas.aspx, accessed on
feb 10, 2014.
[18] Moffatt S, White M, Mackintosh J, Howel D. Using quantitative and qualitative data in
health services research - what happens when mixed method findings conflict? BMC Health
Services Research, 2006, 6, 28.
[19] Oncentrahttp://www.nucletron.com/en/ProductsAndSolutions/Pages/OncentraExternalBeam.aspx,
accessed on 7 may, 2014
[20] Dice LR. Measures of the amount of ecologic association between species. Ecology 1945;
26(3):297-302.
[21] Zijdenbos A, Dawant B, Margolin R, Palmer A. Morphometric analysis of white matter
lesions in MR images: method and validation. IEEE Trans Med Imaging 1994; 13(4):716724.
[22] Morenoa A, Takemura CM, Colliotc O, Camarad O, Blocha I. Using anatomical knowledge
expressed as fuzzy constraints to segment the heart in CT images, Pattern Recognition 2008;
41:2525-2540.
[23] Mevislab - http://www.mevislab.de/, accessed on 7 may, 2014.
[24] Heimann T, van Ginneken B, Styner M, Arzhaeva Y, Aurich V, Bauer C, Wolf I.
Comparison and evaluation of methods for liver segmentation from CT datasets. IEEE
A.Ramkumar et al.
79
Trans Med Imaging 2009; 28(8):1251-1265.
[25] El Mendili MM, Chen R, Tiret B, Pélégrini-Issac M, Cohen-Adad J, Lehéricy S, Pradat PF,
Benali H. Validation of a semiautomated spinal cord segmentation method. Journal of
Magnetic Resonance Imaging  2014; 00:1-6.
Anjana Ramkumar was born in a small town called as Renukoot, in
state called as Uttar Pradesh in north India. Ramkumar hold a Bachelor’s
degree in Medical Radiological Technology (4years) from the Amrita
Institute of Medical Sciences and Research Centre, Kochi, India in 2011.
After her Bachelors, she moved to the United Kingdom for her Masters
in Medical Physics at the University of Surrey in 2012 September. In
2012 November, Ramkumar joined the Faculty of Industrial Design
Engineering, Delft University of Technology for her PhD study.
Along with her bachelor’s studies she had training at the radiotherapy
department, radiology and nuclear medicine departments as a
technologist for 3 years. Her bachelor thesis was about “Comparison of
gross tumour volumes obtained using auto-contoured program PETVCAR versus manual contouring”. During her masters she did her
internship at the Brighton Sussex cancer centre for few weeks and then
she worked on her maser’s research at the same hospital on the topic,
“commissioning of 4D CT”. At present, her research work is related to
contouring, but focusing more on user-centred design.
80
Nanotechnology in Cancer
Nanoparticle technology: future
opportunities in cancer treatment
Suneil Jain1*, Karl T. Butterworth1
1
*
Centre for Cancer Research and Cell biology, Queens University Belfast
s.jain@qub.ac.uk
Abstract: Gold nanoparticles are emerging as promising agents for cancer therapy and are being
investigated as vehicles for drug delivery, agents for photothermal therapy, image contrast and
radiosensitisation. This review introduces the field of nanotechnology with a focus on recent gold
nanoparticle research which has led to early phase clinical trials. In particular the increasing
preclinical evidence for gold nanoparticles as sensitizers with ionizing radiation in vitro and in
vivo is discussed.
Index Terms — gold nanoparticles, nanotechnology, radiosensitisers, image guidance.
INTRODUCTION
Nanotechnologies can be defined as the design, characterization, production and application of
structures, devices and systems by controlling shape and size at nanometre scale [1]. The
potential wide ranging applications and benefits of nanomaterials are well recognized in the
literature with some commentators speculating the impact of nanotechnology to far exceed that of
the Industrial Revolution, projecting to a market of $1 trillion by 2015 [2,3]. In medicine, much
research interest is focused on the use of nanoparticles to enhance drug delivery. However,
nanoparticles have addition wide ranging applications including in-vitro diagnostics, novel
biomaterial design, bioimaging, therapies and active implants [4].
REVIEW
Gold nanoparticles (GNPs) are very small particles (1-100 nm in diameter) that exist in a nonoxidized state. In the late 20th century techniques including transmission electron microscopy
(TEM) and atomic force microscopy (AFM) enabled direct imaging of GNPs and subsequently
improved control over nanoparticle properties including size, geometry and surface
functionalization [5]. In recent years there has been an explosion in GNP research with a rapid
increase in GNP publications in diverse fields including imaging, bioengineering and molecular
biology. It is probable that this is directly related to a similar increase in the broader field of
nanotechnology associated with increased governmental awareness and funding with
simultaneous rapid progress in chemical synthesis and molecular biology [6]. GNPs exhibit
unique physicochemical properties including surface plasmon resonance (SPR) and the ability to
bind amine and thiol groups which allows surface modification and applications in biomedicine
[7]. Surface functionalization of nanoparticles is an area of intense research at present with rapid
progress being made towards the development of biocompatible, multifunctional particles for use
in cancer diagnosis and therapy [2].
There is intense interest in modifying existing drugs to improve pharmacokinetics, thereby
S. Jain et al.
81
reducing non-specific side-effects and enabling higher dose delivery to target tissues. An
important demonstration of the potential of multifunctional GNPs for drug delivery was the use
of 5 nm GNPs as a delivery vehicle, covalently bound to cetuximab as an active targeting agent
and gemcitabine as a therapeutic pay-load as a systemic treatment for pancreatic cancer [8]. The
epidermal growth factor receptor (EGFR) is overexpressed in up to 60% of pancreatic cancers
and the combination of cetuximab and gemcitabine has been investigated in Phase II trials of this
disease [9].
Hyperthermia is known to induce apoptotic cell death in many tissues and has been shown to
increase local control and overall survival in combination with radiotherapy (RT) and
chemotherapy in randomized clinical trials [10-12]. Hyperthermia is normally used in
combination with other treatments including RT and can be delivered externally, interstitially or
endo-luminally with heat generation by radiofrequency waves, microwaves or ultrasound [13]. A
novel approach to improve thermal therapy involves tumour specific targeting of metal
nanoparticles used in combination with a non-ionizing electromagnetic radiation source such as a
laser. When the laser is applied to the nanoparticle loaded tumour there is highly efficient energy
conversion due to electron excitation and relaxation which increases the temperature of metal
nanoparticles resulting in increased therapeutic efficacy [14]. Furthermore, lasers can be
specifically tuned to the SPR frequency of nanoparticles, which varies with the size, shape and
composition of the nanoparticle [15]. Most research has used gold nanoshells composed of100
nm silica cores with a 15 nm gold coating, which shifts the resonance peak to the near infrared
region (650-950 nm) where blood and tissue are maximally transmissive [16].
Physicochemical properties of GNPs including small size, biocompatibility, high atomic number
(high-Z) and the ability to bind targeting agents mean they have potential as image contrast
agents. Contrast materials such as iodine, improve the definition of heavily vascularized tumour
tissue by increasing photoelectric photon absorption enabling improved accuracy of tumour
diagnosis, staging and aiding volume definition in RT planning [17]. 1.9 nm GNPs (Aurovist™)
have demonstrated increased retention times and superior contrast to iodine with resolution of
vessels as small as 100 µm in an ectopic breast tumour model imaged with a 225 kVp
mammography unit 2 minutes to 24 hours post-injection [18]. . Despite high initial blood
concentrations of GNPs (10 mg/ml blood), no hematological or biochemical abnormalities were
detected at 11 or 30 days post-injection. Quantitative pharmacodynamics demonstrated that
GNPs were renally excreted with blood gold concentration falling in a biphasic manner with a
50% drop from 2 to 10 minutes followed by a further 50% reduction from 15 minutes to 1.4
hours. In contrast, tumour levels at 24 hours were 64% of peak levels,15 min post-injection
suggesting nanoparticle extravasation into tumour tissue. Improved retention times and contrast
could allow detection of smaller tumours at staging, aid image guided RT and allow intratumoural GNP dose quantification.
GNPs have the potential to improve contrast with structural imaging modalities including
Computed Tomography (CT) and Magnetic Resonance Imaging (MRI), it is possible that
functionalized GNPs could be useful in the field of molecular imaging to give in vivo information
on the metabolic activity of cancer and the expression of molecular markers. Positron Emission
Tomography (PET) is the most frequently used functional imaging modality in clinical use and
its benefits over standard imaging have been well demonstrated [19-21]. To date, CT has not
been used as a molecular imaging modality as iodine cannot be conjugated to molecular proteins.
Targeted nanoparticles, including super-paramagnetic nanoparticles and GNPs, are now being
developed to improve imaging with MRI and CT [22,23].
82
Whilst GNP radiosensitisation has been observed in many studies, as discussed below, much of
this work has been phenomenological with the underlying mechanisms of sensitization remaining
unclear. Most researchers have attributed GNP radiosensitisation to the physical process
ofincreased photoelectric photon absorption by high-Z materials at kilovoltage (kV) photon
energies (Figure 1).
Fig. 1. Radiation dose response curves for MDA-MB-231 and DU145 cells with 160 kVp X-rays. The SERs
were 1.41 and 0.92 for MDA-MB-231 and DU145 cells respectively.
GNP radiosensitisation has been modelled by a number of investigators. Cho et al. modelled the
effects of GNPs with an iridium-192 source, kV and MV photon energies [24]. With 140 kVp Xrays and a uniform distribution of 7 mg/ml GNPs a DEF of 2.11 was predicted. However, at MV
energies predicted physical enhancement was extremely low, for example, a physical dose
enhancement of 1% to 7% was predicted with 4 and 6 MV photons with gold concentrations
ranging from 7 mg/ml to 30 mg/ml. McMahon et al. generated a figure of merit to account for
increased radiation absorption in tumours loaded with 1% GNPs, demonstrating that tumours up
to 4 cm deep could preferentially be treated with kV photons. However, this study did not
consider increased radiation dose to skin due to loss of radiation build-up, which may be doselimiting [25]. Furthermore, a commercial kV X-ray unit with the ability to deliver intensity
modulated radiation is unlikely to be developed. Patients with localized prostate cancer are often
treated with brachytherapy using iodine-125 (I-125) or palladium-103, which emit γ-rays of
maximum energy 35 keV and 21 keV respectively. Cho et al. specifically modelled I-125
brachytherapy seeds in tumours exposed to 0-18 mg/g GNPs [26]. DEFs of 1.68 were noted a
distance of 1 cm from the I-125 source when 7mg/ml GNP were used.
Radiosensitisation driven solely by this physical mechanism would predict no effect at clinically
relevant megavoltage (MV) energies where Compton interactions are dominant [24]. The
hypothesis has been disproven by several investigators who have shown significant levels of
radiosensitisation at MV energies [27,28]. In vitro 1.9 nm GNPs (Aurovist™) in combination
with 250 kVp radiation were shown to prolong survival in tumour-bearing mice [29]. In the first
experiment Balb/c mice bearing EMT-6 murine breast cancer tumours received a single dose of
30 Gy using 250 kVp radiation alone or in combination with high concentrations of GNPs (1.35g
Au/kg) injected IV 5 minutes prior to irradiation. Radiation alone induced tumour growth delay,
however radiation and GNPs actually led to a dramatic reduction in tumour growth when
assessed 1 month after treatment.
For clinical translation and optimized efficacy, it is essential to know the importance of
S. Jain et al.
83
nanoparticle properties such assize, concentration, surface coating and distance from target
material such as nuclear DNA. These relationships along with a complementary knowledge of the
range, energy and type of secondary species released from the nanoparticle, eg. short range low
energy electrons, Auger electrons, photoelectrons or characteristic X-rays, and their variation
with primary photon energies would enable the rational design of GNPs for use with radiation.
The field of nanomedicine remains relatively immature with its full potential in the clinical
landscape not yet fully realized. Many new nanocomplexes are being developed for cancer
therapy which has much potential to improve standard of care. However, for these new agents to
be translated to clinic, there is a distinct need for well designed, safe and timely clinical trials. .
Currently, only 1 GNP therapy, CYT-6091, has reached early phase clinical trials. CYT 6091 is
a 27 nm citrate coated GNP bound with thiolated PEG and TNF-α (Aurimmune) which has the
dual effect of increasing tumour targeting and tumour toxicity [30].
CONCLUSION
GNPs have many properties that are attractive for use in cancer therapy. They are small, can
penetrate widely throughout the body and preferentially accumulate at tumour sites. Importantly,
they can bind many proteins and drugs and can be actively targeted to cancer cells overexpressing cell surface receptors. Whilst they are biocompatible, it is clear that GNP preparations
can be toxic in in vitro and in vivo systems. GNPs have a high atomic number, which leads to
greater absorption of kV X-rays and provides greater contrast than standard agents. They resonate
when exposed to light of specific energies producing heat that can be used for tumour selective
photothermal therapy. GNPs have been shown to cause radiosensitisation at kV and MV photon
energies. The exact mechanism remains to be elucidated, but may be physical, chemical or
biological. Many questions need to be answered before GNP complexes enter routine clinical
use.
ACKNOWLEDGMENT
We acknowledge funding from Cancer Research UK, Friends of the Cancer Centre and Men
Against Cancer to carry out this work.
REFERENCES
[1] ASTM E 2456-06. Terminology for Nanotechnology, (2006).
[2] Nel A, Xia T, Madler L, Li N. Toxic potential of materials at the nanolevel. Science 2006;
311:622-627.
[3] R.F. Service. American Chemical Society meeting. Nanomaterials show signs of toxicity.
Science 2003; 300:243.
[4] Wagner V. The emerging nanomedicine landscape. Nat Biotechnol 2006; 24:1211-1217.
[5] Eigler DM, Schweizer EK. Positioning single atoms with a scanning tunnelling microscope.
Nature 1990; 344:524-526.
[6] Chen H, Roco MC, Li X, Lin Y. Trends in nanotechnology patents. Nat Nano 2008; 3:123125.
[7] Shukla R, Bansal V, Chaudhary M, Basu A, Bhonde RR, Sastry M. Biocompatibility of
gold nanoparticles and their endocytotic fate inside the cellular compartment: a microscopic
84
overview. Langmuir 2005; 21:10644-10654.
[8] Patra CR, Bhattacharya R, Mukhopadhyay D, Mukherjee P. Fabrication of gold
nanoparticles for targeted therapy in pancreatic cancer. Adv Drug Deliv Rev 2010; 62:346361.
[9] Kullmann F, Hollerbach S, Dollinger M, Harder J, Fuchs M, Messmann H, et al. Cetuximab
plus gemcitabine/oxaliplatin (GEMOXCET) in first-line metastatic pancreatic cancer: a
multicentre phase II study. Br J Cancer 2009; 100(7):1032-1036.
[10] Issels RD, Lindner LH, Verweij J, Wust P, Reichardt P, Schem BC, et al. Neo-adjuvant
chemotherapy alone or with regional hyperthermia for localised high-risk soft-tissue
sarcoma: a randomised phase 3 multicentre study. Lancet Oncol 2010; 11:561-570.
[11] van der Zee J, González D, van Rhoon GC, van Dijk JDP, van Putten WLJ, Hart AAM.
Comparison of radiotherapy alone with radiotherapy plus hyperthermia in locally advanced
pelvic tumours: a prospective, randomised, multicentre trial. Lancet 2000; 355:1119-1125.
[12] Vernon CC, Hand JW, Field SB, Machin D, Whaley JB, Van Der Zee J, et al. Radiotherapy
with or without hyperthermia in the treatment of superficial localized breast cancer: Results
from five randomized controlled trials. Int J Radiat Oncol Biol Phys 1996; 35(4):731-744.
[13] Wust P, Hildebrandt B, Sreenivasa G, Rau B, Gellermann J, Riess H, et al. Hyperthermia in
combined treatment of cancer. Lancet Oncol 2002; 3:487-497.
[14] Cherukuri P, Curley SA. Use of nanoparticles for targeted, noninvasive thermal destruction
of malignant cells. Methods Mol Biol 2010; 624:359-373.
[15] El-Sayed IH, Huang X, El-Sayed MA. Surface plasmon resonance scattering and absorption
of anti-EGFR antibody conjugated gold nanoparticles in cancer diagnostics: applications in
oral cancer. Nano Lett 2005; 5:829-834.
[16] Lal S, Clare SE, Halas NJ. Nanoshell-enabled photothermal cancer therapy: impending
clinical impact. Acc Chem Res 2008; 41:1842-1851.
[17] Essig M, Debus J, Schlemmer HP, Hawighorst H, Wannenmacher M, van Kaick G.
Improved tumour contrast and delineation in the stereotactic radiotherapy planning of
cerebral gliomas and metastases with contrast media-supported FLAIR imaging.
Strahlenther Onkol 2000; 176:84-94.
[18] Hainfeld JF, Slatkin DN, Focella TM, Smilowitz HM. Gold nanoparticles: a new X-ray
contrast agent. Br J Radiol 2006; 79:248-253.
[19] Dwamena BA, Sonnad SS, Angobaldo JO, Wahl RL. Metastases from non-small cell lung
cancer: mediastinal staging in the 1990s--meta-analytic comparison of PET and CT.
Radiology 1999; 213:530-536.
[20] Vansteenkiste JF. FDG-PET for lymph node staging in NSCLC: a major step forward, but
beware of the pitfalls. Lung Cancer 2005; 47:151-153.
[21] Vansteenkiste JF, Stroobants SG, De Leyn PR, Dupont PJ, Bogaert J, Maes A, et al. Lymph
node staging in non-small-cell lung cancer with FDG-PET scan: a prospective study on 690
lymph node stations from 68 patients. J Clin Oncol 1998; 16:2142-2149.
[22] Harisinghani MG, Barentsz J, Hahn PF, Deserno WM, Tabatabaei S, van de Kaa CH, et al.
Noninvasive detection of clinically occult lymph-node metastases in prostate cancer, N Engl
J Med 2003; 348:2491-2499.
[23] Debouttiere PJ, Roux S, Vocanson F, Billotey C, Beuf O, Favre-Reguillon A, et al. Design
of gold nanoparticles for magnetic resonance imaging. Advanced Functional Materials
S. Jain et al.
85
2006; 16:2330-2339.
[24] Cho SH. Estimation of tumour dose enhancement due to gold nanoparticles during typical
radiation treatments: a preliminary Monte Carlo study. Phys Med Biol 2005; 50:N163-173.
[25] McMahon SJ, Mendenhall MH, Jain S, Currell F. Radiotherapy in the presence of contrast
agents: a general figure of merit and its application to gold nanoparticles. Phys Med Biol
2008; 53:5635-5651.
[26] Cho SH, Jones BL, Krishnan S. The dosimetric feasibility of gold nanoparticle-aided
radiation therapy (GNRT) via brachytherapy using low-energy gamma-/x-ray sources. Phys
Med Biol 2009; 54:4889.
[27] Jain S, Coulter JA, Hounsell AR et al. Cell-specific Radiosensitisation by Gold
Nanoparticles at Megavoltage Radiation Energies. Int J Radiat Oncol Biol Phys 2011;
79:531-539.
[28] Chithrani DB, Jelveh S, Jalali F et al. Gold nanoparticles as radiation sensitizers in cancer
therapy. Radiat Res 2012; 173:719-728.
[29] Hainfeld JF, Slatkin DN, Smilowitz HM. The use of gold nanoparticles to enhance
radiotherapy in mice. Phys Med Biol 2004; 49:N309-N315.
[30] Libutti S, Paciotti G, Myer L, Haynes R, Gannon W, Walker M, et al. Results of a
completed phase I clinical trial of CYT-6091: A pegylated colloidal gold-TNF
nanomedicine. J Clin Oncol 2009; 27(15S):3586.
Suneil Jain is a Consultant and Senior Lecturer at Queens University Belfast,
UK. He completed his medical degree from the same institution in 1999. He
completed a PhD in radiobiology and nanotechnology in 2010 also from QUB.
He completed training in Clinical Oncology at the Northern Ireland Cancer
Centre. His international fellowship was carried out at University of Toronto
researching stereotactic radiotherapy in lung and prostate cancer. He leads the
prostate SABR group in Northern Ireland and is a member of the UK NCRI
Prostate Clinical Studies Group. He is interested in modelling modern
radiotherapy treatments in vitro and in vivo and in the utilisation of
nanotechnology to enhance the effects of radiation therapy.
Dr Jain has been a recipient of an ASCO merit award. He holds grant funding
from Movember, Prostate Cancer UK and the NI R+D office.
86
The ART of translation
The ART of translation: from research
to clinical application
Marcel Verheij MD PhD1* and Jan-Jakob Sonke PhD1
1
*
Department of Radiation Oncology, The Netherlands Cancer Institute, Amsterdam
m.verheij@nki.nl
Abstract: Translational research refers to the transfer of a new scientific concept from its basic
preclinical stage to a successful clinical application. This process is bidirectional as clinical
observations should also be used to re-evaluate and improve their underlying experimental
models. An inherent element of translational research is the use of patient material. This should,
however, be interpreted widely and not be restricted to biological material. In fact, all
information derived from patients – including digital and epidemiological data – is suitable for
translational research. Acknowledging translational research as an essential tool to advance
oncology, many research groups and funding organizations focus on this area.
Index Terms — Radiotherapy, Translational Research, Adaptive, Preclinical.
TRANSLATIONAL RESEARCH AT NKI
The Netherlands Cancer Institute (NKI) is a comprehensive cancer center combining hospital and
research laboratories under one roof in a single independent organization with one board of
Directors responsible for both clinic and research. The institute has a long tradition of integrating
basic science and cancer care, and has developed several strategies to accommodate translational
research since its foundation in 1913 [1]. NKI stimulates part-time research appointments of staff
clinicians by “twinning” them to basic scientists on joint translational projects. A translational
research fellowship program offers to young clinicians after completion of their specialty
training, a 2-3 years fellowship in basic research labs to build-up their oncology careers. In
addition, internal start-up funding is provided to generate preliminary data and support project
proposals. Furthermore, all clinical trial proposals are screened for potential translational research
elements. A Translational Research Board, consisting of staff MDs and PhDs with a track record
in translational research, coordinates these activities.
Although many regard translational oncology as interplay between fundamental biological and
clinical research, this is certainly not a complete description. In radiation oncology, translational
research includes many other areas of “basic” research, such as molecular biology,
bioinformatics, imaging, software development, epidemiology and physics. The department of
Radiation Oncology at NKI has structured its translational physics research activities in
multidisciplinary groups to ensure an optimal two-way interaction between clinicians and
physicists and to provide a discussion forum for linking relevant clinical questions to innovative
solutions. The software development for image-guided CT-based position verification integrated
with the linear accelerator represents an opposite example [2] and has been essential for Adaptive
RadioTherapy, the latest piece in art of modern radiotherapy. Building on the practice-changing
studies on the interaction between cisplatin and radiation in tumour and normal cells [3] and new
insights in molecular effects of radiation [4], biology-driven translational research is facilitated at
M. Verheij et al.
87
the departmental level by resident-PhD programs, combined clinical-research appointments and
focused start-up funding.
IMAGE GUIDANCE: A TRANSLATIONAL ART
Fig. 1. Adaptive Radiation Therapy (ART). Imaging, planning and dosimetric information acquired during
treatment, is fed back into the treatment chain for plan adaptation.
For decades, radiation therapy has suffered from poor imaging quality, limiting the detailed
definition of treatment volumes and organs at risk. Moreover, positional and anatomical changes
over the course of therapy could not be detected or corrected. This has forced radiation
oncologists to use generous margins to compensate for these geometric uncertainties, thereby,
however, increasing the risk of inducing normal tissue toxicity. With the introduction of
computed tomography (CT) and in-room imaging techniques, dose planning and delivery have
become more accurate, allowing safe dose escalation. The recent incorporation of functional
imaging modalities like PET and fMRI and the option to integrate motion (4D imaging), provides
a next step towards further optimizing image-guided radiotherapy.
The aforementioned geometrical uncertainties that limit the precision and accuracy of radiation
therapy include setup errors, posture change, organ motion, deformations and treatment response.
Consequently, the actually delivered dose typically deviates from the planned dose. To minimize
the deleterious effects of geometrical uncertainties, adaptive radiation techniques (ART) aim to
characterize the patient’s specific variation through an image feedback loop and adapt the
patients’ treatment plan accordingly (Fig. 1). Adaptive radiation therapy research therefore
includes improving in-room imaging, patient variability characterization, treatment plan
modification and outcome modelling. Additionally, the adaptive radiation therapy framework is
prototyped pre-clinically using a dedicated small animal irradiator.
88
The ART of translation
In Room Imaging
Fig. 2. Clinical applications of CBCT image guidance.
A cone beam computed tomography (CBCT) scanner integrated with a linear accelerator captures
the patients’ anatomy just prior to treatment and is a powerful tool for image-guided
radiotherapy. Respiratory motion, however, induces artefacts in CBCT such as blurring and
streaks that limit the image quality of CBCT in the thorax and upper abdominal region. At the
NKI, we have developed a respiratory correlated procedure for CBCT, which allows verifying
the mean position, trajectory, and shape of a moving tumour (and/or normal organs) just before
radiation treatment is delivered (Fig. 2). Such verification reduces respiration induced
geometrical uncertainties, enabling safe delivery of 4D radiotherapy with small margins. In-room
imaging can also be applied to reduce radiation exposure to normal tissue. Because an increased
risk of cardiovascular-related morbidity and mortality has been observed after breast or thoracic
wall irradiation, we established the feasibility, cardiac dose reduction, and the influence of the
setup error on the delivered dose for fluoroscopy-guided deep inspiration breath hold (DIBH)
irradiation using cone-beam CT for irradiation of left-sided breast cancer patients [5].
SMALL ANIMAL IRRADIATION: µIGRT
Novel developments in radiotherapy depend ultimately on clinical trials to demonstrate their
efficacy. Due to the improvements in image guidance, however, a new therapeutic window is
opening that can only slowly be explored. For instance, the higher precision allows a better
sparing of normal tissues, but to translate that into a higher tumour dose to improve control is
scary because the effect of the remaining hot spots on the normal tissue are unknown. By
translating the human image guidance solution back to small animal research, it now becomes
possible to explore high-dose high-precision treatments in preclinical work in combination with
all sorts of smart drugs. In addition, the volume effect of irradiation normal tissue can be further
M. Verheij et al.
89
explored. To support this work, a commercial small animal irradiation unit has been interfaced
with our XVI image guidance software. The system demonstrates a very high accuracy of 0.1
mm and sharp treatments fields that can be collimated down to 1 mm or less. As a result, it is
now possible to selectively irradiate very small parts of the animals rather than a ‘half mouse’ as
was the state of the art until a few years ago. A start has been made with the first studies, looking
into the effect of partial heart and lung irradiation, and quantifying response timelines after reirradiation of orthotopically implanted breast tumours (Fig. 3).
Fig. 3. Small animal µIGRT system (X-RAD 225 CX, PXI, North Branford, USA).
REFERENCES
[1] www.historad.nl/en
[2] Sonke JJ, Zijp L, Remeijer P, van Herk M. Respiratory correlated cone beam CT. Med Phys
2005; 32(4):1176-1186.
[3] Schaake-Koning C, van den Bogaert W, Dalesio O, Festen J, Hoogenhout J, van Houtte P,
Kirkpatrick A, Koolen M, Maat B, Nijs A, Renaud A, Rodrigus P, Schuster-Uitterhoeve L,
Sculier J-P, van Zandwijk N, Bartelink H. Effects of concomitant cisplatin and radiotherapy
on inoperable non-small-cell lung cancer. N Engl J Med 1992; 326(8):524-530.
[4] Verheij M, Vens C, van Triest B. Novel therapeutics in combination with radiotherapy to
improve cancer treatment: rationale, mechanisms of action and clinical perspective. Drug
Resist Updat 2010; 13(1-2):29-43.
[5] Borst GR, Sonke JJ, den Hollander S, Betgen A, Remeijer P, van Giersbergen A, Russell
NS, Elkhuizen PH, Bartelink H, van Vliet-Vroegindeweij C. Clinical results of imageguided deep inspiration breath hold breast irradiation. Int J Radiat Oncol Biol Phys 2010;
78(5):1345-1351.
90
Fast analysis and fusion of MR spectroscopy
Enabling fast analysis and fusion of
MR spectroscopy imaging
Miguel Nunes1*, Benjamin Rowland2, Matthias Schlachter1, Soléakhéna Ken2, Kresimir
Matkovic1, Anne Laprie2, Katja Bühler1
1
*
2
mnunes@vrvis.at
Abstract: Magnetic Resonance Spectroscopy Imaging (MRSI) contains spectral information per
data point regarding concentrations of metabolites of interest in in-vivo tissue. In radiotherapy,
doctors use MRSI data to make treatment decisions for patients. We present a system that is able
to flexibly and easily depict information contained in MRSI and give the possibility to fuse
related data acquired by several modalities. First experiments with real-world medical data
indicate the usefulness of our system and how, by combining insights from different types of
visualizations, it is possible to achieve better delineations of tumour regions. Opinions of domain
experts denote the positive impact of our system in integrating MRSI in radiotherapy planning
workflow and how the understanding of MRSI data can quickly increase.
Index Terms — MR Spectroscopy, Visual Analytics, Fusion, Radiotherapy.
INTRODUCTION
Magnetic Resonance Spectroscopy Imaging (MRSI) is a non-invasive molecular imaging
technique providing a spectral range of active biomarkers per sample, where each biomarker
indicates a certain concentration of a specific molecule present in a sub-volume of the tissue
being analysed. In medicine, MRSI has been primarily used to study brain and prostate cancers as
well as other metabolic functions of the human body. For instance, in glioblastoma multiforme
(GBM) cases, it is known that cancerous tissue is present when the ratio between choline and NAcetyl-Aspartate (NAA) biomarkers is equal or higher than 2 [1]. This ratio is used as a threshold
to create segmentations so higher doses of radiation are delivered in such areas. These regions are
called Biological Target Volumes (BTV) and are designed using functional imaging [2], for they
represent areas with radioresistence and are thought to be responsible for the relapse of patients.
However, the analysis and visualization of MRSI data, and respective BTV generation, is limited
by the lack of proper tools or by the quality of its acquisition and pre-processing.
Existing tools to pre-process and analyse MRSI data, such as LCModel [3], visualize MRSI data
as metabolite colour maps or ratio maps, which are fixed and do not allow extraction of
additional information. Alternatively, each voxel can be seen as a histogram depicting the
concentrations of metabolites. Another tool that is able to include MRSI data into clinical
workflow is SIVIC [4]. This tool is able to pre-process MRSI raw data into DICOM format of 3D
metabolite maps. However, SIVIC visualization options are limited to the rendering of MRSI
data colour maps together with anatomical images and no statistical inquiries are possible.
Visual analytic tools have the power to boost the understanding of highly complex data, and
M. Nuneset al.
91
alternate visualizations and fusion of more modalities can give better support for medical
decision making. Three studies by Feng et al. on visualization of MRSI data allowed an approach
to this data which combined rendering of anatomical slices and statistical inspection of
metabolite values. Firstly, glyph rendering depicting properties of metabolites allowed visual
value estimation and the identification of relationships between metabolites raw values [5]. Later,
plotting metabolite concentration values into parallel coordinates visualization and using a linear
function brushing helped identifying linear relationships between pairs of variables. The addition
of a slice rendering system to visualize cubes of selected MRSI voxels together with anatomical
images and glyph rendering brought light to correlations between certain metabolites [6]. In a
more recent work by the same authors [7], scatter plots were included to enhance the analysis of
MRSI data. In these three works, only raw values of metabolite concentration were used,
however, these values are not standardized and can only be meaningfully used as ratios.
Furthermore, the creation of complex voxel signatures or generation of new values for better
understanding of tissue characteristics were not addressed.
Currently, no solution enables doctors to quickly access all the power of MRSI and its integration
into radiotherapy treatment planning workflow. The challenges of quickly generating metabolic
ratios, the analysis of these ratios to other functional or anatomical data and methods to easily
select and relate different datasets values have not yet been solved. This work expands current
state of the art by enabling the comparison of any given metabolite dataset, supporting both quick
BTV generation and fast but meaningful analysis of MRSI metabolite data.
A system was implemented to answer the needs to easily and flexibly access MRS data. It is
composed by the visual analytics tool ComVis [9] and a medical image processing framework
MIKT [8]. ComVis provides multiple linked views (histograms, scatter plots, parallel
coordinates, etc.) that allow users to interact with multivariate data by brushing sets of plotted
values. MITK is a framework designed for medical imaging processing that contains numerous
plugins. It was extended with two new plugins: a TCP/IP communication plugin and a rendering
plugin, which renders axial, sagittal and coronal views supporting BTV visualization.
Fig. 1: ComVis-MITK system workflow
The workflow of this system can be seen in Fig. 1. In short, pre-processed data can be registered
and resampled in or outside MITK, and then stored in the MITK Data Manager. Delineations of
regions of interest can also be imported or made in the MITK segmentation plugin. Loaded data
can be instantly visualized in our rendering plugin. Through the communication plugin, we can
convert and send medical data into a ComVis Data Table. Here, data can be plotted in linked
92
views for statistical analysis where it can be brushed and related. Via a Data Fusion plugin, it is
possible to generate ratios from data present in the Data Table that can be instantly analysed.
Brushing values in ComVis generates a message to MITK in form of a binary mask which can be
later visualized as segmentations. ComVis was expanded with new types of brushes and it also
allows smooth brushing for uncertainty visualization by assigning values between 0 and 1 to
voxels present inside the region (Fig. 2a). The result of smooth brushing is a mask with opacity
values equal to its probability (Fig. 2b). Finally, new brushes, including a convex hull generator,
were also added to the scatter plot view.
(a)
(b)
(c)
(d)
Fig. 2: Smooth brushing in ComVis (a) and respective visualization in MITK (red contour with different
intensity related to membership probability) (b). Brushed values of CNR in a histogram (c) and its
visualization in MITK red contour matches perfectly the currently used method (d).
Our solution was evaluated by five experienced medical doctors and physicist from the Institut
Claudius Regaud (ICR). Data of six patients with GBM from a multi-centre phase III clinical trial
called Spectro-Glio was used. For this study, we used contrast-enhanced T1 Gadolinium, FLAIR
and 1H MRSI. It was decided to restrict quantified metabolites to choline, creatine and NAA. In
case of artefacts are present after pre-processing, affected voxels are discarded. Metabolites were
broken down into single valued datasets so MITK would be able to load them. Also, datasets
were already aligned during acquisition and all volumes were resampled in MITK to the
respective T1 Gadolinum dataset. An additional delineation was performed in MITK to restraint
the amount of data to be sent to ComVis.
RESULTS
Five use-cases were developed while working with the expert users. These use-cases indicate
how this system enables an easy and interactive analysis of MRSI. It also demonstrates how the
process of integrating MRSI in radiotherapy treatment planning workflow is achieved.
Analysis and BTV computation
MRSI metabolites datasets are loaded into MITK and sent to ComVis. The choline/NAA ratio
(CNR) is instantly calculated with the use of the Data Fusion plugin. CNR values can then be
plotted in any view. For achieving a BTV equal to the one previously manually done by doctors,
the user only needs to plot the CNR values in a histogram and brush the columns that have the
desired ratio values (Fig. 2c). Then, one segmentation per brush is automatically sent to MITK
and instantly displayed (Fig 2d). The time to perform this operation was around 2 minutes
representing an enormous speedup compared to the current workflow at ICR, which takes around
100 minutes per patient.
Tumour Signature
The combination of different linked views in ComVis granted the power to evaluate different
M. Nuneset al.
93
relationships among MRSI metabolites. By creating new ratios (choline/creatine and
NAA/creatine), users were able to realize that each patient has its own set of metabolite
properties. Brushing the original CE region users were able to visualize which voxels were
selected in the scatter plot of the previously generated ratios (Fig. 3). Users discovered that
voxels with very similar signatures were not included in the original CE region. By making use
of the convex hull generator, these extra voxels were selected and then depicted in MITK
rendering plugin. These extra voxels showed a new region of high tumour activity. This new
information can later be compared to relapsing images to confirm if any association between
relapse and this new selected region exists. This feature was regarded as one of the most
interesting as it allowed to visualize and interpret data both in a statistical and a visual ways.
(a)
(b)
(c)
(d)
Fig. 3: Original CE region selected (a) and respective visualisation in MITK (white contour) (b). Convex
hull around original CE data (c) and respective render of new area, in red, in MITK(d).
Personalized Analysis
In our system, it is possible to analyse how different patients present different tumour signatures
(Fig. 4). The existence of such variety goes in line with what has already been demonstrated in
clinical studies, pointing to the necessity of individualized treatment, as anomalous values for one
patient can be considered normal for another. Our system allows brushing and analysing different
kinds of tissue (necrosis, tumour and healthy tissue) in order to better evaluate what ratios values
are indicators of cancer. Allowing such analysis was considered as the major contribution of this
work.
Fig. 4: scatter plots of 3 patients with different choline/creatine and NAA/creatine ratios distribution values.
Tissue classification
Smooth brushing allows the association of probability values to voxels inside a certain region. In
this way, it is possible to create regions which might be associated with healthy tissue, necrosis or
tumour even if they do not fit in the pre-defined signature. These probabilities can later be
compared to data of relapsing patients. This has been noted by the users as an initial step to create
automatic cancer detection algorithms.
Parallel Coordinates
Studying the evolution of ratios across different exams for each patient can be realized by
94
plotting the ratios in parallel coordinates, in ComVis. Each line representing a 3D voxel can be
selected and compared to the following exams, given that images are aligned and correctly
resampled. Plotting data in this fashion enables the study on the behaviour of ratios evolution in
relapsing patients. This opens the door to large clinical trials to better understand the factors
involved in relapse and, possibly, how to avoid it.
DISCUSSION
Extensions made to MITK and ComVis address the challenges previously noted. This system
manages to load metabolic data into MITK and convert it to ComVis data format so statistical
and numerical analysis is performed. Brushing linked views enable complexity reduction over the
analysis of metabolites and its ratios. It is also allowed to compare any given dataset per voxel,
contributing to a better definition of tissue signatures. Fusion operations to relate datasets was
achieved and yields fast computation of new values. Lastly, the individualization of treatments
was provided by quickly generating meaningful segmentations relating anatomical to functional
data. It was noted by users the ease and flexibility delivered by this system together with the time
it took to study each patient, which was definitively lower than current practise. The visual
analytics enhances the access to MRS data compared to single rendering of the same data in
glyphs or multiplanar reconstruction views.
CONCLUSION
Regarded as the key benefit of our system, we provide medical staff the opportunity to analyse,
relate and visualize MRS data in a novel way. Linking MITK, our plugin renderer and ComVis
answers the current medical requirements, bringing, in one case, the same results obtained with
current commercial software but in a much shorter time. Real-world medical cases of how this
system can introduce flexibility and novelty in future radiology MRSI studies by permitting the
discovery of new MRSI metabolites’ relationships and tumour tissues’ signatures. These cases
showed that different patients have different ratio values, pointing in the direction of personalized
analysis in GBM cases. Regarding future work, it is planned to extend the Data Fusion plugin.
The introduction of other modalities such as fMRI or PET can bring more information about
functional areas of the brain and tissue which could impose adaptations in the planning treatment
and improve the quality of the signatures of voxels.
ACKNOWLEDGMENT
REFERENCES
[1] Laprie A, Catalaa I, Cassol E, McKnight TR, Berchery D, Marre D, Bachaud J-M, Berry I,
Moyal EC-J. Proton magnetic resonance spectroscopic imaging in newly diagnosed
glioblastoma: Predictive value for the site of postradiotherapy relapse in a prospective
longitudinal study. Int J Radiat Oncol Biol Phys 2008; 70(3):773-781.
[2] Ken S, Vieillevigne L, Franceries X, Simon L, Supper C., Lotterie J-A, Filleron T, Lubrano
V, Berry I, Cassol E, Delannes M, Celsis P, Cohen-Jonathan EM, Laprie A. Integration
M. Nuneset al.
95
method of 3d mr spectroscopy into treatment planning system for glioblastoma imrt dose
painting with integrated simultaneous boost. Radiat Oncol, 2013; 8(1).
[3] Provencher SW. Estimation of metabolite concentrations from localized in vivo proton nmr
spectra. Magnet Reson Med 1993; 30(6): 672–679.
[4] Crane JC, Olson MP, Nelson SJ: Sivic: Open-source, standards-based software for dicom
mr spectroscopy workflows. J Biomed Imag 2013; 12.
[5] Feng D, Lee Y, Kwock L, Taylor II RM. Evaluation of glyph-based multivariate scalar
volume visualization techniques. Proc 6th Symp Appl Perception Graph Vis 2009; 61-68.
[6] Feng D, Kwock L, Lee Y, Taylor II RM. Linked exploratory visualizations for uncertain mr
spectroscopy data. SPIE Proceedings 2010; 7530: 753004
[7] Feng D, Kwock L, Lee Y, Taylor II RM. Matching visual saliency to confidence in plots of
uncertain data. IEEE Trans Vis Comput Graph 2010; 16(6): 980–989.
[8] Wolf I, Vetter M, Wegner I, Nolden M, Bottger T, Hastenteufel M, Meinzer HP. The
medical imaging interaction toolkit (MITK): a toolkit facilitating the creation of interactive
software by extending VTK and ITK. Med Imag 2004; 16-27.
[9] Matkovic K, Freiler W, Gracanin D, Hauser H. Comvis: A coordinated multiple views
system for prototyping new visualization technology. Inf Vis 2008; 215–220.
Miguel Nunes was born in Braga, Portugal, in 1985. He received his MSc degree
in Informatics Engineering from the University of Minho, Braga, Portugal in
2009.
After graduation, he worked as a Researcher at CCG Guimarães, and later as a
Consultant for Deloitte and Logica in Lisbon. He is currently employed at VrVis
Zentrum für Virtual Reality und Visualisierung Forschungs-GmBH in, Vienna,
Austria, as an Early Stage Researcher.
96
4D PET/CT visualization in radiotherapy planning
4D PET/CT visualization in
radiotherapy planning
Matthias Schlachter1*, Tobias Fechter2, Ursula Nestle2 and Katja Bühler1
1
*
2
schlachter@vrvis.at
Abstract: In radiation treatment (RT) planning medical doctors need to consider a variety of
information sources for biological target volume delineation. The validation and inspection of the
defined target volumes and the resulting RT plan is a complex task, especially in the presence of
moving target areas as it is the case for tumours of the chest and the upper abdomen. A 4DPET/CT visualization system may become a helpful tool for validating RT plans. We define
major goals such a visualization system should fulfil to provide medical doctors the necessary
visual information to validate tumour delineation, and review the dose distribution of a RT plan.
We present an implementation of such a system, and present results of how such a system can be
used to validate a plan for a lung cancer patient.
Index Terms — Medical visualization, volume rendering, 4D PET-CT, RT Dose.
INTRODUCTION
For lung cancer, the most prominent functional imaging system in use is PET along with the CT
as the anatomical imaging modality. PET imaging with the 18f-fludeoxyglucose (18FDG) tracer
is an accurate diagnostic method for non-small lung cancer, and used for the delineation of the
gross tumour volume (GTV) [1]. Respiration causes target areas to move which cannot be
captured by the planning CT (only a static image) and 4D PET/CT can be used to image patients
under free breathing conditions. Tumour delineation can be done on each breathing phase
captured by 4D PET/CT. The union of the contouring can be used to define the internal target
volume (ITV) [1] representing the lesion over the whole breathing cycle. The inspection of target
volumes is usually done slice-wise and often combined with a video of maximum intensity
projection of the 4D data sets. However, this makes it hard to capture the real 3D motion of
target areas, and might give false impressions about tumour coverage by the defined target
volumes. Therefore, a 4D-PET/CT visualization system can assist RT planning and validating
treatment plans, especially in the presence of moving structures like tumours of the chest and the
upper abdomen.
In this paper we present a 3D multi-modal visualization framework which focuses on the
validation and inspection of the RT plans in the presence of moving target areas. In order for a
3D visualization to assist physicians to evaluate and validate a RT plan, we define the following
major goals (use cases) which need to be addressed:
1. Support for 4D (3D+t) PET and CT data sets and fusion of these image modalities: PET and
CT signals should be fused in a 3D rendering. Support for changing time bins should be
provided for giving access to the whole breathing cycle of the patient.
M Schlachter et al.
97
2. Visualization of structure sets: Defined structures such as GTV, ITV or organs at risk
(OAR) should be included and combined with the 3D visualization of PET and CT for
evaluating the spatial configuration and ensuring optimal coverage of moving target areas.
3. Visualization of dose information: Visualizing dose information as iso-dose surfaces
together with defined structures like the GTV, ITV or OARs should allow evaluating the
spatial configuration and coverage of moving target areas complementary to dose volume
histograms.
4. Clipping and/or masking parts of the volume: Hiding parts of the volume which might not
be relevant in the current situation (e.g. remove CT signal inside a structure set and show
only the PET signal) should be supported.
5. Interactivity and pre-processing: There should be no pre-processing involved such as resampling data sets to the same size or offline volume fusion into a new data set. The
parameters (data sets, structure sets, and visual appearance) of the visualization should be
modifiable on-the-fly.
The proposed visualization framework performs fusion of 4D PET/CT images, combined with
defined target volumes and segmentation information of OARs. Furthermore, the visualization of
dose volumes provides the necessary information for a visual review and validation of a
computed treatment plan.
Data
The data sets consist of four types: PET, CT, Segmentation and Dose Volumes. CT data sets are
either the planning CT or the 4D CT. The planning CT has a voxel size of 1.17mm x 1.17mm x
3mm and pixel dimensions of 512 x 512 x 107. Structure sets and target volumes are binary
volumes representing OAR segmentation (or margins around an OAR), as well as delineated
target volumes (GTV, ITV). They are all in reference to the planning CT, and therefore have the
same voxel size and data dimensions as the planning CT. The 4D CT consists of 10 time bins
with a voxel size of 1.17mm x 1.17mm x 2mm and pixel dimensions of 512 x 512 x 77. The 4D
FDG-PET data set has a voxel size of 4mm x 4mm x 4mm and pixel dimensions of 144 x 144 x
45 consisting of 10 time bins. The dose volume holds the relevant dose information in reference
to the planning CT and was exported from the planning system software. It has a voxel size of
2.5mm x 2.5mm x 3mm with pixel dimensions of 212 x 119 x 107.
Visualization System
Our visualization framework is integrated in the Medical Imaging Interaction Toolkit (MITK)
[3]. MITK provides a platform with a plug-in system and combines functionality of VTK [4] and
ITK [5]. It provides DICOM data import, visualization and interaction. Figure 1 shows the GUI
of the MITK platform. Part 1 of figure 1 shows already available functionality of MITK. This
includes a data set manager, image navigator (3D+time navigation) and 2D slice views. The
integration of our multi-modal rendering implementation is realized via a MITK plug-in which is
the connection between MITK and our rendering core (see fig. 2). Figure 1 part 3 shows the GUI
of our MITK plug-in. It communicates with our rendering core via a VTK interface (see fig. 2),
and is responsible for setting and changing parameters such as data sets, their visual appearance
and parameters for clipping and fusion. Finally part 2 integrates the result of our 3D rendering
and replaces the standard 3D view of MITK.
98
Fig. 1: Screenshot of the MITK platform with the integration of 3D multi-modal rendering.
The 3D multi-modal rendering framework is mostly implemented in CUDA [2], and consists of
three main parts (see fig. 2): the data store module, the rendering module and an interface to
VTK. The data store is responsible for storing volumes in GPU memory and makes them
available to our rendering module. Data sets are organized in a unified coordinate system which
takes into account spatial transformations between data sets. The core of the rendering module is
based on ray-casting with front-to-back blending [6], and responsible for PET-CT fusion, binary
volume rendering and dose volume visualization.
Fig. 2: Overview of MITK integration and rendering framework.
The functionality of the 3D multi-modal rendering consists of three main parts: fusion of PET
and CT, rendering of binary volumes and iso-dose rendering of dose volumes. The fusion of
PET-CT is done by taking a linear combination of colour and opacity of both volumes and can be
combined with information from binary volumes to define regions of interest where only
information from one modality should be visualized (e.g. only PET inside the ITV). Binary
volumes which represent target volumes and segmentation of OARs are visualized by rendering
their outline as a surface. Color and opacity values can be assigned to each binary volume
individually. Dose information is rendered as iso-dose surfaces. The respective dose parameters
can be specified as a list of values in grey units. The combination of all three parts produces the
final result of the rendering.
RESULTS
Our 3D multi-modal rendering framework combines information of PET, CT, segmentation and
dose information into one final result. This, together with the integration into the MITK platform
implements our previously defined goals 1-5 (see fig. 3 and 4). A qualitative result of PET-CT
fusion and iso-dose rendering is shown in fig 3 (left image). The iso-dose is rendered using four
M Schlachter et al.
99
defined dose values and combined with PET-CT fusion. Clipping is used to hide parts of the
thorax and iso-dose surfaces which would occlude the PET signal of the target area. The right
image of fig. 3 shows the same patient, however, the PET signal was disabled and binary volume
rendering of the ITV and a safety margin around the trachea was added. The parameters for
clipping and dose are the same as for the left image.
Fig. 3: Result of 3D renderings. (left) PET-CT fusion and iso-dose rendering. (right) CT combined with
binary volume and iso-dose rendering. Clipping was applied to hide parts of the thorax.
Figure 4 shows a combination of fusion and binary volume rendering of different breathing
phases of the PET. The structure set represents the ITV. Parts of the thorax are hidden by
applying clipping. Masking is applied inside the ITV to show only the PET signal. The left and
right part show the PET signal of different phases of the patients’ breathing cycle. The phase can
be changed interactively in the GUI and allows seeing the PET signal “move” inside the ITV.
Fig. 4: CT combined with different breathing phases of PET (left, right). The structure set represents the
ITV. Inside the ITV only the PET signal is visualized. Clipping was applied to hide parts of the thorax.
Making all this available to medical doctors gives them a set of tools, which they can use to
interactively explore and validate a RT plan. The proposed functionality can be applied to a
multitude of scenarios including: checking the spatial configuration of target volumes defined on
volumes of different modalities (GTV of CT and ITV of 4D PET), checking the coverage of the
ITV with the 4D PET signal (respiratory motion of the tumour area) or the spatial configuration
of dose areas inside OARs.
CONCLUSION
A visualization system which fulfils the goals defined in the introduction may become a helpful
tool for validating tumour delineations and RT plans. We implemented such a system by
integrating a custom 3D multi-modal visualization framework into the MITK platform.
100
ACKNOWLEDGMENT
REFERENCES
[1] Nestle U, Weber W, Hentschel M, Grosu AL. Biological imaging in radiation therapy: role
of positron emission tomography. Phys Med Biol 2009; 54(1):R1.
[2] NVIDIA Corporation. NVIDIA CUDA C Programming Guide. 2011.
[3] Wolf I, Vetter M, Wegner I, Nolden M, Bottger T, Hastenteufel M, Schobinger M, Kunert
T, Meinzer HP. The medical imaging interaction toolkit (MITK): a toolkit facilitating the
creation of interactive software by extending VTK and ITK. Med Imag 2004; 16-27.
[4] Schroeder W, Martin K, Lorensen N. Visualization toolkit: an object-oriented approach to
3D graphics. 4th Edition, Kitware, 2006.
[5] Ibanez L, Schroeder W, Ng L, Cates J. The ITK software guide. 2005.
[6] Kruger J, Westermann R. Acceleration techniques for GPU-based volume rendering. Proc
14th IEEE Visualization 2003 (VIS'03); 38.
Matthias Schlachter received the Dipl-Inf. degree in computer science from the
University of Freiburg, Germany, in 2009.
He is a Marie Curie early-stage researcher for the SUMMER project and is
currently working at VRVis, Zentrum für Virtual Reality und Visualisierung
Forschungs-GmBH, in Vienna, Austria. His current research interests are in
medical visualization with a focus on image fusion and uncertainty visualisation
of multi-modal data.

Available here

Transcription

Similar documents

DynaCAD Breast Brochure

Segmentation,Targeting and Positioning

Software developments in CT scanning

B2G Segmentation Strategies

Database Marketing Solutions - Anchor Computer

segmentation of cytological images using color and

Foreword - touchONCOLOGY

VLA Presentation 23 Oct 2015 - Virginia Library Association

Milestone: Extracting Map Features from Satellite Images

Angular/Depth Dependent Analysis