Diaphragm border detection in coronary X-ray - CVC

Computerized Medical Imaging and Graphics 38 (2014) 296–305
Contents lists available at ScienceDirect
Computerized Medical Imaging and Graphics
journal homepage: www.elsevier.com/locate/compmedimag
Diaphragm border detection in coronary X-ray angiographies:
New method and applications
Simeon Petkov a,b,∗ , Xavier Carrillo c , Petia Radeva b , Carlo Gatta a
a
b
c
Centre de Visió per Computador, Bellaterra, Spain
Universitat de Barcelona, Barcelona, Spain
University Hospital “Germans Trias i Pujol”, Badalona, Spain
a r t i c l e
i n f o
Article history:
Received 25 August 2013
Received in revised form
12 December 2013
Accepted 20 January 2014
Keywords:
X-ray
Angiography
Diaphragm
Myocardial Blush Grade
DSA
Vesselness
a b s t r a c t
X-ray angiography is widely used in cardiac disease diagnosis during or prior to intravascular interventions. The diaphragm motion and the heart beating induce gray-level changes, which are one of the
main obstacles in quantitative analysis of myocardial perfusion. In this paper we focus on detecting the
diaphragm border in both single images or whole X-ray angiography sequences. We show that the proposed method outperforms state of the art approaches. We extend a previous publicly available data set,
adding new ground truth data. We also compose another set of more challenging images, thus having
two separate data sets of increasing difficulty. Finally, we show three applications of our method: (1) a
strategy to reduce false positives in vessel enhanced images; (2) a digital diaphragm removal algorithm;
(3) an improvement in Myocardial Blush Grade semi-automatic estimation.
© 2014 Elsevier Ltd. All rights reserved.
1. Introduction
1.1. Motivation
In diseases related to heart malfunctioning, it is important to
ensure proper blood supply to the heart and estimate the healthiness of the myocardial tissue. For this purpose, medical doctors
insert a catheter into the affected coronary artery and inject a
radio-opaque liquid through it. The liquid flows through the arteries and perfuses the myocardium. This process is recorded as an
angiography video sequence using X-ray technology. Fig. 1 shows
an exemplar angiographic image. The arteries are clearly visible
as they are filled with the contrast liquid. The diaphragm is also
visible as a darkened area due to the fact that it is a thick muscle. Usually, other structures like bones or gas are visible as bright
or dark areas with varying gray level intensity and strength of
the contours that delineate them. When contrast liquid perfuses
myocardium, it is seen as a gray staining, which is brighter than
arteries and diaphragm. The motion of the diaphragm follows the
patient’s breathing pattern; since the diaphragm often overlaps
with the myocardium, it complicates both visual and automatic
inspection of the myocardial perfusion.
Digital removal of the diaphragm could help medical doctors
when visually estimating myocardium healthiness. Additionally,
several automatic or semi-automatic methods for myocardial perfusion estimation have been proposed in the last seven years
and all of them are negatively affected by the diaphragm motion.
The method in [1] use pre-processing to compensate for the
diaphragm movement and the ones in [2–8] explicitly require that
the diaphragm is not present or moving while image acquisition is
performed. It has to be noted that not all patients can hold breath
for the time required to record a complete sequence for myocardial
perfusion analysis, i.e. at least 7 s. In [9], which is based on Digital
Subtraction Angiography (DSA), authors have the same requirement that patient breathing is avoided for the time of recording.
Moreover, in the context of vessels segmentation in X-ray images,
the diaphragm border is often misclassified as a vessel [10–13].
∗ Corresponding author at: Centre de Visió per Computador, Edifici O, Campus
UAB, 08193 Bellaterra, Barcelona, Spain. Tel.: +34 693980478.
E-mail addresses: spetkov@cvc.uab.es, simeon.petkov.bg@gmail.com (S. Petkov).
0895-6111/$ – see front matter © 2014 Elsevier Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.compmedimag.2014.01.003
1.2. State of the art
To the best of our knowledge, two methods for automatic
diaphragm detection in cardiac X-ray angiographies have been proposed so far in [14,15]. In [14], authors model the diaphragm as
an arc of a circle. A pre-processing step removes narrow contrast
objects like arteries and the catheter by means of morphological
closing operator. Then, a Canny edge detector defines an edgeness
S. Petkov et al. / Computerized Medical Imaging and Graphics 38 (2014) 296–305
297
contrast liquid injection, it requires that a small number of images
are acquired at different cardiac cycles and breathing phases to
serve as static masks for the background estimation.
1.3. Contribution
Fig. 1. An exemplar frame from a coronary angiography sequence.
map for the pre-processed image. For any arc in the image plane,
authors assign a score – the sum of the edgeness values of the
pixels composing the arc. The circle that maximizes that score is
the optimal prediction. The initial result is refined with active contours (a.k.a. snakes) if a confidence measure indicates that it is not
good enough. In [15], authors adopt a similar approach as in [14]
to remove arteries and highlight edges. Then, a set of paths is constructed by tracking edges from one frame to the next. K-means
clustering divides the paths in three clusters. The method keeps
only the paths that follow the breathing pattern by selecting the
cluster of highest quality paths as defined in [15], Section 2.4. The
geometric model for the diaphragm border in [15] is a parabola;
the final step of the method is to find the optimal parabola for each
frame by removing outlying paths.
The method in [15] has one main drawback – it cannot make a
prediction for a single frame before processing the whole sequence,
while the method in [14] operates individually on each frame. The
models used in the two methods show that none of them has the
flexibility to represent the diversity of diaphragm borders. The circle is too simple and extensive search over circular arcs often leads
to non-plausible detections. The parabola is not flexible enough to
model diaphragm border curves, since it is symmetric along a single
axis and this assumption is not verified for all diaphragm borders.
Although the following two papers do not present an automatic
diaphragm detection method, they are worth mentioning. In [16],
a method to track anatomical curves in X-ray sequences is presented. The method requires manual initialization of the curve to be
tracked and clearly outperforms a classical optical flow technique.
In [17], authors present a method that estimates the “background”
layer of an X-ray angiography sequence. The goal is to perform
a form of digital subtraction in angiographies, which highlights
coronary arteries. The method is based on a Bayesian framework,
which combines tracking and modeling of the background. Prior to
Our paper contributes with a new diaphragm border detection
method that outperforms previous ones (Section 4). The method
we propose has two main novelties compared to previous methods. Firstly, we make use of a priori knowledge for the shape of the
diaphragm border, which reduces the possibility of non-plausible
predictions caused by edges that do not belong to the diaphragm
border. Secondly, our method is based on increasing flexible polynomial model to improve the detection accuracy.
We also provide a quantitative evaluation of different models
for the diaphragm border (Section 2.1). The methodology for digital diaphragm removal from X-ray angiographies (Section 5.2) is
another contribution. In addition, we extended the publicly available data set adding new images that increase the diversity of cases.
2. Diaphragm border detection
Our method is composed by a training phase and three main
steps, as depicted in Fig. 2. The trained model introduces a probability criterion for the diaphragm border shape. In the first step of
the method, a morphological pre-processing removes the arteries
and the catheter. The second step computes an edgeness map by
means of first order vertical derivatives within a Gaussian pyramid
[18]. The last step estimates the optimal diaphragm border, maximizing two criteria – (1) the diaphragm border shape probability
and (2) the normalized edgeness value collected by the pixels that
belong to the border. The proposed method has three parameters:
– The size of the structuring disk for the morphological operator –
RM .
– The standard deviation of the normal probability distribution
expressing the diaphragm border positional uncertainty – G .
– The regularization factor for the diaphragm shape probability –
.
Section 2.1 explains the rational for using a polynomial curve
to model the diaphragm border and following subsections explain
in detail each of the method modules. Section 2.6 describes the
validation we performed for tuning the three method parameters.
2.1. Evaluating different models for the diaphragm border
To compare different diaphragm border models, we fitted various choices to the annotated ground truth curves of data sets A
and B (described in Section 3). We evaluated quantitatively a circular model R2 = (x − xc )2 + (y − yc )2 and polynomial curves of the
Fig. 2. Block scheme of our method.
298
S. Petkov et al. / Computerized Medical Imaging and Graphics 38 (2014) 296–305
Fig. 3. Morphological removal of dark and thin structures (right) on an X-ray image (left). The radius of the structuring disk for the operator is 27 pixels.
N
form y =
a xi , where N ∈ {2, 3, 4, 5, 6, 7}. Polynomials of 0th
i=0 i
and 1st order are bad choices because the diaphragm border is
physiologically not plausible to appear as a straight line.
Table 1 contains the quantitative results for each model. We
used two error measures: the Mean Minimal Distance (dMMD )
and the Hausdorff Distance (dHD ). The Mean Minimal Distance
gives information about the precision of the prediction while the
Hausdorff Distance is an indicator of the robustness. Rigorous
description of these measures is provided in Section 4. The numerical comparison indicates that the parabola is a better choice than
the circle. Furthermore, results consistently improve as the polynomial degree increases. Considering this we use polynomial model
for the diaphragm border.
2.3. Pre-processing: digital artery removal
When the arteries are filled with contrast liquid, they usually
produce stronger edges than the diaphragm border. These edges
could mislead our diaphragm detection, since it is based on edge
analysis. We use a morphological closing operator to remove dark
and thin structures. The structuring element of the operator is a
disk and its radius is one of the parameters of our method. The
effect of applying the morphological operator is shown in Fig. 3;
it removes arteries and preserves edges that resemble diaphragm
borders. The resulting circular artifacts have lower contrast than
the arteries which is enough to reduce the possibility of suboptimal
diaphragm edge analysis.
2.4. Edgeness computation
2.2. Diaphragm border shape training
The variation in diaphragm shape is little with respect to different patients and projections. Considering this, we estimate the
probability distribution of the diaphragm border shape over the
model parameters, using Gaussian Mixture Models. As explained
in Section 2.5 we start estimating the diaphragm border using the
parabolic model y = a0 + a1 x + a2 x2 . Parameter a2 represents the
broadness/narrowness of the parabola and whether it is concave
or convex. The a1 parameter affects the horizontal shift and the
scaling of the parabola and a0 specifies its vertical translation. We
are interested in building a probabilistic model for the shape of the
diaphragm regardless its vertical position, so we estimate the joint
probability p(a1 , a2 |D) using a two-dimensional Gaussian Mixture
Model and the ground truth of a certain training data D. Estimating the joint probability density for all the polynomial parameters
except a0 is still valid for N > 2, as a0 specifies the vertical displacement of the curve, while the other parameters specify its shape.
To highlight consistent edges we compute an edgeness map on
the morphologically pre-processed image. First-order scale-space
vertical derivative is applied as
D (x, y) = I(x, y) ×
dHD
Avg ± Std
Circle
2nd degree polynomial
3rd degree polynomial
4th degree polynomial
5th degree polynomial
6th degree polynomial
7th degree polynomial
47.28
15.77
9.48
5.76
4.47
3.64
3.12
±
±
±
±
±
±
±
72.80
10.45
6.74
4.14
2.97
2.23
1.74
dMMD
Avg ± Std
8.91
3.25
1.97
1.16
0.93
0.73
0.67
±
±
±
±
±
±
±
19.68
1.76
1.16
0.66
0.50
0.34
0.27
(1)
where I(x, y) ∈ R is the input image, × denotes the convolution
operator, G(y; 0, 2 ) is a zero-mean Gaussian and is the scale
parameter [19]. The set of scales for the derived Gaussians ˚ =
{1, 2, 4, 8, 16} is defined in octaves of pixels so that it covers all
possible sizes of edges that could be produced by the diaphragm
border. To make the results for different scales comparable, we
apply the Lindeberg normalization [19], i.e. multiplication by the
scale .
In all standard projections for recording X-ray angiographies,
the diaphragm is situated below the diaphragm border, so we modify the edge computation to nullify edges that correspond to lighter
pixels below and darker pixels above:
Table 1
Quantitative results when fitting different models to the ground truth. The error
measures used for performance evaluation are defined in Section 4.
∂G(y; 0, 2 )
,
∂y
D̃ (x, y) =
D (x, y)
if
D (x, y) > 0
0
if
D (x, y) ≤ 0.
(2)
We combine the maps for all scales D̃ (x, y) into a single map by
averaging the values over the number of scales:
Ẽ(x, y) =
1 D̃ (x, y).
|˚|
(3)
∈˚
Finally, to ensure the edgeness E(x, y) ∈ [01], we apply the following non-linear transformation:
E(x, y) = 1 − exp(−Ẽ(x, y)).
(4)
S. Petkov et al. / Computerized Medical Imaging and Graphics 38 (2014) 296–305
299
Fig. 4. A morphologically pre-processed frame (left) and its Edgeness map (right).
Fig. 4 shows an example of an edgeness map. The pixels with
high edgeness values consistently delineate the two edges that look
like a diaphragm border. The rest of the pixels with positive edgeness are scattered around the image and produce weaker signal
than the edges resembling a diaphragm border.
2.5. Diaphragm border estimation
We estimate the values {â0 , . . ., âN } that define the optimal
polynomial curve of Nth degree, which delineates the diaphragm
border. Let X be the width and Y the height of a frame. We minimize
the following cost function:
Q (a0 , . . ., aN ) = −
1
|R|
N
᏾
(xr ,yr ) ∈
E(xr , yr ) − p(a1 , . . ., aN |D),
(5)
a-priori shape
data
where R = {(xr , i=0 ai xri )} for xr ∈ {1, . . ., X} is the set of pixels
constructing a polynomial curve. To exclude points falling outside
the image boundaries, if yr ∈
/ [1Y ] for some xr , the pair (xr , yr ) is
removed from R. The data term represents the criterion for maximizing the average edgeness value for the pixels that belong to
the curve. The a priori shape term makes use of the a priori knowledge about the diaphragm shape (Section 2.2); it maximizes the
joint probability for the polynomial parameters with respect to the
estimated joint probability density on a certain training data D.
The regularization factor for the diaphragm shape probability is a
parameter of our method. As it is shown in Fig. 4, the edgeness map
E is not precise in highlighting the diaphragm curve. We express the
positional uncertainty using a normal probability distribution. This
can be quickly implemented by applying a Gaussian filtering on the
Edgeness map. The standard deviation of the Gaussian is another
parameter of our method.
As stated in Section 2.1, diaphragm border is not prone to appear
as a straight line. According to the results from Table 1 the parabola
(N = 2) is the polynomial with the lowest order that achieves good
balance between precision and robustness. We start estimating the
diaphragm border with N = 2. To avoid falling into local minima
when minimizing the cost function (5), we construct a set of M
hypotheses drawn by the estimated probability densities for a1 and
a2 . The bootstrapping for a0 covers the range of vertical positions so
that the parabolic curves fall into the frame spatial support. For initialization we select the triplet {a0 ,a1 ,a2 } that maximizes the data
term of the cost function (5). The minimization of the cost function
(5), by means of standard gradient descent algorithm, estimates the
optimal values â0 , â1 and â2 . To improve the precision of the detection, we iteratively increase the polynomial order N by one degree,
initializing the parameter values to the estimated in the previous
Table 2
The mean and the standard deviation of the estimated values for the method
parameters.
Parameters
Mean ± Std
RM
G
27.46 ± 7.39
8.42 ± 1.85
0.013 ± 0.017
iteration. The new parameter aN is initialized to zero. By minimizing
the cost function (5) again, we find the polynomial curve of order
N that models the diaphragm border. This iterative process can be
repeated up to any desired order.
2.6. Parameters tuning
As Section 2 specifies, our method has three parameters that
require tuning. Prior to testing we estimate their optimal values
with an extensive search that optimizes method performance on
validation data. Section 3 describes the data sets and Section 4
explains how they are being distributed between training, validation and testing. The mean and the standard deviation of the
estimated values for the method parameters are listed in Table 2.
The tuning process considered also zero as an optional value
for each parameter. Setting a parameter to zero is equivalent to
omit the corresponding step. Zero was never estimated as optimal
value for none of the parameters. This observation shows the need
of morphological pre-processing, positional uncertainty modeling
and a priori knowledge in our method.
3. Material
We used the publicly available set proposed in [15] (a.k.a. data
set A). To increase its diversity, we extended it with additional
77 frames (from 39 patients) for a total of 93 frames (from 50
patients).1 In some cases it is hard to delineate the diaphragm
even for specialists. The most common difficulty is when the border is not clearly visible or if its starting and/or ending points are
vague. In other cases the diaphragm border is well seen but it has
an unexpected shape and position which complicates the decision
if it delineates a diaphragm or another visible structure. Considering these observations and to provide interesting insights during
testing, we composed another set of more difficult frames – data
set B.2 It contains 47 frames (from 26 patients).
1
Data set A is available at https://sites.google.com/site/diaphragmdetection/
engineering-docs.
2
Data set B is available at https://sites.google.com/site/diaphragmdetection/
engineering-docs/validation-set-b.
300
S. Petkov et al. / Computerized Medical Imaging and Graphics 38 (2014) 296–305
Table 3
Quantitative results on data set A. All measures are in pixels.
dHD
dMMD
Avg ± Std
Min
19.78 ± 28.05
19.78 ± 28.05
O1 vs O2
O2 vs O1
92.53 ± 107.41
89.43 ± 97.49
117.37 ± 107.15
[14]
[14] (with snakes)
[15]
Our (N = 2)
Our (N = 3)
Our (N = 4)
Our (N = 5)
Our (N = 6)
Our (N = 7)
46.82
46.04
45.68
45.66
45.66
45.65
±
±
±
±
±
±
58.04
58.09
58.14
58.15
58.15
58.15
Max
Avg ± Std
Min
Max
2.83
2.83
183.17
183.17
2.31 ± 3.66
1.71 ± 1.59
0.74
0.71
33.08
15.65
17.03
15.65
5.39
482.15
468.00
378.08
37.62 ± 48.84
28.31 ± 43.04
41.65 ± 53.75
5.10
3.84
1.28
236.24
237.66
229.01
6.23
5.31
5.12
5.24
5.22
5.23
498.74
498.73
498.77
498.78
498.78
498.78
10.33
10.20
10.16
10.16
10.16
10.16
±
±
±
±
±
±
1.98
1.93
2.04
2.04
2.04
2.04
296.38
296.38
296.39
296.40
296.40
296.40
30.27
30.28
30.29
30.29
30.29
30.30
Table 4
Quantitative results on data set B. All measures are in pixels.
dHD
dMMD
Avg ± Std
Min
37.03 ± 58.05
37.03 ± 58.05
O1 vs O2
O2 vs O1
Max
Avg ± Std
Min
Max
3.16
3.16
378.19
378.19
2.44 ± 2.52
4.21 ± 11.96
0.72
0.74
14.36
83.59
[14]
[14] (with snakes)
[15]
153.56 ± 143.61
136.19 ± 138.80
172.76 ± 79.63
21.95
17.03
20.12
520.03
518.99
425.88
55.86 ± 67.30
51.88 ± 76.83
57.75 ± 53.35
10.71
9.04
3.38
288.44
350.07
177.43
Our (N = 2)
Our (N = 3)
Our (N = 4)
Our (N = 5)
Our (N = 6)
Our (N = 7)
114.27
113.10
112.74
112.71
112.69
112.69
±
±
±
±
±
±
16.24
9.70
9.70
9.70
9.70
9.70
532.79
532.78
532.78
532.78
532.78
532.78
36.46
36.00
35.87
35.86
35.86
35.86
±
±
±
±
±
±
4.03
2.49
2.30
2.28
2.27
2.27
352.11
352.64
352.69
352.73
352.73
352.73
101.51
102.05
102.23
102.27
102.27
102.28
To compute the inter-observer variability, the ground truth for
all additional frames was annotated independently by two experts.
Angiography sequences have been acquired using is a Philips Allura
Xper FD20. The average pixel resolution has been estimated to be
0.34 mm × 0.34 mm. The C-arm’s primary and secondary angles in
data set A vary from −42◦ to 97◦ and from −22◦ to 31◦ respectively.
For data set B the primary and secondary angles vary from −43◦ to
107◦ and from −18◦ to 28◦ .
4. Evaluation
We used the evaluation protocol proposed in [15], which is
based on curve-to-curve distance errors. It is composed of two error
measures: the Mean Minimal Distance (dMMD ) and the Hausdorff
Distance (dHD ). The definition of dMMD measure is:
dMMD (P, GT ) =
1 mind(i, j),
|P|
j ∈ GT
(6)
i∈P
where P is the set of all points from the predicted diaphragm curve,
GT is the set of all points from the ground truth curve and d(i,j) is
74.16
74.36
74.43
74.43
74.44
74.44
the Euclidean distance between two points i and j. The definition
of dHD is:
dHD (P, GT ) = max{sup inf d(i, j), sup inf d(i, j)},
i ∈ P j ∈ GT
j ∈ GT i ∈ P
(7)
which is the Euclidean distance between the two most remote
points from each of the sets. It has to be noted that while dHD
is symmetric, dMMD is not. The rational for combining two error
measures is based on the conclusion from [15] that each of the
measures represents different performance aspects. The measure
dHD is very sensitive to predicted points laying far from the ground
truth points. However, it gives little information about the overall precision of the prediction. This makes it useful for measuring
robustness. The other measure, dMMD , is an indicator of the average
precision of the prediction.
We followed a cross-validation technique based on LOPO
(Leave-One Patient Out). Given a data set, we take the data for
one patient out and split the rest of the data in two parts; half
of the images are used to train the probabilistic model, and the
other half for validating different values for the method param-
Fig. 5. The performance of our method on data set B while changing the method parameters around their optimal values. The first row of plots shows the average dMMD and
the second row the average dHD .
S. Petkov et al. / Computerized Medical Imaging and Graphics 38 (2014) 296–305
eters. Then we switch the training and validation subsets and
repeat the last step. Finally we test the data for each patient, by
using all remaining data for training and assigning to the method
parameters the average values estimated during the validation. The
process is repeated so that each patient data is used only once for
testing.
Tables 3 and 4 show the quantitative results respectively on data
set A and B.
The first two rows in each table contain the inter-observer variability. Since dMMD is not symmetric, the results of the first observer
are evaluated against the results of the second observer and vice
versa. The method in [14] in general performs better than the
method in [15], despite the fact that it achieves worse minimum
and maximum errors. The performance of the method in [15] on
data set A decreases compared to the evaluation performed in
[15]; this indicates that the method does not generalize well on
new images. Even with the lowest degree polynomial model, our
method achieves better performance than both [14,15]. If we iteratively increase the polynomial degree from the 2nd to the 4rd the
quantitative results improve on both data sets; for degrees higher
than 4th the overall performance of our method stabilizes. Fig. 5
shows the advantage of incrementally increasing the flexibility of
a polynomial model over directly applying a polynomial model of
higher than the second degree. The inter-observer variability as
well as prediction errors for data set B are higher, since data set B
has been designed to have more challenging cases, highlighting the
weak points of our and other methods.
To show how parameter tuning affects the detection results,
we measured the method performance on data set B with different initializations of the three input parameters. Our choices
for parameter values are based on their estimated values during
301
Fig. 6. Plot of dMMD on the accumulated data from data sets A and B for two testing
scenarios. Solid line is the dMMD if the degree of the polynomial model is iteratively
increased and the dashed line is the dMMD if the method estimates the diaphragm
border using directly higher than the second polynomial degree.
method validation (Table 2). The plots in Fig. 5 show that in general,
the detection is stable if parameters change around their optimal
values. The sensitivity of the prediction to changes in the regularization factor is the lowest; this is expected, since the bootstrapping
makes use of the a priori shape knowledge.
Fig. 6 plots dMMD on the accumulated data from data sets A and
B for two testing scenarios. Solid line is the dMMD if the degree of
the polynomial model is iteratively increased and the dashed line
is the dMMD if the method estimates the diaphragm border using
directly higher than the second polynomial degree. The plot clearly
Fig. 7. Visual results of our prediction together with the predictions of state of the art and manually annotated ground truth. Example (a) is from the data set A and examples
(b), (c) and (d) are from data set B. The dashed line is our prediction, the dotted line is the prediction of [15], the ‘+’ signs represent the prediction of [14] and the thick solid
line is the ground truth marked by one of the specialists.
302
S. Petkov et al. / Computerized Medical Imaging and Graphics 38 (2014) 296–305
Fig. 8. The effect of improving a vesselness map using our diaphragm prediction: (a) is the original frame, (b) is the vesselness map after applying the filter from [10] and (c)
is the improved result after the postprocessing.
shows the advantage of iteratively incrementing the flexibility of
the model.
Regarding computational cost, the complexity of the method in
[15] is O(TZ3 ), where T is the number of frames in a sequence and
Z is the number of pixels in a frame. The complexity of the method
in [14] and the one we propose is O(Z). The method in [15] has the
highest complexity mainly because it needs to process the whole
sequence before making a prediction. Our method is comparable in
terms of time complexity to the method in [14] but outperforms it
in terms of prediction accuracy (Tables 3 and 4).
Fig. 7 shows four visual results of our prediction, together with
the predictions of [14,15] and the ground truth. Example (a) is from
data set A and shows that our method performs better than the
state of the art. Examples (b), (c) and (d) show cases from data set
B, in which the diaphragm border does not touch the image plane
borders. All predictions for these cases are suboptimal; in addition
to the bad detection of the diaphragm border curves, their starting
and ending points are not determined properly.
Finally, to validate the proposed method effectiveness, we performed Analysis of Variance (ANOVA) comparing our performance
on data set A to the performance of the other methods. The sample
mean errors are significantly different with p < 0.01.
5. Applications of the method
As stated in Section 1.1, the proposed method could improve
other algorithms for coronary angiography processing. We provide
three examples:
1. Removal of artifacts caused by the diaphragm border.
2. Digital diaphragm removal from X-ray angiographies.
3. Improvement of the semi-automatic Myocardial Blush estimation in [3].
5.1. Vesselness filter improvement
The vessel enhancement filter in [10] often detects the
diaphragm edge as a false positive vessel. Fig. 7(b) shows the
filter output for the frame in Fig. 7(a). We used the diaphragm
border detection method to postprocess the vesselness map. Let
V(x,y) be the vesselness value for pixel (x,y) and is the area
that spans 15 pixels above and below the predicted diaphragm
border. We attenuated the vesselness for each pixel in as
following:
Ṽ (x, y) = V (x, y)A(x, y)|(x, y) ∈ ˝,
(8)
where the attenuation factor is:
A(x, y) = | sin(∠(rp , rv ))|.
N
(9)
Here rp rp (1, i=1 iâi xi−1 ) is the tangent vector to the
diaphragm border and rv rv (x, y) is the orientation of the tubular structure at pixel (x,y). To compute rp and rv we used first
order derivative and Hessian matrix, respectively. Considering that
A ∈ [0, 1], if V(x,y) is small before the improvement, it will remain
small after the improvement. This is sufficient to prevent introducing any artifact. If the two vectors rp and rv are perpendicular
(meaning that a vessel is crossing the diaphragm border), then the
vesselness value will stay the same. If the two vectors are parallel
(an indication that a false positive detection is highly probable),
the vesselness will become zero. Fig. 7(c) shows the improvement in the vesselness map from Fig. 7(b). It can be seen that
the false positives caused by the diaphragm are almost completely
S. Petkov et al. / Computerized Medical Imaging and Graphics 38 (2014) 296–305
303
Fig. 9. The effect of digital diaphragm removal on four cases. Column (a) contains the original images and column (b) contains the images after the diaphragm has been
removed. The third column shows the result of applying Digital Subtraction Angiography (DSA) to the original image. The fourth column shows how the visualization of
arteries and myocardial perfusion improves when we apply DSA on the original image after removing the diaphragm from it.
removed while the vessels are preserved. Improving the general
robustness of the vesselness filter from [10] is out of the scope
of this paper – our goal here is to show how a simple algorithm, based on diaphragm detection, improves the result of the
filter.
= [1X] × [1Y ] × [1T ] ⊂ N3 . X, Y and T are the size of the angiography in each dimension. Let ⊂ be the subset of pixels below
or on the diaphragm border and ¯ = \ . We calculate the average gray values for and ¯ , respectively G and G¯ . Considering
the fact that the X-ray technology is multiplicative when displaying
two overlapping objects [20], the proportion:
5.2. Digital diaphragm removal
In our algorithm for digital diaphragm removal, we look at an
angiography as a volume S(x, y, t) ∈ R, which has a compact support
=
G
G
(10)
304
S. Petkov et al. / Computerized Medical Imaging and Graphics 38 (2014) 296–305
Fig. 11. Two plots of the gray level variation through the whole angiography in a
fixed region of the myocardium with size 15 × 15 pixels. Plot (a) is the gray level
before the diaphragm removal and plot (b) shows the same information after the
diaphragm has been removed.
removing the diaphragm – the arteries (and in the third case the
myocardial perfusion) are better outlined against the background.
5.3. Myocardial Blush Grade estimation improvement
Fig. 10. Improvement of the staining space descriptor in [3]: (a) depicts the staining
space descriptor for an angiography sequence; (b) depicts the same descriptor after
the diaphragm has been digitally removed. The range in which the diaphragm border
moves is delineated with white lines. The whiter the pixel, the higher is the staining
in the area.
gives the factor by which the diaphragm introduces darkening. We
use to digitally remove the darkening from the part of the volume
that contains the diaphragm as:
Ŝ(x, y, t) =
S(x, y, t),
if (x, y, t) ∈ ¯
S(x, y, t),
if (x, y, t) ∈ .
(11)
Fig. 8 shows the effect of digital diaphragm removal on four
cases. Column (a) contains the original images and column (b)
contains the images after the diaphragm has been removed. DSA
is a well-known method to highlight arteries [21]. Column (c)
shows the result of applying DSA to the original image. Column (d)
shows the result when we apply DSA on the images after digitally
Myocardial Blush Grade (MBG) is a subjective score for evaluating different levels of myocardial perfusion [22]. In X-ray videos the
perfusion of contrast liquid is seen as gray staining. In [3], authors
propose four descriptors of the myocardial staining pattern. The
diaphragm movement affects negatively the performance of the
method if the range covered by the diaphragm border overlaps
with the myocardial tissue; hence we chose an X-ray angiography in which the diaphragm and the myocardium partially overlap.
Fig. 9(a) shows the map for the staining space descriptor generated by the method in [3]. Fig. 9(b) depicts the same descriptor if
we remove the diaphragm prior to MBG analysis. The difference
in the visual results shows the improvement if the diaphragm is
digitally removed; most of the artifacts caused by the breathing
motion are removed while at the same time the myocardial perfusion is retained. Since the prediction is not precise at pixel level,
not all artifacts related to the diaphragm are removed.
Fig. 10 shows two plots of gray level variation through an
entire angiography for a fixed region of the myocardium with size
15 × 15 pixels. Plot (a) is the gray level variation before the digital
diaphragm removal and plot (b) shows the same information after
it. The plot on the left shows that the patient inhales and exhales
three times during the angiography. Each breathing cycle is visible
as a peak to bright gray level which represents the inhaling while
the diaphragm goes down. Then a drop-down to dark gray level follows, representing the exhaling while the diaphragm goes up and
S. Petkov et al. / Computerized Medical Imaging and Graphics 38 (2014) 296–305
covers the region. In the plot on the right the gray level variations
due to the diaphragm have been drastically reduced (Fig. 11).
6. Conclusion
We proposed an automatic method to delineate the diaphragm
border in X-ray angiography images and showed three possible
applications: (1) digital removal of the diaphragm from X-ray
angiographies, (2) improvement of the vesselness filter from [10]
and (3) an improvement of the method for semi-automatic myocardial blush estimation from [3]. Our algorithm advances the state of
the art by adopting a probabilistic approach. Iteratively increasing the flexibility of the polynomial model allows to improve the
performance of the method.
Several factors play an important role in our detection procedure. The predicted curve may not correctly delineate the
diaphragm border if the border induces weak edge or the morphological pre-processing does not eliminate other dark structures
than the diaphragm. The same effect may occur if the probabilistic
model has not learned a curve with a similar shape to the one that is
being detected. Ineffective bootstrapping procedure to find proper
initialization of the cost function (5) or bad tuning of the method
parameters could also hinder proper convergence.
Future work will be devoted to overcome current limitations in
the detection. In some cases, the diaphragm is seen as more than
one distinct borders and the assumption that each diaphragm border touches the image plane borders is incorrect. To achieve errors
that are close to inter-observer variability any method should first
estimate the number of diaphragm borders and the starting and
ending points for each of them. In addition, if there is no visible
diaphragm a method should not predict one.
References
[1] Condurache A, Aach T, Kaiser A, Radke P. User-defined ROI tracking of the
myocardial blush grade. In: 7th IEEE SSIAI. 2006. p. 66–70.
[2] Gatta C, Valencia JDG, Ciompi F, Rodriguez-Leor O, Radeva P. Toward robust
myocardial blush grade estimation in contrast angiography. In: IbPRIA. 2009.
p. 249–56.
[3] Gil D, Rodriguez-Leor O, Radeva P, Mauri J. Myocardial perfusion characterization from contrast angiography spectral distribution. IEEE TMI
2008;27(5):641–9.
[4] Liénard J, Vaillant R. Quantitative tool for the assessment of myocardial perfusion during X-ray angiographic procedures. In: Proceedings of the 5th ICFIMH,
ser FIMH ‘09. Berlin, Heidelberg: Springer-Verlag; 2009. p. 124–33.
[5] Pijls N, Uijen G, Hoevelaken A, Pijnenburg T, Leeuwen K, Fast J, et al. Mean
transit time for videodensitometric assessment of myocardial perfusion and the
concept of maximal flow ratio: a validation study in the intact dog and a pilot
study in man. International Journal of Cardiac Imaging 1990;5(2–3):191–202,
http://dx.doi.org/10.1007/BF01833988 [Online].
[6] Boyle AJ, Schuleri KH, Lienard J, Vaillant R, Chan MY, Zimmet JM, et al. Quantitative automated assessment of myocardial perfusion at cardiac catheterization.
American Journal of Cardiology 2008;102(8):980–7 [Online]. Available:
http://www.sciencedirect.com/science/article/pii/S0002914908010114
[7] Korosoglou G, Haars A, Michael G, Erbacher M, Hardt S, Giannitsis
E, et al. Quantitative evaluation of myocardial blush to assess tissue level reperfusion in patients with acute st-elevation myocardial
infarction: incremental prognostic value compared with visual assessment. American Heart Journal 2007;153(4):612–20 [Online]. Available:
http://www.sciencedirect.com/science/article/pii/S0002870307000531
[8] Haude M, Caspari G, Baumgart D, Ehring T, Schulz R, Roth T, et al. X-ray densitometry for the measurement of regional myocardial perfusion. Basic Research
in Cardiology 2000;95(3):261–70, http://dx.doi.org/10.1007/s003950050189
[Online].
[9] Ungi T, Zimmermann Z, Balázs E, Lassó A, Ungi I, Forster T, et al.
Vessel masking improves densitometric myocardial perfusion assessment. International Journal of Cardiovascular Imaging 2009;25(3):229–36,
http://dx.doi.org/10.1007/s10554-008-9374-5 [Online].
305
[10] Frangi AF, Niessen WJ, Vincken KL, Viergever MA. Muliscale vessel enhancement filtering. In: MICCAI ‘98: Proceedings of the first international conference
on medical image computing and computer-assisted intervention. London, UK:
Springer-Verlag; 1998. p. 130–7.
[11] Hernandez-Vela A, Gatta C, Escalera S, Igual L, Martin-Yuste V, Sabate M,
et al. Accurate coronary centerline extraction, caliber estimation, and catheter
detection in angiographies. IEEE Transactions on Information Technology in
Biomedicine 2012;16(6):1332–40.
[12] Tagizaheh M, Sadri S, Doosthoseini A. Segmentation of coronary vessels by
combining the detection of centerlines and active contour model. In: 2011 7th
Iranian in Machine Vision and Image Processing (MVIP). 2011. p. 1–4.
[13] Jandt U, Schäfer D, Grass M, Rasche V. Automatic generation of
3d coronary artery centerlines using rotational X-ray angiography.
Medical Image Analysis 2009;13(6):846–58. Includes Special Section on Computational Biomechanics for Medicine [Online]. Available:
http://www.sciencedirect.com/science/article/pii/S1361841509000668=0pt
[14] Condurache A, Aach T, Eck K, Bredno J, Stehle T. Fast and robust diaphragm
detection and tracking in cardiac X-ray projection images. In: Proc. SPIE. 2005.
p. 1766–75.
[15] Petkov S, Romero A, Suarez XC, Radeva P, Gatta C. Robust and accurate
diaphragm border detection in cardiac X-ray angiographies. In: Proceedings
of the 3th STACOM, part of MICCAI. 2012.
[16] Cao Y, Wang P. An adaptive method of tracking anatomical curves in X-ray
sequences. In: Proceedings of the 15th international conference on Medical
Image Computing and Computer-Assisted Intervention – Volume Part I, ser.
MICCAI’12. Berlin, Heidelberg: Springer-Verlag; 2012. p. 137–80.
[17] Zhu Y, Prummer S, Wang P, Chen T, Comaniciu D, Ostermeier M. Dynamic
layer separation for coronary dsa and enhancement in fluoroscopic sequences.
In: Proceedings of the 12th International Conference on Medical Image Computing and Computer-Assisted Intervention: part II, ser. MICCAI ‘09. Berlin,
Heidelberg: Springer-Verlag; 2009. p. 877–84.
[18] Anderson CH, Bergen JR, Burt PJ, Ogden JM. Pyramid methods in image processing. RCA Engineer 1984;29(November–December (6)).
[19] Lindeberg T. Principle for automatic scale selection. In: RIT, Tech. Rep.; 1998.
[20] Meijering E, Niessen W, Viegever M. Retrospective motion correction in digital subtraction angiography: a review. IEEE Transactions on Medical Imaging
1999;18(1):2–21.
[21] Meaney T, Weinstein M, Buonocore E, Pavlicek W, Borkowski G, Gallagher J,
et al. Digital subtraction angiography of the human cardiovascular system. AJR
American Journal of Roentgenology 1980;135(6):1153–60.
[22] van ‘t Hof AW, Liem A, Suryapranata H, Hoorntje JC, de Boer MJ, Zijlstra F. Angiographic assessment of myocardial reperfusion in patients treated with primary
angioplasty for acute myocardial infarction: myocardial blush grade. zwolle
myocardial infarction study group. Circulation 1998;97(June (23)):2302–6.
Simeon Petkov graduated as a bachelor in Informatics in 2009 at Sofia University
(Bulgaria). In 2010 he was enrolled in a Master course in Artificial Intelligence at
Barcelona University. The next year he started working in Computer Vision Center on
a project for automatic myocardial perfusion estimation. In 2012 he completed the
master course and started a Ph.D. research on simultaneous modeling, registration
and segmentation in medical imaging. The funding for the Ph.D. program is provided
by FI national scholarship.
Xavier Carrillo, M.D., is an invasive cardiologist graduated in Medicine in 2003 at
University of Barcelona. He works in Germans Trias i Pujol University Hospital in
Badalona (Barcelona) and he is a Ph.D. student at Universitat Autonoma of Barcelona.
The main topics of his scientific work are Acute Coronary Syndromes and medical imaging. He is a member of European and Spanish cardiology societies. He is a
member of AIM code committee at Catalan Health Department.
Dr. Petia Radeva (Ph.D. 1998, Universitat Autònoma de Barcelona, Spain) is a senior
researcher and associate professor at the University of Barcelona. She is the head of
Barcelona Perceptual Computing Laboratory (BCNPCL) at the University of Barcelona
and the head of MiLab of Computer Vision Center (www.cvc.uab.es). Her present
research interests are on development of learning-based approaches (in particular,
statistical methods) for computer vision and medical imaging. She is currently leading the projects: Machine learning tools for large scale object recognition, Audience
measurements, Intestinal Motility Analysis in Wireless Endoscopy, Automatic Stent
Detection in IVUS, Polyp detection, etc.
Carlo Gatta obtained the degree in Electronic Engineering in 2001 from the Università degli Studi di Brescia (Italy). In 2006 he received the Ph.D. in Computer Science
at the Università degli Studi di Milano (Italy), with a thesis on perceptually based
color image processing. In September 2007 he joined the Computer Vision Center at
Universitat Autonoma de Barcelona (UAB) as a postdoc researcher working mainly
on medical imaging. He is member of the Computer Vision Center and the BCN Perceptual Computing Lab. He is currently a senior researcher at the Computer Vision
Center, under the Ramon y Cajal program. His main research interests are image
processing, medical imaging, computer vision, machine learning and contextual
learning.