Theory in segmentation

.1 Segmentation in normalized cuts method

a digital image into multiple reg

rpose algorithms and techniques have been developed for image seg

ed by Shi and Malik in 1997[23] . In this

2

Segmentation refers to the process of partitioning

ions. The goal of segmentation is to simplify or change the representation of an image into something that is more meaningful and easier to analyze [22]. Image segmentation is typically used to locate objects and boundaries (lines, curves, etc.) in images. The result of image segmentation is a set of regions that collectively cover the entire image, or a set of contours extracted from the image. Each of the pixel in a region are similar with respect to some characteristic or computed property, such as color, intensity, or texture.

Several general-pu

mentation. Since there is no general solution to the image segmentation problem, these techniques often have to be combined with domain knowledge in order to effectively solve an image segmentation problem.

The “normalized cuts” method was first propos

method, the image being segmented is modeled as a weighted undirected graph.

Each pixel is a node in the graph, and an edge is formed between every pair of pixels.

The weight of an edge is a measure of the similarity between the pixels. The image is

l dissimilarity between the

partitioned into disjoint sets by removing the edges connecting the segments. The optimal partitioning of the graph is the one that minimizes the weights of the edges that were removed (the “cut”). Shi’s algorithm seeks to minimize the “normalized cut”, which is the ratio of the “cut” to all of the edges in the set.

The normalized cut criterion measures both the tota

erent groups as well as the total similarity within the groups. The grouping algorithm consists of the following steps:1. Given an image or image sequence, set up a weighted graph G = (V,E) and set the weight on the edge connecting two nodes to be a measure of the similarity between the two nodes. 2. Solve …(D-W)x = λDx for eigenvectors with the smallest eigenvalues. 3. Use the eigenvector with the second smallest eigenvalue to bipartition the graph. 4. Decide if the current partition should be subdivided and recursively repartition the segmented parts if necessary.

A graph G = (V,E) can be partitioned into two dis

oving edges connecting the two parts. The degree of dissimilarity between these two pieces can be computed as total weight of the edges that have been removed. In graph

theoretic language, it is called the cut :

cut A B)( ,

∑

( , ) (2.1)

e one that minimizes this cut value. Although there are an

The optimal of a graph is th

exponential number of such partitions, finding the minimum cut of a graph is a

d by recursively finding the minimum cuts that bise

cut criteria favo

well-studied problem and there exist efficient algorithms for solving it. Wu and Leahy [24] proposed a clustering method based on this minimum cut criterion. In particular, they seek to partition a graph into k-subgraphs such that the maximum

cut across the subgroups is minimized.

This problem can be efficiently solve

ct the existing segments. As shown in Wu and Leahy's work, this globally optimal criterion can be used to produce good segmentation on some of the images.

However, as Wu and Leahy also noticed in their work, the minimum

rs cutting small sets of isolated nodes in the graph. This is not surprising since the cut defined in (1) increases with the number of edges going across the two partitioned parts. Fig. 2.1 illustrates one such case.

Assuming the ance between the

two

Fig 2.1 A case where minimum cut gives a bad partition. [23]

edge weights are inversely proportional to the dist

nodes, we see the cut that partitions out node n1 or n2 will have a very small value.

aper pro

In fact, any cut that partitions out individual nodes on the right half will have smaller cut value than the cut that partitions the nodes into the left and right halves.

To avoid this unnatural bias for partitioning out small sets of points, the p pose a new measure of disassociation between two groups. Instead of looking at the value of total edge weight connecting the two partitions, our measure computes the cut cost as a fraction of the total edge connections to all the nodes in the graph. It’s call the

normalized cut (Ncut):

Where assoc(A,V) is the total connection from nodes in A and

r total normalized association within gro

to all nodes in the graph assoc(B,V) is similarly defined. With this definition of the disassociation between the groups, the cut that partitions out small isolated points will no longer have small Ncut value, since the cut value will almost certainly be a large percentage of the total connection from that small set to all other nodes.

In the same way, it can define a measure fo

ups for a given partition:

and assoc(B,B) are total weights of wit

Where assoc(A,A) edges connecting nodes

hin A and B, respectively. We see again this is an unbiased measure, which reflects how tightly on average nodes within the group are connected to each other. Another

important property of this definition of association and disassociation of a partition is

criteria that we seek in our grouping algorithm, minimizing disassociation between the groups and maximizing the association within the groups , are in fact identical and can be satisfied simultaneously. In our algorithm, we will use this normalized cut as the partition criterion.

.2 Gradient Vector Flow

rs, are curves defined within an image domain that can mo

ugh the spatial domain of an image to min

2

Snakes [25], or active contou

ve under the influence of internal forces coming from within the curve itself and external forces computed from the image data. The internal and external forces are defined so that the snake will conform to an object boundary or other desired features within an image. Snakes are widely used in many applications, including edge detection , shape modeling [26-27], segmentation [28-29].

A traditional snake is a curve, that moves thro

imize the energy functional: The external energy function Eext is derived from the its sma

=

∫

⎢⎣

image so that it takes on ller values at the features of interest, such as boundaries. Given a gray-level image I(x,y) , viewed as a function of continuous position variables (x,y), typical external energies designed to lead an active contour toward step edges are:

where G^σ(x,y)is a two-dimensional Gaussian function with standard deviation and gradient operator. If the image is a line drawing (black on white), then appropriate external energies include

ext

A snake that minimizes E must satisfy the Euler equation :

αx s^''( )−βx s^'''( )− ∇E =0 (2.8)

This can be viewed as a force balance equation

F_int+F_ext^{( )p} =0 (2.9)

The internal force discourages stretching and bending while the external potential force pulls the snake toward the desired image edges. The gradient vector flow snake approach is to use the force balance condition as a starting point for designing a snake.

It define below a new static external force field, which we call the gradient vector flow (GVF) field. To obtain the corresponding dynamic snake equation, we replace the potential force, yielding :

x s t_t( , )=αx s t^''( , )−βx s t^''''( , )+v (2.10) We call the parametric curve solving the above dynamic equation a GVF snake. It is solved numerically by discretization and iteration, in identical fashion to the traditional snake. Although the final configuration of a GVF snake will satisfy the force-balance equation, this equation does not, in general, represent the Euler equations of the energy minimization problem. This is because v(x,y) will not, in general, be an irrotational field . The loss of this optimality property, however, is well-compensated by the significantly improved performance of the GVF snake.

We define the gradient vector flow field V(x,y) = [u(x,y) , v(x,y)] to be the vector field that minimizes the energy functional

( ) ( )

This variational formulation follows a standard principle, that of making the result smooth when there is no data. In particular, we see that when ∇ is small, the energy

f

is dominated by sum of the squares of the partial derivatives of the vector field, yielding a slowly varying field. On the other hand, when ∇ is large, the second term

f

dominates the integrand, and is minimized by setting v = ∇f . This produces the desired effect of keeping v nearly equal to the gradient of the edge map when it is large, but forcing the field to be slowly-varying in homogeneous regions.

The parameter μ is a regularization parameter governing the tradeoff between the first term and the second term in the integrand. This parameter should be set according to the amount of noise present in the image.

We note that the smoothing term —the first term within the integrand by Horn and Schunck in their classical formulation of optical flow [30]. It has recently been shown that this term corresponds to an equal penalty on the divergence and curl of the vector field [31]. Therefore, the vector field resulting from this minimization can be expected to be neither entirely irrotational nor entirely solenoidal.

Using the calculus of variations [32], it can be shown that the GVF field can be

found by solving the following Euler equations :

These equations provide further intuition behind the GVF formulation. We note that in a homogeneous region, the second term in each equation is zero because the gradient of f(x,y) is zero. Therefore, within such a region, and are each determined by Laplace’s equation, and the resulting GVF field is interpolated from the region’s boundary, reflecting a kind of competition among the boundary vectors [33].

Chapter 3 Method

3.1 Clinical Experiment

The image raw data is gathered by Dr.Chang in National Taiwan University Hospital.

This clinical experiment is performance in 1.5T Siemens MRI system. We use gadolinium as contrast agent and Avastin as the chemotherapy drugs.

After inject the contrast agent, we scan the patient’s lung 100 frames in about 100 seconds, each frame will have four sagittal view images and one axial view image.

t = 1 2 3 98 99 100

+

4 Sagittal lung images 1 axial lung image

Fig 3.1 Image sequence in the clinical experiment.

3.2 Pre-processing in DICOM images

After DCE-MRI experiments, the console will output DICOM format images. If we want avoid motion effect in Mistar software, we should do motion correct operation before the DICOM input to the Mistar software.

Suppose we have two images like figure 3.2. Left side is the reference image (we suppose the tumor position is correct ), and right side is temporal image ( the tumor position will have shift effect because of the motion during scan process ) which we want to correct it.

Reference Image Temporal Image

Figure 3.2 Reference image and temporal image.

The red circle indicate the tumor ( to simplify the motion problem, we assume the tumor volume in each image is the same with each other ), and we can clearly identify the tumor position (red circle) in reference image and temporal image is quite different. If we input the sequence DICOM image data to the Mistar software, we

Reference image

Temporal image 1

Temporal image 2

Temporal image 3

Temporal image 100

Figure 3.3 Registration problem in sequential images.

To solve this problem, we using correlation method in image registration. For example, if we have two images like figure 3.2. The goal is that we want to put tumor in both images in the same position (fig 3.4).

Figure 3.4 Adjust motion problem by moving temporal image to reference image.

In fact, we select reference image in first time. Then we will find out the proper ROI in reference image (Fig 3.5) making the correlation with temporal images.

Proper ROI

Fig 3.5 Select proper ROI in reference image.

We will get the proper ROI from the reference image to compare with the similar area in the temporal images (Fig 3.6)

Similar Area

ROI (Reference Image)

Temporal Image

Fig 3.6 Get proper ROI from the reference image to compare with the similar area in the temporal images.

To increase the specification in tumor property, we can select the tumor position by

contour by user (dot line) in Fig 3.7

By User defined

Step 1: Select the

Fig 3.7 Contour which decided by user.

tep 2: We set gray level in the area outside the contour be zero (black) in Fig 3.8 S

Fig 3.8 Make outside gray level be zero.

By automatic segmentatio

cut will do operation in Fig 3.9

n method

Step 1: Select the area which normalize

Fig 3.9 Prepare proper ROI for normalize cut.

Step 2: After running normalize cut program, we get a binary image and its corresponding gray level image in Fig 3.10.

Step 3: Run the gradient vector flow (GVF) program to determined the contour in Fig 3.11 ~ Fig 3.13

Fig 3.11 Four maps (test image, edge map, edge map gradient, normalized GVF field) for determined gradient vector flow.

Fig 3.12 Running gradient vector flow program to determined the contour

Fig 3.13 Final image which decided by normalize cut and gradient vector flow.

The similar area should be the possible area which tumor position is inside in the temporal images. Although the tumor position in temporal images is differ with each other, the tumor position in reference image will provide a standard to deal with the problem. Since we already get the ROI in both reference image and temporal image, the maximum correlation will determined the correct position.

After we do the correlation with the reference and temporal image, there exist many correlation coefficients. What we want is the maximum correlation coefficient.

The maximum correlation coefficient represent the most proper tumor position in the temporal images. Once we find the proper position, the next step is to shift the temporal images to the new position (fig3.8).

Fig 3.15 Shift temporal images to the arbitrarily position which correlation value is maximum.

Since we suppose the tumor is rigid, the reference image will change to get the most similarity in tumor shape. That is, the image which was corrected will be the next reference image. Under the assumption, we suppose to correct total temporal images in the whole image sequences.

software Mistar to accomplish the DCE-MRI study and entify the difference between image data with motion correct or not.

After we lunch the DICOM data from MRI console, it shows the lung image in both gittal and axial view in the upper right during the software window. In order to find e arterial input function (AIF), we select the axial view and set the ROI ( yellow uare ) in the aorta to get the AIF. The upper right shows the 100 time frames signal

.

3.3 Mistar software processing

We use the commercial id

sa th sq

intensity in aorta which used to be the AIF

Besides select the AIF, we also decided the area which need to be calculated in the software. The selection of the area should be include the tumor and not exceed to much to waste the processing time (fig 3.17).

Fig 3.17 Select processing area to in Mistar software. If we select large area, it will spend more time to finish the calculation.

When we decided the curve of AIF and the region which needed to be execute, then we can push the processing button to run the entire calculation like fig 3.18.

Fig 3.18 Processing the calculation (blue image is the area which selected to calculate).

After Processing, we can get the DCE-MRI parameter maps . For example: The Upper left is kep; the upper right is Ve; the lower left is K^trans; the lower right is Vp.

Fig 3.19 DCE-MRI parameter maps (upper right is ve, upper left is kep, lower right is vp, and lower left is k^trans).

Chapter 4 Results

4.1 Results from Mistar software

After we finish the processing in Mistar software, the parameter maps will show us the information about the DCE-MRI.

First, we should select the most proper tumor contour in the maps. We can console the T1-contrast image for reference to adjust the correct contour (fig 4.1).

T1-Contrast enhance Image T1-Weighted Image

Fig 4.1 Select tumor contour by doctor to make sure the area is exactly in tumor position.

When we se aps

can be also determinate like the fig 4.2.

lect the proper contour about the tumor, the DEC-MEI parameter m

K^trans Map Kep Map

Ve Map Vp Maps

Fig 4.2 Four parameters maps results which calculated by Mistar software, the yellow line in the pictures e tumor position and shape decided by doctor.

The Mistar software allows us to output the detail values in each pixel which is in the range of tumor contour. Because we want to know the information in entire tumor,

are th

we should calculate the values in all slices. Here we have four slices in the sagittal view, and there should be four different parameter maps ex: K^trans.

The following table shows the data which we collect in this experiment. There are eleven patients in it. Some of them having twice or three times cases, even four times is include.

Chart No. Age Gender date of MRI K

^trans

(mean) K

^trans

(SD) K

_ep

(mean) K

_ep

(SD) V

_e

(mean) V

_e

(S D) V

_p

(me an) V

_p

(S D)

Pixel Np.

Volume

F779 52 F before 20080110 172.8782 86.5904 1263 511.1793 146.4887 69.9464 38.0 40 6 22.3202 37 92 26.24 after 156.4238 119.044 1355 1439 273.98 421.8603 46.7 21 3 35.2522

before 20080131 127.1347 77.6199 1299 669.2082 110.8461 84.0982 31.8 77 4 31.1185 37 35 25.87 after 114.408 86.1253 1268 871.3224 123.993 136.2469 35.5 29 3 31.9941

before 20080220 110.3947 48.4889 969.2542 502.518 164.1305 126.0281 24.6 48 4 16.2942 20 85 14.44 after 122.5904 51.9391 996.4077 492.7662 166.7213 120.7124 24. 14 2 16.7437

M329 62 M before 20070526 98.0442 95.8564 1164 1004 101.6869 118.3307 23.3 01 4 23.8693 102 41 76.69 after 90.019 92.0413 1024 949.2185 116.109 150.226 23.5 81 6 23.6555

before 20070801 44.4853 55.767 710.7123 1003 122.2082 199.0388 29.1 40 1 29.3651 47 31 38.2 after 34.1365 60.8422 457.9362 936.201 165.4887 284.2617 24.7 33 9 29.3889

before 20071003 57.8705 73.7613 850.1242 1062 83.4321 121.6247 21.1 14 6 25.1096 28 26 28.18 after 45.9646 86.4798 742.3719 1526 74.1359 163.2148 25.9 40 9 33. 4157

before 20071121 45.1077 54.8354 822.802 922.8821 56.2496 75.8963 21.0 38 4 21.8096 50 25 43.64 after 47.6603 59.579 721.9954 923.8309 85.7731 139.319 22.7 38 9 27.2716

M871 49 M before 20080118 96.9632 57.9461 1636 879.2621 83.6799 92.9254 31.7 94 2 20.1439 41 55 36.09 after 93.7745 67.0807 1544 1006 98.8046 113.8152 33.3 87 5 22.6192

before 20080212 52.9786 43.0451 1014 813.8075 92.4277 113.5992 23.4 99 4 21.2353 24 29 21.09

after 59.7205 61.7942 1128 888.0041 77.1618 83.8275 21.4 06 3 19.161

Chart No. Age Gender date of MRI K

^trans

(mean) K

^trans

(SD) K

_ep

(mean) K

_ep

(SD) V

_e

(mean) V

_e

(SD) V

_p

(mean) V

_p

(SD)

Pixel Np.

Volume

M141 68 M before 20070529 100.9661 60.5172 781 .2155 519.3872 214.7719 97.9482 1 17.6092 14.2926 3424 23.71 after 97.9588 57.7668 787 .2339 567 .8119 22 9.6203 209.1 20.4229 16.4177

before 20070815 87.1577 70.3808 1225 858.1167 89.4633 92.6278 20.2625 17.9677 2042 14.13 after 81.0984 70.6965 1037 825.8859 122.3457 144.6699 20.1102 18.1868

before 20071031 99.6389 85.1381 1175 869.0652 119.4608 104.4188 21.06 18.8583 983 7.94 after 74.6541 65.4323 870.0458 794.2426 167.2279 234.8201 23.5371 28.6724

M740 60 M before 20080325 111.5457 86.375 1948 1276 93.0266 123.4674 34.8995 23.8812 1840 15.97 after 8 8.8685 86.7014 1623 1569 1 98.7152 87.2727 2 41.1228 28.7461

before 20080421 1 07.3975 5 6.2021 1639 762 .2981 80.0312 66.3173 20.6098 15.1292 961 9.58 after 93.5109 52.6339 1620 1038 72.6608 77.4495 23.7804 19.9829

before 20080509 76.3165 48.3744 1164 779 .1787 1 08.6878 120.3225 16.566 11.9694 1166 11.63 after 72.7015 67.1446 1089 1382 171.6475 299.9304 16.8148 17.546

M826 58 M befor e 20080402 1 21.7588 10 5.2475 1279 862.7111 1 22.5373 104.345 31.1425 28.5017 10228 70.82 after 120.7957 119.5485 1206 979.0496 148.0378 184.8912 33.3206 32.2619

before 20080429 2 33.9865 2 17.0597 1574 983.4117 1 60.6663 06.3205 1 49.0664 38.8932 7993 55.34 after 2 32.8203 2 41.7068 1453 1143 3 06.2463 460.7253 56.6718 56.2376

before 20080520 67.4415 63.5726 66 0.581 545.4191 1 86.8765 159.4787 23.2834 22.3548 4704 32.56

after 95.6312 1 21.7479 958 .7119 985.875 2 22.0485 66.6306 2 27.6603 2 6.8991

Chart No. Age Gender date of MRI K

^trans

(mean) K

^trans

(SD) K

_ep

(mean) K

_ep

(SD) V

_e

(mean) V

_e

(SD) V

_p

(mean) V

_p

(SD)

Pixel Np.

Volume

M469 77 M before 20061030 219.3056 167.3221 1625 934.4481 128.5896 90.6103 58.9645 40.7381 5275 52.6 after 1 94.4157 1 70.0952 1393 1150 164.4108 203 .3105 60.8883 47.0387

before 20061205 40.5627 44.0976 680 .6431 781.0105 85.153 36.4305 1 33.1725 31.6663 3444 34.34 after 45.6771 58.8942 645.5044 835.5118 76.261 118.7002 27.6519 29.4886

before 20070111 89.1366 1 00.7679 794 .7162 801.439 146.5321 202.2965 45 .5301 43.4019 5591 38.62 after 80.3615 112.4534 722.1975 922.8178 136.6053 208.5487 48.299 50.8263

F928 58 F before 20080119 238.59 12 3.8953 2185 863 .2497 115.288 49.7335 55.4066 29.0878 7765 53.75 after 2 26.4178 1 38.2651 1881 1122 217.97 305.2919 66.1131 39.5759

before 20080214 89.6262 61.2078 1545 809.7152 76.6474 70.779 44.6246 21.4567 6817 47.19 after 51.7822 45.7289 1079 1003 88.0855 127.0202 53.5089 29.2859

before 20080306 1 06.0227 48.1355 1542 633.4481 79.243 41.985 3 5.8371 16.0028 749 5.18 after 1 09.0774 57.4687 1311 690 .5937 134.7704 136.7922 37.5701 18.058

M372 41 M before 20070602 205.999 98.3014 2052 978.0806 116.5311 75.6308 37.5079 23.2589 3099 21.46 after 180.6705 104.3126 1672 1016 153.9742 148.2009 40.5608 32.3914

before 20070816 81.0472 42.49 1073 634.5683 106.9744 90.8074 21.0072 14.3647 1249 8.65

after 80.5124 42.4462 1002 683.3029 131.9744 128.8156 22.1954 15.6253

Chart No. Age Gender date of MRI K

^trans

(mean) K

^trans

(SD) K

_ep

(mean) K

_ep

(SD) V

_e

(mean) V

_e

(SD) V

_p

(mean) V

_p

(SD)

Pixel Np.

Volume

M664 47 M before 20070801 69.9793 44.6357 1133 754.1266 113.6001 1 25.8711 26.2577 19.9017 1688 1.68 1 after 53.2725 41.0073 853 .5101 795 .0104 145.8181 195.008 30.8104 23.3339

before 20071003 35.2455 43.9684 700.7417 796.9888 60.7749 83.7166 32.3862 22.407 1173 8.11 after 28.7025 39.6834 424.3495 632.6747 91.1705 154.0047 35.1117 25.7059

M807 44 M befor e 20070725 1 78.2295 145.7388 1810 1176 130.5924 123.2023 9 2.6686 45.5533 1521 13.21 after 142.2906 146.8902 1538 1629 123.5095 134.204 98.4313 56.2298

before 20071011 8 1.6464 54.1193 1231 795.6572 1 12.3391 09.3154 1 31.6754 15.0083 345 3.45 after 70.7594 56.9641 1028 917 .3103 16 2.9246 233.9392 36.3101 22.3327

e 4.1 ter ch patie u and de

Tabl Four parame values in ea nt incl de the mean standard viation.

4 ata aly

e c ab otherapy treatment can be divided into

three terms: Before (Baseline) treatment, ent (after the first

co of an (the final study before stopping chemo-

therapy).

e divided the patient into two groups. Group A: The treatment include only before treatm B: the tment include before treatment, im diat t an t. We can plot the histogram about the four

DCE-MRI param ent.

e follo shows these parameters which the patient in group A. The

v al a no number of pixels in the area which we

consider is tumor, while the horizontal axis means the normalized histograms of

am tude R ,Kep, p).

.2 D an sis

Th ommon catalog out the cancer chem

Immediate after treatm urse chemotherapy) d final treatment

W

ent and final treatment; Group trea

Ve,V me e after treatmen d final treatmen

eters to comment the response of treatm g fi

Th win gure

ertic xis shows the rmalize of the

pli in each DCE-M I parameters (K^trans

Group A (M372)

K^trans K_ep

V V_{e p} Group A (M807)

K^trans K_ep

V_e V_p

Group A (M664)

K^trans K_ep

V_e V_p Group A (M871)

K^trans K_ep

V_e V_p

Group B (F779)

K^trans K_ep

V_e V_p Group B (M469)

K^trans K_ep

V_e V_p

Group B (M826)

K^trans K_ep

V_e V_p Group B (M141)

K^trans K_ep

V_e V_p

Group B (F928)

K^trans K_ep

V_e V_p Group B (M740)

K^trans K_ep

V_e V_p

Chapter 5 Discussion

5.1 Problems in Data Analysis

According to the results present in 4.2, we find almost every patient in group A shows the reasonable results in the figure about eter. The curve after motion correction process will lower and have trend through the vertical axis.

In group B, it also exist the same phenomenon in the distribution curve. But in some patient, the baseline curve (red curve) is in the right side, some final curve (blue curve) is in the middle, and the imm

need to be analysis more detail or analysis in another way to explain it.

Besides the curve position will differ (red, green, blue) in each chemotherapy process (Baseline, Immediate, Final), the height in each curve is also different. That’s because the area under distribution curve have been normalized to be one. And we can find that in most patients, the baseline curve is the lowest while the final curve is the most height in the diagram.

four different param

ediate curve (g een curve)is in the left. These kind of data r

5.2 Inaccuracy Pro

There may exist some reason to explain the results (from parameter maps) which is irst, in our assumption, the tumor shape is solid and

ords, the tumor cha

blems

not totally satisfy with the situation. F

stable (not change with time). But in fact, many tumor’s shape do have some different. Even in the scan processing, the slice will not so exact to cut, so the image in

stable (not change with time). But in fact, many tumor’s shape do have some different. Even in the scan processing, the slice will not so exact to cut, so the image in

在文檔中 動態對比增強磁共振影像參數之估測 : 位移效應之修正 (頁 40-81)