Automatic segmentation of liver PET images

(1)

Abstract

Automation of liver positron emission tomography (PET) image segmentation is proposed in this paper. A new active contour model (ACM), called Poisson Gradient Vector Flow (PGVF), with genetic algorithm (GA) constructs a scheme to automatically find the contour of liver in the PET images. PET is widely used for the clinical purpose, but image quality of PET makes the image segmentation be a tough work.

Three image data sets are tested for evaluating the new segmentation approach of liver PET images. One image data set is adapted from the study of one person with a normal liver. The other two image data sets are adapted from the studies of two patients with abnormal livers. The results show that the regions of interest (ROI) of liver are automatically segmented from the images of three data sets.

Keywords: PET images; Image segmentation; Active contour model; Genetic algorithm

1. Introduction

The imaging formation of positron emission tomography (PET) images[1]can be used to evaluate the organs’ metabolic function because the positron decay[2]takes place due to the metabolic reaction after injection of the radiopharmaceutical substance into human body. The releasing electron and a positron encounter each other and they annihilate to produce two high-energy photons,␥ rays[3]. The␥ rays collected by detectors to generate signals that are used for reconstruction of func-tional PET images. To evaluate the organs’ metabolic function, PET images need to be processed by workstations equipped with the PET imaging machines. However, most software pack-ages installed in these workstations use the seed region growing methods[4]for image segmentation purposes. The segmentation results of PET images are not good enough because of the high noise, low resolution, and blurriness. The cluster analysis[5] method and traditional watershed method[6]were used for PET image segmentations, but the over segmentation problem caused unqualified results. A new automatic segmentation method for

∗_{Corresponding author. Tel.: +886 4 23323000x7802; fax: +886 4 23305539.}

E-mail addresses:[email protected](C.-Y. Hsu),

[email protected](C.-Y. Liu),[email protected](C.-M. Chen).

PET images by using active contour model (ACM)[7–10]is pro-posed in this paper. Active contour models can capture the object boundaries in images and they are widely used in the image seg-mentation problem for their robustness to the noisiness. The application of traditional active contour for PET images focuses on brain tissues[11], but not on liver tissue. Automation of liver segmentation is more difficult than automation of brain tissue segmentation because of organs more asymmetrically located in abdomen slice images. Traditional active contour models with the manual selection of initial contours are not suitable for the automation of liver PET image segmentations. Our pre-vious work[12], Poisson Gradient Vector Flow (PGVF) ACM, is applied on the automatic segmentations of PET image because the initial contour can be automatic selected by the Canny edge detection method from the candidate curves of liver. The num-ber of the candidate curves is influenced by the values of the two parameters, threshold and sigma. However, manually find-ing the optimal values of the two parameters is time consumfind-ing. The genetic algorithm (GA) is used to automatically obtain opti-mal values by designing the fitness function to obtain the longest and the least number of the candidate curves. Appling genetic algorithm once to obtain an initial curve on the first slice of sequential images is enough for one subject study. It saves CPU time using the liver contour of the segmentation result of the last slice as the initial contour for the next slice.

(2)

Fig. 1. The flow chart of the automatic segmentation method of liver PET images.

2. Automation of image segmentation

The flow chart of the image segmentation method is shown inFig. 1.

The first step is to input the original PET image and the second step is to find the edge map by the Canny edge detection and genetic algorithm. The third step is to capture the contour of the liver in the PET image by the PGVF ACM. Because the PGVF ACM method plays an important role, Section2will explain this method in detail. Section2.1is an introduction of the theory and development of ACM which is the basic knowledge of PGVF ACM. The image force of PGVF ACM is introduced in Section 2.2and the theory of Canny edge detection method is presented in Section 2.3. Section 2.4introduces the implementation of the genetic algorithm for finding the parameters of Canny edge detection method.

2.1. The theory of ACM

There are two kinds of the image segmentation methods: one is to take out regions of interest (ROI) pixels from origi-nal image and the other is finding out the ROI contour. Active contour model is a method to find out the ROI contour. Active contour model is presented by Kass et al.[7–10,13,14]. Active contour model was classified into one kind of physics-based technique[15,16]is able to segment the edge of object. Since the active contour is deformable like a snake, then “Snake” is used as the same as the traditional ACM. One drawback of traditional ACM is that the initial circle must be closed to an object and the other drawback is that concave object cannot be segmented accurately. In other words, the accuracy of the traditional ACM needs to be improved. For improv-ing of the drawbacks, Xu and Prince announced Gradient Vector Flow (GVF) ACM[17]in 1999. The GVF ACM devel-oped a new external image force and overcome the defects of traditional ACM. In 2003, Hsu et al. proposed Poisson Gra-dient Vector Flow ACM [12]. The external image force is obtained from the gradient of the function solved by the Poisson equation.

Active contour model describes parametric curves with point position vector ៝X = [x(s), y(s)] which is function of the param-eter s in the range [0, 1]. The energy E of an active contour can be expressed as a functional of ៝X(s): E = 1 0 1 2 α| X₍_s)|2_{+ β|}_X₍_s)|2_{+ E} ext ds (1)

where Eext is the external energy, X= d X/ds and X= d2X/d2s are the first derivative and second derivative of curve point position vector X with respect to the s parameter. The coor-dinates of vector ៝X(s) are a pair of numbers [x(s), y(s)]. Using the variation calculus and Eq.(1), we obtained the Euler equations: αxss+ βxssss+∂E

ext

∂s = 0 (2)

and

αyss+ βyssss+∂Eext

∂s = 0 (3)

where∂Eext/∂s is the external image force field, xssand yssare the second derivatives, xsss and ysss are the forth derivatives of

x and y coordinates respect to the independent variable s. In the traditional ACM, external energy Eext equals to square of intensity gradient:

Eext= |∇I(x, y)|2 (4)

or equals to square of convolution between Gaussian function G_σ with varianceσ and intensity operated by the gradient operator :

Eext= |Gσ∇I(x, y)|2 (5)

The GVF[17]ACM method is modified from the traditional ACM method with an edge map f(x,y) =−Eextand the external image functional Eextis defined as follows:

Eext=

(3)

∇2_{φ(x, y) = f}

edge(x, y) (8)

fedgeis a binary edge map detected by Canny edge method. After

φ(x, y) is computed, the external image force field is computed by the vector flowv = −∇φ(x, y). The finite difference method can be used to obtain the numerical solution.

2.3. The Canny edge detection

Canny edge method[18]is an edge detection method to gen-erate edge image by looking for local maxima of the gradient of grayscale image. The gradient is calculated by using the deriva-tive of a Gaussian filtered grayscale image. The method uses two thresholds, to detect strong and weak edges, and includes the weak edges in the output only if they are connected to strong edges. If you use the edge function of Matlab and specify a scalar for thresh, this value is used for the high threshold and 0.4× thresh is used for the low threshold. This method is there-fore not to be fooled by noise, and more likely to detect true weak edges. One parameterσ of Canny edge method to be selected is the standard deviation of the Gaussian function. Once the edge image obtained, non-maximum suppression algorithm is used to mark the edge element and trace along the gradient in the edge direction. After non-maximum suppression, a thresh-old is used to produce a binary edge image. The black areas are the pixels with the edge strength below the threshold and the white areas are the pixels with the edge strength above the threshold.

2.4. Implementation of genetic algorithm

GAs were invented by Holland [19] and developed by him and his students and colleagues [20]. The GA is an evolutionary computing (EC) inspired by Darwin’s theory of evolution and uses an evolutionary process to solve problems [21,22]. As shown in the previous section, how to choose the values of the threshold and σ parameters is a problem to be solved. The values of threshold and σ parameter form a set of solutions and are separately represented by chro-mosomes. The concept of genetic algorithm applied on the threshold and parameter selection is described in the next para-graph.

Fig. 2. Edge map obtained by Canny edge detection with threshold 0.1 andσ values: (a) 1, (b) 2, and (c) 3.

(4)

Fig. 3. Edge map obtained by Canny edge detection withσ = 1.9 and threshold (a) 0.1, (b) 0.11, (c) 0.23, and (d) 0.4. fitness function is defined as in the following equation:

F = αmax(L)

min(N) (9)

where L and N are length and total pixel number of white contour line, andα is a weighting number 2/3.

3. Results and discussion

There are 3 image data sets with 16 abdomen PET slice images used for testing the new segmentation method. One image data set D1 is adapted from the study of one person with a normal liver that is obtained from the website of the book [23]. The other two image data sets, D2 and D3, are adapted from the studies of two patients with abnormal livers that are supplied by the Chung Shan Medical University Hospi-tal. The size of each image is 128× 128 pixels. It is the first step to find the edge map for the segmentation of the liver. In the next section, there are two examples used to show that it is very difficult to find the optimal edge map by manual selection.

3.1. Manual selection of edge map

The threshold andσ need to be decided for obtaining suitable edge map for PGVF ACM by the Canny edge detection method. The values ofσ influence the edge maps as shown inFig. 2(a)–(c)

by setting the threshold values 0.1 andσ values 1, 2, and 3. The liver contour inFig. 2(b) is more complete thanFig. 2(a) and (c).

Fig. 3is an example to demonstrate that the values of thresh-old influence the results of the edge map. The σ values are fixed at 1.9 and threshold values are 0.1, 0.11, 0.23, and 0.4 in Fig. 3(a)–(d). Although a little variance of theσ values,Fig. 3(a) has more edge lines thanFig. 3(b). The total number of the edge line inFig. 3(a) and (b) is larger thanFig. 3(c) and (d). According to these observations, slight change ofσ or threshold value induces significant influence on the contours in edge map. Because the liver contour line is complete and least number of edge line,Fig. 3(c) is the best binary image used for the edge map. Manual selection of the two suitable parameters is not effi-cient because hundred values of parameters should be tested. To develop automatic method to select suitable parameters is nec-essary and GA is introduced to solve the problem in the next section.

3.2. Automatic selection of edge map

In this section, three test images were selected from the 9th slice of D1, D2, and D3 image data sets, and they were denoted as D1(9), D2(9), and D3(9) as shown inFig. 4. These images were used to demonstrate to the effectiveness of GA applied on the automatic selection of the threshold and σ parameters.

(5)

Fig. 4. The original images of (a) D1(9), (b) D2(9), and (c) D3(9).

The fitness function of the GA is defined as in Eq.(9)and the ranges of threshold andσ are (0, 0.9) and (0.1, 2). The other parameters of GA are given inTable 1.

It is important to choose the generation number because it influences the values of threshold andσ parameters. The gen-eration number as shown in Table 1 that is decided by the convergence test of the fitness function values. The fitness func-tion values vary with the number of generafunc-tion as shown in Fig. 5(a)–(c).

As shown in Fig. 5, the change rate is larger when gen-eration number is less than 100 and is going to be stable as the generation number is increasing. The generation number 100 is chosen as the stopping condition in the following exam-ples.

Fig. 6(a)–(c) was binary images obtained by Canny edge detection with the values of threshold and σ in Table 2 and they can be used as the edge maps for PGVF ACM.

3.3. Segmentation by the PGVF ACM

The edge maps as shown inFig. 6are used by the PGVF ACM to process the image segmentation. Selection of the initial curve is important for PGVF ACM. The location of liver is always at Table 1

The values of parameters are used in genetic algorithm

Parameter name Value

Generation number 100

Bit number 100

Crossover rate 0.7

Mutation rate 0.02

Fig. 5. The fitness function values vary with the number of generation for (a) D1(9), (b) D2(9), and (c) D3(9) images.

the left side of each image as shown inFig. 7. We chose a circle with radius (15 pixels) as an initial curve. The center of each circle is located at the intersect point of the vertical tangent line with the liver contour.Fig. 7shows the initial circles of PGVF ACM for images (a) D1(9), (b) D2(9), and (c) D3(9), and the center coordinates for each initial circle are (26, 73), (40, 70), and (50, 70).

(6)

Fig. 6. The edge maps obtained by Canny edge detection with the parameters shown inTable 2.

Table 2

The values of threshold andσ of image data D1(9), D2(9), and D3(9)

Images Threshold σ

D1(9) 0.2283 1.6353

D2(9) 0.84023 1.1166

D3(9) 0.32536 0.3614

Fig. 7. Initial circles are chosen for images (a) D1(9), (b) D2(9), and (c) D3(9).

In the first 40 iterations, the evolution curves are progressing as shown inFig. 8.

By testing the images D1(9), D2(9), and D3(9), the evolution curves were stopped at the 100th iteration and the shapes and the locations of contours are shown inFig. 9(a), (c), and (e). The inner regions of these contours are shown as the binary images

Fig. 8. Progressing evolution curves of PGVF ACM on (a) D1(9), (b) D2(9), and (c) D3(9).

(7)

Fig. 9. The shape and location of contours (a), (c), and (e) and binary regions (b), (d), and (f) of images D1(9), D2(9), and D3(9).

inFig. 9(b), (d), and (f). The subject of the image data D1(9) is normal and the subjects of D2(9) and D3(9) are abnormal. Comparing withFigs. 9 and 10, radiologists indicated the result ofFig. 9(a) is better thanFig. 9(b) and (c) and these contours almost match the edges of livers.

3.4. Segmentation of sequence images

PGVF ACM has to select the initial curve for continuing the curve evolution. However, for the sequence images PGVF ACM only has to choose an initial curve once because the initial curve of the next image can use the final curve of the last image. As shown inFig. 11(a), the segmentation result of the 9th slice

image is used to be the initial curve for the segmentation of the 10th slice image.Fig. 11(b) shows that curves are progressing at the 40th iteration and the contour is close to the edge of the liver.Fig. 11(c) shows the contour at the 100th iteration and the contour is similar asFig. 11(b).

Comparing to Fig. 11(a), an initial circle as shown in Fig. 12(a) is used as an initial curve for the 10th slice image seg-mentation.Fig. 12(b) shows that curves are progressing at the 40th iteration and the contour is far from the edge of the liver. Comparing withFigs. 11(b) and 12(b), we can conclude that it only needs 40 iterations for curve evolution if using the 9th slice image segmentation result as the initial curve.Fig. 12(c) shows the contour at the 100th iteration and it is similar asFig. 11(c).

(8)

Fig. 10. The manual segmentation results for images (a) D1(9), (b) D2(9), and (c) D3(9).

Fig. 11. The shape and location of the initial curve (a) obtained from result of the 9th slice and progressing results at (b) 40th and (c) 100th iterations.

Fig. 12. The shape and location of the initial curve (a) with a circle and pro-gressing results at (b) 40th and (c) 100th iterations.

(9)

Fig. 14. The segmentation results of (a) 8th, (b) 10th, and (c) 11th slice of D2.

Fig. 15. The segmentation results of (a) 8th, (b) 10th, and (c) 11th slice of D3.

By using the 9th slice image segmentation results as the initial curve, the segmentation results of the (a) 8th, (b) 10th, and (c) 11th slice image for each image data set D1, D2, and D3 are shown inFigs. 13–15.

4. Conclusion

There are 3 image data sets with 16 abdomen PET images used in the experiments. The segmentation results show that the PGVF ACM is suitable for automatic segmentation of the liver PET images. When lots of image data need to be processed, the automatic segmentation by using PGVF ACM with GA can save much time and overcome the defects caused by manual selection of ROI.

the study. We also appreciate Dr. Chenyang Xu for his help-ful discussions during his visit in Taiwan that motivated the development of PGVF active contour model.

References

[1] Harvey A. Ziessman, Janis P. O’Malley, James H. Thrall. Nuclear medicine: the requisites in radiology. 3rd ed. Mosby Elsevier; 2006.

[2] Rachel A. Powsner. Essentials of Nuclear Medicine Physics. Blackwell Publishing; 1998.

[3] Ramesh Chandra. Nuclear Medicine Physics: The Basics. Lippincott Williams & Wilkins; 2004.

[4] Adams R, Bischof L. Seeded region growing. IEEE Trans Pattern Anal Machine Intell 1994;16:641–7.

[5] Wong KP, Feng D, Meikle SR, Fulham MJ. Segmentation of dynamic PET images using cluster analysis. IEEE Trans Nucl Sci 2002;49(February (1)):200–7.

[6] Riddell C, Brigger P, Carson RE, Bacharach SL. The watershed algorithm: a method to segment noisy PET transmission images. IEEE Trans Nucl Sci 1999;46(June (3)):713–9.

[7] Leymarie F, Levine MD. Tracking deformable objects in the plane using an active contour model. IEEE Trans Pattern Anal Mach Intell 1993;15(6):617–34.

[8] Kass M, Witkin A, Terzopoulos D. Snakes: active contour models. Int J Comput Vision 1988;1(4):321–31.

[9] Terzopoulos D, Witkin A, Kass M. Constraints on deformable models: recovering 3D shape and nonrigid motion. Artif Intell 1988;36(1):91–123. [10] Terzopoulos D. On matching deformable models to images, 1986. Tech-nical report 60, Schlumberger Palo Alto Research. Reprinted in topical meeting on machine vision, technical digest series, vol. 12. Washington, DC: Optical Society of America; 1987. p. 160–7.

[11] Mykkänen J, Tohka J, Ruotsalainen U. Delineation of brain structures from positron emission tomography images with deformable models, in the new navigators: from professionals to patients, studies in health technology and informatics. In: Proceedings of the Medical Informatics Europe, vol. 95. St. Malo, France: IOS-Press; June 2003. p. 33–8.

[12] Hsu CY, Chen SH, Wang KL. Active contour model with a novel image force field. In: Conference CVGIP2003. 2003. p. 47783.

[13] Terzopoulos D, Fleischer K. Deformable models. Visual Comput 1988;4(6):306–31.

[14] Z. Yu, C.L. Bajaj. Image Segmentation Using Gradient Vector Diffusion and Region Merging, Proceedings of the 16th International Conference on Pattern Recognition (ICPR’02), vol. 2. Quebec, Canada, August 2002. p. 941–944.

[15] Fischler MA, Elschlager RA. The representation and matching of pictorial structures. IEEE Trans Comput 1973;22(1):67–92.

[16] Widrow B. The rubber-mask technique. Pattern Recogn 1973;5:175–211. [17] Xu C, Prince JL. Snakes shapes, and gradient vector flow. IEEE Trans

(10)

[18] Canny J. A computational approach to edge detection. PAMI 1986;8(November 6):679–98.

[19] Holland JH. Genetic algorithms and the optimal allocation of trials. SIAM J Comput 1973;2:88–105.

[20] Holland JH. Adaptation in natural and artificial systems. Ann Arbor: Uni-versity of Michigan Press; 1975.

[21] Fogel L, Owens A, Walsh M. Articial intelligence through simulated evo-lution. John Wiley and Sons; 1966.

[22] Rechenberg I. Evolution strategie: Optimierung Technisher Systeme nach Prinzipien des Biologischen Evolution. Stuttgart: Fromman-Hozlboog Ver-lag; 1973.

[23] Dhawan AP. Medical image analysis. John Wiley Publications and IEEE Press; 2003.

Chih-Yu Hsu received the M.S. degree in 1993 and the Ph.D. in 1997, both in

the Department of Applied Mathematics from National Chung Hsing University at Taichung, Taiwan, ROC. At present, he is on the faculty of the Department of Graduate Institute of Networking and Communication Engineering, Chaoyang University of Technology, Taichung, Taiwan, ROC, where he is currently an assistant professor. His research interests include medical image and signal processing.

Chun-You Liu was a psychology graduate of Department of Medical Imaging

and Radiological Sciences, Chung Shan Medical University in 2005. He received a Master degrees from Department of Electrical Engineering of National Tai-wan University, Taipei, TaiTai-wan, ROC in 2008 and he is now a Ph.D. student at the Institute of Biomedical Engineering, National Taiwan University, Taipei, Taiwan, ROC. His main interests are the medical image processing and analysis of metallic nanoparticles’ spectra.

Chung-Ming Chen received the B.S. degree in Electrical Engineering from

National Tsing-Hua University, Hsin-Chu, Taiwan, and the M.S. and Ph.D. degrees in Electrical Engineering both from Cornell University, Ithaca, New York, USA. He subsequently joined the Center for Biomedical Engineering, National Taiwan University, Taiwan, where he served as a research assistant professor. He is currently a professor of the Institute of Biomedical Engi-neering, National Taiwan University, Taiwan. His research interests are in medical image analysis, bioinformatics, and computer-aided diagnosis. He is a member of the IEEE and the SPIE. He is an associate editor of Biomedical Engineering—Applications, Basis, and Communication, and sits on the editorial board of The Open Acoustics Journal.