中 華 大 學 碩 士 論 文
利用超像素分組進行模糊影像分割
Blur Image Segmentation using Iterative Super Pixels Grouping Method
系 所 別：資訊工程學系碩士班 學號姓名：M09802023 余冠霖 指導教授：連振昌 博士
中 華 民 國 101 年 8 月
摘要
影像分割技術為影像分析與影像識別的一項基礎技術。影像可以分割成許多相同 之色彩、強度、或紋理之區域。然而，對於邊界較模糊的圖片，傳統影像分割技術方 法有一些問題造成分割處理之不精確，即無法保證區域的連接性，精確地定義區域邊 界，以及決定分割區域之數目。在電腦視覺領域，超像素(Superpixel)已經成為不少 演算法的基礎組件。本篇提出以迭代超像素分組進行影像分割(segmentation)的方法，
利用超像素的特性，可以在初期便可消除模糊影像不準確的邊緣，進而將超像素分組 以達到影像分割的目地。
關鍵字：超像素、影像分割、模糊影像、超像素分組
Abstract
Image segmentation is a fundamental technology in image processing and image understanding. Images are partitioned into many regions with the same color, intensity, or texture homogeneity. However conventional image segmentation methods have some problems on the blur images that make the image segmentation inaccurate. The histogram-based segmentation methods can't ensure the region connectivity, especially on the blur images. The split-merge methods can't define the region boundaries accurately.
Recently, superpixels have become an essential and fundamental preprocess in many computer vision algorithms. By using the superpixels, the accurate region boundaries in the blur images can be obtained. However, the region completeness is still a problem to overcome. In this study, by extending the superpixel segmentation method, the method of Iterative Super Pixels Grouping (ISPG) is proposed to overcome the inaccurate segmentation problem and guarantee the region completeness on the blur images.
Furthermore, the proposed ISPG method can partition the image with an arbitrary number according to the region completeness measure. Experimental results show that the performance of ISPG method outperforms the conventional methods in terms of subjective and quantitative measures.
Keywords: Image segmentation, Superpixel, Iterative Super Pixels Grouping (ISPG)
致謝
首先感謝花費了無以言語形容的心力與辛苦的指導教授 連振昌博士。自大學時 代起，連老師不僅幫助與指導我許多專業領域知識，更在做人處事與態度上的身教和 言教，使學生我獲益良多、受用無窮。以及我的母親與弟弟，為我提供了強大後盾讓 我不必擔心家裡的大小事務。再來感謝與一路上互相照應的實驗室夥伴們：佩蓉、郁 偉、正達、琮瑋、永坤、文楷、珮筠、耀德、政揚、建翔、柏廷、子豪。
最後誠摯與衷心的感謝所有曾經或現在圍繞在我周圍的大家，謝謝你們的關心與 鼓勵。
Contents
摘要 ... 1
Abstract ... 2
致謝 ... 3
Contents ... 4
List of Tables ... 5
List of Figures ... 6
Chapter 1 Introduction ... 7
Chapter 2 Superpixels ... 11
Chapter 3 Iterative Super Pixels Grouping Method ... 14
3.1 Oversegmentation ... 15
3.2 Recursive Labeling ... 20
3.3 Best Match Merging ... 23
3.4 Annexation Merging ... 26
Chapter 4 Experimental Results ... 30
Chapter 5 Conclusion ... 35
Reference ... 36
List of Tables
Algorithm 1 SLIC superpixel segmentation ... 20
Algorithm 2 Labeling ... 21
Algorithm 3 Best Match Merging ... 26
Algorithm 4 Annexation Merging ... 28
Table 1 Accuracy for each method. The accuracy for each algorithm is calculated by averaging the result of 40 test images. ... 31
List of Figures
Fig. 1 An example of image segmentation. Image taken from The Berkeley Segmentation
Dataset and Benchmark[11] ... 8
Fig. 2 A smooth edge of an object ... 9
Fig. 3 Conventional segmentation methods applied on smooth edge, (a) is the original smooth edge, (b) is the result of thresholding, (c) is the result of Sobel edge detection and (d) is the result of K-means with k = 4. ... 10
Fig. 4 An example of pixels in a digital image, image taken from [16]. ... 11
Fig. 5 Examples of superpixel oversegmentation. (a) and (b) taken from The Berkeley Segmentation Dataset and Benchmark[11] ... 13
Fig. 6 Flow chart of ISPG ... 15
Fig. 7 An image oversegmented into superpixels using SLIC [22]. Left part of the image were segmentated to 200 superpixels, 400 in the middle and 800 in the right part of the image. ... 16
Fig. 8 Result of oversegmentation on a blur image ... 18
Fig. 9 Flow chart of SLIC ... 19
Fig. 10 Flow chart of Labeling ... 22
Fig. 11 Result of labeling on a blur images ... 23
Fig. 12 A best match ... 24
Fig. 13 Result of best match merging on a blur image ... 24
Fig. 14 Flow chart of Best Match Merging ... 25
Fig.15 Flow chart of Annexation Merging ... 27
Fig. 16 Result of annexation merging on a blur image ... 29
Fig. 17 Examples of three categories of image. (a) Blur image. (b) Normal image. (c) Low contrast image. ... 30
Fig. 18 An example of ground truth image. ... 31
Fig. 19 A blur image. (a) Original image. (b) The result of GS04 with sigma = 0.5, k = 500, min = 20. (c) The result of ISPG. ... 32
Fig. 20 Another blur image. (a) The original image, (b) The result of GS04 with sigma = 0.5, k = 500, min = 20. (c) The result of ISPG. ... 33 Fig. 21 Images with both smooth and sharp edges with highly textured background.
Column (a) are the original images, column (b) are the result of GS04 with sigma =
Chapter 1 Introduction
In computer vision, in order to analyze the image easily, a process of image segmentation must be introduced. The goal of segmentation is to simplify the representation of an image from pixel grid into something that is more meaningful, i.e. a set of regions or contours. Each pixel in the same region is similar with respect to some characteristic of computed property, such as color, intensity or texture. One or more regions may belong to an object in real world. Connected regions are significantly different with respect to the same characteristics. Image segmentation is typically used to locate objects and boundaries such as lines and curves in images. More precisely, image segmentation is the process of assigning a label to every pixel in an image such that pixels with the same label share certain visual characteristics. Fig. 1 illustrates the result of image segmentation in which a set of segmented regions that collectively cover the entire image, or a set of contours extracted from the image.
In general, the image segmentation methods can be categorized into the histogram-based methods [1], the cluster-based methods [2], and the boundary detection methods [3]. The simplest methods of image segmentation are histogram-based methods.
The key of these methods is to determine the threshold value with the predefined criteria.
For example, the Otsu's method [1] is the typical methods in the histogram-based methods.
In the cluster-based methods, k-means clustering algorithm [2] can partition the image into the specified k clusters. The concept is similar to the expectation-maximization algorithm to find the centers of clusters in the mixtures of Gaussian's models. The main drawback of the k-means algorithm is that the user needs to assign the number of clusters k.
An inappropriate choice of k may yield poor results[2, 4, 5]. Split-and-merge segmentation of quadtree segmentation methods [6, 7, 8] are based on the quadtree partition of an image.
This method starts at the root of the tree that represents the whole image. If the
non-uniform nodes are found, then they are split into four child nodes. Conversely, if four child nodes are homogeneous, they can be merged as a single node.
For the boundary detection methods, edge detection algorithms can be applied as the region segmentation methods. To segment an object from an image requires the closed region boundaries but, the edges identified by edge detection are often disconnected.
Lindeberg and Li [9] developed an integrated method that segments edges into straight and curved edge segments for parts-based object recognition. In [9], a minimum description length (MDL) criterion is optimized by a split-and-merge-like method with candidate breakpoints obtained from complementary junction cues to obtain more likely points at which to consider partitions into different segments. The idea of watershed [10] is to place water sources in each regional minimum, floods the valleys from sources, and builds the barriers when different sources are meeting.
Fig. 1 An example of image segmentation. Image taken from The Berkeley Segmentation Dataset and Benchmark[11]
However, the conventional image segmentation methods have some problems on blur images that make the image segmentation process inaccurate. The histogram-based segmentation methods can't ensure the region connectivity, especially on blur images. The split-merge methods can't define the region boundaries accurately [12]. A blur image is often generated by the defocus aberration. Fig. 2 shows an example of a blur image, a smooth edge may cause the above mentioned methods failed. Fig. 3 shows a smooth edge segmented by conventional approaches. The thresholding method can extract the inaccurate boundary. A lot of noisy regions generated from the Sobel edge detection [13]
show that the boundary detection can’t be used to segment the blur image. However, clustering methods, e.g., K-Means algorithm, can partition the image into many regions including the redundant regions because of the inappropriate choice of cluster number.
Fig. 2 A smooth edge of an object
Inspired by the clustering algorithms, the method of Iterative Super Pixels Grouping (ISPG) is proposed here to overcome the inaccurate region segmentation problems. First, the image is oversegmented into superpixels. By using the superpixels, the inaccurately boundaries especially on blur images can be eliminated. Second, to achieve the region completeness, the superpixel merging with respect to mean intensity or other effective
features is performed. This can ensure the connectivity of the regions. And because of this, ISPG is able to partition the image with an arbitrary number while merging steps here need parameters of smallest superpixel size and distance threshold but not the number of segments on the image.
(a) (b)
(c) (d)
Fig. 3 Conventional segmentation methods applied on smooth edge, (a) is the original smooth edge, (b) is the result of thresholding, (c) is the result of Sobel edge detection and
(d) is the result of K-means with k = 4.
Chapter 2 Superpixels
Many existing algorithms in computer vision use the pixel-grid as the underlying representation. Fig. 4 illustrates a pixel grid of a digital image. However, the pixel-grid is not a natural representation rather an artifact of a digital imaging process. It would be more natural, and presumably more efficient, to work with perceptually meaningful entities obtained from a low-level grouping process. [14, 15]
Fig. 4 An example of pixels in a digital image, image taken from [16].
Superpixels [17] are homogeneous image regions comprised of multiple contiguous pixels. They are obtained by oversegmentating the image. Superpixel algorithms group pixels into perceptually meaningful atomic regions, which can be used to replace the rigid structure of the pixel grid. Ideally, every pixel within each superpixel region will most likely belong to the same real-world object. Each superpixel is a perceptually consistent unit; all pixels in a superpixel are most likely uniform in color and/or texture. Therefore the superpixels are perceptually meaningful. Superpixel representations can improve the computational efficiency for the further merging process; which depends on the spatial complexity. They can reduce the number of distinct spectra to analyze by one or more
orders of magnitude.
Superpixel map reduces the complexity of images from millions of pixels to hundreds of superpixels, and then the objects in the image can be represented more efficiently and meaningfully. Meanwhile, because of the superpixels are results of an oversegmentation, most structures such as the object boundary in the image are conserved. There is very little distortion in moving from the pixel-grid to the superpixel map.
There are many existed approaches [18, 19, 20, 21, 22] can generate superpixels. Each of them has its own advantages and drawbacks that may be better suited to a particular application. For example, if obejct boundary is of paramount importance, the graph-based method of [18] may be an appropriate choice. However, if superpixels are used to construct a graph representation, the methods that can generate a more regular lattice, such as [19, 20], are probably good choices. For many computer vision applications, compact and highly uniform superpixels that respect to image boundaries are highly demanded. For instance, graph-based models such as Conditional Random Fields (CRF) can dramaticaly speed up the computering when switching from pixel-based graphs to superpixel-based graph [21, 22], but irregular superpixels can degrade the performance. Local features such as SIFT extracted from the image at superpixel locations become less meaningful and discriminative if the superpixels are loose or irregular, and learning statistics over cliques of two or more superpixels can be unreliable[22]. The simple linear iterative clustering (SLIC) [22] results effective performance at a lower computational cost in comparison to existing methods [23]. Fig. 5 shows the typical superpixel maps. It is obvious that the object boundaries in the image are already included in the boundaries of related superpixels.
(a) (b)
(c) (d)
Fig. 5 Examples of superpixel oversegmentation. (a) and (b) taken from The Berkeley Segmentation Dataset and Benchmark[11]
Chapter 3 Iterative Super Pixels Grouping Method
In this study, by extending the superpixel segmentation method, the method of Iterative Super Pixels Grouping (ISPG) is proposed to overcome the inaccurate segmentation problem and guarantee the region completeness on the blur images. The proposed method consists of four steps. First, the oversegmentation is applied to the input image and the superpixel map is then generated. Second, a recursive connected component labeling process with a distance threshold Td is used to merge the superpixels on the superpixel map. This step will roughly merge the superpixels that are connected and most like belong to the same region. Third, the best match searching is design to find and merge best matches of superpixels that are smaller than the area threshold Ta iteratively until there is no best match on the superpixel map. A pair of best match is determined by searching the lowest distance among connected superpixel and the must be the superpixels closest to each other and both of them must be smaller than Ta in size. The final step acts like a clean-up process. Iteratively check and merge small superpixels that connected to the ones that are larger than Ta until all superpixels in the superpixel map is larger than Ta. After all steps were done, we transform the superpixel map into a label map and the label map is the result of ISPG segmentation. Fig. 6 describes the flow-chart of the ISPG algorithm.
Input image
Output Oversegmentation
Best Match Merging
Superpixel Labeling
Annexation Merging
Fig. 6 Flow chart of ISPG
3.1 Oversegmentation
A modified Simple Linear Iterative Clustering (SLIC) [22] is used in this study to construct the superpixel map because of the low computational and memory cost. It needs only the number of desired superpixels as the input parameter. Compared to other state-of-the-art algorithms, SLIC obtain better quality and higher computational efficiency [23]. Fig. 7 shows the output of SLIC superpixel in different number of superpixels on the image. It can be seen that the boundaries of image objects were covered by oundaries of superpixels.
Fig. 7 An image oversegmented into superpixels using SLIC [22]. Left part of the image were segmentated to 200 superpixels, 400 in the middle and 800 in the right part of the
image.
SLIC uses CIELAB color space. Only one parameter is required for the SLIC algorithm, i.e., the number of superpixels K. The initialization step begins with sampling K initial cluster centers 𝐶𝑘 = [𝑙𝑘, 𝑎𝑘, 𝑏𝑘, 𝑥𝑘, 𝑦𝑘]𝑇 with 𝑘 = [1, 𝐾] on a regular grid spaced with S pixels. The grid interval 𝑆 = √𝑁𝐾, where N is the total number of pixels in the image.
The centers are then moved to locations of the lowest gradient position in a 3 × 3 neighborhood to avoid placing a center on an edge or a noise pixel. Image gradients are computed as:
𝐺(𝑥, 𝑦) = ‖𝑰(𝑥 + 1, 𝑦) − 𝑰(𝑥 − 1, 𝑦)‖2+ ‖𝑰(𝑥, 𝑦 + 1) − 𝑰(𝑥, 𝑦 − 1)‖2 (1) where, I(x, y) is the LAB color vector corresponding to the pixel at position (x, y), and
‖ ‖2 is the L2 norm. Each center has a search region of 2𝑆 × 2𝑆. Next, in the assignment step, each pixel i is associated with the nearest cluster center whose search region overlaps its location with distance measure D. The distance measure D is defined as follows:
𝑑𝑙𝑎𝑏 = √(𝑙𝑘− 𝑙𝑖)2+ (𝑎𝑘− 𝑎𝑖)2+ (𝑏𝑘− 𝑏𝑖)2
𝑑𝑥𝑦 = √(𝑥𝑘− 𝑥𝑖)2+ (𝑦𝑘− 𝑦𝑖)2 (2)
𝐷 = 𝑑𝑙𝑎𝑏 +𝑚𝑆 𝑑𝑥𝑦
where, Dlab is the distance of LAB color space and Dxy is the distance in xy plane. D is the sum of lab distance and the xy plane distance normalized by the grid interval 𝑆. A variable m is introduced in D allowed to control the compactness of a superpixel. The greater of m,
the more spatial proximity is emphasized and the more compact of the cluster. After all the pixels are associated with the nearest cluster center, the new cluster centers are computed as the average labxy vector of all the pixels belonging to the cluster. Repeat the process of associating pixels with the nearest cluster center and recompute the cluster center iteratively until the moving of cluster centers converges. A few stray labels may remain at the end of this process, i.e., a few pixels in the vicinity of a larger region having the same label but not connected to it. The reason introducing this rare situation is that the clustering process does not explicitly enforce the pixel connectivity. This problem can be overcome by enforcing the connectivity in the last step of SLIC by relabeling disjoint regions with the labels of the largest neighboring cluster. Fig.8 is the result of oversegmentation on a test image. Algorithm 1 summarized the procedures of the SLIC oversegmentation and Fig.
9 shows the flow chart of SLIC algorithm.
In the following, we extract the features including mean intensity, area, and neighboring superpixels for further superpixel merging process. The feature vector is defined as:
𝑆 = [ 𝑘, 𝑘, 𝑆 𝐶𝑘]𝑇 (3) where, Ik is the feature (mean intensity) superpixel 𝑆 𝑘, Ak is the area of the superpixel SPk, and SPCk is the set whose members are superpixels connected to SPk.
Fig. 8 Result of oversegmentation on a blur image
Start
End Set centers with
step interval S
Assign pixels to each centers by distance measure D and
search area of 2Sx2S Move centers to lowest
gradient position in 3x3 grid
Calculate new centers
E > threshold
Max iteration reached
Yes
Yes
No No
Fig. 9 Flow chart of SLIC
Algorithm 1 SLIC superpixel segmentation
01: Place cluster centers 𝐶𝑘= [𝑙𝑘, 𝑎𝑘, 𝑏𝑘, 𝑥𝑘, 𝑦𝑘]𝑇 at regular grid steps interval 𝑆.
02: Move cluster centers to the lowest gradient position in a 3 × 3 neighborhood.
03: repeat
04: for each cluster center 𝐶𝑘 do
05: Assign pixels to the closest cluster centers according to the distance measure from a 2𝑆 × 2𝑆 square neighborhood around the cluster center.
06: end for
07: Compute new cluster centers and residual error 𝐸 {L1 distance between previous centers and recomputed centers}
08: Until E < threshold 09: Enforce connectivity.
3.2 Recursive Labeling
To ensure the region completeness we begin to merge the superpixels with the close features. Here, we apply the recursive connected component labeling algorithm associated with the neighboring superpixel information SPCk to merge the superpixels with the close features recursively. The recursive labeling is summarized in Algorithm 2 and Fig. 10. The recursive connected component labeling algorithm starts with any one of the labeled superpixels, mark a label to the current superpixel, and check the connected ones if their distance is less than Td. Mark the same label to connected superpixels if their distance is less than Td. Queue and do the same operations on all superpixels that had been marked the same label until the region of the same label is fully growth. Then move to next unmarked superpixel and do the same operations recursively until all superpixels in the superpixel map are marked. Here, Fig. 11 shows the result of recursive labeling for the superpixels on Fig. 8.
Algorithm 2 Labeling
01: set all superpixel Labels to 0 02: set CurrentLabel to 1
03: initial a Queue
04: for each superpixel 𝑆 𝑘 do 05: if label of 𝑆 𝑘 != 0 then
06: continue
07: enqueue 𝑆 𝑘
08: repeat
09: for each superpixel 𝑆 𝑞 in the Queue do 10: set Label of 𝑆 𝑞 to CurrentLabel
11: for each superpixel 𝑆 𝑐 that connected to 𝑆 𝑞 do
12: if Label of 𝑆 𝑐 == 0 then
13: continue
14: if distance of 𝑆 𝑐 to 𝑆 𝑞 < 𝑇𝑑 then
15: enqueue 𝑆 𝑐
16: end for
17: end for
18: until Queue is empty 19: CurrentLabel ++
20: end for
Start
End
Set CurrentLabel to 1
Next unlabeled superpixel
Label to CurrentLabel
Find superpixels connected to CurrentLabel superpixels with
distance < Td
Found
More unlabeled superpixels
No
Yes
No Yes
CurrentLabel + 1
Fig. 10 Flow chart of Labeling
Fig. 11 Result of labeling on a blur images
3.3 Best Match Merging
After the recursive labeling process, the neighboring superpixels with close features have been merged together. However, there still have some superpixels with small distance have not been dealt with yet. The idea behind the best match merging process is to search the best match neighboring superpixels in terms of feature distance measure and find new potenial regions that contain several small superpixels. First, to search the superpixel or regions whose area is smaller than the area threshold Ta. A best match pair is a pair of connected superpixel SPa and SPb with SPb is the closest superpixel among all superpixels connected to SPa, and same from SPa to SPb. The pair of best match must satisfy the following conditions:
1. SPa and SPb are connected
2. SPa and SPb are both smaller than Ta
3. SPa is the closest superpixel connected to SPb
4. SPb is the closest superpixel connected to SPa
Fig. 12 shows a best match of superpixels A and B, the numbers in the superpixels on the
image denote the mean intensity to superpixels. Hence, the second step is to find best match pairs by searching the closest superpixels among all superpixels that have area smaller than Ta and then check and merge best matches. Loop back to find best matches again until there is no best matches exist in the superpixel map. A demonstrate of best match merging is showed in Fig. 13. We can observe that the strip in the middle of the image almost forms a complete region. The best match merging process is summarized in Algorithm 3 and Fig. 14.
Fig. 12 A best match
Fig. 13 Result of best match merging on a blur image
Start
Find and Merge Best Matches
Changed Initialize a List
Add all superpixels size < Ta
to the List
List Empty
End No
No
Yes
Yes
Fig. 14 Flow chart of Best Match Merging
Algorithm 3 Best Match Merging 01: repeat
02: initialize List 𝑆 𝑙𝑖𝑠𝑡
03: for each superpixel 𝑆 𝑘 do 04: if 𝑘 < 𝑇𝑎 then 05: add 𝑆 𝑘 to 𝑆 𝑙𝑖𝑠𝑡 06: end for
07: if 𝑆 𝑙𝑖𝑠𝑡 ≤ 0 then
08: break
09: for each superpixel 𝑆 𝑘 ∈ 𝑆 𝑙𝑖𝑠𝑡 do
10: find superpixel 𝑆 𝑡 in 𝑆 𝐶𝑘 that is closest to 𝑆 𝑘 11: if 𝑆 𝑡 ∈ 𝑆 𝑙𝑖𝑠𝑡 then
12: find superpixel 𝑆 𝑡2 ∈ 𝑆 𝐶𝑡 that is closest to 𝑆 𝑡 13: if 𝑆 𝑡2 ∈ 𝑆 𝑙𝑖𝑠𝑡 then
14: if 𝑆 𝑡2 == 𝑆 𝑘 then
15: merge 𝑆 𝑡 and 𝑆 𝑘
16: end if
17: end for
18: if any single pair of superpixels were merged then
19: break
20: loop
3.4 Annexation Merging
In Fig. 13, we can observe that there still have some small regions are not merged because of the feature distance measures are not satisfied to the criteria of “recursive labeling” and “best match merge”. However, this situation degrades the region completebess. Hence, the final merge process in the ISPG called “annexation merging” is proposed to merge these untreated superpixels or regions. The concept of annexation merging is to make those regions larger than Ta to annex remaining connected superpixels
smaller than Ta. This step acts like a clean-up process; the main goal is to clean the remaining smaller superpixels that could be noise or regions that small enough to be ignored. Check all regions smaller than Ta if they are connected to any larger regions;
merge them to the connected regions. If a smaller region is connected to two or more larger regions, merge to the region that is closest in distance. Loop back until there is no any region has area smaller than Ta. The annexation merging process is summarized in Algorithm 4 and Fig. 15. Fig. 16 is the result of annexation merging.
Start
End
Find all small superpixels connected to large
superpixels
Merge them to closest large superpixels
connected
Small superpixels remaining
No Yes
Fig.15 Flow chart of Annexation Merging
Algorithm 4 Annexation Merging 01: repeat
02: initialize List 𝑆 𝑙𝑖𝑠𝑡
03: for each superpixel 𝑆 𝑘 do 04: if 𝑘 < 𝑇𝑎 then 05: add 𝑆 𝑘 to 𝑆 𝑙𝑖𝑠𝑡 06: end for
07: if 𝑆 𝑙𝑖𝑠𝑡 ≤ 0 then
08: break
09: for each superpixel 𝑆 𝑘 ∈ 𝑆 𝑙𝑖𝑠𝑡 do 10: initialize List 𝑆 𝑙𝑖𝑠𝑡_𝑙𝑎𝑟𝑔𝑒𝑟 11: for each superpixel 𝑆 𝑡 ∈ 𝑆 𝐶𝑘 12: if 𝑡 ≥ 𝑇𝑎 then
13: add 𝑆 𝑡 to 𝑆 𝑙𝑖𝑠𝑡_𝑙𝑎𝑟𝑔𝑒𝑟
14: end for
15: if 𝑆 𝑙𝑖𝑠𝑡_𝑙𝑎𝑟𝑔𝑒𝑟 ≤ 0 then
16: continue
17: else
18: merge 𝑆 𝑘 and closest superpixel in 𝑆 𝑙𝑖𝑠𝑡_𝑙𝑎𝑟𝑔𝑒𝑟
19: end if
20: end for 21: loop
Fig. 16 Result of annexation merging on a blur image
Chapter 4 Experimental Results
To analyze the performance of ISPG algorithm three kinds of transmission electron microscopy (TEM) images are segmented and the segmentation accuracy is compare to other methods. The size of these images varies from 59 × 78 to 262 × 252. According the blur degree, we classify the test images into three categories: blur images, normal images(images with both smooth and sharp edge) and low contrast images. Fig. 17 shows the three categories of images.
(a) (b) (c)
Fig. 17 Examples of three categories of image. (a) Blur image. (b) Normal image. (c) Low contrast image.
We compared this approach with a graph-based image segmentation algorithm (GS04) [18]. The Parameters of GS04 were set to the typical values, sigma = 0.5, k = 500, min = 20, where the sigma is used to smooth the input image before segmentation, larger values
for k results in larger components in the result and min is the minimum component size enforced by post-processing.
The quantitative performance analysis is evaluated by calculating ratio of the correct
segmented area to the ground truth area for the regions in the test image, which is defined as:
𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = 𝑁1
𝑟∑ 𝐴𝑟𝑒𝑎𝑆𝑒𝑔𝑖
𝐴𝑟𝑒𝑎𝐺𝑇𝑖 𝑁𝑟
𝑖=0 (4)
where, Nr is the total number of regions in ground truth, 𝑟𝑒𝑎𝐺𝑇𝑖 is the area of ith ground truth region and 𝑟𝑒𝑎𝑆𝑒𝑔𝑖 is the area of ith segmented region. The ground truth for each region is marked manually. Fig. 18 shows an example of ground truth image. Table 1 lists the comparison of the segmentation accuracy between the ISPG and GS04. The accuracy for each algorithm is calculated by averaging the result of 40 test images. It is obvious that our proposed method outperform the GS04 method for the three kinds of test images.
Fig. 18 An example of ground truth image.
Table 1 Accuracy for each method. The accuracy for each algorithm is calculated by averaging the result of 40 test images.
Blur images Normal images Low contrast
ISPG 93.31% 87.92% 53.47%
GS04 85.76% 78.44% 41.78%
Fig. 19 and Fig. 20 show blur images which result best quality of segmentation. GS04 results in acceptable output but the blur boundaries make it loss at some points, note the gap in the middle of the sphere in Fig. 19(b) and some noise-like segments in Fig. 20(b).
(a)
(b) (c)
Fig. 19 A blur image. (a) Original image. (b) The result of GS04 with sigma = 0.5, k = 500, min = 20. (c) The result of ISPG.
(a)
(b) (c)
Fig. 20 Another blur image. (a) The original image, (b) The result of GS04 with sigma = 0.5, k = 500, min = 20. (c) The result of ISPG.
Fig. 21 shows normal images that have both smooth and sharp edges with highly textured background. GS04 extracted too much boundaries on smooth edges and background while ISPG did well. This is because the first step of IPSG, superpixel clustering, had already extracted relatively accuracy boundary.
(a) (b) (c)
Fig. 21 Images with both smooth and sharp edges with highly textured background.
Column (a) are the original images, column (b) are the result of GS04 with sigma = 0.5, k
= 500, min = 20 and column (c) is the result of ISPG
Chapter 5 Conclusion
Superpixels have become an essential tool to the vision community and key building blocks of many computer vision algorithms. In this work, we proposed a simple and easy to implement approach to segment images with boundary more blur than natural images.
By using superpixels, the inaccurately boundaries especially on blur images can be eliminated in early stage of creating super pixels since the result of an oversegmentation contains necessary boundaries and redundant boundaries. To deal with boundaries that we are not interested, the rest of steps are merging steps that merger super pixels with respect to simple mean intensity or other even complex or effective features. This can ensure the connectivity of the regions. And because of this ISPG is able to partition the image with an arbitrary number while merging steps here need parameters of smallest superpixel size and distance threshold but not the number of segments on the image. This showed the use of superpixels on blur images can be effective and accuracy. This approach can still be improved with distance measure on some other features not just intensity. It is hoped that the study will stimulate further study in superpixels and image segmentations.
Reference
[1] Nobuyuki Otsu, “A threshold selection method from gray-level histograms,” IEEE Transactions on Systems, Man and Cybernetics, Vol. 9, Issue 1, pp 62-66 January 1979.
[2] S. P. Lloyd, "Least square quantization in PCM," IEEE Transactions on Information Theory, Vol 28, No. 2, pp. 129-137, March 1982.
[3] T. Lindeberg and M.-X. Li "Segmentation and classification of edges using minimum description length approximation and complementary junction cues", Computer Vision and Image Understanding, Vol. 67, No. 1, pp. 88-98, 1997.
[4] http://en.wikipedia.org/wiki/Segmentation_(image_processing) [5] http://en.wikipedia.org/wiki/K-means
[6] S. L. Horowitz and T. Pavlidis, "Picture Segmentation by a Directed Split and Merge Procedure", Proceedings of the 2nd International Joint Conference on Pattern Recognition, Copenhagen, Denmark, pp. 424-433, 1974.
[7] S. L. Horowitz and T. Pavlidis, "Picture Segmentation by a Tree Traversal Algorithm", Journal of the ACM, Vol. 23, No. 2, pp. 368-388, April 1976.
[8] http://en.wikipedia.org/wiki/Quadtree
[9] T. Lindeberg and M.-X. Li "Segmentation and classification of edges using minimum description length approximation and complementary junction cues", Computer Vision and Image Understanding, Vol. 67, No. 1, pp. 88-98, 1997.
[10] S. Beucher and C. Lantuejoul, "Use of watersheds in contour detection", International Workshop on Image Processing: Real-time Edge and Motion Detection/Estimation, Rennes, France, 1979
[11] D. Martin and C. Fowlkes and D. Tal and J. Malik, "A Database of Human Segmented Natural Images and its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics", Proceedings of the 8th International Conference of Computer Vision, Vol. 2, pp 416-423, July 2001
[12] Chien-Hao Chen and Cheng-Chang Lien, “The Multiresolution and Flooding based RSST (MFRSST) Image Segmentation Method,” Chung Hua Journal of Science and Engineering, Vol. 1, No. 1, pp. 9-16, March 2003.
[13] http://en.wikipedia.org/wiki/Sobel_operator
[14] Xiaofeng Ren and Jitendra Malik, “Learning a Classification Model for Segmentation,” Proceedings of the 9th IEEE International Conference on Computer Vision, Vol. 2 Page 10-17, 2003
[15] http://ttic.uchicago.edu/~xren/research/superpixel/
[16] http://en.wikipedia.org/wiki/File:Pixel-example.png
[17] D. R. Thompson, L Mandrake, M. S. Gilmore and R. Castaño "Superpixel Endmember Detection", IEEE Transactions on Geoscience and Remote Sensing, Vol 48 ,Issue 11, pp 4023-4033, November 2010
[18] P. F. Felzenszwalb and D. P. Huttenlocher, “Efficient Graph-Based Image Segmentation,” International Journal of Computer Vision, Vol. 59, No. 2, September 2004.
[19] J. Shi and J. Malik, “Normalized cuts and image segmentation,” IEEE Transcations on Pattern Analyses and Machine Intelligence, Vol. 22, Issue. 8, pp. 888-905, August 2000.
[20] A. P. Moore, S. J. D. Prince, J. Warrell, U. Mohammed and G. Jones, “Superpixel Lattices,” IEEE Conference on Computer Vision and Pattern Recognition, 2008, pp 1-8, June 2008.
[21] B. Fulkerson, A. Vedaldi, S. Soatto, "Class segmentation and object localization with superpixel neighborhoods", IEEE 12th International Conference on Computer Vision, pp. 670-677, September 2009
[22] A. Radhakrishna, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Susstrunk, “SLIC Superpixels,” Technical Report 149300, EPFL, June 2010.
[23] R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Susstrunk, “SLIC Superpixels Compared to State-of-the-art Superpixel Methods,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Issue 99 Page 1.
