Graph cuts is a method for separating foreground and background and is similar to max-flow min-cut theorem. The difference is that in graph cuts, each vertices has an edge (t-link) connected to source S, an edge connected to sink T and an edge (n-link) between neighboring vertices.
When we use graph cuts, the first step is to select foreground seeds and background seeds. And then we assign edge weights as capacity. The weight of n-link is according the difference of gray value or color between neighboring points. The weight of t-link is according the similarity to foreground seeds and background seeds. If the intensity of a node is like to foreground seeds than the weight of t-link connected to source S is big that means this node has more chance be classified as foreground and vise versa. Finally, according to the weights of t-link and n-link, we can get a cut to partition foreground and background by graph cuts algorithm. Figure1.11 demonstrates the graph cuts.
Short-cut is a well-known problem in graph cuts and mostly occur in thin and long region. Short-cut problem means that the cost of desired boundary is bigger than the short-cut path that misclassification happened. Figure1.12 is an example of the short-short-cut problem.
This problem can be solved by combining geodesic information and graph cuts algorithm [12].
Graph cut in general image and video
Based on [11], many applications of graph cuts were published. Whatever no methods are perfect, therefore, user interactive tools often come with graph cuts [13][14].
Lazy Snapping [13], the picture is pre-segmented by watershed and user only need to use mouse to stroke off about foreground and background regions to define the seeds.
Because the pre-segmented picture takes a watershed region as a node, therefore, nodes in
Figure 1.11: An example of graph cuts. (a) An Image with foreground/background seeds.
(b) The constructed graph for graph cuts. The blue lines are t-links and the green lines are n-links. The thickness presents the weight of each link. (c) According to the weight of t-link and n-link, graph cut algorithm will find an optimal path to segment foreground and background. (d) The classified image.
Foreground seeds Background seeds
S
2S
1Goal of boundary S1
Short-cut S2
Figure 1.12: Short-cut problem of graph cuts. S1 is the goal of boundary and S2 is the short-cut path. Although the edge weights of n-links on the goal of boundary boundary S1
are smaller than those on the short-cut path S2, however, the boundary S1 is to long cause the cost is bigger than short-cut S2. Therefore, graph cuts may result in short-cut S2instead of boundary S1.
Figure 1.13: Lazy Sanpping. (a)The original image. (b)User roughly define the foreground seeds (yellow line) and background seeds (blue line). (c) The blue polygon is the instant visual feedback and have UI tool to edit. (d) Synthesis the graph cuts result and another picture. (The figure is cited from [13].)
graph cuts is reduced. So Lazy Snapping can offer instant visual feedback and polygon tool to refine the result, as shown in Figure1.13. Furthermore, [15], [16], and [17] are also the segmentation methods using graph cuts and watershed.
Video Snap Cut [18] mentioned that graph cuts with global statistics can’t handle com-plexity scene therefore they segment with localized classifiers. The user will roughly define the foreground and background on key frame, and get a initial result. The localized clas-sifiers then surround along the initial result. Each localized clasclas-sifiers will combine local information to get a segmentation, as shown in Figure1.14. These localized classifiers will propagate to next frame based on optical flow and update their local statistic. For refining the result of graph cuts, [16] also provides a tool to compute local color distribution.
Graph cuts in medical image
Multilevel banded graph cuts (BGC) [19] and [21] are two graph cuts applications in medical image. Multilevel banded graph cuts (BGC) [19] run full grid graph cuts with the coarsest image and get an initial object , then refine the initial object in a coarse-to-fine
Figure 1.14: Video Snapping Cut. Illustration of local classifiers. (a) Overlapping clas-sifiers (yellow squares) are initialized along the object boundary (red curve) on frame t.
(b) These classifiers are then propagated onto the next frame by motion estimation. (c) Each classifier contains a local color model and shape model, they are initialized on frame t and updated on frame t+1. (d) Local classification results are then combined to generate a global foreground probability map. (e) The final segmented foreground object on frame t + 1. Original video courtesy of Artbeats. (The figure is cited from [18].)
Figure 1.15: Multilevel Banded Graph Cuts.
manner and graph cuts only operate in a band which between eroded and dilated the object, as shown in Figure1.15. In this way, we can decrease memory usage and time cost but might exclude some thin structures such as blood vessels showed in the experiment of 3-D heart and pulmonary artery CT volume. To tackle the problem of BGC, [20] presented a accurate multilevel BGC by identifying thin structures and adding to BGC system when fining the lower resolution object.
In brain extraction topic, [21] first finding white matter (WM) by region growing and regarding as foreground seeds. Second, decide a threshold mask as background seeds. key point is that the threshold need to cause narrow connection between brain and non-brain tissue. For the graph cuts result, if the threshold is too low, it will lead to residual non-brain. On the contrary, choosing a too high value will exclude brain volume, Figure 1.16
Figure 1.16: Illustration the influence of different threshold values on the quality of initial mask for skull stripping using graph cuts. Too low threshold (second column) leads to in-sufficient separation between brain and non-brain structures, high threshold (right column) results in brain loss. (The figure is cited from [21].)
show the selection of threshold. The disadvantage of this method is the threshold mask may contains some partial brain voxels, hence after apply graph cuts algorithm the method performing closing operation (10mm voxel dilation and 10mm voxel erosion), however we always don’t know how much brain volume is included in background seeds and how much volume need to be recovered.