Evaluation Methods for Image Segmentation

Color segmentation is a crucial step in image analysis and pattern recognition. The performance of color segmentation may significantly affect the quality of an image understanding system. So far, hundreds of color segmentation algorithms have already been developed to deal with various kinds of image-related applications [13][41]. For these color segmentation algorithms, the automatic setting of controlling parameters is usually a difficult task. Currently, these control parameters are often adjusted by the users in an interactive and tiresome manner. Moreover, the selection of control parameters is also image-dependent. For most color segmentation algorithms, there exists no parameter setting that is universally applicable.

On the other hand, it is well known that performance evaluation of segmentation algorithms is critical and essential in the development of image understanding systems. However, as compared with the tremendous efforts spent in the development of segmentation algorithms, relatively fewer efforts have been made on the subject of image segmentation evaluation [42][43][44][45][46]. As shown in Fig. 2.3, Zhang classified existing evaluation methods for image segmentation into three categories [42][43]: 1) analytical methods; 2) discrepancy methods; and 3) goodness methods.

Analytical methods directly evaluate segmentation algorithms by analyzing their principles, requirements, utilities and complexity, etc [42][43]. Due to the lack of a general theory for image segmentation, analytical methods work well only for some particular models or for some desirable properties of the algorithms. Moreover, these analytical methods themselves are seldom used alone. On the contrary, both discrepancy methods and goodness methods evaluate the performance of segmentation by judging the quality of segmentation results directly. Especially, discrepancy methods measure the difference between the segmentation result and a

reference result, which is usually an expected result or a ground truth [47][48]. On the other hand, as illustrated in Fig. 2.3, goodness methods evaluate the segmentation results directly with certain quality measures, without the use of any reference result.

Fig. 2.3 Approaches for Evaluating Image Segmentation [42][43].

Among these three evaluation categories, the third type of methods, the goodness methods, is considered a more practical approach due to its direct evaluation of segmentation results without any user-dependent ground truth [49][50]. In this dissertation, we’ll focus on this type of evaluation methods. Moreover, we believe that, some evaluation principles and formulas in these evaluation methods can be used to facilitate the development of segmentation algorithm. Hence, some goodness measures that had been proposed before are to be used in the development of a segmentation algorithm in this dissertation.

So far, several goodness methods have already been proposed [42][43][49][50].

One type of method is to evaluate segmented results with the use of the “Peak Signal to Noise Ratio” (PSNR) function, which is defined as follows:

( )

( )

where I is an N×M color image f(x,y), and f’(x,y) is denoted as the segmented result, with the color of each segmented region being filled with the average color of that region. Then, with a specified region number, the desired segmented results are defined to be the results with the maximal PSNR value, or equally with the minimum MSE value. In general, this type of method tends to measure the within-region color difference between the original color image and the segmented result. However, due to the lack of consideration regarding the color contrast between adjacent regions, this type of method tends to prefer over-segmented results, which include many small regions. This is because a larger value of MSE would be expected when we merge several small regions into a large one.

To avoid producing overly segmented results, the factor of region area is considered in the approach proposed by Liu and Yang [49]. In their approach, an evaluation function named F function is defined as:

( ) ( ) ∑

where I is the image to be segmented, R is the number of regions in the segmented image, ei is the color error of the ith region, Ai is the area of the ith region, and N, M represent the length and width of the image. Here, ei is defined as the sum of the Euclidean distance in the RGB color space between the color vectors in the original image and the color vectors in the segmented image, in the ith region. Consequently, with a smaller F-function value, the segmented result is regarded to be better.

Although these methods take the factor of region area into account to avoid the

segmented results from including too many small regions, the relation between color difference and region area is not clearly treated. Hence, these F-function based methods may include both over-segmented regions and under-segmented regions at the same time. Furthermore, since these evaluation functions only consider the color differences within regions but ignore the color contrast between regions, they may cause some undesired circumstances. For example, the segmented result with each pixel being labeled as an independent region may be treated as a “perfect” result without any color difference error.

Based on Equation (2.21), two further improved evaluation functions are proposed in [50] that are defined as:

( ) ( ) ∑ [ ( ) ] ∑

where R(Ai) represents the number of regions with the area size Ai. In both equations, the areas of regions are considered to punish these segmentation results with too many small regions. Similarly, the number of segmented regions is also included, aiming to achieve segmented results with an appropriate number of homogeneous regions.

For the above evaluation functions, two primary requirements are adopted to define “preferred” segmentation results: smaller color difference and fewer segmented regions. However, color difference and the number of segmented regions are two very different measures in their physical meanings. The trade-off between these two measures would be very difficult. Moreover, the preferred number of segmented regions could be very different from image to image. When this image-dependent measure is involved in a single evaluation function, it would be rather difficult to

perform segmentation evaluation without having the prior knowledge of the image contents in advance.

In summary, although several goodness functions have already been proposed, most of them are not directly based on human visual perception. Instead, most goodness methods combine several existing measures together to formulate their evaluation functions. The selection and combination of measures are usually subjective. The adjustment of weighting coefficients is often troublesome. Moreover, for most evaluation methods, the quality of segmentation is usually represented in a single function, which mixes together several weakly related measures. Without knowing the erroneous information about the segmented result under evaluation, these approaches could be very unreliable.

CHAPTER 3

Visual Inspection for Mura on LCDs Based on Luminance Contrast

______________________________________________

3.1 Introduction of Automatic Inspection for Mura on LCDs

Recently, Liquid Crystal Displays (LCDs) have received increasing market attention because of the decreasing prices and the improved visual quality. Among several characteristics regarding the visual quality of LCDs, the front-of-screen (FOS) quality of LCDs is essential. Most existing FOS quality tests depend on the perception of human eyes. However, human inspection needs higher labor power and usually causes inefficient and inconsistent inspection. Instead of human inspection, automatic inspection that employs efficient algorithms over photographed images could be a reasonable and reliable way to evaluate the FOS quality of LCDs.

So far, several efforts have been spent on the fulfillment of defect classification

and the establishment of quality evaluation standards [2][9]. For example, the Video Electronics Standards Association (VESA) has been carrying activities related to the standards for classifying defects [9], and the Semiconductor Equipment and Materials International (SEMI) has spent a lot of efforts on the standardization of defect quantification [2]. However, even though there is a high demand of automatic inspection of FOS quality, there are relatively few efforts spent on the development of defect detection algorithms [3][10][11][12].

Fig. 3.1 Cross-section of an active matrix TFT-LCD.

In this chapter, we focus on the development of detection algorithms for the inspection of the FOS quality of an active matrix thin-film transistor liquid crystal display (AM TFT-LCD). Fig. 3.1 illustrates the cross-section of an AM TFT-LCD.

Basically, an LCD display includes two essential components: Cell Unit and Backlight Unit. In the cell unit, there are mainly five elements: liquid crystal, thin-film-transistor (TFT) array, color filters, glasses, and polarizer. In general, the functionality of cell unit is to make the RGB color switching at each pixel controllable.

On the other hand, in the backlight unit, there are four basic elements: lamp, light pipe,

reflective film and optical film. Generally, the functionality of backlight unit is to produce uniform light.

In the inspection of FOS quality, the so-called Mura defects greatly influence the FOS quality [9]. The Mura defects are defined as the visible imperfections of the FOS of a display screen in active use. Mura defects usually appear as a low contrast, non-uniform brightness regions, typically larger than a single pixel [2]. As shown in Table 3.1 Mori et al [10] listed several causes of Mura defects in TFT-LCD. Usually, the manufacturing performance of each component in the cell unit or in the backlight unit would affect the appearance of Mura defects. A superior manufacture process produces fewer Mura defects. Usually, the non-uniformity in various kinds of components induces different kinds of Mura defects.

Table 3.1 Causes of Mura defects on TFT-LCD [10]

Basic Unit Causes of Mura

(1) Non-uniform gap between glasses (2) Non-uniform color of color filter (3) Non-uniform density of liquid crystal Cell unit

(4) Non-uniform thickness of TFT array layer (5) Wrinkled optical filter

Backlight unit (6) Non-uniform lamp’s rays

3.1.1 SEMU Formula Based on Just Noticeable Difference

As mentioned above, Mura indicates the defects as the visible imperfections on the FOS of a display screen. To make standard for the quantification of Mura phenomenon, a committee in SEMI defines a measurement index for Mura in the inspection of FPD (Flat Panel Display) image quality. To define Mura, an ergonomic approach is used to investigate human eye’s sensitivity with respect to Mura by exploring the relation between Mura’s area and contrast. Also, an index, named SEMU (Semi Mura), is defined to express the degree of Mura defects. This SEMU index is defined to be the ratio of the target’s contrast over the contrast at JND. To deduce the SEMU formula, a subjective experiment had been conducted in [3], as shown in Fig. 3.2. In this experiment, a special LCD panel was produced to display 256 gradations in the luminance range between 43 cd/m² and 54 cd/m². A program created synthesized Mura and displayed the Mura at the center of the LCD panel. The observer looked at the screen with both eyes without any artificial pupil. The viewing direction is normal to the center of the LCD panel and the viewing distance is 500 mm. The experiments were performed in a darkroom and the observer could freely adjust the luminance of the displayed Mura by using a numeric keypad. The test began with the gray level of the synthesized Mura set to the same level as that of the background. Then, the observer adjusted the luminance of the Mura to the level where the Mura region was just perceptible and also to another level where the Mura can be clearly detected. The contrast values of the synthesized Mura at both levels were recorded. The contrast at the first level provided the data for the JND (Just Noticeable Difference) contrast, while the contrast at the second level provided the data for the

“Dist (Distinct difference)” contrast.

L + ∆L L + ∆L Fig. 3.2 Experimental conditions and equipment used in the subjective evaluations in [3].

Assume the background luminance is denoted as L and the Mura luminance is denoted as L + ∆L. In this experiment, the contrast of the Mura was defined to be the value of ∆L/L. Various sizes of circle-type and rectangle-type Muras were used in the experiments. The observers include 8 experts, who regularly conduct Mura inspections and analysis, and 8 college students with normal vision (with near vision strength (500mm) of 1.2 or greater for both eyes).

Fig. 3.3 shows the experimental results for Muras larger than 1 pixel, where the horizontal axis is 1/A^0.33 and the vertical axis is the value of contrast. The observers’

JND data are plotted as “ •”, while the Dist data are plotted as “ο”. The bold line was determined by the linear regression of the JND data. This line indicates a strong correlation between the size of Mura and the perceived contrast value of Mura. In this experiment, the larger the area of the Mura is, the smaller the JND contrast becomes.

In other words, Muras are more visible as their area becomes larger. In mathematics, the linear relationship between JND contrast (Cjnd) and A^0.33 was expressed in [3] as

Cjnd = 1.97/A^0.33 + 0.72. (3.1) This equation indicates the important characteristics of the “just noticeable

difference” for LCD Muras.

Fig. 3.3 JND and Dist result of the subjective evaluation in [3].

f(A) = A^0.33, A: Mura area (mm²)

Fig. 3.3 also shows the straight lines for 2.0×Cjnd and 3.0×Cjnd. The Dist contrast data is around the 2.0×Cjnd line. This indicates that not only the JND level but also the visible contrast level has the linear relationship between the contrast level and A^0.33. Hence, the level of Mura defects can be described accordingly.

To evaluate the blemish degree of Cluster Mura defect, the SEMU formula is defined as:

) . .

(197_. 072

33

0 +

=

≡

A C C

SEMU C ^x

jnd

x , (3.2)

where Cx is the average contrast of Mura, Cjnd is the just noticeable contrast of Mura, and A is the area of Mura [2]. A larger value of SEMU indicates a more serious

blemish. In [4] and [5], some modified versions of this SEMU formula have been further proposed. In these modified SEMU formulae, the contrast and the area of Mura defects are still the main factors. Similar to Equation (3.2), either a larger contrast of Mura or a larger area of Mura will cause a higher degree of blemish.

As mentioned above, Mura defects appear as low contrast, non-uniform brightness regions. They are typically larger than a single pixel when the screen is at a constant gray level. Moreover, various kinds of Mura defects are caused by different reasons. It would be difficult to develop a single universal algorithm to detect all kinds of Mura defects. In this dissertation, we consider two major kinds of Mura defects, Cluster Mura and Vertical-Band Mura. These two types of Mura are common in the FOS of LCDs. In Section 3.2, we’ll introduce the photography of the FOS image first. Then, in Section 3.3, the proposed techniques in the detection of Cluster Mura are discussed. In Section 3.4, on the other hand, we discuss the proposed techniques for the detection of Vertical-Band Mura.

3.2 Photography of FOS Images

Fig. 3.4(a) shows the prototype of our automatic inspection system. In this system, we have an LCD panel that is under test, a CCD camera to capture the FOS image of the LCD panel, and a computer to execute Mura detection algorithms. In the inspection process, the LCD panel under test is vertically placed on the apparatus. Each time, the LCD panel is driven by a pattern generator to display a certain gray level image pattern. Then, the camera will capture an FOS image for this specific pattern. The FOS image is transmitted to the computer for storage and then is inspected by a set of Mura detection algorithms.

At markets, 15-inch and 17-inch LCD panels are the mainstream for desktop displays and notebook displays. In fact, the manufacture processes in these two types of LCD panels are quite similar. Hence, the appearances of Mura defects are also similar. Both types of LCD panels can be inspected by this prototype system.

Moreover, to mimic the performance of human inspection, the specifications of the CCD camera are chosen to have a more than 12-bit dynamic range and a spatial resolution of 1532 × 1024 pixels.

(a) Inspection prototype (b) Flowchart of LCD inspection Fig. 3.4 The prototype of the LCD inspection system.

3.2.1 Aliasing

In the photography process, aliasing usually occurs when the lens of camera is well focused on the center of the FOS. Fig. 3.5(a) shows an image with aliasing. Fig. 3.5(b) shows the enlarged image of the portion inside the red rectangular in Fig. 3.5(a). As shown in Fig. 3.5(b), the lattice-like pattern occurs when the camera lens is in focus.

This lattice-like pattern will not only increase the difficulty in automatic inspection but also influence the inspection results of Mura defects.

To eliminate the aliasing effect, several anti-aliasing approaches have already been published or patented [51][52]. These approaches usually require additional accessories and may cause image blurring. Among these approaches, we adopt the de-focusing approach, which may eliminate the aliasing effect without additional accessories. In this approach, we focus the camera lens on the FOS of LCD first, followed by slightly de-focusing the lens till the lattice-like pattern disappears.

Although this method still causes some degree of blurring in the FOS image, this method demands no extra expense. In Fig. 3.5(c), we show the de-focused image of Fig. 3.5(a). It can be easily seen that the lattice-like pattern is suppressed.

Fig. 3.5 Aliasing in the photography process.

3.2.2 Cluster Mura and V-Band Mura

Among several kinds of Mura defects, the detection of two major kinds of Mura defects, Cluster Mura and Vertical Band (V-Band) Mura, will be taken into account in this dissertation. As mentioned before, Mura defects appear as non-uniform brightness regions in the FOS of an LCD panel. Nevertheless, not all kinds of Mura defects appear in a similar manner. For example, Cluster Mura usually appears as a cluster of several points locating within a small area. As shown in Fig. 3.6(a)(c), a round-type Cluster Mura defect locates at (x=750, y=820), while a rectangular-type Cluster Mura defect appears as a narrow region ranging from (x=200, y=720) to (x=600, y=720).

On the other hand, a V-Band Mura usually appears as a wide vertical strip with either brighter or darker brightness with respect to the uniform background. As shown in Fig.

3.6(e), a V-Band Mura is centered around x=850. The cause of V-Band Mura usually comes from non-uniform thickness of components, such as non-uniform thickness of the glasses in the cell unit. In comparison, Cluster Muras usually appear within a local area, while V-Band Muras usually occupy a larger area. The appearance model of a Cluster Mura is quite different from that of a V-Band Mura. It would be impractical to detect both types of Muras based on a single algorithm.

(a) Round-type Cluster Mura (b) Detection result of (a)

(c) Rectangular-type Cluster Mura (d) Detection result of (c)

(e) V-Band Mura

Fig. 3.6 Cluster Mura and V-Band Mura.

In the following sections, we will present the use of two different approaches to deal with these two kinds of Mura defects. These two different approaches actually share the same kernel: a Laplacian of Gaussian (LOG) filter. This LOG filter, which compares the intensity of the target with the intensity of the surroundings, will be demonstrated to be very useful in the development of Mura detection algorithms. To detect Cluster Mura defects, an approach based on 2-D LOG filters will be presented in Section 3.3. To detect V-Band Mura, on the other hand, an approach based on 1-D LOG filters will be presented in Section 3.4.

3.3 Inspection of Cluster Mura

3.3.1 Cluster Mura Detection

In practice, there are two types of Cluster Mura defects: round-type Cluster Mura and rectangular-type Cluster Mura. In the selection of operators to detect these two types of Muras on a constant luminance level, two requirements are demanded: 1) having fewer parameters and 2) being less influenced by the shading of the luminance level.

In practice, there are two types of Cluster Mura defects: round-type Cluster Mura and rectangular-type Cluster Mura. In the selection of operators to detect these two types of Muras on a constant luminance level, two requirements are demanded: 1) having fewer parameters and 2) being less influenced by the shading of the luminance level.