Defect Inspection and Analysis of Color Filter Panel

Defect Inspection and Analysis of Color Filter Panel

Mu-Hsing Wu, Chiou-Shann Fuh, and Hsien-Yei Chen

The Department of Computer Science and Information Engineering National Taiwan University, Taipei, Taiwan

ABSTRACT

In this paper, an experimental automatic color filter surface defect inspection system is described. It is based on the image subtraction method for the purpose of high-speed inspection. The experimental result shows that our algorithm achieves high accuracy and high speed for industrial requirement.

Keywords— Color filter, defect inspection.

Ⅰ. INTRODUCTION

LCDs will be the next combat ground after Taiwan’s computer monitors made of cathode ray tube become the world market leader. In the LCD flat panel production, the easiest way to realize colorization is to make use of the back illuminant and the color filter panel [14]. In order to guarantee the output quality of the LCD flat panel, people have to inspect any killing defect from the surface of the color filter panel. Traditional, human operators simply inspect the color filter panel visually against prescribed standards. The decisions made by these inspectors often involve subjective judgment, in addition to being labor intensive and therefore costly, whereas automatic inspection systems remove the subjective aspects and provide fast, quantitative dimensional assessments. These automatic systems do not get tired, do not suffer burnouts, and are consistent day in and day out. All of this means better quality at a lower cost. In this paper, we develop an automatic color filter surface defect inspection system that is based on the image subtraction method for the purpose of high-speed inspection.

The experimental result shows that our algorithm achieves high accuracy and high speed for industrial requirement.

This paper is organized as follows: Section Ⅱ outlines the classification of defects;

Section Ⅲ describes the inspection framework and hardware requirement; Section Ⅳ details the inspection algorithms and performance report; Section Ⅴ gives some conclusion remarks.

Ⅱ. CLASSIFICATION OF DEFECTS

Generally speaking, the taxonomy of the color filter is usually based on the configurations of the color cell pattern besides the size of the color filter panel. Different kinds of the configurations have been adopted for different purposes. The experimental sample of the color filter panel that we obtained from the Mechanical Industrial Research Laboratories, Industrial Technology Research Institute (MIRL/ITRI) inheres in the stripy configuration type (Fig. 1) that has been adopted for many TFT-LCD (TFT: Thin-Film Transistor) panel manufacturers. The defect inspection algorithm that we develop still performs well for other types of configuration. Throughout this thesis, we will focus on the stripy-configuration-based color filter panel to introduce our inspection system.

In the color filter production, defects are roughly classified into two parts (Fig. 2). One is called macro-defect, and another is called micro-defect [10]. The former defects are classified into “IRO-MURA”, “SIMI”, and “IRO-ZURE”. “IRO-MURA” means unevenness of color.

“SIMI” means a stain on a filter. “IRO-ZURE” means misalignment of color cells. As these defects are slightly big, they can be found by human eyes easily through common microscopes.

The later, called micro-defects, are very small and cannot be found easily by eyesight even through common microscopes. They also can be classified into three categories: the first one is black matrix hole, the second one is color area shortage in filter cell, and the last one is particle.

Figure 1: Stripy configuration of the color cell pattern.

R G B R G B R G B

R G B R G B R G B

R G B R G B R G B

R G B R G B R G B

R G B R G B R G B

R G B R G B R G B

Defects

Macro-Defects

Unevenness of Color (IRO-MURA) Stain (SIMI)

Misalignment of Color Cells (IRO-ZURE) Micro-Defects

Black Matrix Hole

Color Area Shortage in Filter Cell Particle

Figure 2: Classification of the color filter surface defects.

Surface defects of the color filter panel do not only cause visual failure, but also cause electrical failure to operate LCD panel. For example, a grain defect is made by small particle between color filter and ITO (Indium-Tin Oxide) film electrode. If the grain is larger than the distance of ITO film and TFT panel (the distance is usually 5~10 micron), it will short the film electrode and TFT panel circuit. Therefore, it is very important to inspect these defects in the color filter production to increase yield.

Ⅲ Inspection Framework and Hardware Requirement

The inspection framework of our experimental inspection system is illustrated in Figure 3.

Basically, the main hardware components consist of the illumination system, the CNC (Computer Numerical Control) X-Y table, the CCD (Charge-Coupled Device) camera, the frame grabber, and the personal computer.

The lighting technique we use is backlighting—the light source is on the opposite side of the object from the CCD camera. We use a strobe light as the light source of our inspection system, and keep it upright under the CCD camera. For synchronizing the timings of image grabbing and lighting, a control signal is employed to trigger the CCD camera and the strobe light simultaneously. Because the FOV (Field of View) of the CCD camera and the field of illumination of the strobe light both can not cover the color filter panel entirely, the inspection system must not only move the CCD camera and strobe light to scan the color filter panel but also keep the relative position between them (for the purpose of stabilization). Thus we fix the CCD camera and the strobe light on a CNC X-Y table and employ another control signal to trigger the CNC X-Y table to move the CCD camera and the strobe light during the whole inspection process.

Image Signal

CNC X-Y Table CCD Camera

Lens

Color Filter

Transparent Glass Plate Strobe Light

Monitor

Control Signal 2

Inspection Memory

Control Signal 1 Software Frame Grabber

Personal Computer

Figure 3: The inspection framework of our experimental inspection system.

Our experimental inspection system operates as follows. First of all, the operator places and fixes a color filter panel on the transparent glass plate before starting the inspection process.

By triggering the control signal 1 and setting proper x-axis counter value and y-axis counter value of the CNC X-Y table, the inspection system can control the movement of the CCD camera and strobe light precisely. Then, through the control signal 2, the inspection system triggers the strobe light and CCD camera simultaneously for lighting and image grabbing respectively. The analog image signal output from the CCD camera is transferred to the frame grabber via a cable. Then the frame grabber performs an analog-to-digital signal conversion and put the digitized image into a pre-allocated memory buffer. Finally, the inspection software inspects the image and reports the result on the monitor.

We will give a concise description about the main hardware components of our experimental inspection system as follows.

CCD Camera

A JAI CV-M10 monochrome CCD camera with 768 (h) × 576 (v) resolution and 256 gray levels is used in our inspection system for image grabbing. It is equipped with a 1/2-inch CCD sensor that supports 6.4 mm (h) × 4.8 mm (v) sensing area.

Lens

A Moritex MML1-65D micro lens is adopted in our inspection system. Its working distance is 65 mm; depth of field is 1.5 mm. Cooperating with JAI CV-M10 monochrome CCD camera can get 6.4 mm (h) × 4.8 mm (v) field of view equal to the sensing area of the CCD sensor.

Because the image resolution is 768 pixel (h) × 576 pixel (v) and the field of view is 6.4 mm (h) × 4.8 mm (v), we can get the coordinate system mapping that 1 pixel ≒ 8.33 um. (Note that the more accurate datum can be gotten through the process of camera calibration.)

CNC X-Y Table

The CNC X-Y table used in our system has the maximal accumulative movement error ±60 um (namely ±7.2 pixels) in horizontal direction (x-axis) and vertical direction (y-axis) respectively. Hence it should be taken care that the scenes of any two neighbor images must be partially overlapped to tolerate the movement error. Moreover, due to the potential movement error of the CNC X-Y table, the content of any grabbed image cannot be used directly for defect inspection—we should segment the image properly to get what we really need!

Light Source

Suitable lighting facilitates inspection by avoiding the need for complex image-processing algorithms as much as possible. The light source adopted in our system now is the strobe light.

The main consideration is simply “longevity”. The timing of illumination depends on the control signal sent by the inspection system, i.e. the strobe light needs not keep on illuminating continuously and thus can operate normally for a much longer time than other ordinary light source such as fluorescent light.

Frame Grabber

A Matrox Meteor Ⅱ Multi-Channel frame grabber is used in our inspection system for two purposes. The first one is image digitization, i.e. performing analog-to-digital signal conversion. The other one is for the support of external triggering, i.e. the timing of image

grabbing can be controlled.

The judgmental criterion we adopt for the fatal defect of the color filter panel is based on the bounding box of the defect. If the width or height of the bounding box is greater than or equal to 30 um, the defect is presumed fatal. On the other hand, if both the width and height of the bounding box are less than 30 um, the defect is presumed non-fatal and thus can be connived. From Section 2.2, we know that the mapping between the image and world coordinate system is 1 pixel ≒ 8.33 um, i.e. 30um ≒ 3.6 pixels. For convenience, we judge the fatal defect by the width and height of its bounding box in the image coordinate system and revise the judgmental criterion of the fatal defect as below:

For any defect d found in the grabbed image, assume that the width of its bounding box is wb and the height of its bounding box is hb. We claim d is fatal and should be found out by our inspection system if wb＞3 pixels or hb＞3 pixels, otherwise d is non-fatal and thus can be connived by our inspection system.

Ⅳ INSPECTION ALGORITHMS

In this section, we will detail the processes of golden image generation, orientation compensation, image segmentation, and noise removal. The performance report will also be discussed in this section.

Golden Image Generation

Defect inspection by image subtraction method is easily understood. By subtracting the image with defect from the golden image (image without defect), then binarizing the residual image with proper threshold, we can detect the existence of defect from the residual image. A pixel of the residual image is presumed defective if its gray level is greater than the binarizing threshold. By connected components analysis of the binarized residual image, we can group all defective pixels into maximal connected regions called blob. From the judgmental criterion we discussed in Section 2.3, a defective blob is claimed as a fatal defect if the width or height of its bounding box is greater than 3 pixels.

The problem we want to discuss here is how to generate the golden image. Whether the result of defect inspection is correct or not will depend on the quality of the golden image. To ensure the quality of the golden image, we should average a certain amount of non-defective image to reduce the effect of non-constant light source. Before the development of our inspection system, however, who can judge the defectiveness of the color filter images for

golden image generation? The answer is the human operators. But we still cannot count on the subjective judgment of the operator completely—that may shoulder a certain degree of risk.

After the visual examination by human eyes, our inspection system gives the judgment of defectiveness during the golden image generation process.

The golden image does not exist in the beginning of the golden image generation process.

After fixing the visually examined color filter panel on the transparent glass plate, the inspection system triggers the CNC X-Y table to control the precise movement of the CCD camera and strobe light to scan the color filter panel during the whole process of golden image generation. The conceptual diagram of the defect detection is shown in Figure 4, where image I_n and I_n+1 stand for any two neighboring color filter images and image I_G represents the temporal golden image during the golden image generation process.

－ ＝

Image I_n Image I_n+1 Residual Image

Judgment

Temporal Golden Image I_G

Figure 4: The conceptual diagram of the defect detection method during the golden image generation process.

We explain the golden image generation process here. Every time the inspection system gets two neighboring color filter images In and In+1 from the CCD camera, the inspection system subtracts image In+1 from image In and checks if defect exists. Then inspection system will act according to the following rules:

Rule 1. If no defect is found and the temporal golden image IG still does not exist, the

inspection system averages image I_n with image I_n+1 to generate image I_G.

Rule 2. If no defect is found and the temporal golden image IG exists already, the inspection system averages both the image I_n and I_n+1 with the image I_G by the weighted average method to update the image I_G.

Rule 3. If any defect is found and the temporal golden image IG still does not exist, the inspection system will store the images I_n and I_n+1 until the image I_G has been generated.

Then the inspection system will check all possibly defective regions in both the images I_n and I_n+1 by the image I_G to distinguish the non-defective image from the defective image and update the image I_G if there exists a non-defective image.

Rule 4. If any defect is found and the temporal golden image IG exists already, the inspection system will check all possibly defective regions in both the images I_n and I_n+1 by the image I_G to distinguish the non-defective image from the defective image and update the image I_G if any non-defective image exists.

After the golden image generation process, the inspection system will save the golden image to disk for later defect inspection.

Orientation Compensation

The orientation of the golden image may be different from the orientation of the inspected image because they come from different color filter panels. Although all color filter panels are fixed on the transparent glass plate of our inspection system before performing the golden image generation process or the defect inspection process, each of them may have its own orientation respectively. For accuracy, even if the difference between any two orientations is very small, we have to compensate for the orientation before image subtraction.

For quantification, we define the orientation of a color filter panel as the angle θ between its registration mark and the horizontal line in the registration mark image shown in Figure 5.

The orientation-compensated task can be divided into the orientation teaching process and the orientation computing process. The orientation teaching process should be done only once for every batch of the color filter panels with its own specification, but the orientation computing process will be taken for every color filter panel.

In the orientation teaching process, we have two jobs to do. The first one is to record the x-axis counter value and the y-axis counter value of the CNC X-Y table when the CCD camera can capture the registration mark image, and the other one is to store the top-left corner and top-right corner of the registration mark as pattern images. Since the x-axis

Figure 5: The definition for the orientation of the color filter panel is the angle θ between its registration mark and the horizontal line in the registration mark image.

counter value and the y-axis counter value of the CNC X-Y table record the location of the registration mark with respect to the origin of the CNC X-Y table coordinate system, the inspection system can automatically move the CCD camera to capture the registration mark image. With the top-left corner pattern image and top-right corner pattern image of the registration mark, the inspection system can locate the edge of the registration mark to compute the orientation of the color filter panel.

The orientation computing process is applied to every inspected color filter panel once to compute its orientation, i.e. the angle θ between its registration mark and the horizontal line in the registration mark image. For the following color cell images of the color filter panel, the inspection system will compensate for the orientation of each image with the image rotation automatically.

The orientation computing process can be classified into six steps as discussed below:

Step 1. Grab the registration mark image.

Step 2. Locate the top-left corner and top-right corner of the registration mark respectively by the normalized cross correlation [3, 4]: r(u,v)











 



 















 



 



  









 







 



  







 

 

 

 

 

 

 

 

 

 

 

 

 

 

2

0 0 0 0

2 2

0 0 0 0

2

0 0 0 0

0 0

) , ( )

, ( )

, ( )

, (

) , ( )

, ( )

, ( ) , (

m

i n

j m

i n

j m

i n

j m

i n

j

m i

n j m

i n j m

i n j

j i M j

i M N

v j u i I v

j u i I N

j i M v

j u i I j

i M v j u i I N

(1)

where N mn; m is the width of pattern image, n is the height of pattern image, w is the width of searching image; h is the height of searching image; r( vu, ) is the correlation coefficient,













Step 3. Locate the rectangular region that the edge of the registration mark must be located according to the positions of the top-left corner and the top-right corner found in Step 2 (see Figure 6).

Figure 6: The rectangular region surrounded by the dotted line represents the rectangular region that the edge of the registration mark must be located.

Step 4. Binarize the rectangular region with proper threshold such as 128, and then perform edge detection to find out all edge pixels in the rectangular region (Fig. 7).

Figure 7: The result of edge detection.

Step 5. Perform line fitting to the edge pixels found in Step 4 with the least squares method to

get the line equation as expressed below.

b aX

Y  (2)

where ₂

1 1

2

1 1 1



 





 





















n i

i n

i i

n i

i n

i i n

i i i

x x

n

y x y x n

a ,

n x a y b

n i

i n

i

i





 ^{1 1} , n is the number of edge pixels,





Step 6. Calculate the orientation with respect to the horizontal line and record it.

After the orientation computing process, we can rotate the image back (Eq. 3) to compensate for the orientation. Figure 8 shows the rotated registration mark image.



 







 



 





 









y x y

x







 cos sin

sin

cos (3)

Figure 8: Rotated registration mark.

Image Segmentation

Even though we have resolved the problem of orientation, we still cannot subtract one color cell image from another directly to generate golden image or inspect defects. Due to the potential movement error of the CNC X-Y table, the image coordinates of the color cells are not fixed and we cannot guarantee the contents of any two neighboring images to be the same.

In order to solve the problem, we have to segment the grabbed image properly to get the

sub-images with the same content. Thus the real golden image and the inspected image both get smaller size than the grabbed image.

Not only in the golden image generation process but also in the defect inspection process any grabbed image from the CCD camera should be applied to the procedures of orientation compensation and image segmentation before image subtraction.

The concept of our solution can be expressed in Figure 9. For the purpose of image registration, we introduce a combination of neighboring R-cell, G-cell and B-cell as the fiducial cell. The fiducial cell will be stored as a pattern image in advance and restricted the searching area within a proper range around the fiducial cell itself to avoid the ambiguous condition that more than one fiducial cell exist in the searching area.

To define the rectangular region of the inspected image, we must observe the following rules: (Suppose the width and height of the grabbed image are ImageWidth and ImageHeight respectively with the range from 0 to ImageWidth-1 and 0 to ImageHeight-1)

G B R G B R G B R G

Inspected Image

Searching Range Fiducial Cell

Grabbed Image

Figure 9: The concept of our solution to image segmentation.

Rule 1. Both the top-left corner coordinate (x_min,y_min) and the bottom-right coordinate )

,

(x_max y_max should satisfy the following conditions to tolerate the movement error of

the CNC X-Y table:

8 x_{m i n} (4) 8 ymin (5)

max ImageWidth9

x (6)

max ImageHeight9

y (7) Rule 2. The horizontal displacement x and vertical displacement y of the CNC X-Y table should be assigned equal to the width and height of the inspected image in world coordinate system as below except for the last row and column of the inspected image matrix shown in Figure 3.10:

33 . 8 ) 1 ( _max  _min  



x x x (Unit: um) (8) 33

. 8 ) 1 ( _max  _min 



y y y (Unit: um) (9)

Rule 3. The content of all inspected images should be exactly the same with the golden image except the defective ones.

After the teaching for the fiducial cell and the inspected image, Besides the width and height of the inspected image, we have to compute and record the relative offset between the rectangular fiducial cell and inspected image. After we get the image coordinate of the fiducial cell by pattern matching method, we can segment the grabbed image properly to get the inspected image.

Noise Removal

For defect inspection, we perform image subtraction and set proper threshold to binarize the residual image. All foreground pixels of the residual image are regarded as defective pixels, but not all of them constitute the fatal. Moreover, some fatal defects may be just the noises introduced by the image grabbing system and non-constancy of the strobe light [9]. An example is shown in Figure 10.

(a) (b)

(c) (d)

Figure 10: (a) Golden image. (b) Inspected image. (c) Binarized residual image. (d) Inspection result that finds not only the fatal defects but also the edge noise.

We find that all fatal noises exist around the edge of color cells and they all have long and thin shape, i.e. the width or height of their bounding box is greater than 3 pixels. In the golden image generation process, we eliminate the noises by binary opening method.

Applying the binary opening operator to the residual image with a 3×3 kernel, all edge noise can be eliminated as shown in Figure 11.

(a) (b)

Figure 11: (a) Binarized residual image. (b) The binarized residual image after applying binary opening.

Although binary opening can eliminate edge noise, however, there exist a problem that

some real fatal defects with long and thin shape may also be eliminated. Thus we create 2-pixel-widthed edge map based on image projection (Fig. 12) after golden image generation process to eliminate edge noise during inspection process.

Figure 12. The edge map.

Performance

The inspection time is critical for industrial application, but it is related to many conditions such as the hardware, the size of the inspected area, and the inspection parameters. Table 4.1 shows the approximate time profile of our inspection procedure on a personal computer with PentiumⅢ-450MHz CPU. The size of the inspected area is 724 pixel (h) × 522 pixel (v).

Moreover, the time of image grabbing and CNC X-Y table movement is not contained in the time profile.

The Inspection Steps Time (ms)

1. Image Rotation 32

2. Image Segmentation 5

3. Image Subtraction and Binarization 30

4. Defect Analysis 35

Table 4.1: The approximate time profile of our inspection procedure.

Ⅴ. CONCLUSION

Total 102

In this thesis, we have proposed an adaptive and efficient method for color filter surface defect inspection. For speed consideration, our inspection method is based on the image subtraction method. In order to reduce the false alarm rate due to the edge noise introduced by the image grabbing system and light source, we also proposed a new noise elimination method within the defect analysis process.

The strobe light with non-uniformity and non-constancy property may be a potential killer of the inspection accuracy and repeatability, so we need more experiments to appraise the effect of the light source. Suitable lighting facilitates inspection. We will look for a more adaptive illumination system as fast as possible.

Another research target is parallel processing. If the inspection system can scan the inspected color filter panel with more CCD cameras and process all grabbed images at the same time with multiple processors, the thoughput of defect inspection can be much better!

With the hardware support from the Mechanical Industrial Research Laboratories, Industrial Technology Research Institute (MIRL/ITRI), the research on the color filter surface inspection will keep improving in the future.

ACKNOWLEDGEMENT

This research was supported by the National Science Council of Taiwan, R.O.C., under Grant NSC 88-2213-E-002-031, by Mechanical Industry Research Laboratories, Industrial Technology Research Institute, under Grant MIRL 893K61CE1, by the EeRise Corporation, Tekom Technologies, Arima Computer, and Ulead Systems.

REFERENCE

[1] M. Bennamoun, “Edge Detection: Problems and Solutions,” IEEE International Conference on Systems, Man, and Cybernetics, Vol. 4, pp. 3164-3169, 1997.

[2] J. C. Bezdek and Y. Attikiouzel, “A Geometric Approach to Edge Detection,” IEEE Transactions on Fuzzy System, Vol. 6, pp. 52-75, 1998.

[3] G. Bonmassar and E. L. Schwartz, “Improved Cross-Correlation for Template Matching on the Laplacian Pyramid,” Journal of the Pattern Recognition Letters, Vol. 19, pp.

765-770, 1998.

[4] R. Brunelli and T. Poggio, “Template Matching: Matched Spatial Filters and Beyond,”

Journal of the Pattern Recognition, Vol. 30, pp. 751-768, 1997.

[5] R. C. Gonzalez and R. E. Woods, Digital Image Processing, Addison Wesley, Reading, MA, 1992.

[6] R. M. Haralick and L. G. Shapiro, Computer and Robot Vision, Vol. 1, Addison Wesley, Reading, MA, 1992.

[7] R. M. Haralick and L. G. Shapiro, Computer and Robot Vision, Vol. 2, Addison Wesley, Reading, MA, 1993.

[8] Kubotek, Optics: Optical Inspection System, Kubotek, Osaka, 1999.

[9] K. C. Lee, H. J. Song, and K. H. Sohn, “Detection-Estimation Based Approach for Impulsive Noise Removal,” Journal of the Electronics Letters, Vol. 34, pp. 449-450, 1998.

[10] K. Nakashima, “Hybrid Inspection System for LCD Color Filter Panels,” Proceedings of IEEE Instrumentation and Measurement Technology Conference, Hamamatsu, Japan, Vol. 2, pp. 689-692, 1994.

[11] Orbotech, LC-3000 Series, Orbotech, Yavne, Israel, 1999.

[12] Orbotech, Specifications LC-3090, Orbotech, Yavne, Israel, 1999.

[13] Orbotech, LC-3050, Orbotech, Yavne, Israel, 1999.

[14] S. Terada and Y. Shono, “Optical Approach for LCD Inspection,” Proceedings of International Conference on Optical Fabrication and Testing, Vol. 2576, pp. 105-114, 1995.