In our experiment, we create some synthetic images to test whether that our system is really working out or not. The image resolutions of these synthetic images are 512 512 pixels. However, the synthetic images are sharper than the real PCB images. Therefore, we apply the average filter to blur the synthetic images. If we apply the average filter with larger size, the synthetic image we generated will be much vaguer.
4.1 Defect Detection
Our defect detection is focus on how to segment defects from images. If we can segment the defects more precisely, we can classify defects more correctly. PCB images can be roughly divided into two types, periodical and non-periodical images.
Therefore, we will discuss the result of defect detection with the non-periodical image in Sec. 4.1.1 and the periodical in Sec. 4.1.2.
4.1.1 Non-periodic Image
For non-periodic images, we have two images in the input, reference image and test image to find defects. The reference image is an ideal image with no defects, and the test image is a synthetic image which we put eight types of defects described in Chapter 3 in the test image generated. These eight types of defects are “open,”
“mouse bite,” “pinhole,” “missing conductor,” “short,” “spur,” “missing hole,” and
“excess copper.” To generate synthetic images to be more similar to the real PCB images, since the synthetic images are sharper than the real PCB images, we apply an
31
7 7 average filter with every entry having value 1
49 on the synthetic images. The synthetic reference image and the synthetic test image are shown in Fig. 4.1(a) and Fig. 4.1(b), respectively. On the other hand, the blurred reference image and blurred test image, are shown in Fig. 4.1(c) and Fig. 4.1(d), which are blurred by 7 7 average filter.
(a) (b)
(c) (d)
Fig. 4.1. The synthetic images. (a) The synthetic reference image. (b) The synthetic test image. (c) The synthetic reference image which is blurred by 7 7 average filter.
(d) The synthetic test image which is blurred by 7 7 average filter.
32
Image alignment is the first step to detect the defects. Therefore, we have to align the test images to the reference image, as shown in Fig. 4.1(c) and Fig. 4.1(d). The result of image alignment is shown in Fig. 4.2. In this case, we can find the displacement ( x, y) between these two images is (10, 10) and the reference image is on the left-upper side of the test image, according to the method mentioned in Sec.
2.2.1.
(a) (b)
Fig. 4.2. The result of image alignment. (a) The aligned reference image. (b) The aligned test image.
After aligning the test image with the reference image, the next step is to segment defects from images. We apply the same threshold T 128 to the aligned reference image and the aligned test image to acquire the binary reference image and binary test image, as shown in Fig. 4.3(a) and Fig. 4.3(b), respectively. Then we subtract the reference image from the test image producing the excess pixel image of
33
positive pixel difference and producing the missing pixel image of negative pixel difference as shown in Fig. 4.3(c) and Fig. 4.3(d). However, these two images may contain many noisy pixels thus we apply the minimal connected threshold of 10 to reduce the noisy pixels. Namely, if the pixel numbers of the connected region is less than 10 pixels, this region will be defined as noise, as mentioned in Sec. 2.5. Finally, we can obtain two clear images, clear missing pixel image and clear excess pixel image, as shown in Fig. 4.3(e) and Fig. 4.3(f), respectively.
(a) (b)
(c) (d)
34
(e) (f)
Fig. 4.3. The result of defect detection. (a) The binary reference image. (b) The binary test image. (c) The missing pixel image. (d) The excess pixel image. (e) The clear missing pixel image. (f) The clear excess pixel image.
At the end of defect detection, we record the left-upper coordinates, number of pixels and trueness of each defect blob. Table I and Table II show the defect detection result of clear missing pixel image and clear excess pixel image.
TABLE I
THE DEFECT DETECTION RESULT OF MISSING PIXEL IMAGE
Left-upper coordinates of defect blob Defect index
x y number of pixels in
defect blob defect trueness 1m 102 105 362 True 2m 140 152 344 True 3m 210 207 317 True 4m 328 94 327 True
35
TABLE II
THE DEFECT DETECTION RESULT OF EXCESS PIXEL IMAGE
Left-upper coordinates of defect blob Defect index
x y number of pixels in
defect blob defect trueness 1e 39 141 100 True 2e 79 84 110 True 3e 141 119 813 True 4e 303 147 398 True
4.1.2 Periodic Image
For periodic images, we only have to input a test image to find the defects, as shown in Fig. 4.4(a). This test image is also a synthetic image and filtered by 7 7 average filter. Therefore, we apply the method of MAD, as mentioned in Sec. 2.2, to find out the period ( x, y)(50,128). Then we left shift the test image with period
x 50
to generate the new reference image. Fig. 4.4 shows the result of image alignment in test image.
(a) (b)
36
(c)
Fig. 4.4. The result of image alignment. (a) The test image. (b) The new reference image with left shifting period x50 (c) The aligned test image.
After acquiring the reference image, we also apply the image subtraction method to detect defects. We apply the same threshold T 128 for aligned reference image and aligned test image and then we can acquire the binary reference image and binary test image, as shown in Fig. 4.5(a) and Fig. 4.5(b). After that, we subtract these two images to generate the missing pixel image and the excess pixel image, as shown in Fig. 4.5(c) and Fig. 4.5(d). In Fig. 4.5(c) and Fig. 4.5(d), we can detect defects but could also contain noisy pixels. Therefore, we apply the threshold num10 to reduce the noisy pixels, as mentioned in Sec. 2.5. Finally, we can obtain two clear image, clear missing pixel image and clear excess pixel image, in Fig. 4.5(e) and Fig.
4.5(f).
37
(a) (b)
(e) (f)
(c) (d)
Fig. 4.5. The result of defect detection. (a) The binary reference image. (b) The binary test image. (c) The missing pixel image. (d) The excess pixel image. (e) The clear missing pixel image. (f) The clear excess pixel image.
38
Table III and Table IV show the defect detection result of clear missing pixel image and clear excess pixel image. Different from non-periodic images, the reference image of periodic image is generated by left shifting the test image with period x50, so the reference and the test image of periodic image have the same number but the different location of the defects. Hence, we can observe that the total number of defects in both missing pixel image and excess pixel image is sixteen not eight, as shown in Table III and Table IV.
39 [10] to classify the eight types of defects: “open,” “mouse bite,” “pinhole,” “missing conductor,” “short,” “spur,” “missing hole,” and “excess copper.” We will discuss the classification result of the periodical image in Sec. 4.2.1 and the non-periodical in Sec.
4.2.2.
4.2.1 Non-periodic Image
After detecting defects, we can acquire the clear missing pixel image and the clear excess pixel image, as shown in Fig. 4.6(a) and Fig. 4.6(b). In order to classify
40
the type of each defect, we extract the three features, “defect state,” “boundary state,”
and “number of state transition,” from the outer boundary of each defect. Therefore, we have to extract the boundary of each defect at first, as shown in Fig. 4.6(c) and Fig.
4.6(d).
(a) (b)
(c) (d)
Fig. 4.6. The result of defect classification. (a) The clear missing pixel image. (b) The clear excess pixel image. (c) The outer boundary of the clear missing pixel image.
(d) The outer boundary of the clear excess pixel image.
41
After acquiring the boundaries of each defect in both the clear missing pixel image and the clear excess pixel image, we can extract the feature of each defect and then classify these defects into their own type, as mentioned in Sec. 3.3. After classifying all these defects, we will record the left-upper coordinates, NOST, BS and defect type of each defect. Table V and Table VI show the classification result of clear missing pixel image and clear excess pixel image. In Table VI, we can observe that there is an unknown defect type. The reason is that in our defect detection algorithm, we only use a threshold T to acquire the binary reference image and the test reference image. However, using the same threshold T to binarize whole images could not be suitable for each defect in the reference image and test image. Therefore, we acquire the wrong NOST 8 and misclassification in this case.
42
TABLE VI
THE DEFECT CLASSIFICATION RESULT OF EXCESS PIXEL IMAGE
Left-upper coordinates of defect blob Defect index
x y NOST BS defect type
1e 39 141 0 Excess copper 2e 79 84 8 Unknown
(Missing hole) 3e 141 119 4 Short 4e 303 147 2 Spur
4.2.2 Periodic Image
For periodic image, we apply the same classification method as non-periodic image used. We also extract the outer boundary of each defect, as shown in Fig. 4.7.
(a) (b)
43
(c) (d)
Fig. 4.7. The result of defect classification. (a) The clear missing pixel image. (b) The clear excess pixel image. (c) The outer boundary of the clear missing pixel image.
(d) The outer boundary of the clear excess pixel image.
After acquiring the boundaries of each defect in both the clear missing pixel image and the clear excess pixel image, we can extract the feature of each defect and then classify these defects into their own type, as mentioned in Sec. 3.3. Table VII and Table VIII show the classification result of clear missing pixel image and clear excess pixel image. In Table VIII, we can observe that there is a misclassification defect. The reason is that in our defect detection algorithm, we only use a threshold T to acquire the binary reference image and the test reference image. However, using
the same threshold T to binarize whole images could not be suitable for each defect in the reference image and test image. Therefore, the regions of the defects are not very correct and then cause the wrong NOST and misclassification.
44
45
4e 211 424 2 Spur 5e 255 52 0 Excess copper
(Caused by periodical) 6e 258 176 2 Spur
(Missing hole) 7e 329 418 0 Missing hole
(Caused by periodical) 8e 406 273 0 Excess copper
46