Using Modified Bagging and Boosting Algorithms in Multiple Classifiers System for Remote Sensing Image Classification

(1)

Volume 12, No.3, September 2007, pp. 241-256

Using Modified Bagging and Boosting Algorithms in Multiple Classifiers System for Remote Sensing

Image Classification

Y. C. Tzeng

¹

Kun-Shan Chen

²

N. S. Chou

³

ABSTRACT

In this paper, modified Bagging and Boosting voting methods were proposed in the multiple classifiers system for terrain classification of remote sensing images. The improvement is achieved by introducing a confidence index to reduce the ambiguities among the targets being classified. Performance of the proposed multiple classifiers system was tested using fused radar and optical images. Experimental results show that the classifier is able to substantially improve the classification accuracy.

Key Words: multiple classifiers system, Bagging and Boosting, remote sensing image

1. Introduction

A multiple classifiers system (MCS), which combines the outputs of several classifiers, has been widely used to produce a better performance in varied applications (Tumer and Ghosh, 1996, Kittler and Roli, 2000). Its aim is mainly to provide a merging method that effectively takes advantages of individual classifier while bypass its weaknesses. Benediktsson and Swain (1992) presented a consensus theory to combine single probability distributions to summarize estimates from multiple experts with the assumption that the experts make decisions based on Bayesian decision theory. A statistical analysis method was augmented to include mechanisms to weight the influence of the data sources in the classification.

Wolpert (1992) introduced the stacked generalization scheme for minimizing the classification error rate for

one or more classifiers. The outputs from classifiers were combined in a weighted sum with weights that are determined by the individual performance of the classifiers. Serpico and Roli (1995) utilized a structured neural network in classification of remote sensing images. The structured neural network may be equivalently interpreted as a hierarchical arrangement of “committees” that accomplish the classification task by checking a set of explicit constraints on input data. Benediktsson et al., (1997) successfully applied a parallel consensual neural network in classification/data fusion of remote sensing and geographic data. The input data were transformed several times and subsequently used as if they were independent inputs, followed by being classified, weighted and combined to reach consensual decision.

Carpenter et al. (1997) proposed the use of fuzzy ARTMAP neural networks for vegetation

Received Date: Jan. 16, 2007 Revised Date: Sep. 21, 2007 Accepted Date: Sep. 21, 2007 1 Professor, Dept. of Electronics Engineering, National United University

2 Professor, Center for Space and Remote Sensing Research and Dept. of Information Engineering, National Central University

3 Ph. D Candidate, Dept. of Information Engineering, National Central University

(2)

classification from Landsat thematic mapper (TM) and terrain data. A voting strategy improves the prediction and assigns the confidence estimates by training the system repeatly on different orderings of an input set.

Datcu et al. (2002) adopted the concept of dependence trees for the integration of information through estimation of probability distributions to apply a statistical approach to the classification of remote sensing data. An Nth order binary distribution was approximated by a product of (N-1) second-order class conditional distributions. Then, a nonparametric method based on Gaussian kernels was used to estimate the second-order class conditional distributions. More recent developments of MCS can be found in Nikunj C. Oza et al. (2005).

2. Background

2.1 Multiple Classifier System

The expectation of using multiple classifiers system is that the classifiers, trained differently, will converge to local minima on the error plane and their overall performance can be improved by combining the individual outputs in effective way.

Consider that in a case of single classifier that is trained based on pre-selected training set. Let x be an input vector of the classifier, D denote the desired response. The functional relationship between x and D can be mathematically expressed by a regressive model (White, 1989)

ε +

= f(x)

D (1) where f(·) is a deterministic function of its argument vector, and  is the residual error, which is assumed to be a random variable with zero mean and with positive probability of occurrence, and is uncorrelated with the regression

function f(x). The regression function f(x) can be regarded as the conditional mean of the model output D, as given by f(x) = E[D|x]. Assume that F(x) is the functional link between input and output, to be realized by the classifiers. The effectiveness of F(x) as a predictor of the desired response D can be measured by the mean-square error between F(x) and the conditional expectation E[D|x], defined by

( ) [ ]

( )

[

F x EDx ²

]

(E[F( )x] E[ ]Dx)² E

[

(F( )x E[F( )x])²

]

E − = − + − (2)

Equation (2) may be viewed as the average of the estimated error between f(x) and the approximation function F(x), evaluated over the entire training data domain. The first term on the right hand side of (2) is the squared bias of the average of the approximating function F(x), measured with respect to the regression function f(x). Hence, this bias may be regarded as an approximation error of the function F(x). The second term on the right hand side is recognized as the variance of the approximation function F(x).

The objective of the multiple classifiers system is therefore to find an effective combination method that reduces the bias and/or the variance of the approximation function F(x). Taking x to be a fixed input and D as output, then, (2) can be rewritten as

^D^x

)

²

E − ≥ _EA − (4) Further performing integration both sides of (4) over the joint x and D, we may observe that the

(3)

mean-squared error of FEA(x) is less than that of F(x). Because the bias of the ensemble-averaged function FEA(x) is exactly the same as that of the approximation function F(x), its variance is hence less than that of the function F(x). In fact, a simple ensemble averaging combiner, where the outputs of different classifiers are linearly combined to produce an overall output, proves to be able to reduce the variance (Naftaly et. al., 1997). As far as the individual classifier is concerned, the bias is reduced at the cost of variance. But, subsequently, the variance can be reduced by means of ensemble averaging (weighted sum) of the classifiers’

outputs, leaving the bias unchanged. Two voting approaches, boosting [16,17] and bagging [3], have been extensively used and analyzed. Both generate by resampling training sets from the original data set to the learning algorithm which builds up a base classifier for each training set.

They are briefly outlined below.

2.2. The Bagging Algorithm

The Bagging algorithm works with a training sample of fixed size. The samples are resampled according to a given probability distribution. The error is calculated with respect to the fixed training sample. The algorithm constructs many different bags of samples by performing boostrapping iteratively. Each bag is a set of training samples and is collected by randomly and uniformly resampling of the original training set. The algorithm then applies a base classifier to classify each bag. Finally, it performs some types of average over the classifications of each sample via a voting. For easy understanding, a pusedo-code for Bagging algorithm is presented as follows.

Input: A training set X with N samples, where

each sample xj is of class Dj

∈ { 1 , K , J }

and J is the number of classes predicted, base classifier Y, and number of classifiers n.

for i = 1 to n

Xi = bootstrapped bag from X Yi = Y (Xi)

endfor for k = 1 to J

∑

⁼

= _i_:_Y₍_x₎ _k1

k _i

Z endfor

( ) x

_k

Z

_k

B = arg max

Output: The multiple classifiers B.

Because the algorithm does not change the distribution of the samples, all the classifiers bear equal weights during the voting. It has been shown that if the base classifier is unstable, then Bagging may improve the classification accuracy significantly. However, if the base classifier is stable, the Bagging may adversely deteriorate the classification accuracy because each classifier receives lesser of the training data. The Bagging algorithm reduces the variance of the classification but exercises little effect on the bias of the classification [Breiman, 1996]. Its main advantage is that it can improve the classification accuracy significantly if the base classifier is properly selected. It is also not very sensitive to noise in the data.

2.3. Boosting Algorithm

Boosting algorithm, similar to Bagging algorithm, works with a fixed training sample, but assumes that the weak learning algorithm can receive weighted examples. The input-output function F(x) realized by a classifier can be viewed as a hypothesis. The value F(x) can be interpreted

(4)

as a randomized prediction of the label of x that is 1 with probability F(x) and 0 with probability

) (

1 − F x

. From Freund and Schapire (1997), a strong learning is an algorithm that, given

0 , > δ

ε

and access to random examples, outputs with probability 1−δ a hypothesis with error at most ε. On the other hand, a weak learning algorithm satisfies the same conditions but only for

ε ≥ 1 / 2 − γ

where

γ > 0

is either a constant, or decreases as 1/p where p is a polynomial in the relevant parameters. The error is calculated with respect to the weighted examples.

Further detail discussion is referred to (Freund and Schapire, 1997). In the beginning of a Boosting method, all the training samples are designated to have the same weight. Then, the weights are subsequently adjusted in such a way that the incorrect classified samples have more weight than those correct ones. A pusedo-code is given below.

Input: A training set X with N samples, where each sample xj is of class Dj

∈ { 1 , K , J }

and J is the number of classes predicted, base classifier Y, and number of classifiers n.

for j = 1 to N

N x

w (

_j

) = 1 /

endfor for i = 1 to n

Xi = X Yi = Y (Xi)

∑

( )≠

=

j j

ix D

Y

j j

i w x

e _: ( )

if ei > 0.5 then abort

(

_i

)

i

i =e −e

β 1

for j = 1 to N

if Yi(xj) = Dj then

i j

j

w x

x

w ( ) = ( ) β

endfor

∑

=w _jw x_j

w ( )

endfor for k = 1 to J

( )

∑

= β

= iY x k i

k _i

Z _: log1

endfor

( )

x _kZ_k

B =argmax

Output: The multiple classifiers B

As is known, Boosting algorithm has a tendency to reduce both the variance and the bias of the classification. Its main advantage is that in many cases it increases the overall accuracy of the classification. On the other hand, its major problem is that it usually does not perform well in terms of accuracies when there is noise in the data.

3. Improved Weighting in MCS

As shown in Fig. 1, the output of each classifier used in the ordinary Bagging and Boosting weighted multiple classifiers systems is encoded (class label).

An unencoded output of a classifier is numeric value while an encoded output of a classifier is class labeled.

The output Yi of each classifier is a class label and is aggregating by voting. However, if the output is a numeric value, a procedure is required to replace the voting mechanism by the average of Yi. Because the unencoded classifier contains richer information than that of encoded classifier, we must modify Bagging and/ or Boosting algorithm for use in multiple classifiers system. The output vector Yi = [yi1 yi2 … yiJ] of each classifier is without encoding. An unencoded classifier assigns a numeric value in between 0 and 1 to each output to represent the degree of an input xj belonging to the

(5)

corresponding class. The closer the value yik is to 1, the more likely the input xj belongs to class k. Thus, the fused output of class k for the Bagging and/or Boosting algorithms can be generally written as an average (weighted sum) of yik over i

∑

=

i i ik

k

w y

Z

(5)

where the weight wi is assigned according to the performances of each classifier. Simple weighted sum strategy may work well for most applications. However, poor classification performance usually results when the ambiguities among classes are high. For example, in a two classifiers system, if an input is classified as class k by classifier a and class l by classifier b, the weighted sum for each class is defined as

bk b ak a

k w y w y

Z = + and

bl b al a

l w y w y

Z = + (6) If Zk is greater than Zl, class k is voted in the final. This may be done by the following threshold criteria:

l l

k

k w y w y w y

y

w₁ ₁ + ₂ ₂ > ₁ ₁ + ₂ ₂ or (7) 0

) (

)

(w₁y₁_k+w₂y₂_k − w₁y₁_l +w₂y₂_l > (8) On the other hand, if Zk is smaller than Zl, class l is voted. However, if the difference between Zk and Zl is sufficiently small, it is difficult to discern class k and class l, leading to poor classification. To account for this ambiguity, a confidence index is proposed to enhance class boundary and thus to increase the classification accuracy. By this, the fused output of class k now becomes

∑

− =

=

ik ky k

i i il

i i ik

k wy wy

Z : argmax (9)

The subscripts of the second term are defined as l = argmaxl yjl,

j ≠ i

and l ≠k . The

confidence index is then defined as

al a bk b ak a

k w y w y w y

c = + − and

bk b bl b al a

l w y w y w y

c = + − (10) It can be interpreted as that the higher the value ck is, the more confident the class k is predicted by classifier a. If ck is greater than cl, it votes for class k. This is equivalent to modifying (7), (8), respectively, as

bk b bl b al a al a bk b ak

ay wy wy wy wy wy

w + − > + − or (11)

) (

)

(w_ay_ak+w_by_bk − w_ay_al+w_by_bl > w_ay_al−w_by_bk (12) At this point, we may consider the confidence index as an adaptive thresholding technique. An example was given to illustrate the effect of confidence index. The number of classes predicted is assumed to be 6 and number of classifiers is 3.

For a pixel known as class 4, the outputs of the three classifiers are shown in Table 1. It is classified as class 3, 4, and 1 by using classifier 1, 2, and 3, individually. The weight of each classifier for the Bagging and Boosting algorithms are listed in Table 2. The fused outputs and final results for different method used are given in Table 3. If the original Bagging or Boosting algorithm (class labeled or numeric) is used, the pixel has been misclassified as class 3. However, the pixel has been correctly classified as class 4, if the concept of confidence index (the modified Bagging or Boosting algorithm) is applied.

Finally, a winner-takes-all was adopted to select a proper class. This enables us to use Kappa coefficient (Lillesand and Kiefer, 1993) for accuracy evaluation. To this end, the base classifier used was a dynamic learning neural network[Tzeng, et.al 1994, Chen et. al, 1995, 1996]. Its output can be encoded (class labeling) or unencoded (numeric value) and is well suited for

(6)

our problems treated here. A dynamic learning neural network (DL) (Tzeng et al., 1994) was adopted as the base classifier. DL is a modified MLP neural network. After two modifications: 1) every node at input layer and all hidden layers are fully connected to the output layer and 2) the activation function is removed from each output node are made, the output of the network is expressed as a linear function (weighted sum) of the polynomial basis vectors of the inputs in terms of the polynomial basis function (Chen and Manry, 1993). Hence, the Kalman filtering technique (Brown and Hwang, 1983; Bierman, 1977) can be utilized to solve the linear equations to adjust the weights of the neural network and to minimize the training error. It has been shown that DL is an effective and efficient tool for both multispectral and SAR images classifications (Chen et al., 1995;

Chen et al., 1996).

4. Experimental Results

In this paper, a plantation area in Au-Ku on the east coast of Taiwan was chosen as test site. Two sets of imagery data, airborne SAR (AIRSAR) and MODIS/ASTER Simulator (MASTER), were acquired during the NASA/JPL PACRIM II campaign over Taiwan area (Tzeng and Chen, 2005). All the AIRSAR and MASTER data were pre-processed into the same coordinate system. Three L-band linearly polarized channels (hh, vv, and hv) of the AIRSAR data were selected for testing. For MASTER data set, three spectral bands (bands 3, 5, and 8 which correspond to the R, G, and IR bands corresponding to SPOT sensor, respectively) were chosen as the second data source. The image sets of the test sites are displayed in Fig. 2 (MASTER) and Fig.3 (SAR). Fig.

4 is a ground truth map collected during the campaign

and serves as classification accuracy check.

The test site contained six ground cover types:

sugar cane A, sugar cane B, bare soil, rice, grass, and seawater, where sugar cane A and B were discriminated by their growth stages. The total number of training and verifying pixels for each class was listed in Table 4. In what follows, the classification results using various approaches described in previous section were discussed in detail.

4.1. AIRSAR Data

The classification performance of using a single classifier was evaluated first and was served as a reference. In this case, the DL neural network was configured to have 3 input nodes (accepting three polarization channels hh, vv, and hv), 1 hidden layer with 10 hidden nodes, and 6 output nodes, representing six terrain covers to be classified. The classification matrix was given in Table 5. In this case, the overall accuracy was 75.44% and the kappa coefficient was about 0.69. Because some pixels of the class 4 (rice) were misclassified to class 2 (sugar cane B) and some pixels of the class 3 (bare soil) were misclassified to class 4 (rice), a not so satisfactory accuracy was produced. To apply the multiple classifiers system, the number of classifiers n used was selected to be 3. All the base classifiers (i.e. DL) were devised to have identical structure configurations, just as those for the single classifier case, and were trained on the same data set, but differently initialized.

For both the original and modified Bagging weighted multiple classifiers systems, the training set for each classifier was collected by randomly and uniformly resampling to about 63.2% of the whole data set.

Table 6 displayed the classification matrix of using the original Bagging weighted multiple classifiers system.

The overall accuracy obtained was 80.95% with kappa coefficient about 0.7611. Besides, the classification

(7)

result of using the original Boosting weighted multiple classifiers system was shown in Table 7 where it indicated that the overall accuracy was 81.23% with kappa coefficient 0.7645. It can be seen that a considerable amount of the misclassified pixels were recovered. Both original Bagging and Boosting algorithms significantly improve the classification accuracy slightly. Upon applying the concept of confidence index, the classification result of using the modified Bagging weighted multiple classifiers system was shown in Table 8. Its overall accuracy was 82.85% and its kappa coefficient is 0.7847, while, the classification matrix of using the modified Boosting weighted multiple classifiers system was given in Table 9. Its overall accuracy was up to 85.40% and its kappa coefficient was 0.8161. It is obvious that both the modified Bagging and Boosting algorithms obtain better performance. In particular, the misclassified pixels in class 4 (rice) into class 2 (sugar cane B) were effectively recovered, but only a few misclassified pixels in class 3 (bare soil) were corrected.

Nevertheless, substantial improvement of classification performance using Boosting and Bagging algorithm is evident.

4.2. MASTER Data

The same procedure applied for MASTER data was in order. When applied to the second data source (MASTER), DL was configured with 3 input nodes (representing R, G, and IR), 6 output nodes (representing 6 cover types), and 2 hidden layers with 40 hidden nodes each. The classification matrix was given in Table 10. It was seen that the overall accuracy was only 65.01% with kappa coefficient 0.549, a quite poor performance. The classification accuracy was so low mainly because that too many pixels of the class 1 (sugar cane A) were adversely misclassified as class 5 (grass) and vise visa, while

pixels of the class 4 (rice) were misclassified as class 1 (sugar cane A). Table 11 and Table 12, respectively, give the classification result of using the original Bagging and Boosting weighted multiple classifiers systems. Its overall accuracy was 84.20% with kappa coefficient of 0.7955 for Bagging, and 83.92% with kappa coefficient 0.7942 for Boosting. Both were relatively comparable and able to improve the classification performance. We found from the classification matrix that most of the misclassified pixels of class 1 (sugar cane A) into class 5 (grass) were adequately recovered. However, still a large amount of class 4 (rice) pixels misclassified into class 1 (sugar cane A), and class 5 (grass) pixels misclassified as class 1 (sugar cane A) remained recovered. We now turn to the modified Bagging and Boosting algorithms. The classification accuracy and kappa coefficient were increased to 95.37%/0.9409 and 95.74%/0.9456, respectively, for Bagging and Boosting. The classification matrices were given in Table 13 and Table 14. It was evidently seen that, for both modified Bagging and Boosting algorithms, most of the misclassified pixels were effectively recovered.

Both algorithms offer similarly super classification performance.

To this end, we demonstrated the terrain cover classification of remote images using various classification methods. We summarized the classification performance of various algorithms in Table 15. The MCS system with proper voting algorithms generally offers better performance as compared to direct and single classification. For AIRSAR data, the ambiguities existing among classes were moderate. The classification performances of using modified Bagging and Boosting algorithms were slightly better than those of using original Bagging and Boosting algorithms. For the second data source, MASTER data, the ambiguities among classes

(8)

were extremely high, as revealed from its covariance matrices (not shown here). In such case, the classification performance offered by the modified Bagging and Boosting is highly satisfactory.

5. Conclusions

In this paper, multiple classifiers system was applied for terrain covers classification of remote sensing. The Bagging and Boosting algorithm were investigated as weighting policy. A confidence index is defined to account for the ambiguities among classes. By making use of the concept of confidence index, the modified Bagging and Boosting weighted multiple classifiers system were presented. The classification performances of utilizing the original and modified Bagging and Boosting weighted multiple classifiers systems to the application of remote sensing image classification were then demonstrated. Experimental results show that the classification performance is considerably improved.

In addition, the multiple classifiers systems of using modified Bagging and Boosting algorithms are superior to those of using original Bagging and Boosting algorithms, particularly when the ambiguities among classes are high.

References

J. A. Benediktsson and P. H. Swain, 1992.

“Consensus theoretic classification methods,”

IEEE Transaction on Systems, Man, and Cybernetics, vol. 22, pp.688-704.

Benediktsson J. A., Sveinsson J. R., Ersoy O. K., and Swain P. H., 1997, Parallel consensual neural networks. IEEE Transaction on Neural Networks, 8, 54-64.

Bierman, G. J., 1977, Factorization Methods for Discrete Sequential Estimation, (New York:Academic).

Breiman, L., 1996, “Bagging Predictors”, Machine Learning, vol. 24, no. 2, pp. 123-140.

R. E. Schapire, 1999. A Brief Introduction to Boosting, in Proc. 16^th Int. Joint Conference on Artificial Intelligence.

R. E. Schapire, 2003. “The boosting approach to machine learning: An overview,” in D. D.

Denison, M. H. Hansen, C. Holmes, B.

Mallick, B. Yu, editors, Nonlinear Estimation and Classification, Springer.

Brown, R. G. and Hwang P. Y. C., 1983,

Introduction to Random Signals and Applied Kalman Filtering, (New York:Wiley).

Carpenter, G. A., Gjaja M. N., Gopal S., and Woodcock C. E., 1997, ART neural networks for remote sensing: vegetation classification from Landsat TM and terrain data. IEEE Transactions on Geoscience and Remote Sensing, 35, 308 - 325

Chen, K. S., Tzeng Y. C., Chen C. F., and Kao W. L., 1995, Land-cover classification of multispectral imagery using a dynamic learning neural network. Photogrammetric Engineering and Remote Sensing, 61, 403-408.

Chen, K. S., Huang W. P., Tsay D. H., and Amar F., 1996, Classification of multifrequency polarimetric SAR imagery using a dynamic learning neural network. IEEE Transaction on Geoscience and Remote Sensing, 34, 814-820.

Chen, M. S. and Manry M. T., 1993, Conventional modeling of the multilayer perceptron using polynomial basis functions. IEEE

Transactions on Neural Networks, 4, 164 - 166.

(9)

Congalton, R. G. and Kass Green, 1998. Assessing the Accuracy of Remotely Sensed Data:

Principles and Practices, CRC.

Datcu M., Melgani F., Piardi A., and Serpico S. B., 2002, Multisource data classification with dependence tree. IEEE Transaction on Geoscience and Remote Sensing, 40, 609-617.

Freund, Y. and R. E. Schapire, 1997, “A decision-theoretic generalization of online elearning and an application to boosting,”

Journal of Computer and System Sciences, vol.

55, no. 1, pp. 119-139.

Josef Kittler, Fabio Roli (Eds.), 2000. Proceedings of Multiple Classifier Systems, First International Workshop, MCS 2000, Cagliari, Italy, June 21-23.

Lillesand, T. M. and Kiefer R. W., 1993, Remote Sensing and Image Interpretation, (New York:

Wiley).

Naftaly U., N. Intrator, and D. Horn, 1997, “Optimal ensemble averaging of neural networks,”

Network, vol. 8, pp. 283-296.

Nikunj C. Oza, Robi Polikar, Josef Kittler, and Fabio Roli(Eds.), 2005. Proceedings of Multiple Classifier Systems: 6th International Workshop, MCS 2005, Seaside, CA, USA, June 13-15.

Serpico S.B. and Roli, F., 1995, Classification of multisensor remote-sensing images by structured neural networks. IEEE Transactions on Geoscience and Remote Sensing, 33, 562 – 578.

Tumer K. and Ghosh J., 1996, Analysis of decision boundaries in linearly combined neural classifiers. Pattern Recognition, 29, 341-348.

Tzeng, Y. C. and Chen K. S., 2005, Image Fusion of SAR and Optical Data for Terrain

Classification with a Variance Reduction Technique. Optical Engineering, accepted.

Tzeng, Y. C., Chen K. S., Kao W. L., and Fung A.

K., 1994, A dynamic learning neural network for remote sensing applications. IEEE Transaction on Geoscience and Remote Sensing, 32, 1096-1102.

White H., 1989. “Learning in artificial neural networks: A statistical perspective,” Neural Computation, vol. 1, pp. 425-464.

Wolper, D. H., 1992, Stacked generalization.

Neural Networks, 5, 241-259.

(10)

Table 1 Typical unencoded and Encoded Outputs unencoded Output Yi (Numeric) Classifier i

yi1 yi2 yi3 yi4 yi5 yi6

Yi

a 0.12 0.01 0.61 0.34 0.17 0.01 3

b 0.21 0.01 0.18 0.63 0.01 0.15 4

c 0.58 0.22 0.45 0.27 0.01 0.01 1

Table 2 Classifier

i

Weights wi (Boosting) Weights wi

(Bagging)

a 2.27 1840 b 1.41 1680 c 0.17 1694

Table 3

Fused Output Z Method

Z1 Z2 Z3 Z4 Z5 Z6

Encoded Output B Boosting

(Class Labeled) 0.17 0 2.27 1.41 0 0 3

Boosting

(Numerical) 0.67 0.07 1.72 1.71 0.4 0.24 3

Modified

Boosting 0.54 0.07 0.67 1.16 0.4 0.24 4

Bagging

(Class Labeled) 1694 0 1840 1694 0 0 3

Bagging

(Numerical) 1556.12 407.88 2187.1 2141.38 346.54 287.34 3 Modified

Bagging 336.44 407.88 1340.7 1486.18 346.54 287.34 4

(11)

Table 4 Data Set Summary

Table 5 AIRSAR Classification Matrix (Single Classifier)

Table 6 AIRSAR Classification Matrix (Original Bagging Weighted)

Class Label Ground Cover Types Number of Training Pixels Number of Verification Pixels

1 Sugar Cane A 1110 660

2 Sugar Cane B 288 126

3 Bare Soil 540 568

4 Rice 300 310

5 Grass 343 269

6 Sea Water 595 225

class 1 2 3 4 5 6 Producer's

1 650 7 2 1 0 0 98.48

2 18 108 0 0 0 0 85.71

3 0 4 294 270 0 0 51.76

4 0 137 8 165 0 0 53.23

5 3 7 52 0 207 0 76.95

6 0 0 16 0 5 204 90.67

User's 96.87 41.06 79.03 37.84 97.64 100.0 overall accuracy = 75.44%; kappa coefficient = 0.6935

1 653 7 0 0 0 0 98.94

2 18 108 0 0 0 0 85.71

3 0 1 365 200 2 0 64.26

4 0 111 17 182 0 0 58.71

5 3 6 23 1 236 0 87.73

6 0 0 16 0 6 203 90.22

(12)

Table 7 AIRSAR Classification Matrix (Original Boosting Weighted)

Table 8 AIRSAR Classification Matrix (Modified Bagging Weighted)

Table 9 AIRSAR Classification Matrix (Modified Boosting Weighted)

1 656 4 0 0 0 0 99.39

2 22 104 0 0 0 0 82.54

3 0 0 318 249 1 0 55.99

4 0 53 14 243 0 0 78.39

5 3 4 33 3 226 0 84.01

6 0 0 8 0 11 206 91.56

1 659 0 0 1 0 0 99.85

2 24 99 0 2 1 0 78.57

3 0 0 290 269 9 0 51.06

4 0 0 17 293 0 0 94.52

5 3 1 23 5 237 0 88.10

6 0 0 6 0 9 210 93.33

1 659 1 0 0 0 0 99.85

2 23 101 1 1 0 0 80.16

3 0 0 346 203 19 0 60.92

4 0 5 20 285 0 0 91.94

5 3 2 16 4 244 0 90.71

6 0 0 6 0 11 208 92.44

(13)

Table 10 MASTER Classification Matrix (Single Classifier)

Table 11 MASTER Classification Matrix (Original Bagging Weighted)

Table 12 MASTER Classification Matrix (Original Boosting Weighted)

1 422 65 0 9 164 0 63.94

2 14 112 0 0 0 0 88.89

3 43 0 506 1 18 0 89.08

4 96 2 41 137 34 0 44.19

5 268 0 0 0 1 0 0.37

6 0 0 0 0 0 225 100.0

1 648 11 0 0 1 0 98.18

2 14 112 0 0 0 0 88.89

3 43 0 460 64 1 0 80.99

4 68 0 0 242 0 0 78.06

5 136 0 0 0 133 0 49.44

6 0 0 1 0 2 222 98.67

1 649 10 0 0 1 0 98.33

2 14 112 0 0 0 0 88.89

3 41 0 395 129 3 0 69.54

4 35 0 3 272 0 0 87.74

5 101 0 1 0 167 0 62.08

6 0 0 4 0 5 216 96.00

(14)

Table 13 MASTER Classification Matrix (Modified Bagging Weighted)

Table 14 MASTER Classification Matrix (Modified Boosting Weighted)

Table 15 Summary of classification performance of various algorithms performance

Method

AIRSAR Data Overall accuracy/kappa

MASTER Data Overall accuracy/kappa

Single Classifier 75.44%/ 0.6935 65.01%/0.549

Bagging Weighted MCS 80.95%/0.7611 84.20%/0.7955

Boosting Weighted MCS 81.23%/0.7645 83.92%/0.7942

Modified Bagging Weighted MCS 82.85%/0.7847 95.37%/0.9409

Modified Boosting Weighted MCS 85.40%/0.8161 95.74%/0.9456

1 646 8 0 3 3 0 97.88

2 10 114 0 2 0 0 90.48

3 15 0 538 12 3 0 94.72

4 0 0 3 307 0 0 99.03

5 26 0 12 0 231 0 85.87

6 0 0 0 0 3 222 98.67

1 648 7 1 3 1 0 98.18

2 12 111 0 2 1 0 88.10

3 12 0 541 12 3 0 95.25

4 0 0 2 308 0 0 99.35

5 21 0 12 0 236 0 87.73

6 0 0 0 0 3 222 98.67

User's 93.51 94.07 97.30 94.77 96.72 100.0

overall accuracy = 95.74%; kappa coefficient = 0.9456

(15)

Classifier 1

Classifier n Xn

Bagging/

Boosting Algorithms Yn

Fused output Z

Fig. 1. Bagging and/or Boosting weighted multiple classifiers system.

Data Set X1

Encoded output Y1

Winner Takes All

Encoded output

B

Fig. 2 MASTER images of different bands combination

RGB: 0.66μm,0.54μm,0.46μm (left), 0.80μm, 0.66μm, 0.54μm (middle), 9.10μm, 1.67μm, 0.54μm (right)

Fig. 3 An L-band AIRSAR image of the test site Fig. 4 Ground truth map of the test site R: hh, G:hv, B: vv

(16)

多分類器系統中 Bagging and Boosting 法則的改進

曾裕強

¹

陳錕山

²

周念湘

³

摘要

本文針對多分類器系統中提出一修正後的 Bagging and Boosting 票決方式來改善遙測影像中地物分類的精度，並藉由引進一信心指標，多分類器系統可以增加各分類器成間的差異度或降低模糊度。我們利用雷達影像與光學影像的融合來測試多分類器系統的分類性能。實驗結果顯示新的多分類器系統可大幅提升整體的分類精度。

關鍵詞:多分類器系統、遙測影像

收到日期:民國 96 年 01 月 16 日修改日期:民國 96 年 09 月 21 日接受日期:民國 96 年 09 月 21 日

1國立聯合大學電子工程學系教授

2國立中央大學太空及遙測研究中心暨資訊工程學系教授

3國立中央大學資訊工程學系博士候選人