According to the results of experiments, using pruned model to sample partially does deliver better performance than the original, complete sampling approach if there are
disparate scales among building blocks. Moreover, for the uniformly scaled problems in which the linkage are always completely sensible, the proposed method uses just nearly twice as many function evaluations as the original approach. While this performance is already reasonable considered that only half of the population is used for model building, we would like to point out another direction of thinking concerning this constant overhead:
most problems that we are dealing with are not uniformly scaled. That is, we can expect more often the case would be that the subproblems are of unequal salience. Thus, it may be more practical to look at black-box optimization problems with this caution in mind and not assume the linkage would be completely sensible.
In this study, we focused on the scaling difficulties and their influences on the ability of EDAs to appropriately identify building blocks. However, at a higher scope, our at-tempt was trying to resolve an important issue which was rarely addressed: what if the information contained in the given population is inevitably insufficient? The approach to solve this problem was proposed and successfully implemented for ECGA. It may be adopted and carried over to other EDAs such that more flexible and robust EDAs can be developed.
Bibliography
[1] H. M¨uhlenbein and G. Paaß, “From recombination of genes to the estimation of distributions I. binary parameters,” in PPSN IV: Proceedings of the 4th International Conference on Parallel Problem Solving from Nature. London, UK: Springer-Verlag, 1996, pp. 178–187.
[2] P. Larra˜naga and J. A. Lozano, Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation, ser. Genetic algorithms and evolutionary computa-tion. Boston, MA: Kluwer Academic Publishers, October 2001, vol. 2, ISBN: 0-7923-7466-5.
[3] M. Pelikan, D. E. Goldberg, and F. G. Lobo, “A survey of optimization by build-ing and usbuild-ing probabilistic models,” Computational Optimization and Applications, vol. 21, no. 1, pp. 5–20, 2002.
[4] D. E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning.
Boston, MA, USA: Addison-Wesley Longman Publishing Co., Inc., 1989.
[5] ——, The Design of Innovation: Lessons from and for Competent Genetic Algo-rithms. Norwell, MA, USA: Kluwer Academic Publishers, 2002.
[6] G. Harik, “Linkage learning via probabilistic modeling in the ECGA,” Illinois Genetic Algorithms Laboratory, University of Illinois at Urbana-Champaign., IlliGAL Report No. 99010, 1999.
[7] J. H. Holland, Adaptation in natural and artificial systems. Cambridge, MA, USA:
MIT Press, 1992.
[8] D. E. Goldberg, B. Korb, and K. Deb, “Messy genetic algorithms: Motivation, anal-ysis, and first results,” Complex Systems, vol. 3, no. 5, pp. 493–530, 1989.
[9] H. Kargupta, “SEARCH, polynomial complexity, and the fast messy genetic algo-rithm,” Ph.D. dissertation, University of Illinois, 1995.
[10] G. Harik, “Learning gene linkage to efficiently solve problems of bounded difficulty using genetic algorithms,” Ph.D. dissertation, University of Illinois, 1997.
[11] Y.-p. Chen, Extending the scalability of linkage learning genetic algorithms: Theory and practice, ser. Studies in Fuzziness and Soft Computing. Springer, 2006, vol.
190, ISBN: 3-540-28459-1.
[12] H. Kargupta, “The gene expression messy genetic algorithm,” in International Con-ference on Evolutionary Computation, 1996, pp. 814–819.
[13] M. Munetomo and D. Goldberg, “Identifying linkage by nonlinearity check,” Illi-nois Genetic Algorithms Laboratory, University of IlliIlli-nois at Urbana-Champaign., IlliGAL Report No. 98012, 1998.
[14] M. Munetomo and D. E. Goldberg, “Identifying linkage groups by nonlinearity/non-monotonicity detection,” in Proceedings of the Genetic and Evolutionary Computa-tion Conference GECCO-99, W. Banzhaf, J. Daida, A. E. Eiben, M. H. Garzon, V. Honavar, M. Jakiela, and R. E. Smith, Eds., vol. 1. Orlando, Florida, USA:
Morgan Kaufmann, 13-17 1999, pp. 433–440.
[15] R. B. Heckendorn and A. H. Wright, “Efficient linkage discovery by limited probing,”
Evolutionary Computation, vol. 12, no. 4, pp. 517–545, 2004.
[16] S. Baluja, “Population-based incremental learning: A method for integrating genetic search based function optimization and competitive learning,” Pittsburgh, PA, USA, Tech. Rep., 1994.
[17] G. R. Harik, F. G. Lobo, and D. E. Goldberg, “The compact genetic algorithm,”
IEEE Transactions on Evolutionary Computation, vol. 3, no. 4, p. 287, November
[18] J. de Bonet, C. Isbell, and P. Viola, “MIMIC: Finding optima by estimating probabil-ity densities,” in Advances in Neural Information Processing Systems, M. C. Mozer, M. I. Jordan, and T. Petsche, Eds., vol. 9. The MIT Press, 1997, pp. 424–430.
[19] S. Baluja and S. Davies, “Using optimal dependency-trees for combinational opti-mization,” in ICML ’97: Proceedings of the Fourteenth International Conference on Machine Learning. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1997, pp. 30–38.
[20] M. Pelikan and H. M¨uhlenbein, “The bivariate marginal distribution algorithm,”
in Advances in Soft Computing - Engineering Design and Manufacturing, R. Roy, T. Furuhashi, and P. K. Chawdhry, Eds. London: Springer-Verlag, 1999, pp. 521–
535.
[21] M. Pelikan, D. E. Goldberg, and E. Cant´u-Paz, “BOA: The Bayesian optimization algorithm,” in Proceedings of the Genetic and Evolutionary Computation Confer-ence GECCO-99, W. Banzhaf, J. Daida, A. E. Eiben, M. H. Garzon, V. Honavar, M. Jakiela, and R. E. Smith, Eds., vol. I. Orlando, FL: Morgan Kaufmann Pub-lishers, San Fransisco, CA, 13-17 1999, pp. 525–532.
[22] R. Etxeberria and P. Larra˜naga, “Global optimization using bayesian networks,” in Proceedings of the Second Symposium on Artificial Intelligence (CIMAF-99), A. O.
Rodriguez, M. S. Ortiz, and R. S. Hermida, Eds., Habana, Cuba, 1999, pp. 332–339.
[23] H. M¨uhlenbein and T. Mahnig, “FDA: A scalable evolutionary algorithm for the optimization of additively decomposed functions,” Evolutionary Computation, vol. 7, no. 4, pp. 353–376, 1999.
[24] H. M¨uhlenbein and R. H¨ons, “The estimation of distributions and the minimum relative entropy principle,” Evolutionary Computation, vol. 13, no. 1, pp. 1–27, 2005.
[25] D. Thierens, D. E. Goldberg, and ˆA. G. Pereira, “Domino convergence, drift and the temporal salience structure of problems,” in Proceedings of the 1998 IEEE Interna-tional Conference on Evolutionary Computation. IEEE Press, 1998, pp. 535–540.
[26] D. E. Goldberg, K. Deb, and J. H. Clark, “Genetic algorithms, noise, and the sizing of populations,” Complex Systems, vol. 6, no. 4, pp. 333–362, 1992.
[27] D. E. Goldberg and M. Rudnick, “Genetic algorithms and the variance of fitness,”
Complex Systems, vol. 5, no. 3, pp. 265–278, 1991.
[28] D. E. Goldberg, K. Deb, and B. Korb, “Messy genetic algorithms revisited: Studies in mixed size and scale,” Complex Systems, vol. 4, no. 4, pp. 415–444, 1990.
[29] F. G. Lobo, D. E. Goldberg, and M. Pelikan, “Time complexity of genetic algorithms on exponentially scaled problems,” in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2000), D. Whitley, D. Goldberg, E. Cant´u-Paz, L. Spector, I. Parmee, and H.-G. Beyer, Eds. Las Vegas, Nevada, USA: Morgan Kaufmann, 10-12 2000, pp. 151–158.
[30] Y.-p. Chen and D. E. Goldberg, “Convergence time for the linkage learning genetic algorithm,” Evolutionary Computation, vol. 13, no. 3, pp. 279–302, 2005.
[31] K. Deb and D. E. Goldberg, “Analyzing deception in trap functions,” in Foundations of Genetic Algorithms 2, 1993, pp. 93–108.
[32] ——, “Sufficient conditions for deceptive and easy binary functions,” Annals of Math-ematics and Artificial Intelligence, vol. 10, no. 4, pp. 385–408, 1994.
[33] J. Rissanen, “Modelling by shortest data description,” Automatica, vol. 14, pp. 465–
471, 1978.
[34] T. M. Cover and J. A. Thomas, Elements of information theory. New York, NY, USA: Wiley-Interscience, 1991.
[35] T. M. Mitchell, Machine Learning. McGraw-Hill Higher Education, 1997.