T2(k) is expressed as
T2(k) = 1 k
Xk
i=1
(T2(i − 1) + T2(k − i)) + (k + 1)2m2τ + 2mτ, (B.13)
where T2(0) ≤ (3m + 4 − 4 ln(m + 1))(m + 1)τ . Similar to the manipulation of Eq. (B.1) in Appendix B.1, by setting ak= T2(k)/(k + 1), we have
ak= ak−1+ (3k2+ k)m2τ + 2mτ k(k + 1) ,
= ak−1+ (3 − 2
k + 1)m2τ + 2(1 k − 1
k + 1)mτ,
≈ a0 + (3k + 2 − 2 ln(k + 1))m2τ + 2(ln k − ln(k + 1) + 1)mτ. (B.14) Substituting a0 = T2(0) into Eq. (B.14), we obtain
ak≤ (3k + 5 − 2 ln(k + 1))m2τ
+(9 + 2 ln k − 2 ln(k + 1) − 4 ln(m + 1))mτ + (4 − 4 ln(m + 1))τ. (B.15) Since ak = T2(k)/(k + 1), we have
T2(k) ≤ (3k + 5 − 2 ln(k + 1))(k + 1)m2τ
+(9 + 2 ln k − 2 ln(k + 1) − 4 ln(m + 1))(k + 1)mτ
+(4 − 4 ln(m + 1))(k + 1)τ. (B.16)
Bibliography
[1] A. K. Jain, M. N. Murty, and P. J. Flynn, “Data clustering: A review,” ACM Computing Surveys, vol. 31, no. 3, pp. 264–323, 1999.
[2] R. Xu and D. Wunsch, “Survey of clustering algorithms,” IEEE Trans. Neural Networks, vol. 16, no. 3, pp. 645–678, 2005.
[3] D. A. Reynolds, T. F. Quatieri, and R. B. Dunn, “Speaker verification using adapted Gaussian mixture models,” Digital Signal Processing, vol. 10, no. 1-3, pp. 19–41, 2000.
[4] D. A. Reynolds and R. C. Rose, “Robust text-independent speaker identification us-ing Gaussian mixture speaker models,” IEEE Trans. Speech and Audio Processus-ing, vol. 3, no. 1, pp. 72–83, 1995.
[5] S.-S. Cheng, Y.-Y. Xu, H.-M. Wang, and H.-C. Fu, “Automatic construction of regression class tree for MLLR via model-based hierarchical clustering,” Lecture Notes in Computer Science, pp. 390–398, 2006.
[6] H. C. Fu, H. Y. Chang, Y. Y. Xu, and H. T. Pao, “User adaptive handwriting recog-nition by self-growing probabilistic decision-based neural networks,” IEEE Trans.
Neural Networks, vol. 11, no. 6, pp. 1373–1384, 2000.
[7] R. Wehrens, L. M. C. Buydens, C. Fraley, and A. E. Raftery, “Model-based clus-tering for image segmentation and large datasets via sampling,” Journal of Classi-fication, vol. 21, no. 2, pp. 231–253, 2004.
[8] G. J. McLachlan, R. W. Bean, and D. Peel, “A mixture model-based approach to the clustering of microarray expression data,” Bioinformatics, vol. 18, no. 3, pp. 413–422, 2002.
[9] J. C. Mar and G. J. McLachlan, “Model-based clustering in gene expression mi-croarrays: an application to breast cancer data,” International Journal of Software Engineering and Knowledge Engineering, vol. 13, no. 6, pp. 579–592, 2003.
[10] C. Fraley and A. E. Raftery, “How many clusters? Which clustering method?
Answers via model-based cluster analysis,” The Computer Journal, vol. 41, no. 8, pp. 578–588, 1998.
[11] C. Fraley and A. E. Raftery, “Model-based clustering, discriminant analysis, and density estimation,” Journal of the American Statistical Association, vol. 97, no. 458, pp. 611–631, 2002.
[12] S. Zhong and J. Ghosh, “A unified framework for model-based clustering,” Journal of Machine Learning Research, vol. 4, no. 6, pp. 1001–1037, 2003.
[13] C. Fraley and A. E. Raftery, “Bayesian regularization for normal mixture estimation and model-based clustering,” Journal of Classification, vol. 24, no. 2, pp. 155–181, 2007.
[14] M. S. Oh and A. E. Raftery, “Model-based clustering with dissimilarities: A Bayesian approach,” Journal of Computational and Graphical Statistics, vol. 16, no. 3, pp. 559–585, 2007.
[15] M. J. Symons, “Clustering criteria and multivariate normal mixture,” Biometrics, vol. 37, pp. 35–43, 1981.
[16] S. Ganesalingam, “Classification and mixture approach to clustering via maximum likelihood,” Applied Statistics, vol. 38, no. 3, pp. 455–466, 1989.
[17] G. Celeux and G. Govaert, “A classification EM algorithm for clustering and two stochastic versions,” Computational Statistics & Data Analysis, vol. 14, no. 3, pp. 315–332, 1992.
[18] J. D. Banfield and A. E. Raftery, “Model-based Gaussian and non-Gaussian clus-tering,” Biometrics, vol. 49, no. 3, pp. 803–821, 1993.
[19] A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum likelihood from incom-plete data via the EM algorithm,” J. Roy. Statistical Soc. B, vol. 39, 1977.
[20] J. A. Bilmes, “A gentle tutorial of the EM algorithm and its application to pa-rameter estimation for Gaussian mixture and hidden Markov models,” Technical Reports TR-97-021, International Computer Science Insitute, April 1998.
[21] L. Xu and M. Jordan, “On convergence properties of the EM algorithm for Gaussian mixtures,” Neural Computation, vol. 8, no. 1, pp. 129–151, 1996.
[22] G. J. McLachlan and T. Krishnan, The EM algorithm and extensions. New York:
John Wiley, 1977.
[23] N. Ueda and R. Nakano, “Deterministic annealing EM algorithm,” Neural Net-works, vol. 11, no. 2, pp. 271–282, 1998.
[24] Y. Ishikawa and R. Nakano, “Landscape of a likelihood surface for a Gaussian mix-ture and its use for the EM algorithm,” in Proc. Int. Joint Conf. Neural Networks, pp. 1434–1440, 2006.
[25] Y. Ishikawa and R. Nakano, “EM algorithm with PIP initialization and temperature-based selection,” in Proc. Int. Conf. Knowledge-Based Intelligent In-formation and Engineering Systems, Part III, pp. 58–66, 2008.
[26] M. Takada and R. Nakano, “Multi-thread search with deterministic annealing EM algorithm,” in Proc. Int. Joint Conf. Neural Networks, pp. 1034–1038, 2002.
[27] M. Takada and R. Nakano, “Threshold-based dynamic annealing for multi-thread DAEM and its extreme,” pp. 501–506, 2003.
[28] F. Pernkopf and D. Bouchaffra, “Genetic-based EM algorithm for learning Gaussian mixture models,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 8, pp. 1344–
1348, 2005.
[29] U. Fayyad, C. Reina, and P. S. Bradley, “Initialization of iterative refinement cluster-ing algorithms,” in Proc. Int. Conf. Knowledge Discovery and Data Mincluster-ing, pp. 194–
198, AAAI Press, 1998.
[30] N. Ueda, R. Nakano, Z. Ghahramani, and G. Hinton, “SMEM algorithm for mixture models,” Neural Computation, vol. 12, no. 9, pp. 2109–2128, 2000.
[31] S. J. Young and P. Woodland, “State clustering in hidden Markov model-based con-tinuous speech recognition,” Computer Speech and Language, vol. 8, no. 4, pp. 369–
383, 1994.
[32] N. Vlassis and A. Likas, “A greedy EM algorithm for Gaussian mixture learning,”
in Neural Processing Letters, pp. 77–87, 2002.
[33] C. Biernacki, G. Celeux, and G. Govaert, “Choosing starting values for the EM al-gorithm for getting the highest likelihood in multivariate Gaussian mixture models,”
Comput. Stat. Data Anal., vol. 41, no. 3-4, pp. 561–575, 2003.
[34] R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification. Wiley-Interscience Publication, 2000.
[35] T. Hastie, R. Tibshirani, and J. Friedman, The elements of statistical learning.
Springer, 2001.
[36] W. D. Furman and B. G. Lindsay, “Testing for the number of components in a mixture of normal distributions using moment estimators,” Comput. Stat. Data Anal., vol. 17, no. 5, pp. 473–492, 1994.
[37] G. J. McLachlan and N. Khan, “On a resampling approach for tests on the num-ber of clusters with mixture model-based clustering of tissue samples,” Journal of Multivariate Analysis, vol. 90, pp. 90–105, 2004.
[38] L. F. James, C. E. Priebe, and D. J. Marchette, “Consistent estimation of mixture complexity,” The Annsls of Statisitcs, vol. 29, no. 5, pp. 1281–1296, 2001.
[39] A. P. Benavent, F. E. Ruiz, and J. M. S. Martinez, “EBEM: An entropy-based EM algorithm for Gaussian mixture models,” in Proc. Int. Conf. Pattern Recognition, vol. 2, 2006.
[40] K. P. Burnham and D. R. Anderson, Model Selection and Inference: A Practical Information-Theoretic Approach. New York: Springer-Verlag, 1998.
[41] G. Schwarz, “Estimation the dimension of a model,” Annals of Statistics, vol. 6, no. 2, pp. 461–464, 1978.
[42] A. E. Raftery, “Bayesian model selection in social research,” Sociological Methodol-ogy, vol. 25, pp. 111–163, 1995.
[43] C. Biernacki, G. Celeux, and G. Govaert, “Assessing a mixture model for clustering with the integrated completed likelihood,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 7, pp. 719–725, 2000.
[44] J. Rissanen, “Modelling by shortest data description,” Automatica, vol. 14, pp. 465–
471, 1978.
[45] J. J. Oliver, R. A. Baxter, and C. S. Wallace, “Unsupervised learning using MML,”
in Proc. 13th Int. Conf. Machine Learning, pp. 364–372, 1996.
[46] A. Corduneanu and C. M. Bishop, “Variational Bayesian model selection for mixture distributions,” in Proc. Int. Workshop on Artificial Intelligence and Statistics, 2001.
[47] C. M. Bishop, Pattern Recognition and Machine Learning. Secaucus, NJ, USA:
Springer-Verlag New York, Inc., 2006.
[48] T. Kohonen, Self-organizing Maps. Springer, 2001.
[49] T. Kohonen, “The self-organizing maps,” Neurocomputing, vol. 21, no. 1-3, pp. 1–6, 1998.
[50] C. Bishop, M. Svens´en, and C. Williams, “The generative topographic mapping,”
Neural Computation, vol. 10, no. 1, pp. 215–234, 1998.
[51] V. V. Tolat, “An analysis of Kohonen’s self-organizing maps using a system of energy functions,” Biological Cybernetics, vol. 64, no. 2, pp. 155–164, 1990.
[52] E. Erwin, K. Obermayer, and K. Schulten, “Self-organizing maps: ordering, con-vergence properties and energy functions,” Biological Cybernetics, vol. 67, no. 1, pp. 47–55, 1992.
[53] Y. Cheng, “Convergence and ordering of Kohonen’s batch map,” Neural Computa-tion, vol. 9, no. 8, pp. 1667–1676, 1997.
[54] S. P. Luttrell, “Self-organization: A derivation from first principles of a class of learning algorithm,” in Proc. IEEE Int. Joint Conf. Neural Networks, pp. II–495–
II–498, 1989.
[55] S. P. Luttrell, “Code vector density in topographic mappings: Scalar case,” IEEE Trans. Neural Networks, vol. 2, no. 4, pp. 427–436, 1991.
[56] T. Graepel, M. Burger, and K. Obermayer, “Phase transitions in stochastic self-organization maps,” Physical Review E, vol. 56, no. 4, pp. 3876–3890, 1997.
[57] T. Graepel, M. Burger, and K. Obermayer, “Self-organizing maps: Generalizations and new optimization techniques,” Neurocomputing, vol. 21, 1998.
[58] T. Heskes, “Self-organizing maps, vector quantization, and mixture modeling,”
IEEE Trans. Neural Networks, vol. 12, no. 6, pp. 1299–1350, 2001.
[59] T. W. S. Chow and S. Wu, “An online cellular probabilistic self-organizing map for static and dynamical data sets,” IEEE Trans. Circuit and Systems, Part I, vol. 51, no. 4, pp. 732–747, 2004.
[60] S. Wu and T. W. S. Chow, “Prsom: A new visualization method by hybridizing multidimensional scaling and self-organizing map,” IEEE Trans. Neural Networks, vol. 16, no. 6, pp. 1362–1380, 2005.
[61] S. P. Luttrell, “A Bayesian analysis of self-organizing maps,” Neural Computation, vol. 6, no. 5, pp. 767–794, 1994.
[62] F. Anouar, F. Badran, and S. Thiria, “Probabilistic self-organizing map and radial basis function networks,” Neurocomputing, vol. 20, no. 1-3, pp. 83–96, 1998.
[63] J. Lampinen and T. Kostiainen, “Generative probability density model in the self-organizing map,” in Self-self-organizing neural networks: Recent advances and applica-tions (U. Seiffert and L. Jain, eds.), pp. 75–94, Physica Verlag, 2002.
[64] M. M. V. Hulle, “Joint entropy maximization in kernel-based topographic maps,”
Neural Computation, vol. 14, no. 8, pp. 1887–1906, 2002.
[65] M. M. V. Hulle, “Maximum likelihood topographic map formation,” Neural Com-putation, vol. 17, no. 3, pp. 503–513, 2005.
[66] J. J. Verbeek, N. Vlassis, and B. J. A. Kr¨ose, “Self-organizing mixture models,”
Neurocomputing, vol. 63, pp. 99–123, 2005.
[67] J. Sum, C. S. Leung, L. W. Chan, and L. Xu, “Yet another algorithm which can generate topography map,” IEEE Trans. Neural Networks, vol. 8, no. 5, pp. 1204–
1207, 1997.
[68] L. D. Consortium, 2001 NIST Speaker Recognition Evaluation Corpus.
http://www.ldc.upenn.edu/.
[69] D. A. Reynolds, “Speaker identification and verification using Gaussian mixture speaker models,” Speech Communication, vol. 17, pp. 91–108, 1995.
[70] S. Chen and P. S. Gopalakrishnan, “Speaker, environment and channel change de-tection and clustering via the Bayesian information criterion,” in Proc. DARPA Broadcast News Transcription and Understanding Workshop, (Lansdowne, VA), pp. 127–132, Feb. 1998.
[71] M. Cettolo, M. Vescovi, and R. Rizzi, “Evaluation of BIC-based algorithms for audio segmentation,” Computer Speech and Language, vol. 19, pp. 147–170, 2005.
[72] S. S. Cheng, H. M. Wang, , and H. C. Fu, “A model-selection-based self-splitting Gaussian mixture learning with application to speaker identification,” EURASIP Journal on Applied Signal Processing, vol. 2004, no. 17, pp. 2626–2639, 2004.
[73] S.-S. Cheng, H.-C. Fu, and H.-M. Wang, “Model-based clustering by probabilistic self-organizing maps,” To appear in IEEE Trans. Neural Network.
[74] S.-S. Cheng, H.-M. Wang, and H.-C. Fu, “BIC-based audio segmentation by divide-and-conquer,” in Proc. Int. Conf. Acoustics, Speech and Signal Processing, pp. 4841–
4844, 2008.
[75] Y. Linde, A. Buzo, and R. M. Gray, “An algorithm for vector quantizer design,”
IEEE Trans. Commun., vol. 28, no. 1, pp. 84–95, 1980.
[76] S. C. Chu and J. Roddick, “Pattern clustering using incremental splitting for non-uniformly distributed data,” in Proc. Int. conf. Knowledge-Based Intelligent Infor-mation Engineering Systems and Allied Technologies, pp. 1037–1041, 2001.
[77] T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning:
Data Mining, Inference, and Prediction. Springer, 2002.
[78] T. Kohonen, S. Kaski, K. Lagus, J. Salojarvi, V. P. J. Honkela, and A. Saarela,
“Self organization of a massive document collection,” IEEE Trans. Neural Networks:
Special Issue on Neural Networks for Data Mining and Knowledge Discovery, vol. 11, no. 3, pp. 574–585, 2000.
[79] K. Lagus, S. Kaski, and T. Kohonen, “Mining massive document collections by the WEBSOM method,” Information Sciences, vol. 163, pp. 135–156, 2004.
[80] L. Gupta and S. Das, “Texture edge detection using multi-resolution features and SOM,” in Proc. IEEE Int. Conf. Pattern Recognition, 2006.
[81] M. Kurimo, “Training mixture density HMMs with SOM and LVQ,” Computer Speech and Language, vol. 11, no. 4, pp. 321–341, 1997.
[82] M. Kurimo, “Fast latent semantic indexing of spoken documents by using self-organizing maps,” in Proc. Int. Conf. Acoustics, Speech and Signal Processing, pp. 2425–2428, 2000.
[83] M. Kurimo, “Thematic indexing of spoken documents by using self-organizing maps,” Speech Communication, vol. 38, no. 1-2, pp. 29–44, 2002.
[84] B. D. Ripley, Pattern Recognition and neural networks. Cambridge University Press, 1996.
[85] R. E. Kass and A. E. Raftery, “Bayes factors,” Journal of the American Statistical Association, vol. 90, no. 430, pp. 773–795, 1995.
[86] A. M. T. Tantrum and W. Stuetzle, “Hierarchical model-based clustering of large datasets through fractionation and refractionation,” in Proc. SIGKDD, 2002.
[87] HTK, Hidden Markov Model Toolkit. http://htk.eng.cam.ac.uk/.
[88] B. L. Pellom and J. H. L. Hansen, “An efficient scoring algorithm for Gaussian mixture model based speaker identification,” Signal Processing Letters, IEEE, vol. 5, no. 11, pp. 281–284, 1998.
[89] C. Miyajima, Y. Hattori, K. Tokuda, T. Masuko, T. Kobayashi, and T. Kitamura,
“Speaker identification using Gaussian mixture models based on multi-space prob-ability distribution,” Proc. of ICASSP, vol. 84, pp. 847–855, 2001.
[90] L. Wang, K. Chen, and H. S. Chi, “Capture interspeaker information with a neural network for speaker identification,” IEEE Trans. Neural Networks, vol. 13, pp. 436–
445, 2002.
[91] F. Bimbot, J.-F. Bonastre, C. Fredouille, G. Gravier, I. Magrin-Chagnolleau, S. Meignier, T. Merlin, J. Ortega-Garc´ıa, D. Petrovska-Delacr´etaz, and D. A.
Reynolds, “A tutorial on text-independent speaker verification,” EURASIP J. Appl.
Signal Process., vol. 2004, no. 1, pp. 430–451, 2004.
[92] The VIMAS speech codec. http://www.vimas.com.
[93] C. Ambroise and G. Govaert, “Constrained clustering and Kohonen self-organizing maps,” Journal of Classification, vol. 13, no. 2, pp. 299–313, 1996.
[94] K. Rose, E. Gurewitz, and G. C. Fox, “Vector quantization by deterministic anneal-ing,” IEEE Trans. Inform. Theory, vol. 38, no. 4, pp. 1249–1257, 1992.
[95] K. Rose, “Deterministic annealing for clustering, compression, classification, regres-sion, and related optimization problems,” Proceedings of The IEEE, vol. 86, no. 11, pp. 2210–2239, 1998.
[96] A. Asuncion and D. Newman, UCI Machine Learning Repository. 2007.
[97] H. Meinedo and J. Neto, “Audio segmentation, classification and clustering in a broadcast news task,” in Proc. Int. Conf. Acoust., Speech, Signal Processing, (Hong Kong, China), pp. II–5–II–8, Apr. 2003.
[98] P. C. Woodland, M. J. F. Gales, D. Pye, and S. J. Young, “The development of the 1996 HTK broadcast news transcription system,” in Proc. DARPA Speech Recognition Workshop, (Lansdowne, VA), pp. 73–78, Feb. 1997.
[99] J. F. Bonastre, P. Delacourt, C. Fredouille, T. Merlin, and C. J. Wellekens, “A speaker tracking system based on speaker turn detection for NIST evaluation,” in Proc. Int. Conf. Acoust., Speech, Signal Processing, (Istanbul, Turkey), pp. 1177–
1180, Jun. 2000.
[100] S. E. Tranter and D. A. Reynolds, “An overview of automatic speaker diarization systems,” IEEE Trans. Audio, Speech and Language Processing, vol. 14, no. 5, pp. 1557–1565, 2006.
[101] M. Siegler, U. Jain, B. Raj, and R. Stern, “Automatic segmentation, classifica-tion and clustering of broadcast news audio,” in Proc. DARPA Speech Recogniclassifica-tion Workshop, (Chantilly, VA), pp. 97–99, Feb. 1997.
[102] A. Tritschler and R. A. Gopinath, “Improved speaker segmentation and segments clustering using the Bayesian information criterion,” in Proc. Eur. Conf. Speech Communication and Technology, (Budapest, Hungary), pp. 679–682, Sep. 1999.
[103] P. Sivakumaran, J. Fortuna, and A. M. Ariyaeeinia, “On the use of the Bayesian information criterion in multiple speaker detection,” in Proc. Eur. Conf. Speech Communication and Technology, (Aalborg, Denmark), pp. 795–798, Sep. 2001.
[104] P. Delacourt and C. J. Wellekens, “DISTBIC: A speaker-based segmentation for audio data indexing,” Speech Communication, vol. 32, no. 1, pp. 111–126, 2000.
[105] J. W. Hung, H. M. Wang, and L. S. Lee, “Automatic metric-based speech segmen-tation for broadcast news via principal component analysis,” in Proc. Int. Conf.
Spoken Language Processing, (Beijing, China), pp. 121–124, Oct. 2000.
[106] R. Bakis, S. Chen, P. S. Gopalakrishnan, R. Gopinath, S. Maes, L. Polymenakos, and M. Franz, “Transcription of broadcast news shows with the IBM large vocab-ulary speech recognition system,” in Proc. DARPA Speech Recognition Workshop, (Chantilly, VA), pp. 67–72, Feb. 1997.
[107] B. W. Zhou and J. H. L. Hansen, “Efficient audio stream segmentation via the combined T2 statistic and Bayesian information criterion,” IEEE Trans. Speech and Audio Processing, vol. 13, no. 4, pp. 467–474, 2005.
[108] S. S. Cheng and H. M. Wang, “A sequential metric-based audio segmentation method via the Bayesian information criterion,” in Proc. Eur. Conf. Speech Com-munication and Technology, (Geneva, Switzerland), pp. 945–948, Sep. 2003.
[109] H. M. Wang, B. Chen, J. W. Kuo, and S. S. Cheng, “MATBN: A Mandarin Chinese broadcast news corpus,” Int. Journal of Computational Linguistics and Chinese Language Processing, vol. 10, no. 2, pp. 219–236, 2005.
[110] NIST, 1998 DARPA/NIST HUB-4 broadcast news evaluation test data.
http://www.nist.gov/speech/tests/bnr/hub4 98/hub4 98.htm.
[111] U. Manber, Introduction to Algorithms: A Creative Approach. Addison-Wesley, 1989.