In this research project, we have proposed an easily-extensible factor framework for the adaptability of software assessment models and their associated techniques. In order to demonstrate the practicability of this framework, we have also chosen one of the important factors, namely the number of assessment layers required for reaching the decision in an assessment model, for this experiment. We first classified these software assessment models into two types: one-layered and multi-layered software assessment models, and compared their assessment accuracy and efficiency. Based on this research results, we have written a paper and submitted it to the Journal of Information and Software Technology for its consideration for the publication in September, 2006.
Reference
1. Standish Group, CHAOS Demographics and Project Resolution, Third Quarter 2004, Web Link:http://standishgroup.com/sample_research/PDFpages/q3-spotlight.pdf.
2. CMU/SEI CMMI Product Team, CMU/SEI-2002-TR-012: Capability Maturity Model Integration (CMMI-SE/SW/IPPD/SS, V1.1, Staged Representation), Mar. 2002.
3. ISO/IEC, 15939: Software Engineering-Software Measurement Process, 2002.
4. ISO/IEC, 14598: Information Technology-Software Product Evaluation, 1998-2001.
5. ISO/IEC, 15504: Information Technology-Software Process Assessment, 2003-2006.
6. J. Bansiya, and C.G. Davis, A hierarchical model for object-oriented design quality assessment, IEEE Transactions on Software Engineering, Vol. 28, Issue 1, pp.4-17, Jan.
2002.
7. R. Paul, Metric-based neural network classification tool for analyzing large-scale software, Proceedings of Fourth International Conference on Tools with Artificial Intelligence, pp.108-113, 10-13 Nov. 1992.
8. T.M. Khoshgoftaar and D.L. Lanning, A neural network approach for early detection of program modules having high risk in the maintenance phase, Journal of Systems and Software, Vol. 29, pp.85-91, 1995.
9. R. Kumar, S. Rai and J.L. Trahan, Neural-network techniques for software-quality evaluation, Proceedings of Annual Reliability and Maintainability Symposium, pp.155-161, 19-22 Jan. 1998.
10. I. Anagnostopoulos, C. Anagnostopoulos, V. Loumos and E. Kayafas, Classifying web pages employing a probabilistic neural network, IEE Proceedings-Software (see also IEE Proceedings-Software Engineering), Vol.151, Issue 3, pp.139-150, 7 June 2004.
11. J. Moses, Bayesian probability distributions for assessing measurement of subjective software attributes, Information and Software Technology, Vol. 42, Issue 8, pp.533-546, 15 May 2000.
12. T.M. Khoshgoftaar and N. Seliya, Analogy-based practical classification rules for software quality estimation, Empirical Software Engineering, Vol. 8, Issue 4, pp.325-350, 2003.
13. V. Rodriguez and W.T. Tsai, Evaluation of software metrics using discriminate analysis, Journal of Information and Software Technology, Vol. 29, Issue 3, pp.245-251, 1987.
14. T.M. Khoshgoftaar, E.B. Allen, K.S. Kalaichelvan and N. Goel, Early quality prediction:
a case study in telecommunications, IEEE Software, Vol. 13, Issue 1, pp.65-71, Jan.
1996.
15. M. Pighin and R. Zarnolo, A predictive metric based on discriminant statistical analysis, Proceedings of 9th International Conference on Software Engineering. ACM Press, May 1997.
16. H. SungBack and K. Kapsu, An empirical study on identifying fault-prone module in large switching system, Proceedings of Twelfth International Conference on Information Networking, pp.415-418, 21-23 Jan. 1998.
17. Y. Ping, T. Systa and H. Muller, Predicting fault-proneness using OO metrics. An industrial case study, Proceedings of Sixth European Conference on Software
Maintenance and Reengineering, pp.99-107, 11-13 March 2002.
18. J.C. Esteva, Learning to recognize reusable software modules using an inductive classification system, In Proceedings of 5th Jerusalem Conference on Information Technology, pp.278-285, 22-25 Oct. 1990.
19. A.A. Porter and R.W. Selby, Empirically guided software development using metric-based classification trees, IEEE Software, Vol. 7, Issue 2, pp.46-54, 1990.
20. T.M. Khoshgoftaar, E.B. Allen, L.A. Bullard, R. Halstead and G.P. Trio, A tree-based classification model for analysis of a military software system, 1996. Proceedings of IEEE High-Assurance Systems Engineering Workshop, pp.244-251, 21-22 Oct. 1996.
21. V.R. Basili, S.E. Condon, K.E. Emam, R.B. Hendrick and W. Melo, Characterizing and modeling the cost of rework in a library of reusable software components, Proceedings of 19th international conference on Software engineering, ACM Press, May 1997.
22. T.M. Khoshgoftaar, and N. Seliya, Software quality classification modeling using the sprint decision tree algorithm, In Proceedings of 14th International Conference on Tools with Artificial Intelligence, Washington, DC, USA: IEEE Computer Society, pp.365-374, November 2002.
23. C. Ebert, Classification techniques for metric-based software development, Software Quality Journal, Vol. 5, Issue 4, pp.255-272, 1996.
24. L.C. Briand, W.M. Thomas and C.J. Hetmanski, Modeling and managing risk early in software development, Proceedings of 15th International Conference on Software Engineering, pp.55-65, 17-21 May 1993.
25. T.M. Khoshgoftaar and E.B. Allen, Logistic regression modeling of software quality, International Journal of Reliability, Quality and Safety Engineering, Vol. 6, Issue 4, pp.303–317, 1999.
26. G. Denaro, S. Morasca and M. Pezzè, Validation and verification: Deriving models of software fault-proneness, Proceedings of 14th International Conference on Software Engineering and Knowledge Engineering, ACM Press, July 2002.
27. R. Hochman, T.M. Khoshgoftaar, E.B. Allen and J.P. Hudepohl, Evolutionary neural networks: A robust approach to software reliability problems, Proceedings of Eighth International Symposium on Software Reliability Engineering, pp.13-26, 2-5 Nov. 1997.
28. D.F. Schenker and T.M. Khoshgoftaar, The application of fuzzy enhanced case-based reasoning for identifying fault-prone modules, Proceedings of Third IEEE International High-Assurance Systems Engineering Symposium, pp.90-97, 13-14 Nov. 1998.
29. T.M. Khoshgoftaar, N. Seliya and L. Yi, Genetic programming-based decision trees for software quality classification, Proceedings of 15th IEEE International Conference on Tools with Artificial Intelligence, pp.374-383, 3-5 Nov. 2003.
30. A.A. Porter and R.W. Selby, Evaluating techniques for generating metric-based classification trees, Journal of Systems and Software, Vol. 12, pp.209-218, 1990.
31. T.M. Khoshgoftaar, D.L. Lanning, and A.S. Pandya, A Comparative study of pattern recognition techniques for quality evaluation of telecommunications software, IEEE Journal on Selected Areas in Communications, Vol. 12, Issue 2, pp.279-291, 1994.
32. K.E. Emam, S. Benlarbi, N. Goel and S.N. Rai, Comparing case-based reasoning classifiers for predicting high risk software components, Journal of Systems and Software, Vol. 55, pp.301-320, 2001.
33. M. Reformat, W. Pedrycz and N.J. Pizzi, Software quality analysis with the use of computational intelligence, Journal of Information and Software Technology, Vol. 45, Issue 7, pp.405-417, 1 May 2003.
34. M.Y. Cheng, S.C. Cheung and T.H. Tse, Towards the application of classification techniques to test and identify faults in multimedia systems, Proceedings of Fourth International Conference on Quality Software, pp.32-40, 2004.
35. T.M. Khoshgoftaar and N. Seliya, Comparative assessment of software quality classification techniques: An empirical case study, Empirical Software Engineering, Vol.
9, Issue 3, pp.229–257, 2004.
36. T.M. Khoshgoftaar, and N. Seliya, The necessity of assuring quality in software measurement data, In Proceedings of 10th International Symposium on Software Metrics, pp.119-130, 14-16 Sept. 2004.
37. P. Bellini, I. Bruno, P. Nesi and D. Rogai, Comparing fault-proneness estimation models, Proceedings of 10th IEEE International Conference on Engineering of Complex Computer Systems, pp.205-214, 16-20 June 2005.
38. S. Zhong, T.M. Khoshgoftaar and N. Seliya, Analyzing software measurement data with clustering techniques, IEEE Intelligent Systems, Vol. 19, Issue 2, pp.20-27, Mar-Apr 2004.
39. N.F. Schneidewind, Software metrics model for integrating quality control and prediction, Proceedings of 8th International Symposium on Software Reliability Engineering, IEEE Computer Society, Albuquerque NM USA, pp.402-415, 1997.
40. T.M. Khoshgoftaar and E.B. Allen, A practical classification rule for software quality models, IEEE Transactions on Reliability, Vol. 49, Issue 2, pp.209-216, 2000.
41. T.M. Khoshgoftaar, N. Seliya and A. Herzberg, Resource-oriented software quality classification models, Journal of Systems and Software, Vol. 76, Issue 2, pp.111-126, May 2005.
42. P. Burman, A comparative study of ordinary cross-validation, v-fold cross-validation and the repeated learning-testing methods, Biometrika, Vol. 76, pp.503-514, 1989.
43. A. Saha, Neural Network Models in Excel for Classification and Its Tutorial, Web Link:
http://www.geocities.com/adotsaha/NNinExcel.html.
44. SPSS Inc., SPSS v.13 Tutorial, 2004.
45. J.R. Quinlan, C4.5 Release 8, 1997, Web Link: http://www.rulequest.com/Personal/.
46. J.R. Quinlan, C4.5: Programs for Machine Learning. San Mateo, California: Morgan Kaufmann, 1993.
47. W.M. Han and S.J. Huang, An empirical analysis of risk components and performance on software projects, Journal of Systems and Software, In Press.