In this section, we implement the software system to support collaborative testing with our proposed methodology, so called collaborative testing system. We evaluate the performance of our approach by three experiments. The architecture of our collaborative testing system is shown in Figure 8. The system contains six main modules, ―User Interface‖, ―Test Guiding Module‖, ―User Session Module‖, ―Bug Report System‖, ―Admin Module‖, and ―Graph Construction‖, and three databases,
―User Profile Database‖, ―User Session Log Database‖, and ―Dependence Graph Database‖. The ―User Interface‖ provides interact interface to participants. The module ―Graph Construction‖ transforms web application into a corresponding dependence graph. The ―Admin Module‖ is the administration functions provided to system administrator to monitor the testing, as shown in Figure 9. The ―Test Guiding Module‖ provides a testing objective for testers. In Figure 10, we describe the testing flows for collaborative testing system. The module ―Bug Report System‖ provides bug report function for tester. The ―User Session Module‖ records user session log and change the testing state immediately.
43
Figure 8. Collaborative testing system module architecture
Figure 9. Administration windows for User session logs
45
5.2 Experiments and Results
5.2.1 Experiment I - Dependence graph construction
In this experiment, we transform three web applications ―BookStore‖ 、
―Classifieds‖[http://www.gotocode.com/] and an e-learning platform called
―OOLAYScience‖ into dependence graph respectively according to our proposed approach, and the statistic result is shown in Table 8 and Figure 11. According to these result, it is found out that the complexity of dependence relationships of Page-Level is much higher than Function-Level and Code-Level.
Table 8. Dependence graph statistic result Application Name & Level Type Quantity BookStore
Page-Level Node 25
Edge 152
Function-Level Node 352
Edge 350
Function-Level Node 253
Edge 293
Code-Level Node 541
Edge 912
OOLAYScience
Page-Level Node 43
Edge 284
Function-Level Node 74
Edge 174
Figure 11. Dependence graph statistic result
47
5.2.2 Experiment II – Performance Evaluate Simulation
In general, folksonomy testing needs a large amount of human resource to perform test. In this paper, we implement a simulating system to evaluate the performance of our proposed approach. We propose three experiments with different parameter value to show the relationship between parameter and result. In our simulation each virtual testers has own trustworthy weight from 0.1 to 1, and accesses pages with different execution time. We simulate the testing activities of testers and collect testing session logs to perform our folksonomy-base approach. We guiding tester with our own method.
In the first experiment, we try to find out the relation between total completion time and tester quantity by using the same web application ―BookStore‖. Figure 12、
Figure 13 and Figure 14 show the charts with different support threshold value with 10、20 and 30, that means a page called a confirm page if and only if every nodes in the Code-Level can be execute by 10、20 and 30 times. Observing these charts we can conclude some conclusions below.
(1) The experimental results show that the evaluation of our approach is more efficient and well performance than traditional one. The average reduction rate for testing effort is almost 90%.
(2) The more testers participate in testing, the shorter completion time we get. But the reduce rate also grow much slowly with more testers.
Figure 12. ―BookStore‖ web application with 10 support threshold
Figure 13. ―BookStore‖ web application with 20 support threshold
49
In the second experiment, we try to find out the relation between total completion times with different support threshold by using the same web application ―BookStore‖.
Figure 15 and Figure 16 show the charts with different testing approach,
―Non-Guiding‖ and ―Guiding‖ approach respectively. Observing these two charts we can conclude that total completion time is almost in direct ratio with different value of support threshold. The much higher threshold value, the higher completion time we get.
Figure 15. ―BookStore‖ web application with Non-Guiding testing approach
Figure 16. ―BookStore‖ web application with Guiding testing approach
In the third experiment, we try to compare the total completion time with different web application execute by two kind of testing approach, and with the 100 testers and 10 support threshold show on Figure 17. According to Figure 17 we can find out that
―OOLAYScience‖ web application has the biggest different between ―Non-Guiding‖
and ―Guiding‖ approach. We refer all these results to the complexity of web application.
According to the result of ―Experiment I - Dependence graph construction‖, we can know ―OOLAYScience‖ web application has the maximum of nodes in the Page-Level.
Because of this property, some pages existed in the ―OOLAYScience‖ web application will much harder to execute by tester because the execution path which include those pages is too long to execute from beginning to end. Therefore, the effect of ―Guiding‖
approach will much obvious also.
Figure 17. Costing time comparing with different web application
51
5.2.3 Experiment III – Performance Evaluation for Folksonomy-based Collaborative Testing
In this experiment, we create a real testing environment of ―BookStore‖ web application provide for folksonomy testers which existed on the social network
―Facebook‖ to participate our testing. This experiment cans division into two different testing approaches, ―Non-Guiding‖ and ―Guiding‖ respectively, but same as other conditions, and the value of support threshold in this experiment is 10. Figure 10 shows the testing flows of collaborative testing system with ―Guiding‖ approach. The difference between ―Non-Guiding‖ and ―Guiding‖ is the guiding hint and the bug hint suggestions show on Figure 10, ―Non-Guiding‖ approach non-exist these suggestions.
In this experiment, we hide three bugs in the ―BookStore‖ web application, vote image error、registration E-mail error and modify shopping quantity error. Figure 18 shows the information about this experiment, more testers will report more bug reports, but still existed much garbage bug reports, and in this experiment, ―Non-Guiding‖ and
―Guiding‖ approach has the same ability in the bug detection, ―Non-Guiding‖ approach find out image error and modify shopping quantity error, ―Guiding‖ approach find out image error and registration E-mail error.
Figure 18. Statistics of tester quantity and bug report about two kind of approach
Figure 19 shows the costing time of all testers spend on each page, to observe this result of experiment we can understand ―AdvSearch‖ and ‖MyInfo‖ is the bottleneck in testing.
Figure 20 shows the total completion time comparing of two different guiding approaches, and the reduction rate for testing effort is almost 90% also.
Figure 19. Costing time of all testers spend on each page
53
Chapter 6. Conclusion
In this thesis, we propose our folksonomy-based approach to support collaborative testing for Web application. In construction phase, we define the 3-level dependence graphs to represent the structures of web application. The different dependence graph means different usability testing, page-level testing 、 function-level testing and code-level testing. According to the dependence graphs, we can realize the structures of web application and help us assign jobs in testing. According to the real testing situation, we model the job assignment problem with a mathematics formulation, and prove that the problem is an NP-Complete problem. By reducing the ―Optimum representative set‖[39] problem to our problem. We propose our approach to support testing with the collaborative idea. Finally, we implement a collaborative testing system to evaluate our proposed model, the experiment results show that our approach is effective and well performance in testing and defect revealing. And the average reduction rate for testing effort is almost 90%.
References
[1] A. A. Andrews, J. Offutt, and R. T. Alexander, ―Testing Web applications by modeling with FSMs‖, Software Systems and Modeling, Jul. 2005, pp. 326-345.
[2] Andreas Hotho, Robert J¨aschke, Christoph Schmitz, and Gerd Stumme.
Information retrieval in folksonomies: Search and ranking. In York Sure and John Domingue, editors, The Semantic Web:Research and Applications, volume 4011 of Lecture Notes in Computer Science, p.411–426, Heidelberg, June 2006. Springer.
[3] B. Boehm and A. Egyed, "Software Requirements Negotiation: Some Lessons Learned," in the 20th International Conference on Software Engineering(ICSE'98), Kyoto, Japan, 1998, pp.503-507.
[4] Ben Lund, Tony Hammond, Martin Flack, and Timo Hannay. Social Bookmarking Tools (II): A Case Study - Connotea. D-Lib Magazine, April 2005.
[5] B. S. Lerner, L. J. Osterweil, Stanley M. Sutton Jr., and A. Wise, "Programming Process Coordination in Little-JIL Toward the Harmonious Functioning of Parts for Effective Results," in European Workshop on Software Process Technology, 1998.
[6] Briand, L.C., Wust, J., Daly, J.W. and Porter, D.V. Exploring the Relationships between Design Measures and Software Quality in Object-Oriented Systems. The Journal of Systems and Software, 51, pp. 245-273, 2000.
[7] C. Fu, B. Ryder, A. Milanova, and D. Wonnacott, ―Testing of Java Web Services for Robustness,‖ Proc. ACM SIGSOFT Int’l Symp. Software Testing and Analysis, pp.
23-34, 2004.
[8] C.-H. Liu, D.C. Kung, and P. Hsia, ―Object-Based Data Flow Testing of Web Applications,‖ Proc. First Asia-Pacific Conf. Quality Software, pp. 7-16, 2000.
55
[10] Chidamber, S.R. and Kemerer, C.F. A Metrics Suite for Object-Oriented Design.
IEEE Trans. on Soft. Eng., pp. 476-493, 1994.
[11] D. Benz, K. Tso, and L. Schmidt-Thieme. Automatic bookmark classification: A collaborative approach. In Proceedings of the Second Workshop on Innovations in Web Infrastructure (IWI 2006), Edinburgh,Scotland, 2006.
[12] D. C. Kung, C. H. Liu, P. Hsia, ―An object-oriented Web test model for testing Web applications‖, in Proceedings of IEEE 24th Annual International Computer Software and Applications Conference, Taipei, Taiwan, Oct. 2000, pp. 537-542.
[13] D. Jeffrey and N. Gupta, ―Test Suite Reduction with Selective Redundancy,‖ Proc.
21st IEEE Int’l Conf. Software Maintenance, pp. 549-558, 2005.
[14] D. Leon, W. Masri, and A. Podgurski, ―An Empirical Evaluation of Test Case Filtering Techniques Based on Exercising Complex Information Flows,‖ Proc. 27th Int’l Conf. Software Eng., pp. 412-421, 2005.
[15] D. Leon and A. Podgurski, ―A Comparison of Coverage-Based and Distribution-Based Techniques for Filtering and Prioritizing Test Cases,‖ Proc. 14th Int’l Symp. Software Reliability Eng., pp. 442-453, 2003.
[16] E. J. Whitehead, Jr. and Y. Y. Goland, "WebDAV: A Network Protocol for Remote Collaborative Authoring on the Web," in 6th European Conference on Computer Supported Cooperative Work (ECSCW'99), Copenhagen,Denmark, 1999, pp. 291-310.
[17] F. Ricca and P. Tonella, ―Analysis and Testing of Web Applications‖, In Proceedings of the 23rd International Conference on Software Engineering, Toronto, Ontario, Canada, 2001, pp.25-34.
[18] G. A. D. Lucca and A. R. Fasolino, ―Testing Web-based Applications: The State of the Art and Future Trends‖, Information and Software Technology, 2006(48):
1172-1186.
[19] G. A. Bolcer and R. N. Taylor, "Endeavors: a Process System Integration
Infrastructure," in 4th International Conference on the Software Process (ICSP'96), Brighton, UK, 1996, pp. 76-89.
[20] G. Di Lucca, A. Fasolino, F. Faralli, and U.D. Carlini, ―Testing Web Applications,‖
Proc. 18th IEEE Int’l Conf. Software Maintenance, pp. 310-319, 2002.
[21] Gilad Mishne. Autotag: a collaborative approach to automated tag assignment for weblog posts. In WWW ’06: Proceedings of the 15th international conference on World Wide Web, pages 953–954, New York, NY, USA, 2006. ACM Press.
[22] Hutchens, D.H. and Basili, V.R. System Structure Analysis: Clustering with Data Bindings. IEEE Trans. on Soft. Eng., 11, pp. 749-757,1985.
[23] H. Halpin, V. Robu, and H. Shepard. The dynamics and semantics of collaborative tagging. In Proceedings of the 1st Semantic Authoring and Annotation Workshop (SAAW’06), 2006.
[24] H. Miao, Z. Qian, and B. Song, ―Towards automatically generating test paths for Web application Testing,‖ International Sympothium on Theoretical Aspects of Sotware Engineering, 2nd IFIP/IEEE, 2008.
[25] Horwitz, S., Reps, T., and Binkley, D. Interprocedural slicing using dependence graphs. ACM Trans. on Programming Languages and Systems, 22, pp. 26-60, 1990.
[26] J. A. Jones and M. J. Harrold, ―Test Suite Reduction and Prioritization for Modified Condition/Decision Coverage,‖ IEEE Trans. Software Eng., vol. 29, no. 3, pp. 195-209, Mar. 2003.
[27] Jon M. Kleinberg. Authoritative sources in a hyperlinked environment. Journal of the ACM, 46(5):604–632, 1999.
[28] J. Offutt, J. Pan, and J. Voas, ―Procedures for Reducing the Size of Coverage-Based Test Sets,‖ Proc. 12th Int’l Conf. Testing Computer Software, pp.
111-123, 1995.
[29] J. Offutt and W. Xu, ―Generating Test Cases for Web Services Using Data Perturbation,‖ Proc. Workshop Testing, Analysis, and Verification of Web Services,
57
[32] L. Dusseault, WebDAV: Next-Generation Collaborative Web Authoring, Prentice Hall PTR, 2003.
[33] L. Wakeman and J. Jowett, PCTE: The Standard for Open Repositories: Prentice Hall, 1993.
[34] M. Benedikt, J. Freire, and P. Godefroid, ―VeriWeb:Automatically Testing Dynamic Web Sites‖, In Proceedings of 11th International World Wide Web Conference, Honolulu, HI, USA, May 2002, pp. 654-668.
[35] M. Dubinko, R. Kumar, J. Magnani, J. Novak, P. Raghavan, and A. Tomkins.
Visualizing tags over time. In Proc. of the 15th International WWW Conference, Edinburgh, Scotland, 2006.
[36] M. Harder, J. Mellen, and M.D. Ernst, ―Improving Test Suites via Operational Abstraction,‖ Proc. 25th Int’l Conf. Software Eng., pp. 60-71, 2003.
[37] M. J. Harrold, R. Gupta, and M. L. Soffa, ―A Methodology for Controlling the Size of a Test Suite,‖ ACM Trans. Software Eng. and Methodology, vol. 2, no. 3, pp.
270-285, July 1993.
[38] M. Wang, J. Yuan, H. Miao, and G. Tan, ―A static analysis approach for automatic generating test cases for Web applications,‖ International Conference on Computer Science and Software Engineering, 2008.
[39]M.R. Garey , D.S. Johnson, In: V. Klee (Ed.), Computers and intractability, a guide to the theory of NP-completeness, Freeman, New York, 1979.
[40] Peter Mika. Ontologies Are Us: A Unified Model of Social Networks and Semantics. In Yolanda Gil, Enrico Motta, V. Richard Benjamins, and Mark A.
Musen, editors, ISWC 2005, volume 3729 of LNCS, pp.22–536, Berlin Heidelberg, November 2005. Springer-Verlag.
[41] S. M. Master and A. Memon, ―Call Stack Coverage for Test Suite Reduction,‖ Proc.
21st IEEE Int’l Conf. Software Maintenance, pp. 539-548, 2005.
[42] S. Elbaum, G. Rothermel, S. Karre, and M. Fisher II, ―Leveraging User Session Data to Support Web Application Testing,‖ IEEE Trans. Software Eng., Vol. 31, No.
3, pp. 187-202, Mar. 2005.
[43] Selby, R.W. and Basili, V.R. Analyzing Error-Prone System Structure. IEEE Trans.
on Soft. Eng., pp. 141-152, 1991.
[44] Stevens, W.P., Myers, G.J. and Constantine, L.L. Structure Design. IBM Systems Journal, 13, pp. 231-256, 1974.
[45] T. Y. Chen and M. F. Lau, ―Dividing Strategies for the Optimization of a Test Suite,‖ Information Processing Letters, vol. 60, no. 3, pp. 135-141, Mar. 1996.
[46] Tony Hammond, Timo Hannay, Ben Lund, and Joanna Scott. Social Bookmarking Tools (I): A General Review. D-Lib Magazine, April 2005.
[47] William G. J. Halfond and Alessandro Orso, ―Improving test case generation for Web applications using automated interface discovery‖, ESEC/FSE’07 Sep., 2007, pp. 145-154.
[48] X. Jia and H. Liu, ―Rigorous and automatic testing of Web applications‖, In 6th IASTED International Conference on Software Engineering and Applications, Nov.
2002, pp. 280-285.
[49] Y. Deng, P. Frankl, and J. Wang, ―Testing Web Database Applications,‖ SIGSOFT Software Eng. Notes, Vol. 29, No. 5, pp. 1-10, 2004.
[50] Zhichen Xu, Yun Fu, Jianchang Mao, and Difu Su. Towards the semantic web:
Collaborative tag suggestions. In Proceedings of the Collaborative