Learning assessment is an important and essential part in the process of learning.
With the growth of the computer and the Internet technology, the learning assessment via the computer and the Internet, called Computer-Based Testing (CBT) has become a trend especially in the information technology (IT) certification exam. In software skill certification, such as MS Word certification [1], there are usually two phases of tests. The first phase is to test the examinee’s declarative knowledge, the knowledge of knowing what, which is usually the traditional single-choice or multiple-choice testing. The second phase is to test the examinee’s procedural knowledge, the knowledge of knowing how, called Performance Based Testing (PBT) [2-4]. In PBT, the examinees have to perform some real world tasks by using specific software to demonstrate their software operating skills. For example, the examinee might be asked to complete the following task “At the beginning of the document, insert a Table of Contents showing two heading levels.” in MS Word 2003 certification exam. To complete this task, the examinee has to perform a sequence of actions on MS Word to get the required results and the examinee’s software operating skills can be assessed during the process of accomplishing this task. Besides, PBT allows the examinees to come up with the correct answer using different approaches. That is, the examinee can solve a problem by showing how to navigate through a sequence of processes to get the required results.
Currently, the examinees have to use actual software to realize PBT in software skill certification. Besides, since the actions of most software performed by the examinee during the testing are not recorded, the evaluation of the examinee’s software operating skills which only can be manually done by the teacher is costly
and time-consuming. In recent years, a testing platform for PBT, called the PBT tester, on some specific software which can provide adequate functionality of the software to complete the required tasks and track the examinee’s action sequence were proposed as a cost effective way to realize PBT in software skill certification. However, to build a PBT tester for some specific software, the control flows of testing scenario should be simulated in the tester so that the software output can be correctly visualized according to the examinee’s sequence of actions. Thus, the programming efforts of building a PBT tester are costly. Besides, the reusability for the new PBT test items is also a critical issue for the PBT tester developer.
With our observation, using software to perform a sequence of actions to complete the task seems like a navigation process from the starting point of the software run-time state to the final state to get the required results. Our idea is to use Finite State Machine (FSM) to model what the examinee performs during the PBT in software skill certification, where the current state of FSM represents the software run-time status, and the transitions of FSM represent the actions the examinee can perform. Once the examinee performs an action in a certain state, the corresponding state transition will be triggered to move from the current state to the next state and then the PBT tester will visualize the next software state. Based on this concept, a set of regular grammar, called the Functional Specification Language (FSL), is defined to describe the software run-time status and the transitions of the PBT tester according to the functionality of specific software. Thus, a parser generator can be applied to generate corresponding software tester based on the given FSL, where the grammar rule of FSL can be used to model the action sequence performed by the examinee, the non-terminals of FSL represent the software run-time status, and the terminals of FSL represent the actions the examinee can perform. Besides, action symbols are attached to each terminal symbol to trigger the corresponding action routines, such as the
routines for visualizing the next software run-time status and recording the actions performed by the examinee. To effectively visualize the next software run-time status, the visualization is divided into backgrounds and foregrounds, where the visualization action routine substitutes the foregrounds according to the action performed by the examinee, and the backgrounds do not need to be changed.
To evaluate the correctness, reusability, and expressive power of proposed scheme, we have implemented a prototypical system in the web environment. In this thesis, several test cases are designed to evaluate the expressive power of proposed scheme. According to the feedbacks of the teachers and the tester developers, we may conclude that the proposed scheme is workable and beneficial for them.
The remainder of the article is organized as follows. In Chapter 2, we introduce some related works about the traditional PBT tester and authoring tools for creating the PBT tester. Then, the proposed Tester Generator Scheme and the application of PBT tester are described in Chapter 3 and Chapter 4 respectively. Chapter 5 discusses the system implementation and experiments. Finally, Chapter 6 gives the conclusion and future work.