Chapter 2 Related works
2.2 Programming learning behavior modeling
Model tracing
To support the programming learning, previous studies [22][61] proposed the algorithm animation to demonstrate the execution process of program. With the progress of the program development kit, most of the debugging tools allow the program to be traced step by step and assist the programmer discover the bugs with less effort. However, these tools are basically designed for programmers who can trace the program by themselves, since the provided messages are too difficult for novice learners to understand.
To support the programming learning, the Intelligent Tutoring Systems such as LISP Tutor [3] or DISCOVER [60] are proposed with the interactive practice system using model tracing approach. In the debugging test, the designed buggy patterns are placed in the program and the novice learner is asked to discover them. In [40], a tutoring system with program debugging interface was proposed. However, for the
novice learners who can‟t write the correct program, how to assess their possible misconception from mistake symptoms and provide the appropriate remedial learning is also an interesting and challenging issue.
Programming learning with context of game design
To motivate the students‟ learning interest, the game design is widely accepted as an effective way [47][52]. The researches about game design with learning models such as the learning through project-based learning [20] or problem-based learning [64] with innovative programming laboratories were proposed. These researches show that the interesting game design as learning scenario can effectively engage the students‟ learning. With our observation, the common attractive point of these learning models is that the game design learning content provides interesting scenario instead of abstract concepts provided by traditional lecturing approach.
The narrative and storytelling has a long history of use in structuring, organizing and communicating human. The benefits of using cases and stories for instruction have been demonstrated in many studies. In [77], the narrative-based interactive learning environment generates the cases from knowledge base to support the training of novice in decision making. In [39], the Case-Based Reasoning approach has been applied to support the construction of cognitive model of simulation-based learning system and serious games. These studies used the knowledge base system to support the generation of learning cases with meaningful scenario for the students.
Social network assisted programming learning
Besides the syntactic level learning, the training of problem solving skills such as the learning through project-based learning [11][17][20][65] or problem-based
9
learning [35][64] with innovative programming laboratories were investigated. The interesting learning context such as game design was adopted to motivate the learners‟
engagement. As even small projects are usually implemented in teamwork, the collaboration among members becomes a new issue. Thus, researches based on the social-culture constructivism were proposed to provide the collaborative programming environments [14][15][17][51][56][58][60][70] or the peer assessment activities [9][44][69]. The collective, collaborative learning tools such as discussion board, e-mail, etc are integrated in the learning platform. The assessment through portfolio analysis usually applied to model the behaviors of students [16][18][41][50][63][67].
Chapter 3
Self-organizing Behavior Modeling
Problem solving is the most important capability for learners in Programming Language subject. While given the problem, students need to firstly analyze the problem and select an appropriate problem solving strategy for further flow chart design and code implementation. Therefore, how to provide the adaptive learning guidance for problem analysis is our concern.
3.1 Behavior modeling problem
Current students‟ programming learning activities such as programming forum discussion, online testing, reading online technical articles, download sample codes, content sharing, etc. are getting more and more popular. To provide the adaptive learning guidance for problem analysis, the learning contexts such as visualization tools, inquiry learning companions, technical documents, etc. are important resources in Web 2.0-based e-Learning environment. The platforms of social network service, web forum, social bookmark, Wikipedia, web games, etc. can act as the potential activity resources to build the students‟ behavior model. As Shown in Figure 3.1, the students‟ behaviors of learning diagnosis, project-based game creation, inquiry learning, and knowledge sharing activities can be obtained from the accessing of resource contexts and interaction or collaboration with social contexts.
11
Figure 3.1 The e-learning activities with different contents and contexts
However, the multidisciplinary knowledge and the changing of learning community on the Web 2.0 platforms make the students‟ behaviors more complex.
Therefore, how to acquire and update the learning behavior models becomes an important and challenging issue. In this dissertation, the learning behavior modeling problem is defined as followings.
Definition 3.1 The Learner Behavior Modeling Problem (LBMP)
Given the learning content and context of learning activity, LBMP is how to model the students‟ behaviors to provide adaptive learning guidance.
To simplify the discussion, the inquiry-based programming learning scenario is proposed with three learning activities as shown in Figure 3.2 and the behavior modeling are discussed in Chapter 4, Chapter 5 and Chapter 6, respectively. While
students participate in the inquiry-based programming e-learning, they firstly participate in the self-assessment activity on the online testing system to discover the possible misconceptions. While the misconception for specific programming topic is discovered, the learner can inquire the solution with other community members on the Web forum by posting questions. After several iterations of discussions, while the students figure out the misconception, they can share the learned knowledge on the Web content repository by uploading the learning reports or extending the concept ontology to annotate the new discovery. Besides, the students can retrieve the contents related to the learned concepts and adds their comments for the contents.
Figure 3.2 The inquiry-based programming e-learning
Students on different platforms usually have different behaviors due to different contexts. Therefore, under different learning contexts, the behavior modeling problem can be reduced to the problem solving strategy formulation and realization problem for self programming learning, the trustworthy experts modeling problem for inquiry learning, and the consensus building problem for folksonomy-based knowledge sharing activity.
13
3.2 The idea
With our observations, students could have meaningful learning actions when the purposes of the actions in specific learning context are obtained. To model this property in the learning platform, the knowledge-based programming learning platform is proposed to provide learning guidance based on the high level knowledge structure as shown in Figure 3.3. The learning content ontology is used to construct the metadata and assessment rule for the learning status of students. From the obtained portfolio, the guidance rules of learning platform can be extended by the ontology evolving process called ontology crystallization.
Figure 3.3 The knowledge-based programming learning platform
With the knowledge-based learning platform, the knowledge engineering approach is applied to obtain the self-organizing behavior model as shown in Figure 3.4. There are four steps in the self-organizing model which are the knowledge representation to define the extensible ontology structure, the knowledge acquisition
to extend the knowledge from folksonomy and maintain the stability using the proposed Folksonomy-based Delphi method, the knowledge reasoning to provide learning guidance using the assessment rules generated from constructed Ontology and the knowledge retaining to discover new behaviors and maintain the stability by proposed cascading data mining.
Figure 3.4 The knowledge engineering for self-organizing behavior modeling
Accordingly, to build the students‟ behavior model, the Purpose-based Ontology is built to model the purpose of the actions. The definition of Purpose-based Ontology is as followings.
Definition 3.2 The Purpose-based Ontology (O) is defined as O = (P, C, A, R), where
P={p1, p2, …, pn} is a finite set of purpose nodes in the Purpose Layer to represent predefined purposes in the domain.
15
For each purpose pi, the concepts Ci={pi.c1, pi.c2, …}is a finite set of topic nodes in the Concept Layer to represent different topics discussed in the forum. Topic nodes are linked to the corresponding purpose node by the “A Part OF” relations.
For each purpose pi and concept cj, the action Aijk={ pi.cj.a1, pi.cj.a2, …} is a finite set of nodes in the Action Layer to represent the linkages associated to the original action log. The action nodes are linked to the corresponding concept node by the
“Instance Of” relations.
The Purpose-based Ontology can be instantiated to different concepts definition in different learning contexts. Next, the ontology-based learner behavior modeling approach is proposed to analyze the frequent action patterns and organize the obtained patterns with the structure of ontology as learning behaviors. Therefore, the learning guidance issues under different contexts can be resolved by the following behavior modeling approaches to provide the adaptive learning guidance based on the built behavior models.
Under the context of intelligent tutoring system, the Generalized Model Tracing approach is proposed to organize the diagnosis results of different program model tracing with Problem Solving Strategy Ontology. The diagnosis result can be used to provide the learning guidance for problem solving strategy. The learning guidance applications of adaptive programming misconception diagnosis and game rule tuning learning activity are investigated to evaluate the self-assessment behavior model.
Under the context of learning forum, the Cascading Topic Clustering Algorithm and Self-organized Ontology Maintenance Scheme are proposed to organize the forum experts‟ inquiry activities with the Purpose-based Ontology. Building the forum experts‟ behavior model can provide trustworthy expert finding service for
inquiry-based learning. The learning guidance application of trustworthy expert finding service for inquiry learning on the programming learning forum is investigated to evaluate the inquiry behavior model.
Under the context of collaborative constructed sharable content repository, the IRT-Based Metadata Reengineering Scheme is proposed to evaluate the effectiveness of folksonomy tags and resolve the synonym, redundancy and incompleteness problem of metadata by the domain taxonomy. Accordingly, the tag effectiveness value can detect the conflict and provide the tagging guidance to resolve the consensus building problem to obtain the well-tagged metadata. The learning guidance applications of metadata reengineering and intelligent solution retrieval system are investigated to evaluate the knowledge sharing behavior model.
17
Chapter 4
Behavior modeling of self-assessment activity on online testing system
Comprehending the program examples is important for students in the course of Introductory to Programming to develop their problem solving capability. With our observation, well-trained programmers usually have the capability to explain the problem solving strategic concept of program examples. For example, the linear recurrence strategy is used in the Selection Sort Program to reduce the original sorting problem of n elements into the sub-problem of sorting n-1 elements. The recurrence strategy can be used in a series of programming designs such as quick sort, binary search, printing multiplication table, etc. In this chapter, the behavior modeling to provide adaptive learning guidance for problem analysis is discussed.
4.1 Programming behavior modeling to provide adaptive testing and learning guidance
To model the programming learning behavior, the Bloom‟s taxonomy [10] can be used to classify the programming learning achievement into different capability levels.
The first is knowledge level in which the students can recognize the syntax and statement of program. The second is comprehension level in which the students can understand the semantic flow chart of program. The third is application level in which the students can understand the problem solving strategy used in the algorithm [21].
The fourth is problem analysis level in which the learner can analyze the properties of problem and select suitable problem solving strategy. However, most students can
only understand the knowledge level of the program examples. Thus, students are usually frustrated by the high barrier of program designing in the advanced programming course.
In the researches of behavior modeling on programming testing, the model tracing approach [3][40][60] is the most well-known. The traditional model tracing approach is based on a sample program of specific problem. Before testing, the student‟s possible coding actions should be modeled already. Thus, the testing process would interactively ask student to write down the code step by step. During the model tracing process, once the student encounters troubles in some step, the corresponding remedial instructions can be triggered to assist the student to finish the program coding. However, traditional model tracing approach can only support the intra-program behavior assessment e.g., it can only provide the syntax and semantic level instructions of single program statement.
To solve the problem, this chapter applied the pedagogical theory of Vygotsky‟s Zone of Proximal Development to support students learning the problem solving strategy with the metacognitive scaffolding. The Problem Solving Strategy Ontology is constructed to connect the relationship between low level programming statements and high level problem solving strategic concept.
Definition 4.1 Problem Solving Strategy Ontology (PSSO) There are three layers in PSSO:
Problem Strategic Layer: the algorithms of specific problem for students want to learn.
Semantic Layer: the pragmatic semantic concepts that may be the misconceptions in comprehension level for students.
19
Figure 4.1 Problem Solving Strategy Ontology (PSSO)
Lev Vygotsky‟s theory of Zone of Proximal Development (ZPD) is a core of the theory of cognitive development which explores the development of the psychological function by analyzing the relationship between actual development level learning and developing. That is to say, we want to know the students' current status and their potential capabilities in the learning.
Students often have the primitive semantic knowledge, but they mostly do not have the pragmatic semantic knowledge. To refer to the ZPD theory, we thought that the students have the misconceptions in program understanding, because there is the knowledge boundary in the comprehension level of programming learning, as shown in Figure 4.2.
Knowledge
Figure 4.2 Knowledge Boundary in programming learning domain
Therefore, based on the Problem Solving Strategy Ontology, the classical program examples such as selection sort, bubble sort, binary search, etc are collected as learning scaffolding for training the recurrence strategy. In addition, the primitive program examples of the textbook are collected to provide the diagnosis of students‟
actual development level. The collected program examples are tagged with concepts of the Problem Solving Strategy Ontology.
According to the tagged program examples, the students‟ programming behaviors including frequent used pattern and error patterns can be built from mining the programming testing log. Once the behavior is built, the adaptive learning guidance service can be provided to assist students during the training of problem solving.
4.2 The behavior mining to discover error patterns
In this section, the debugging testing activity is proposed to assess the students‟
debugging capability. There are predefined buggy patterns as followings.
21
1). The detection of syntax error that violates the grammar of programming language.
Missing token: student can detect the token omitted in the statement such as the missing token “;” omitted in the end of the statement “printf("hello world!")”.
Extra token: student can detect the redundant token which should be removed such as extra token “;” in statement “for(….); printf(“*”);”.
Illegal delimiter: student can understand that delimiter is a sequence of one or more characters used to specify the start and end boundary between separate regions of program expressions such as the curly brackets “{“ and
“}”.
2). The detection of misuse token error that happened in the mapping of pseudo code to program.
Misuse operator: student can detect the wrong usage of comparison operator “==” with assignment expression “=” in the statement “if(a=1)”.
Misuse operand data type: student can detect the incorrect operand usage with wrong data type or constraint such as integer variable “a” is mismatched with the wrong data type in the statement “int a = „c‟; ”.
3). The detection of logical error that happened in the design of pseudo code for solving a given problem.
Unawareness of operator precedence: student can detect the computational priority of operators such as the programming for formula “(a+b)*h/2” may be written as “Area_Trapzium = a+b*h/2;” where the unawareness of operator precedence may cause the wrong result.
Absence of boundary condition of some variable: student can detect the boundary condition of variables such as the “divided by zero” error in statement “b=0; a=1000/b;”.
Unawareness of infinite loop: student can detect if the stopping criterion of
“for/while” or “if” the branching condition of expression is unreachable such as the infinite loop in statement “for(i=1, i<10, i--)” or “i=1;
while(i>1){ i++; }”.
Accordingly, the buggy patterns database can be constructed and indexed by the
taxonomy of the Problem Solving Strategy Ontology. Buggy patterns can be embedded in the selected programs and the students are asked to detect those buggy patterns by tracing the program to facilitate the error detection assessment. Two examples of “selection sort” and “factorial number generation” programs with embedded buggy patterns are shown in Example 4.1.
Example 4.1 Examples of program with embedded buggy patterns
With the constructed buggy patterns database, the buggy programs needed for error detection assessment can be provided. As shown in Figure 2, the example of selection sort with Program No. Q001 is embedded with buggy patterns “misuse token” in line 5, the “infinite loop” in line 8, and the “missing token” in line 15; the example of factorial number generation with Program No. Q002 is embedded with buggy patterns “Misuse operand data type” in line 7, the “Misuse operator” in line 10 and line 12.
Figure 4.3. Buggy programs of “selection sort” and “factorial number generation”
Definition 4.2 The student’s portfolio
23
The portfolio of a student is composed of a set of testing results: P = {s1, s2, s3, …}
where symptom si=(Program_no, line_no, position_no, symptom_id) represents the novice student‟s mistake on program detection. The Program_no, line_no and position_no denote the position information of the occurred symptoms. The symptom_id denotes the identification of buggy pattern.
Example 4.2 The portfolio
Assume that the student failed to detect the “misused operator” or “infinite loop”
buggy patterns in program Q001, then the portfolio is {(Q001, 5, 10, Bug_misused_token), (Q001, 8, 1, Bug_infinite_loop)}.
Thus, the assessments of different symptoms are as follows.
Assessment based on symptoms of syntax error: the symptoms of syntax error are including “missing token”, “extra token”, and “illegal delimiter”. The root causes for these symptoms can be most likely identified as misconception in
“program structure” if the student only has symptoms in illegal delimiter;
“unfamiliar with the statement” if the student fails in most of the symptoms; and
“incautious” if the student only has symptom in partial program.
Assessment based on symptoms of misuse token: the symptoms of misuse token are including “misuse operator” and “misuse operand data type”. The root cause can be mostly identified as misconception in the “operator” if the student only has symptom in program operator; “the operand data type” if the student only has the corresponding symptoms; “fail in the mapping of pseudo code to program” if the learner has both symptoms.
Assessment based on symptoms of logical error: the symptoms of logical error
are including “unawareness of operator precedence”, “absence of boundary condition of variable”, and “unawareness of infinite loop”. The root cause can be mostly identified as the corresponding misconception only when learner passes the syntax assessment but has the logical error symptoms.
Therefore, if the major symptoms in novice‟s portfolio matched the symptoms of repertory grid, it can be identified as major root cause misconception. Besides, cooperating with other minor symptoms, the possible misconceptions can also be
Therefore, if the major symptoms in novice‟s portfolio matched the symptoms of repertory grid, it can be identified as major root cause misconception. Besides, cooperating with other minor symptoms, the possible misconceptions can also be