Chapter II: LITERATURE REVIEW
3.3. Instruments
3.3.4. In-Class Observation List
The observation list was revised from the previous research proposed by Lan, Sung, and Chang (2007). The list consists of three learning-related behavior and learning-unrelated behavior. The in-class observation list can be consulted in Appendix D.
3.3.5. Questionnaire about Students’ Attitude toward Applying Comic in Learning Mandarin Chinese.
The questionnaire is based on Lund (2001) Measuring Usability with the USE.
Initially, the questionnaire was adapted from a document that contains three dimensions: usefulness, satisfaction and easiness. The questionnaires consist of multiple choice and essay items. The multiple choices are the usefulness dimension (6 items), easiness dimensions - which are elaborated into ease of using (6 items) and ease of learning (4 items), and the last dimension is satisfaction (6 items). Each question offers 4 different scales from strongly agree (4), agree (3), not agree (2) and strongly not agree (1). The essay item gives opportunity for the respondents to write down their feelings. Pearson validity and Cronbach ‘s Alpha reliability was applied to
this questionnaire. Table 3-3 to 3-5 provides the reliability and the validity of the questionnaire.
39
Table 3-3. Questionnaire’s Reliability in General The Value of Cronbach’s Alpha Number of Items
.977 22
Table 3-4. Questionnaire’s Reliability in Each Dimension
Dimension
Question number
Number of Item
Value of Cronbach’s Alpha
實用性 Usefulness 2, 8, 11, 12,
15, 18 6 .934
容易性 Easiness
易用性 Ease of Using
4, 7, 9, 10, 16,
21 6 .919
.956 易學性
Ease of Learning
3, 6, 14, 20 4 .908
滿意度 Satisfaction 1, 5, 13, 17,
19, 22 6 .951
Then every single item was validated by using the Pearson correlation bivariate with the significant correlation of p<.05 and p<.01. And the results are as follows.
40
Table 3-5. Validation’s Value Question
Item Value Question
Item Value
Q1 .370** Q12 .414**
Q2 .398** Q13 .552**
Q3 .603** Q14 .518**
Q4 .472** Q15 .540**
Q5 .442** Q16 .678**
Q6 .406** Q17 .559**
Q7 .543** Q18 .448**
Q8 .644** Q19 .498**
Q9 .530** Q20 .282*
Q10 .508** Q21 .417**
Q11 .365** Q22 .407**
*p<.05 **p<.01 ***p<.001`
3.3.6. Interview List
There are 6 questions about the application of online comic and collaborative learning. The questions are:
41
Q1. Can you create comics using Toondoo?
Q2. Do you like create comics using Toondoo?
Q3. Do you practice Toondoo at home?
Q4. Do you like this kind of activity?
Q5. If you are interested in Toondoo, will you tell about it to your friends? How?
Q6. What are the advantages of using Toondoo?
3.3.7. Hardware (PC Computers)
There were 18 sets of computers in the computer lab which were connected to the internet. On the keyboard, the Simplified Chinese Language feature has been added to the language choice so that the students can type Mandarin characters during the treatment. An introduction and short training course are given to the experimental groups prior to the commencement of the exercise.
3.4. Procedure
To get both qualitative and quantitative data, the mixed method has been applied in this research. Figure 3-1 shows the cycles about this mixed method in collecting data.
42
Figure 3-1. Mixed Research Model
Before the experiment took place, the students, teacher, principal and vice principal were interviewed about the conduct of the experiment. Then the pretest on vocabulary performance was conducted. The vocabulary pretest was designed to get students’ scores before the experiment took place. This test consisted of vocabulary
they had learnt since the start of school and the vocabulary they would learn during the semester in one academic year.
After finishing the pretest, the experimental group 1 or individual comics group and experimental group 2 or collaborative learning group undertook two weeks training. It was called individual comics group as this group created online comics individually. The class took place in the computer room, where students were
43
instructed to create comics reflecting the themes offered by the teacher. Some vocabularies were provided to help the students formulate phrases or short sentences in the comics (caption). Observation checklist was taken and a video camera was set up to document students’ activities. The experimental group 2 or collaborative learning group did not only create online comics, but also applied the collaborative learning. Small groups consisting of 3-4 students were working together to create their own comics. There were 貓(cat)，青蛙(frog)，狗(dog)，蛇子(snake)，and 老 虎(tiger) group. The class took place in the computer room where students were
instructed to create comics reflecting the themes offered by the teacher.
The treatment was conducted for two teaching units in a month. The first theme was 保護環境 and the second 搬家了. The rubric writing based on ESL English Writing percentile scale (Jacob et al, 1981) was applied for grading the comics created by the students of the three different groups. This rubric consists of 5 items; they are 1) content/ idea development, 2) organization, 3) vocabulary/ word choice, 4) language use/ grammar, and 5) mechanics.
After the experiment, the posttest of vocabulary performance was conducted.
The vocabulary posttest was designed to measure the students’ scores after the experiment had taken place. This test consists of vocabulary they had learnt since the start of school, and some new vocabulary they learnt during the experiments. Finally,
44
the last steps of distributing the questionnaires to collect qualitative data on students’
perspectives on using technology in Budi Utama Multilingual School, using comics to improve the development of vocabulary competence in Mandarin Chinese classes, and conducting the collaborative learning in Mandarin Chinese, were done. Only
students of individual comics and collaborative learning group done this questionnaire of students’ perspective toward comics learning. Control group students do not need
to do the questionnaire as they do not get any online support during the experiment.
To underline the qualitative data, the students also have been interviewed.
3.5. Data Collection
The quantitative data was collected from the scores of students’ pre and posttest vocabulary performance.
The qualitative data will consist of 1) students comic creation of two teaching units, 2) video recording to document in-class activity, 3) in-class observation in an observation checklist, and 4) questionnaire, and 5) interview.
3.6. Data Analysis
During the experiment, both quantitative and qualitative data were gathered. The quantitative data were students’ scores from pretest and posttest on Mandarin
vocabulary performance. A two-way analysis of covariance was conducted to see if
45
the three different groups performed differently, while the covariate is the students’
summative score in the previous semester. The score from the two different teachers on the comics’ creation of each group was also collected as part of the quantitative
data. Then this score will be analyzed using one-way anova, while qualitative data consists of in-class observation about students behaviors. Data were collected from the video recording, and then rated by two raters, noted on the in-class observation list;
later on the data was analyzed by using Chi-Square analysis. Students’ attitude toward comics online learning and collaborative learning data were also gathered as part of the qualitative data. Statistic descriptive was conducted to analyze this qualitative data.
46
CHAPTER IV
RESULTS
This chapter provides the result of the experiment which include the result of the experiment of 3 groups consisting of pre and posttest of Mandarin vocabulary performance and comics creation; the result of in-class observation, questionnaire and interview. A summary is also provided in this chapter.
4.1. Mandarin Chinese Vocabulary’s Performance
This section displays the development of Mandarin Chinese vocabulary of the control group, the individual comics group, and collaborative learning group. The two-way analysis of covariance was applied to their vocabulary performance between pre and posttest, while the covariate was the score of previous semester and the comics’ creation of three different groups.
4.1.1. Pretest and Posttest of Vocabulary Performance
Before and after the treatment, all students were given individual test about their vocabulary performance. Alpha was set at the .05.
47
Table 4-1 lists the descriptive statistics results of the pre and posttest vocabulary performance. Regarding to the score of pretest and posttest, the homogeneity test was significant (F(4.790)=.001, p<.05). Based on the data listed on the Table 4-1, there was improvement of mean between pretest and posttest in each group.
Table 4-1. The Mean and Standard Deviation (SD) of Pretest and Posttest’s Score
Test Control Group (N=16)
Individual Comics (N=14) Collaborative Learning (N=16)
M SD M SD M SD
Pretest 60.68 17.12 60.00 12.61 60.25 13.65
Posttest 63.43 19.57 62.21 15.64 68.18 11.37
Table 4-2 presents a summary of the two-way analysis of covariance on students’ scores in pretest and posttest of vocabulary performance, with the score of previous semester as the covariates. The table shows that the interaction effect between test and group is not significant (F(0.91)=0.409, p>.05). This means that there are no differences varied according to levels. The group factor is not significant (F(0.88)=0.41, p>.05). This means that no difference exists between the scores between pretest and posttest in the three groups. The test factor is significant (F(9.67)=0.007, p<.05), this means that the grouping of control group, individual comics, and collaborative learning gives significantly different score between pre
48
and posttest. Test within group (1) which is control group is not significant (F(0.85)=0.359, p>.05) neither is the test within group (2) which is individual comics (F(0.48)=0.489, p>.05). This means that both groups still made some improvement due to the training, but does not vary. While the test within group (3) which is collaborative learning is significant (F(7.10)=.009, p<.05), means that the experiment significantly benefited students’ vocabulary performance in collaborative learning group.
Table 4-2. Two-Way Analysis of Covariance of Vocabulary Performance Source of Variation Type III sum
of squares
df Mean square
F Sig.
Within Cells 6037.84 85 71.03
Test by Group 135.98 2 67.99 .91 .409
Group 131.97 2 65.99 .88 .419
Group within Test (1) 2.69 2 1.35 .02 .981
Group within Test (2) 304.42 2 152.21 2.14 .124
Test 458.11 1 458.11 9.67 .007**
Test within Group(1) 60.50 1 60.50 .85 .359
Test within Group(2) 34.32 1 34.32 .48 .489
Test within Group(3) 504.03 1 504.03 7.10 .009**
Error 710.79 15 47.39
Note *p<.05 **p<.01 ***p<.001`
4.1.2. Comic Creation
This section provides the comics creation done by control group, individual comics, and collaborative learning. First, each comics creation will be scored based
49
on writing rubric (Jacobs et als, 1981), and then after being scored by two teachers, the final comics score stood as the dependent variable in one-way analysis of variance.
Firstly, the score of two teachers were collated by operating the Pearson correlation. With the significance value of .992** meant that the score of two teachers was highly correlated with p<.01 (2-tailed), therefore the score of these teachers can be used. The Table 4-3 shows scores from two teachers on comics creation.
Table 4-3. Pearson Correlation of Two Teachers’ Score on Comics Creation
Teacher 1 Teacher 2
Rater 1 Pearson Correlation 1 .992**
Rater 2 Pearson Correlation .992** 1
**. Correlation is significant at the .01 level (2-tailed).
Table 4-4 presents the 5 items of writing rubric and the percentage of score got by each group which are control group, individual comics, and collaborative learning. With the maximum score of 5 for each items, the collaborative learning got the highest score for each criterion. Consecutively, the mean of the content/ idea development was 3.6 out of 5 in collaborative learning; 1.9 in control group; and 1.7 in individual comics. The item of organization, the mean got by collaborative
50
learning was 1.7; individual comics was 0.7; and control group was 0.6. The mean in vocabulary or word choice, the highest was 2.2 by collaborative learning; 1.2 by individual group; and 1.1 by control group. The mean in the item of language use/
grammar was 2.1 for collaborative learning; 1.0 for individual comics; and 0.9 for control group. The mean of the last item which is mechanics was 2.2 for collaborative learning; 1.1 for individual comics,; and 0.6 for control group.
51
Table 4-4. Items of Writing Rubric
Items
52
Comic creation done by control group, individual group, and collaborative learning students are then being commented by the teachers. Figures 4-1 to 4-3 show some examples of teachers’ comment to students’ comic creation.
Control Group’s Comic Creation
Writing Rubrics
Content/ Idea Development The plot of the scene unclear delivered. The student, female student wanted to tell that after her friend left, she met a new friend, and then they become close friend.
The used of 我的 xin*), instead of 新
Organization Opening, development, and closing is developed although was so limited.
Vocabulary/ Word Choice There was no feeling expression (難過，開心)
Language Use/ Grammar 搬到……*)
Mechanics All sentences was written in Chinese character, except xin*)
Figure 4-1. Example of Control Group’s Comic Creation
53
Individual Comic Creation
Writing Rubrics
Content/ Idea Development The student is a female student. The idea is not about 搬家, but rather than 旅行.
新家破*)should be 新加坡。
Organization There is no completed story
Vocabulary/ Word Choice The student mixed about 要 and 有; mentioned 新加坡, 真 的啊，but no other vocabulary.
Language Use/ Grammar 我也有去*) should be 我也要去。
Mechanics There is no development in the story. And it is impossible for people to say good bye just too close to the aircraft.
Figure 4-2. Example of Individual Comics’ Comic Creation
54
Collaborative Learning’s Comic Creation
Writing Rubrics
Content/ Idea Development The main character is chosen by a male student Story can be developed a bit longer
Some exclamation word were not written in Mandarin such as “yipi”, and “uh”
Organization There is an opening, development, but still need closing Vocabulary/ Word Choice There is no feeling expression (開心，難過)
Language Use/ Grammar No related story between picture 4 and 5 Mechanics More Chinese character is applied in the comics
Figure 4-3. Example of Collaborative Learning’s Comic Creation
Table 4-5 presents a summary of the one-way analysis of variance on students’
scores of comics creation. The score stands as the dependent variables. The table of homogeneity variance was insignificant (F(33.5)=0.498, p>.05). This means that
55
there is difference scores among the three groups due to the different treatment. The variance between group is significant (F(33.5)=0.000, p<.001, this means that the different treatment in comics creation for control group, individual comics, and collaborative learning gives different score.
This analysis then is followed by the post hoc analysis. Table 4-6 serves the post hoc analysis that show the collaborative learning students outperformed and significant towards the other 2 groups. The individual comic shows insignificant, neither does the control group.
Table 4-5. One-way Analysis of Variance of Comic Creation Source of Variation Sum of
Squares
df Mean Square
F Sig.
Between Groups 428.826 2 214.413 33.523 .000***
Within Groups 275.027 43 6.396
Total 703.853 45
Note *p<.05 **p<.01 ***p<.001`
56
Table 4-6. Post Hoc Analysis
(I) Students_Group (J) Students_Group
Sig. 95% Confidence Interval
57
Table 4-7. Chi Square Analysis Result
Groups Value df
Asymp. Sig.
(2-sided)
Control Group Pearson Chi-Square 495.000 25 .000***
Likelihood Ratio 230.067 25 .000
Linear-by-Linear
Association 72.094 1 .000
N of Valid Cases 99
Individual Comics Pearson Chi-Square 500.000 25 .000***
Likelihood Ratio 319.851 25 .000
Linear-by-Linear
Association 84.017 1 .000
N of Valid Cases 100
Collaborative Learning Pearson Chi-Square 400.000 25 .000***
Likelihood Ratio 235.901 25 .000
Linear-by-Linear
Association 76.320 1 .000
N of Valid Cases 100
Note *p<.05 **p<.01 ***p<.001`
In a brief, in-class observation can be seen in the Table 4-8 below.
Table 4-8. The Frequency of the Learning Behavior (in %)
Group Learning-Related Behavior Learning-Unrelated Behavior Create
58
Table 4-8 presents the frequencies of the behavior from control group, individual comics, and collaborative learning. The Chi-Square analysis results X²=
56.50>18.307, df=25 reveals that there are significant differences in the learning–
related behavior and learning-unrelated behavior. The frequencies of learning-unrelated behavior such us playing with other, moving around, and playing alone in the collaborative learning group is the lowest in compare with the individual comics and control group. The control group students behave better than the individual comics. In learning-related behavior the collaborative learning group got the highest percentage, followed by control group, and the last one was the individual comic
4.3. Students’ Perception about Comics Learning
By the end of the experiment, the students from three groups were given questionnaire about the usability of comic. Students’ perspective about Comics learning is presented in the Table 4-8. Based on the mean of each group the collaborative learning students show the highest in every dimension. Then the individual comics students show highest mean at every dimension to be compared with the control group students. The data also notes that satisfaction dimension was
59
the highest mean among the three groups. The control group and individual comics have the same preference towards their perception that, usefulness of comics learning has the lowest mean. While towards the collaborative learning students, the easiness was the lowest dimension.
Table 4-9. Usability of Comic Learning Questionnaire Dimensions Control Group
(N=16) Individual
Comics (N=14) Collaborative Learning (N=16)
M SD M SD M SD
The whole questionnaire 3.122 0.678 3.259 0.652 3.396 0.626 Usefulness 2.990 0.692 3.208 0.649 3.426 0.606 Easiness Ease of Using
3.185 0.693 3.223 0.692 3.262 0.692 Ease of Learning
Satisfaction 3.188 0.658 3.381 0.597 3.574 0.535
The percentage of the usability of the online comics learning is presented in the Table 4-10 below. In the dimension of usefulness, the collaborative learning got higher percentage of 27.5, then the individual comics of 26.11. However, individual comics got higher percentage on the easiness of using online comics of 22.79, then collaborative learning of 21.89. In the dimension of satisfaction, the collaborative learning was more satisfactory group with the percentage of 28.69, then individual comics of 27.84.
60
Table 4-10. Usability of Online Comic (in %)
No Dimension
GROUP Individual
Comics (N=14)
Collaborative Learning
(N=16) 1 Usefulness 26.11 27.5
2 Easiness 22.79 21.89
3 Satisfaction 27.84 28.69
Besides multiple questions, the questionnaire also provided with the essay.
Based the other opinion about the application of online comics in individual comics group, there are 12 student express their satisfaction in operating online comic and 2 students did not give any other comments. Somehow, there are 8 students who also said that the internet connection was too bad, and 1 student said that the computer needs to be upgraded, 3 students said need much more time to create online comics. While in the collaborative learning group 10 students shows their satisfaction by rating that online comics is fun, interesting and that they love working on online comics. Somehow 7 students state that the internet connection was very unsupported, 7 students mention about the limited time given, 2 students say that online comics was easy, and 1 students declares that online comics is fun but a bit difficult.
61
4.4. Interview
There are also interview to get deeper information from the individual comics and collaborative learning students about the application of online comics and collaborative working. Table 4-11 listed the interview report.
Table 4-11. Interview
Questions Individual Comics Group (N=14)
There are 29% of students from individual comics who did not practice at home due to their class activity. While there are 25% of students from collaborative learning who did not practice online comics due to the absence of internet
62
connection, and that their parents or other family members do not allow them to operate any online activity at home.
Question 4 has been elaborated based on individual comic activity and collaborative learning activity. 50% of individual comics students wish to know what about collaborative learning, or prefer to work together rather than working alone as they cannot solve the problem all by themselves. There are 40%
collaborative students said they do not enjoy the collaborative activity, due to the unfamiliarity of students towards working together, discussion, and individuality preference.
There are 42% of individual comics who will introduce about online comics to their friends while they are doing their homework, playing together, or when practice their wushu 武術(martial art) activity. While 58% said they do not have time to introduce it to their friend, and they forgot to do so. While there are 75% of collaborative students told about the online comics to their friends, bother/ sister, neighbor, or relatives when they play together. Their also introduce about the online comic to their relatives. 25% said they have not introduced about the online comics, but there was no specific reason.
Question 6 about the advantages of applying Toondoo can be categorized into 3
63
aspects. They are aspect of entertainment (cool, fun, interesting), learning support, and simplicity (easy to use, can create comics). Consecutively there are 43% of
aspects. They are aspect of entertainment (cool, fun, interesting), learning support, and simplicity (easy to use, can create comics). Consecutively there are 43% of