2. Dependent Variable
3.3 Material
The science text will contain two topics. In the first section of instruction, an approximately 800-words text on “Cell Hypothesis” (in Chinese) will be used with three paragraphs to present three important concepts. The instructional Power-Point presentation includes 11 slides with ten pictures to reveal the importance and procedure of the drawing construction. The second topic in the instruction will be
“Circulatory System”. This text consisted approximately 1300 words, divided into six paragraphs about three main concepts. (Appendix A and Appendix B).
2. Booklets
Every participant will receive the text about Circulatory System and the same
22
booklet contains all the questionnaires and tests (4 pages). Additionally, the booklet of DIPE group includes extra 5 pages of the drawing instructing “Draw for science understanding” outline with a sample text of Cell Theory for drawing practice.
3. Illustration feedback
The participants in group DIPE, DIP and DI had obtained the Illustration of Circulatory System (heart, vessel and circulatory system) after the drawing. The illustration had labeled the keyword; however, had not labeled the directions and functions, which represents the selecting and internal supports. (Appendix C)
4. Prompting question
The participants in DIPE and DIP obtained the Prompting Questions of Circulatory System (heart, vessel and circulatory system) after the drawing. This support had included 5 prompting questions to guide the learners checking their figures and words. What’s more, conditions with Prompting questions would also receive the directions and functions of the illustration. Overall, the Prompting Question support could regard as a propositional support (with functions and directions) and also a metacognitive support (which ask student to check their original figure), which is similar with selecting, internal and external supports. (Appendix D)
5. Explicit instruction
The participants in DIPE received a 30-min explicit instruction about “how to draw efficiently to learn” before drawing. Participants were all taught the skills for reading and drawing with selecting, organizing and integrating. They all have a
23
chance to practice the topic “cell hypothesis” and teachers would give the feedback after the learners drawing.
3.4 Measure 1. Pretest
The Pretest includes demography questionnaire, prior knowledge test, pre-mental model evaluation, and a motivation questionnaire. .
(1) Demography questionnaire
It will collect the background information of the participants, such as id, school, place of residence, gender, educational background and the investigation of the whether learner had taken any the bio-related course.
(2) Pre-factual knowledge test
The purpose of this test is to examine participants' concepts about the terms and basic structures of Circulatory System. The correct answer in one blank gains 1 point and the highest score would be 14 (see Appendix E).
(3) Questionnaire on current motivation (QCM. Rhieinberget et al, 2001)
The motivation of learning science will be measured by Questionnaire of Current Motivation. This questionnaire is composed with 18 items of 7-point Likert scale.
(4) Pre-mental model test
This assessment is intended to test knowledge about structure of the circulation system and dynamic function conveyed in the text by means of drawing.
The hand-drawing about the components, the whole system and the functions is regarded as the drawer’s mental model which reveals the level of the learner’s structure of knowledge. Learners were instructed to draw representational illustration (real life like) and every instructional procedure were all presented in
24
appendix H. Please draw a diagram with the quality that it could be included into science textbook. Draw Heart. Draw Blood and Vessel. Draw the Circulatory System of human being”. The mental model coding is based on the results of Butcher and Chi (Butcher, 2006; Gadgil et al., 2012) and researcher’s pilot study which would be distinguished into eight levels (in Table 1). The higher represents the higher accuracy and more constructive. A highly accurate and constructive drawing (mental model) would consist of correct visual elements, connections between elements, structures, functions, and the dynamic relations to form the systems. The pre-mental model were score by the researcher, a high school biological teacher and a student assistant from department of biomedical imaging and radiological science with reliable interrater of ICC=.98. Participants with no drawing was regarded as no understanding of the target-text and gained 0 points in level zero. Level 7 is the most complete mental model of the learners and reflected the one had totally understand the circulatory system and could earn maximum7 points.
25
Table 3.3 Mental model coding
score Level Explanation Illustration example 0 No loop No understanding No illustration 1 Ebb and flow/
26
Table 3.2 Mental model coding (cont.)
4 Double loop 1 Only contains whole circulatory (with heart and lungs) and without any descriptions.
5 Double loop 2 Contains whole circulatory (with heart and lungs) but with incorrect or incomplete purpose and description.
6 Double loop 3 Demonstrate a complete circulatory system with two circles and
descriptions.
7 Double loop 4 Demonstrate a complete circulatory system with two circles and
exhaustive purpose , directions and
description (which did not found in pilot study)
27
2. Accuracy
Accuracy is the product of the treatment and reflected the quality that the learner conducting LGD. This will be coded to examine the level of understanding.
Rating methods are totally the same as the pre-mental model test in the pretest. The accuracy were score by the three experts and with reliable interrater of ICC (intraclass correlation) =.97. Participants could earn maximum7 points.
3. Posttest
The posttest score reflected the quality of learning outcome. I distribute three posttests to measure the dependent variables: Retention test, Transfer test and Post-mental model assessment. The Posttest took place right after the treatment.
(1) Post- transfer Test
There will be 5 open-ended questions to test the learner’s ability to transfer and apply the human circulation knowledge listed in the material to new situations.
An example of the question states “John fell off and bruised, why didn’t he infected by bacteria? Why did he stop bleeding? Please try to explain and try to write down the reasons” The accuracy were score by the three experts and with reliable interrater of ICC=.90. Three points reflects the best understanding for each item so that participants could earn maximum 15 points in post-transfer test.
(Appendix F)
(2) Post-retention Test
This multiple-choice test is intended to assess memory of the factual knowledge or recognition of the important concepts in the materials. An example is”How many organs involve in the whole complete human circulatory system?
28
An example is” What is the main purpose of the lung circulatory? (A)Gas exchange (B) Deliver the nutrients (C) Against the pathogens (D) expel the blood (Item A is correct). Please see Appendix G.
(3) Post-mental model test
The drawing construction test was intended to assess student’s comprehension and mental model by constructing illustrated representation. This drawing is intended to measure the learner’s mental model after the treatment.
Every rating method was completely the same as in pretest treatment. The accuracy were score by the three experts and with reliable interrater reliability (Intra-class correlation, ICC) = .98. Participants could earn 7 points maximally.
4. Statistic
This research is tended to assess whether LGD is a useful learning strategy. I emphasized the effect of the strategy and compare with the posttest between different experimental conditions. I will conduct the statistic data by ANOVA to test if the main effects are significant and if there are interaction effect. Additionally, to investigate the influence of the treatment, I also conducted multiple regression analysis to investigate the association between the variables.
3.5 Procedure
Participants were randomly assigned to one of four conditions. Each learner was seated in individual seat. The participants from DIPE group were extra taught a lesson called “draw for science understanding” to effective use the Learner-Generative Drawing strategy for 30 minutes before the treatment. Next, the learners were given the demography questionnaire and the prior knowledge pre-test,
29
pre-mental model test and QCM (15 minute). After completing the pretest, students would receive the text and booklets for conducting LGD task. Group DIPE, DIP and DI were given appropriate supports after 20minutes and were able to revise their drawing by compare with the external supports. The Drawing group could have completely 30 minutes to draw. After the treatment, the posttest took place immediately; every participant could have 35 minutes to finish their posttest.(Appendix H)
Table 3.4 Procedure of the experiment
1. Grouping Randomly assign 96 participants into four conditions.
2. Instruction DIPE conditions would receive an instruction of “Draw for science understanding” about the drawing skills and “Cell Hypothesis”.
3. Pretest Demography questionnaire
Pre-factual knowledge test
Pre-mental model test Questionnaire on current motivation 4. Treatment DIPE
DIP
DI
D
5. Posttest Post-retention test
Post-transfer test
Post-mental model test
30
Chapter.4 Result
4.1 Tests of prior group differences
Before analyzing the main effect of the treatment, this study tested group differences of participants’ backgrounds, prior knowledge and motivation. The examination on group differences of gender was tested by χ2 analysis and the result showed that there was no significance among four groups (χ2=1.50, p=.22). Age difference was not found significance (F= (3, 92) =.49, p=.69, partial η2=.16) when testing by the analysis of variance (ANOVA).
Additionally, prior knowledge (pretest factual knowledge) (F= (3, 92) =.23, p=.87, partial η2=.008),pre-mental-model score (F= (3, 92) =.78, p=.51, partial η2=.025), and score of the Questionnaire of Current Motivation (QCM) (F= (3, 92) =2.49, p=.07, partial η2=.077) were all insignificant between groups. The descriptive statistics of pretest factual knowledge, pre-mental-model and motivation is demonstrated in Table 4.1.
Overall, the results indicate similarity among groups in their gender and age compositions, prior knowledge, and motivation.
4.2 Treatment effects
To examine the hypotheses of this study, an analysis of variance (ANOVA) was used to test the main effect on accuracy, retention, transfer and post mental model.
Table 4.2 summarizes the descriptive statistics of the post-tests and post mental model score among four groups. Accuracy is rated by the drawing products when the
31
participants were receiving treatment, it did not showed a significant difference between grops, F = (3, 92) = 1.64, p = .18, partial η2=.06 (n.s.).
Table 4.1 Descriptive statistics of prior knowledge and motivation among four groups.
Prior knowledge and motivation
DI group: Students were asked to draw a figure and provided with a feedback illustration.
D group: Students were only asked to draw a figure. They did not receive any feedback or instruction.
Table 4.2 Descriptive statistics of retention test, transfer test and post mental model
DI group: Students were asked to draw a figure and provided with a feedback illustration.
D group: Students were only asked to draw a figure. They did not receive any feedback or instruction.
32
Concerning the score of the retention posttest (multiple-choice items), the DI group achieved lower than the other three groups of DIPI, DIP and D, F = (3, 92) = 4.56, p < .05, partial η2=.129 (Table 4.3). A Games-Howell post hoc analysis shows that only retention score of the Group DI was significantly worse than it in the Group DIPI; retention scores in the groups DIPI, DIP, DI did not show statistical differences.
Table 4.3 ANOVA summaries of group differences on retention, transfer and post
In addition, there shows a significant treatment main effect on transfer test score, F= (3, 92) = 3.02, p < .05, partial η2=.09. The Scheffe post hoc analysis showed that there is no remarkable difference between groups DIPI and DIP (transfer MDIPI = 9.15, transfer MDIP group = 9.07, t = .07, p = 1).
Regarding participants’ performance on post mental model, the result demonstrates a significant treatment main effect, F= (3, 92) =3.77, p < .05, partial
33
η2=.109. A Game-Howell test shows that there is no significant difference between Groups DIPE and DIP again (post mental model MDIPI = 5.41, MDIP = 5.15, t = .19, p
= .97). The LGD instruction seemed not effective to help DIPE students achieve higher than the DIP. However, MDIPI is significantly higher than MDI (t = 2.35, p
= .05).
4.3 The comparative effects of prior knowledge, motivation and treatment on accuracy
Multiple regressions were used to explore the effect of pre-test score, pre mental model score, QCM and treatment on accuracy. The multicollinearity diagnosis (correlation, tolerance and VIF) did not show any exceptional value. The result of this analysis had summarized in table3. The overall model R2=.26, reflected the strength of relationship between predictive variable and dependent variable was statistically significant, F (4, 88) = 7.59, p < .001. The model explained 26% variance of accuracy through prior knowledge, motivation and treatment.
Table 3 also shows the standardize estimate of regression coefficient and t value for each predictor as it entered the model. The effect reflected the standardized unit change in a predictor, controlling for other factors. Treatment, QCM and pre mental model had significantly impacted the results of accuracy; however, the pre-test did not.
This result indicates that a participant with higher pre mental model and pre-motivation and receiving more supports during LGD process would perform better on accuracy. In conclusion, the supports (feedback illustration and prompting questions) seem to increase learner’s drawing accuracy during treatment; however, motivation towards the task and the participants’ integrative functional prior knowledge also play important roles in using this strategy for active learning.
34
Table.4.4 The predictive effects of treatment, motivation (QCM), prior knowledge on accuracy in a multiple regression analysis
Predictor Variable B B(SE) β t p
Treatment -.490 .177 -.255 -2.759 .007
QCM .045 .023 .184 1.991 .050
Pre-mental model .686 .161 .450 4.270 .001
Pre-factual knowledge test
.037 .065 -.060 -.569 .571
Model R2 = .26 F (4, 88) = 7.59, p < .001
4.4 Qualitative analysis of the individual participant
As mentioned in the method chapter, there was a pilot test conducted in a class of junior high school students asking them to learn human circulatory system with LGD.
Their drawing products were used to construct a coding system for the evaluation of drawing products in the current study with non-biology major post-secondary students.
Though the age and education level of the pilot study participants were different from that of the participants in this study; however, the range (level 0 to 7) of drawing quality which demonstrates the conceptual understanding of human circulatory system remained the same. I also found that the best drawing product quality (level 7), which no one achieved in the pilot study, appeared in this study. Table 4.4 depicts selected typical drawing products for 7 levels (e.g., 0 = circulatory system with no loop, 1 = ebb and flow/Atrial and ventricular, 2 =single loop, 3 = single loop with lungs or single loop with wrong description, 4 = double loop-1, 5 = double loop-2, 6 = double loop-3, 7 = double loop-4) of coding in this study.
35
Figure 4.1: Mental model coding list 0:
No loop
1:
Ebb and flow/ Atrial and ventricular
2:
Single loop
3:
Single loop with lungs or single loop with wrong
description
36
Figure 4.1: Mental model coding list (cont.) 4:
Double loop 1
5:
Double loop 2
6:
Double loop 3
7:
Double loop 4
37
Eight cases selected from four treatment groups were listed in below for a further descriptive analysis. The observation of learners’ serial drawings could be used as another approach to validate the results obtained from the statistical analyses.
Group DIPE
Drawing products of participant A: a gradual LGD benefit learner
The drawing products of participants A showed that before DIPI treatment (drawing, illustration feedback, prompting questions and LGD instruction), the pre mental model was in the low level of 1; while the accuracy score during treatment showed pretty good improvement to a level of 5. But, from the observation note the author jot down, the author found that this individual mainly took note when reading and might allocate less time in the integration of text and drawing. When it came to the posttest, the participants had enough time to integrate text and image materials, the post mental model revealed a highest level of conceptual understanding.
Figure 4.2: Drawing products of participant A
Pre-mental model: 1 Accuracy: 5 Post-mental model: 7
38
Drawing products of participant B: a dramatic LGD benefit learner
This is the learner with the most dramatic transition pattern across the study procedure and there were quite many participants showed this remarkable change in group DIPI.
Participants B had barley no understanding in the beginning test (pre mental model) but during the treatment the accuracy score showed B participant’s drawing achieved the best quality level of 7 and in the post-mental model the conceptual understanding was still the best. Even in the transfer test B gained 14 points.
Figure 4.3: Drawing products of participant B
Pre-mental model: 0 Accuracy: 7 Post-mental model: 7
Group DIP
Drawing products of participant C: learner with high quality knowledge
In the group DIP (drawing, illustration feedback and prompting questions), the participant C demonstrated having a high quality of knowledge in pre-mental model and there were no room for the progress. The content of participant C’s pre-mental model showed good understanding about component and organization of the circulatory system, only function of circulatory is missing. When it comes to the treatment accuracy, participant C had added the function and detailed description of
39
the circulatory system, which make the treatment accuracy and post-mental model perfect.
Figure4. 4: Drawing products of participant C
Pre-mental model: 6 Accuracy: 7 Post-mental model: 7
Group DI
Drawing products of participant D:could be give-up or poor understanding with poor motivation.
The participant D displayed poor understanding (level 0 to show no conceptual understanding of human circulatory system) all the way through pre-mental model, treatment to post-mental model. It demonstrated that this participant did not engaged in learning during the experiment.
40
Figure 4.5: Drawing products of participant D
Pre-mental model: 0 Accuracy: 0 Post-mental model: 0
Drawing products of participant E: an un-interpretable learner
Participant E did demonstrate a level 2 understanding of human circulatory system.
However, the drawing product during the treatment process dropped back to level 0. I infer that he was not willing to engage in leaning. At the end of the experiment, the participant E displayed a level 4 understanding in the post-mental model. The learning transition is hard to interpret.
Figure 4.6: Drawing products of participant E
Pre-mental model: 2 Accuracy: 0 Post-mental model: 4
41
Group D
Drawing products of participant F: a learner once gets progress but regress eventually.
The drawing products of participant F show that he/she learned and integrated circulatory knowledge during the treatment (from level 2 in the first drawing to level 6 in the treatment); nevertheless, in the post-mental model the conceptual understanding dropped down to level 3. In group D, some participants displayed the same pattern as well. I speculate that due to the non-stop experiment tasks, the learner would probably exhaustive during the highly demanding activity and perhaps refuse for the further mental effort investment in the posttest.
Figure 4.7: Drawing products of participant F
Pre-mental model: 2 Accuracy: 6 Post-mental model: 3
Drawing products of participant G: a no-improvement learner
There were no up and downs of participant G. Obviously this participant did not benefit from the strategy, which the learner might failed to grasp the main point of drawing. This pattern showed frequently in Group DI and Group D.
42
Figure 4.8: Drawing products of participant G
Pre-mental model: 4 Accuracy: 4 Post-mental model: 4
Drawing products of participant H: a verbal strategy learner
The participant G demonstrated a category of learners who might be not good at drawing or tended to be a text learner. In the observation note from the author, there seemed to be some leaners like to write text note and had shown little acquisition from drawing. However, the participants with low drawing ability seemed to be in the groups DI and D. Unfortunately, we are not able to investigate if this classification of learner could learn to draw through the DIPI instruction.
Figure 4.9: Drawing products of participant H
Pre-mental model: 4 Accuracy: 4 Post-mental model: 5
43
Chap.5 Conclusion and Discussion
5.1 Summary of the results
This study, based on previous research finding assumed a positive effect of Leaner-Generated Drawing strategy in science learning. The author also extended the research questions to test the relative effectiveness of different supporting conditions accompanying LGD. First of all, the result showed that explicit instruction (DIPE) is apparently more effective than receiving illustration with LGD (DI) in all outcome measures, from retention test, transfer test to post mental model, for non-science major college students. Conversely, receiving illustration along with LGD demonstrated the poorest understanding among four experimental groups. The
This study, based on previous research finding assumed a positive effect of Leaner-Generated Drawing strategy in science learning. The author also extended the research questions to test the relative effectiveness of different supporting conditions accompanying LGD. First of all, the result showed that explicit instruction (DIPE) is apparently more effective than receiving illustration with LGD (DI) in all outcome measures, from retention test, transfer test to post mental model, for non-science major college students. Conversely, receiving illustration along with LGD demonstrated the poorest understanding among four experimental groups. The