Integrating e-learning into the Direct-instruction Model to enhance the effectiveness of critical-thinking instruction

全文

(1)Instr Sci (2009) 37:185–203 DOI 10.1007/s11251-007-9048-z. Integrating e-learning into the Direct-instruction Model to enhance the effectiveness of critical-thinking instruction Yu-Chu Yeh. Received: 16 August 2006 / Accepted: 12 November 2007 / Published online: 22 November 2007 Springer Science+Business Media B.V. 2007. Abstract The Direct-instruction Model favors the use of teacher explanations and modeling combined with student practice and feedback to teach thinking skills. Using this paradigm, this study incorporates e-learning during an 18-week experimental instruction period that includes 48 preservice teachers. The instructional design in this study emphasizes scaffolding, observational learning, mastery of critical-thinking skills, guided practices, cooperative learning, providing feedback, self-reflection, online discussions, and active participation in an online learning community. This study employs 2 criticalthinking tests, 2 inventories, and 1 open-ended reflection questionnaire; and students’ scores on the pretest and posttest are compared via the Repeated Measure Analysis of Variance. The primary findings are as follows: (a) all participants preferred the instructional design in this study; (b) the experimental instruction effectively improved the preservice teachers’ critical-thinking ability as well as their professional knowledge and personal teaching efficacy concerning critical-thinking instruction; (c) the mechanisms contributing to the effectiveness of the experimental instruction mainly included discussing and sharing, observational learning, self-reflection, guided practice, and the learning community. Keywords E-learning Direct-instruction Model Critical thinking Learning community Online discussion Guided practice Preservice teacher. Introduction Critical thinking is a thought mode for generating knowledge within a changing society (Schroyens 2005), an effective learning strategy (Browne and Meuti 1999; Gadzella and Masten 1998; Halpern 1998; McCarthy-Tucker 2000), a key to emotional intelligence (Elder 1997), and a requirement for business leaders (Harris and Eleser 1997). With the Y.-C. Yeh (&) Institute of Teacher Education, National Chengchi University, 64, Chih-nan Rd., Sec. 2, Wenshan, Taipei 116, Taiwan e-mail: ycyeh@nccu.edu.tw. 123.

(2) 186. Y.-C. Yeh. current emphasis on teaching critical thinking and the recent increase in online education, developing methods for enhancing critical thinking has become important. Some researchers (e.g., Carmen and Kurubacak 2002; Kumta et al. 2003; Leader and Middleton 2003; Loving 2000; Nelson and Oliver 2004) have found that e-learning can foster students’ critical-thinking skills. However, professional development in teaching critical thinking has seldom been addressed. Furthermore, Torff (2005) indicated that preservice teachers typically have low levels of support for using critical-thinking activities. Therefore, preservice teachers require professional development that enhances the skills, knowledge, and confidence necessary to teach critical thinking. Since the Directinstruction Model has been proposed as an effective model for improving critical thinking (Eggen and Kauchak 1996), this study attempts to integrate e-learning into the Directinstruction Model to improve the effectiveness of preservice teachers when teaching critical thinking. Moreover, this study seeks to explore the mechanisms that contribute to such teaching effectiveness.. Critical thinking and effective critical-thinking instruction Research over the past decade has provided numerous definitions for critical thinking (e.g., Bailin et al. 1999; Browne and Meuti 1999; Giancarlo and Facione 2001; Halpern 1998, 2003; McCarthy-Tucker 2000; Paul and Elder 2001). For example, Halpern (1998) argued that critical thinking is purposeful, reasoned, and goal-directed; it is the thinking required to solve problems, formulate inferences, calculate likelihoods, and make decisions. Paul and Elder (2001) proposed that critical thinking is self-disciplined, self-monitored, selfdirected, and self-corrective thinking that typically requires effective communication and problem-solving abilities. To improve the quality of critical thinking, Paul and Elder argued that learners must skillfully take charge of the structures inherent in thinking and impose ten intellectual standards. These intellectual standards are clarity, precision, accuracy, significance, relevance, completeness, logicalness, fairness, breath, and depth. Although various definitions abound for critical thinking, assumption identification, induction, deduction, interpretation, and argument evaluation are commonly tested in multiple-choice tests (Ennis et al. 1985; Facione and Facione 1994; Norris and Ennis 1989; Yeh 2003; Watson and Glaser 1980). To teach critical thinking effectively, teachers must have sound critical-thinking skills, abundant professional knowledge, and a strong sense of personal teaching efficacy. According to Shulman (1987), content knowledge and pedagogical content knowledge are required for effective teaching. ‘‘Content knowledge’’ refers to teachers’ knowledge of the subject matter, while ‘‘pedagogical content knowledge’’ is related to teachers’ knowledge about specific strategies for a particular subject matter. Integrating a review of the literature related to critical thinking (e.g., Halpern 1998, 2003; Harris and Eleser 1997; Hittner 1999; Lawson 1999; McCarthy-Tucker 2000; Yeh 2004), I have redefined these two types of professional knowledge in the context of teaching critical thinking. Accordingly, content knowledge for teaching critical thinking here refers to teachers’ understanding of the definition and constructs of critical thinking, the prerequisites of a good critical thinker, and the factors that affect a student’s ability to develop critical-thinking skills. On the other hand, pedagogical content knowledge for teaching critical thinking involves teachers’ knowledge in designing a suitable curriculum for teaching critical thinking, selecting effective pedagogies for imparting critical-thinking skills, employing effective teacher behaviors, and reliably assessing critical-thinking ability.. 123.

(3) Integrating e-learning into Direct-instruction Model. 187. Personal teaching efficacy is a teacher belief that involves confidence in bringing about student learning (Yeh 2006). The critical thinking construct comprises three elements: critical-thinking skills; critical-thinking dispositions (CT-dispositions); and prerequisite knowledge. Critical-thinking skills encompass deployment of such cognitive and metacognitive skills as analysis, identification of assumptions, interpretation, inference, induction, deduction, and evaluation (Halpern 2003; Norris and Ennis 1989; McCarthyTucker 2000; Paul and Elder 2001). The second element, CT-dispositions, stimulates individuals to apply their critical-thinking skills, as well as to identify key questions, evaluate problems, and seek reasonable answers (Giancarlo and Facione 2001; Leader and Middleton 2003; Paul and Elder 2001). Prerequisite knowledge pertains to knowledge of and experience in the topic or issue being analyzed (Paul and Elder 2001). All three elements are essential to good critical thinking. Therefore, personal teaching efficacy for critical thinking encompasses teachers’ confidence in assisting students to obtain criticalthinking skills, CT-dispositions, and prerequisite knowledge.. Direct-teaching Model, e-learning, and critical thinking instruction The Direct-instruction Model is a teacher-centered paradigm that utilizes teacher explanations and modeling combined with student practice and feedback to teach concepts and skills (Eggen and Kauchak 1996). In this model, content is generally presented in a sequenced and structured manner. Moreover, student performance is usually monitored during the process of instruction (Pressley and McCormick 1995). According to Eggen and Kauchak (2001), the Direct-instruction Model performs best when teachers want to ensure that all students have mastered the essential content and skills. When applied to teaching critical-thinking skills, this model has the following four phases (Eggen and Kauchak 1996, 2001). 1. Introduction and review: The teacher reviews the critical-thinking skills and explores their connection to the students’ background knowledge. The teacher then tries to capture student attention and motivate the students to learn by describing the goals and explaining the value of the critical-thinking skills. 2. Presentation: The teacher presents information such that it can be processed and encoded effectively. Moreover, the teacher models the critical-thinking skills in an interactive manner. 3. Guided practice: The teacher provides students with opportunities to practice the new critical-thinking skills. During this stage, the teacher encourages student interaction and applies the concept of scaffolding to assist students in encoding information to their long-term memory. 4. Independent practice: To enhance students’ retention and transfer of the learned critical-thinking skills, the teacher asks students to practice the skills on their own. This stage focuses on automaticity, which frees working memory, allowing it to focus on application. During this stage, group work for problem-solving and homework can effectively increase learning. A central notion guiding interactive patterns in the Direct-instruction Model is responsibility transfer. Therefore, the concepts of observational learning, modeling, scaffolding, and zone of proximal development (ZPD) should be emphasized. Observational learning and modeling encompasses aspects of behaviors, thinking, and emotions.. 123.

(4) 188. Y.-C. Yeh. Scaffolding refers to instructional support that allows students to perform a skill, which can further lead students to achieve the ZPD—a range of tasks a student cannot yet perform alone but can accomplish when assisted by a skilled partner or instructor (Eggen and Kauchak 2001; Pressley and McCormick 1995). Furthermore, the classroom environment during direct-instruction should be reassuring, democratic, and cooperative (Pressley and McCormick 1995). Although the Direct-instruction Model provides a good framework for improving critical-thinking skills, mastery of such complex thinking skills, related professional knowledge, and personal teaching efficacy requires considerable interactive discussion and practice (Yeh 2004, 2006). These needs challenge the conventional in-class teaching approach in which time and space are restricted. Previous findings (e.g., Carmen and Kurubacak 2002; Kumta et al. 2003; Leader and Middleton 2003; Nelson and Oliver 2004) suggested that online learning provides a natural framework for augmenting critical thinking during teacher education. Online learning comprises synchronous and asynchronous learning. An asynchronous discussion board provides an interface for teaching critical thinking as it embraces conventional assignments and in-class discussion. Moreover, an asynchronous discussion board enables participants to explain opinions based on good reasoning, develop arguments supported by logic and evidence, and reflect on and share ideas with others by making thinking visible (Lin et al. 2003). In addition to online discussions, employing online collaboration and online learning communities enhances the effectiveness of integrating e-learning into the Direct-instruction Model. The advantages of online collaboration and online learning communities are as follows (Ludwig-Hardman and Woolley 2000): • Learners and instructors can create knowledge together via their combined experience. • Learners are encouraged to evaluate complex issues using multiple perspectives. • Learners are individually accountable for their shared work while striving toward group goals so that students help one another and assess one another’s learning. • Learners are provided with opportunities for reflecting on their learning experiences upon others’ input. • Student achievement is enhanced via increased motivation, peer support, communication, and commitment to participating in group work. These advantages are obviously supplemental to in-class instruction. That is, online collaboration and learning communities may facilitate frequent interactive discussion and reflection, and therefore improve learners’ critical-thinking skills, knowledge, and selfefficacy. Therefore, to combine e-learning and the Direct-instruction Model, these elearning characteristics should be considered. Briefly, I hypothesize that integrating elearning into the Direct-instruction Model improves the critical-thinking skills of preservice teachers as well as the professional knowledge and personal teaching efficacy that are deemed essential for teaching critical thinking.. Method Participants Forty-eight preservice teachers enrolled in the Critical Thinking Instruction class comprised the experimental group and 46 preservice teachers not enrolled in the class. 123.

(5) Integrating e-learning into Direct-instruction Model. 189. comprised the control group. All participants were enrolled in a secondary school teacher program. While the preservice teachers in the experimental group were still undergraduates or graduates, those who in the control group had already had a bachelor degree or a master degree. The mean age of the experimental group, 11 males and 37 females, was 21.83 years (SD = 2.54); the largest age group was 21–25 years (46.8%). The mean age of the control group, 15 males and 31 females, was 29.03 years (SD = 5.39).. Instruments The online interface was the e-learning website developed by National Chengchi University, Taiwan. The instruments employed were the Critical Thinking Test, Level II (CTT-II), the Situation-based Critical Thinking Test (SB-CTT), the Inventory of Professional Knowledge for Critical Thinking Instruction (IPK-CTI), the Inventory of Personal Teaching Efficacy in Critical Thinking (IPTE-CT), and a reflection questionnaire. The structure of the e-learning interface consisted of curricular content, curricular information, curricular interaction, individual area, and system area (see Fig. 1). The instructional design in this study required participants to undertake thematic discussions regarding critical-thinking skills; as a result, the ‘‘discussion board’’ under ‘‘curricular interaction’’ was the most commonly used interface. The CTT-II was developed by Yeh (2005) and is based on the Cornell Critical Thinking Test (Ennis et al. 1985) and the Watson-Glasser Critical Thinking Test (Watson and Glasser 1980). It consists of 30 multiple-choice items which are divided evenly into five. Curricular content. Curricular discussion. Discussion room. Online discussion Curricular information. E-learning. Curricular interaction. Individual area. System area. Thematic discussion Group discussion. Discussion board. E-mail to the group. E-mail to TA Questionnaire voting Video conference setup Online video conference. Group video conference setup. Group video conference. Fig. 1 E-learning interface structure. 123.

(6) 190. Y.-C. Yeh. subtests: assumption identification, induction, deduction, interpretation, and argument evaluation. Each item consists of one statement and four multiple-choice answers. An example of assumption identification is as follows: ‘‘To effectively reduce our stress, we’d better learn how to manage our emotions.’’ Assumption 1: The ability to manage self emotions can be learned. Assumption 2: As long as we can manage our emotions, we can effectively reduce our stress. (a) Assumption 1 is valid. (b) Assumption 2 is valid. (c) Both Assumption 1 and Assumption 2 are valid. (d) Neither Assumption 1 nor Assumption 2 is valid. Within a 25-min time limit, the possible total score for the CTT-II is 30 points; in this study, a correct answer is scored as 1 point, and a wrong answer as 0. The average discrimination index of the CTT-II was .35 (with a range of .21–.53) and the average difficulty index was .58 (with a range of .24–.84). Being tested by the five-parameter Item Response Theory (IRT) model, the CTT-II also displayed an average level of difficulty. Furthermore, the test effectively discriminated between the high-ability group (upper 27%) and the low-ability group (lower 27%) on the ability to think critically. The subtest scores and total score were significantly correlated, rs (492) = .352–.665, ps \ .001 (Yeh 2005). The SB-CTT was developed by Yu-Chu Yeh (2005) and is based on Paul and Elder’s (2001) concepts of 10 intellectual standards and 8 reasoning elements. This test consists of a paragraph entitled The Life of Albert and 7 open-ended questions. With the theme ‘‘being happy’’, the paragraph describes Albert’s beliefs and thoughts about life. It also consists of several assumptions and a few issues, such as being happy, pessimistic, selfish, and being irresponsible, as well as believing in fate. After reading the paragraph, the participants are requested to analyze the paragraph from seven perspectives: (1) purpose and information; (2) issues; (3) assumptions; (4) points of view; (5) inferences; (6) implications; and (7) evaluation (Yeh 2005). A consensual assessment by two trained graduate students was employed to score the test in this study. The IPTE-CT scores measure participants’ self-evaluation of their personal teaching efficacy in critical thinking. The IPTE-CT was developed by Yeh and Chen (Chen 2001). Determined by factor analysis, the IPTE-CT comprises two factors: ‘‘efficacy in enhancing dispositions and skills’’ and ‘‘efficacy in improving prior knowledge.’’ As previously determined, Cronbach’s a coefficients for all items (20 items) and the two factors are .89, .88, and .88, respectively (Chen 2001). The test items include statements such as ‘‘I believe that I can make students active critical thinkers through my classroom teaching.’’ Response options range from 1 for ‘‘totally disagree’’ to 6 for ‘‘totally agree.’’ The IPK-CTI scores represent a participant’s self-evaluation of her/his professional knowledge that is required to teach critical thinking. The IPK-CTI, developed by Yeh (1999), consists of two factors that were selected after a factor analysis: ‘‘content knowledge’’ and ‘‘pedagogical content knowledge.’’ The test items include statements such as ‘‘I can create instructional designs for teaching critical thinking.’’ The IPK-CTI utilizes a 6-point Likert scale anchored from ‘‘totally disagree’’ to ‘‘totally agree.’’ As previously determined, Cronbach’s a coefficients for all items (9 items) and the two factors are .95, .92, and .93, ps \ .001, respectively (Yeh 1999).. 123.

(7) Integrating e-learning into Direct-instruction Model. 191. Finally, a reflection questionnaire, consisting of 8 open-ended questions, was developed by the researcher based on the research needs and content of the experimental instruction. Each of the 8 questions is listed in the section ‘‘Results’’.. Procedures and instructional design To determine whether integrating e-learning into the Direct-instruction Model effectively improved preservice teachers’ critical-thinking skills, professional knowledge, and personal teaching efficacy in teaching critical thinking, an 18-week period of experimental instruction focusing on the teaching of critical thinking was given to the experimental group at the beginning of the semester. The control group, however, did not receive any related instruction. Both the experimental group and the control group took the pretest in the first week and completed the posttest during week 18. While the control group only took the CTT-II as the pretest and the posttest, the experimental group received the CTT-II, the SB-CTT, the IPK-CTI, and the IPTE-CT as the pretest and the posttest. The reflection questionnaire was also included in the posttest of the experimental group. The instructional content in this study covered the following topics: (1) introduction to critical thinking; (2) factors that influence critical thinking; (3) critical-thinking skills; (4) evaluation of critical thinking; (5) processes and stages of critical thinking; (6) application of critical-thinking skills; (7) analysis and development of situation-based problems; (8) strategic thinking; (9) models of critical-thinking instruction; (10) effective teaching strategies, such as concept mapping, cooperative learning, discussion, role playing, cubing; and, (11) effective student and teacher behaviors of critical thinking. At the end of the instruction period, participants presented group work regarding instructional design and teaching critical thinking in secondary schools. The experimental instruction in this study emphasized scaffolding, modeling and observational learning, mastery of critical-thinking skills, monitoring processes, guided practices, cooperative learning, feedback, active participation, self-reflection, online discussions, and an online learning community. Based on the Direct-instruction Model, the experimental instruction comprised 4 stages: introduction, presentation, guided practice, and independent practice; e-learning was primarily incorporated in the guided practice and independent practice stages (see Fig. 2). During these two stages, participants completed group assignments and engaged in asynchronous online discussions on the following topics: the five critical-thinking skills in the CTT-II; situation-based problems; and strategic thinking. Group assignments—related to critical-thinking skills and effective teaching strategies—were available for participants to read during the instructional period. That is, online sharing of group work was encouraged. Moreover, participants were free to access the synchronous and asynchronous discussion board for individual and group discussions.. Analyses Since the control group was only used for the evaluation of participants’ improvements on the CTT-II, a mixed design of 2 (within group: pretest vs. posttest) 9 2 (between group: experimental vs. control) Repeated Measure Analysis of Variance was employed. Other analyses which evaluated participants’ improvement on the SB-CTT, the IPK-CTI, and the IPTE-CT were conducted via the one-way (within group: pretest vs. posttest) Repeated Measure Analysis of Variance.. 123.

(8) 192. Y.-C. Yeh. Steps. Emphases. Form of Teaching. Introduction and review. Overview and purpose Motivation. In-class teaching. Presentation. Concrete example Modeling. In-class teaching. Guided practice. Scaffolding Monitoring Discussion & Interaction. In-class and online discussion. Monitoring Mastering thinking skills Home work Feedback Group work Learning community. Independent practice. In-class and online discussion. Fig. 2 Integration of e-learning and the Direct-instruction Model. Results Improvements in critical-thinking skills This study mainly employed the CTT-II to determine whether the experimental group made a greater improvement in their critical-thinking skills than their control group counterparts. However, the study also employed the SB-CTT to assess the influence of the experimental instruction on the experimental group. Figure 3 presents the mean scores for the experimental and control groups on the CTT-II. The mixed design analysis found a significant Test (pretest vs. posttest) 9 Group (experimental vs. control) interaction effect, F (1, 92) = 9.034, p = .003. The following analyses of the simple main effects found a significant Test effect in the experimental group, F(1, 47) = 5.887, p = .019, g2 = .111, but not in the control group, F (1, 45) = 3.745, p = .059, g2 = .077. A comparison of the means showed that while the experimental group improved their mean score from the pretest to the posttest, the mean score of the control group decreased during that time; moreover, the experimental group outperformed the control group on both the pretest and posttest, t (92) = 4.276, p = .000, and t (92) = 6.850, p = .000 (see Table 1). Table 1 Analyses of simple main effects of group at test on the CTT-II Variance. M. SD. N. ANOVA. Comparison 2. MS. F (df). Sig.. g. 21.09. 5.887 (1, 47). .019. .111. 2[1. 25.04. 3.745 (1, 45). .059. .077. 1[2. Experimental Pretest (1). 17.00. 3.10. 48. Posttest (2). 17.94. 2.36. 48. Pretest (1). 14.48. 2.58. 46. Posttest (2). 13.43. 3.81. 46. Control. 123.

(9) Integrating e-learning into Direct-instruction Model. 5 4.5. 4.73 4.6. 4. 4.04. Mean. 4.23. 3.78. 3. Experimental Pretest. 3.85. 3.15. 3.5. 193. 3.15. 3.04. 2.86. 2.5. 2.57 2.46. 2. 2.92 2.67 2.65 2.26. 2.88 2.7 2.17 2.13. 1.5. Experimental Posttest Control Pretest Control Posttest. 1 0.5 atio n Ev alu. nat ion Ex pla. ion. tion De duc. As sum. In d uct. pti. on. 0. Fig. 3 Mean experimental group and control group scores on the CTT-II. The one-way analysis indicated that test (pretest vs. posttest) had a significant effect on the SB-CTT, K = .657, p = .000, g2 = .343. The following test results suggested that participants in the experimental group performed better on the posttest than on the pretest, F (1, 51) = 26.660, p = .000, g2 = .343 (see Table 2).. Improvements in professional knowledge The one-way Repeated Measure Analysis of Variance determined that Test (pretest vs. posttest) had a significant effect on the IPK-CTI, K = .181, p = .000, g2 = .819. A comparison of the means indicated that participants performed better on the posttest than on the pretest. To understand further participant improvement in professional knowledge, a 2 (Factor: content knowledge vs. pedagogical content knowledge) 9 2 (Test: pretest vs. posttest) analysis was employed. Although the interaction effect was not significant, K = .975, p = .316, g2 = .025), both the main effects of Factor (K = .536, p = .000, g2 = .464) and Test (K = .179, p = .000, g2 = .821) were significant. The following test showed that participants performed better on ‘‘content knowledge’’ than ‘‘pedagogical content knowledge.’’ Participants also performed better on the posttest than on the pretest (see Table 3). Table 2 Tests of within-subjects contrasts on the SB-CTT M. SE. Type III SS. df. MS. F. Sig.. g2. Pretest. 25.79. .35. 184.692. 1. 184.692. 26.660. .000. .343. Posttest. 27.67. .25. Source Test. 123.

(10) 194. Y.-C. Yeh. Table 3 Tests of within-subjects contrasts on the IPK-CTI Source. M. SE. Type III SS. df. MS. Kn1. 3.52. .11. 6.404. 1. 6.404. Kn2. 3.13. .12. Pretest. 2.37. .15. 155.187. 1. 155.187. Posttest. 4.29. .12. Sig.. g2. 35.510. .000. .464. 187.768. .000. .821. F. Factor. Test. Note. Kn1, content knowledge; Kn2, pedagogical content knowledge. Improvements in personal teaching Finally, the one-way analysis results found a significant effect of Test (pretest vs. posttest) on the IPTE-CT, K = .732, p = .000, g2 = .268. A comparison of the means again determined that participants performed better on the posttest than on the pretest. To further assess participant improvement in personal teaching efficacy, a 2 (Factor: prior knowledge vs. dispositions and skills) 9 2 (Test: pretest vs. posttest) analysis was performed. The interaction effect (K = .996, p = .681, g2 = .004) and the main effect of Factor (K = .944, p = .114, g2 = .056) were not insignificant. However, the main effect of Test (K = .744, p = .000, g2 = .256) was significant. The following test results indicated that, once again, participants performed better on the posttest than on the pretest (see Table 4).. Mechanisms that contributed to instructional effectiveness At the end of the experimental instruction, the participants answered 8 reflection questions to further identify their opinions about the experimental instruction. The questions and participant responses were as follows: 1. Did the online thematic discussions on the 5 critical-thinking skills contribute to your improvement in critical-thinking skills? Why? Most participants (94%) agreed that the online thematic discussions contributed to the improvement in their critical-thinking skills. The most important reasons were that the discussions provided a broader perspective that reduced their poorly-formed ideas (23%), stimulated ideas through discussion and sharing (21%), and improved their application of critical-thinking skills (21%). The remaining benefits included enhancement of complex thinking skills such as making inferences, induction, and deduction (9%) and rational thinking (9%). However, a few participants (4%) reported that face-to-face discussions would have been more helpful than online discussions. Table 4 Tests of within-subjects contrasts on the IPTE-CT M. SE. Type III SS. df. MS. F. Sig.. g2. Pretest. 4.26. .12. 8.069. 1. 8.069. 15.175. .000. .256. Posttest. 4.68. .08. Source Test. 123.

(11) Integrating e-learning into Direct-instruction Model. 195. 2. Have you read the other group’s work online and what are the main benefits of this? Why? Eighty-seven percent of the participants claimed to have read the other group’s work online. The primary reasons were to improve self-reflection by observing the performance of others (36%), to understand the attitudes and perspectives of others (17%), to provide a chance to practice critical-thinking skills (13%), and to take the other group’s work as inspiration to stimulate new ideas (9%). 3. This class combined e-learning with in-class teaching. Did such an instructional design contribute to the improvement in your critical-thinking skills? Why? Eighty-nine percent of the participants reported that this instructional design contributed to the improvement in their critical-thinking skills. The principal reasons were that discussing questions online increased their opportunities to practice critical-thinking skills (21%), online discussions provided them with opportunities to exchange information (19%), this integration improved their thinking skills and knowledge (15%), and observing the performance of others helped them solve problems on their own (11%). 4. Did the instructional design improve your professional knowledge and ability to teach critical thinking? Why? Most participants (79%) responded positively to this question. The primary supporting reasons were that this instruction provided professional knowledge and practices in critical thinking (34%), an opportunity to apply teaching skills (13%), practice in creating instructional design (11%), and knowledge of teaching strategies (8%). By contrast, those who did not favor the instructional design responded that it did not provide opportunities to practice teaching, e-learning is self-directed learning, and no immediate feedback was provided. 5. Did the instructional design contribute to the improvement in your self-confidence in teaching critical thinking? Why? Most participants (74%) answered positively to this question. The principal reasons included that it improved their instruction skills and self-confidence (28%), it provided increased opportunities to practice, discuss, and express ideas (21%), and the content and process contributed to the improvement in their teaching ability (11%). The few who had a more negative view and did not find the instructional design beneficial claimed that they lacked teaching experience (2%) and had a lack of self-confidence when teaching (2%). 6. Did the instructional design help you in terms of self-reflecting about your criticalthinking skills, attitude, and teaching? Why? Almost all of the participants (92%) responded positively to this question. The main reasons were that discussion contributed to self-reflection (32%), observational learning contributed to self-reflection (28%), and the instructional design contributed to thinking and self-reflection (19%). Those who responded negatively (6%), however, did not offer a specific reason. 7. What were the benefits or advantages of the instructional design? Why? Ninety-eight percent of the participants had a positive attitude toward the instructional design and shared their ideas. The main benefits cited were that it was very convenient for group discussions and for completing and handing in assignments (30%), increased their interaction among classmates (11%), increased their opportunities to review and practice critical-thinking skills outside of the classroom (21%), and increased their opportunities to share reports (19%). Other minor advantages were that the instructional design contributed to group cooperation, self-learning, and effective use of time and resources.. 123.

(12) 196. Y.-C. Yeh. 8. What are your feelings about, or what have you gained from, this class? Why? All of the participants had positive attitudes regarding this class. The primary responses were that they have learned about instructional design and the teaching methods in critical thinking (17%), have benefited substantially from group discussions (13%), have become more able to think rationally when faced with problems (11%), have become better able to share opinions and ideas (6%).. Discussion The effects of integrating e-learning into the Direct-instructional Model It is important to bear in mind that in conducting this research, it was somewhat difficult getting a control group to finish all of the same tests that the experimental group were taking; equally important is the fact that the major objective of this study was to evaluate the preservice teachers’ improvement in critical-thinking skills. For these two reasons, the control group was sampled from a post-bachelor teacher education program and it was only employed when the researcher was testing the participants’ improvement in the major aspects of critical thinking (i.e., the CTT-II). Improvements in other areas, including those in professional knowledge and personal teaching efficacy, were evaluated by comparing the experimental group’s scores in the prestest with those in the posttest. Although a pretest-posttest control group design is generally considered better than a before-and-after design, the before-and-after design is also commonly deemed acceptable when a control group is difficult to get. To compensate for any shortcoming that may have been brought on by the before-and-after design here and to minimize errors that could have been incurred, this study employed 2 complementary instruments: the SB-CTT and the reflection questionnaire. The results of the Repeated Measure Analyses of Variance as well as the content analysis of the participants’ responses in the reflection questionnaire support the hypotheses proposed in this study. Thus, the researcher takes the position that these results attest to the effectiveness of the instructional design and experimental instruction design that were employed in this study. In other words, integrating e-learning into the Direct-instruction Model can substantially enhance teacher effectiveness in criticalthinking instruction. This study employed two tests (the CTT-II and the SB-CTT) to evaluate whether or not the participants had improved their critical-thinking skills during the18-week period of experimental instruction. The analytical results indicated that the experimental group scored considerably higher on the posttest than on the pretest on the CTT-II while the control group did not show this pattern. Moreover, it was found that the experimental group performed significantly better than the control group on the CTT-II. Apart from this, the experimental group greatly improved their mean score on the SB-CTT over the 18week test period. These findings suggest that the experimental instruction in this study was effective in improving the preservice teachers’ critical-thinking skills. Important to note, 89% of the participants reported that the instructional design improved their critical-thinking skills. This study also found that participants significantly improved their professional knowledge (particularly in terms of content knowledge) and personal teaching efficacy in critical thinking. This was later confirmed by the participants’ answers to the open-ended reflective questions, when most participants responded that the instructional design had enhanced their professional knowledge (79%) and personal teaching efficacy (74%). These findings, again, substantiate that the instructional design in. 123.

(13) Integrating e-learning into Direct-instruction Model. 197. this study can effectively improve the professional development of preservice teachers in regards to teaching critical thinking. In the Direct-instruction Model, each teacher assumes the bulk of the responsibility for structuring skills, explaining these skills to students, providing students with opportunities to practice, and providing feedback (Eggen and Kauchak 1996). The experimental findings in this study provide evidence that although the Direct-instruction Model is a teachercentered approach, its four-stage design is in line with that of information-process theory and, therefore, enhances the acquisition of critical-thinking skills. More specifically, the overview and introduction components of the first stage activate sensory memory (Eggen and Kauchak 2001); the well-structured presentation of the course contents in the second stage and the guided practice in thinking skills in the third stage help students record information into their long-term memory (Stein and Carnine 1999; Sweller et al. 1998). Finally, the independent practice in thinking skills in the fourth stage decreases the load on the working memory when applying the thinking skills (Eggen and Kauchak 2001). Furthermore, study findings support claims that group work on problem solving serves to facilitate students’ understanding of the logic required in problem solving (Mevarech 1999) and homework can significantly increase learning (Cooper et al. 1998). The findings in this study also indicate that the integration of asynchronous online learning with in-class teaching is preferred in that it is a powerful approach for improving critical-thinking skills. The enhancement of critical-thinking skills relies on interaction and discussion. In this regard, a few participants in this study stated that the major disadvantage of asynchronous discussion was its inability to provide face-to-face interaction. Added to that, some participants complained that e-learning was limited by its inability to provide immediate feedback and facilitate teaching practice. Prior to the experimental instruction, the researcher in the present study was mindful of these limitations of asynchronous elearning and, to mitigate such drawbacks, this study employed an approach where asynchronous e-learning operates in tandem with traditional instruction. More to the point, during the experimental instruction, in-class discussion followed each thematic online discussion, thereby providing face-to-face interaction and feedback to online discussions. Moreover, a teaching demonstration at the end of the experimental instruction provided opportunities for teaching practice. Halpern (1998) proposed the following four-part model for teaching and learning critical thinking: (a) a dispositional component that prepares learners for cognitive work; (b) instruction in critical-thinking skills; (c) training in the structural aspects of problems and arguments to enhance the transcontextual transfer of critical-thinking skills; and (d) a metacognitive component that consists of checking for accuracy and monitoring one’s progress toward the objectives. The instructional design in this study is, in essence, consistent with this model and has proven effective in improving critical thinking. First, this study encouraged forming an online learning community and utilized cooperative learning to inspire participants. Second, this study used a scaffold model to teach critical-thinking skills via guided and independent practice. Third, this study asked participants to develop a situation-based test to enhance the transfer of critical-thinking skills from the classroom to real-world contexts. Finally, this study employed online thematic discussions that allowed participants to check their logic and thinking and that enabled them to monitor their personal progress. Briefly speaking, the findings in this study support the claims that scaffold-based instruction, a learning community, cooperative learning, online discussion, interaction, and reflection are important for online learning (Bliesener 2006; Brush and Saye 2001; Collison et al. 2000; MaKinster et al. 2006; Rossman 1999; Zydney 2005; Waltonen-Moore et al. 2006). Moreover, the integration of these elements into the Direct-instruction Model. 123.

(14) 198. Y.-C. Yeh. effectively enhanced preservice teacher development of the skills, knowledge, and confidence required to teach critical thinking. Mechanisms that contribute to instructional effectiveness Discussion and sharing This study has demonstrated that the participants were almost unanimous in expressing positive attitudes toward online discussions, and the principal reasons were that online discussion provides a broader perspective, stimulates ideas, improves critical-thinking skills, is convenient in terms of exchanging information, and increases knowledge. Bastiaens and Martens (2000) noted that web-based learning provides opportunities for users to express their ideas and that it stimulates thinking through interpersonal interactions. Eastmond (1995) also argued that learners should be provided an opportunity to interact and reflect and that such learning occurs when an instructor merges online discussions and learning methods in computer-mediated communication. This study confirms the effects of discussion and sharing on e-learning. The participants in this study were requested to engage in online thematic discussions, which created opportunities for interactive discussions and idea-sharing. The group discussion board is a convenient and private space for interaction among group members and is the most frequently used as an interface in this study. There were 7 groups in this study. The discussion frequencies for the 7 groups were 148, 55, 288, 71, 111, 68, and 177, during the 18-week period. As the ability to interact with others, such as discussion, arguing, and expressing ideas, is a key strategy in critical thinking (Norris and Ennis 1989), it seems reasonable to conclude that online discussions are an effective vehicle for improving the critical-thinking skills of preservice teachers. Observational learning To provide opportunities for peer modeling and observational learning, the participants in this study were asked to hand in all assignments to the e-learning interface, and these assignments were always available for them to read; moreover, a group presentation on instructional design and a teaching demonstration were required. Most participants (87%) recalled that they had read other groups’ assignments online and this process had contributed to their reflection, multiple-perspective thinking, perspective-taking, and the generation of novel ideas. Self-reflection, idea-sharing, multiple-perspective thinking, and perspective-taking are skills essential to good critical thinking (Paul and Elder 2001), and observational learning clearly contributes to the cultivation of such skills. When completing a challenging task associated with the application of complex thinking—such as critical thinking—opportunities to learn from the performance and successful experiences of others are especially valuable. Thus, peer-modeling and observational learning can help cultivate critical-thinking skills and upgrade the performance level of preservice teachers, as the theory ZPD suggests. Reflection Maher and Jacob (2006) found that teacher use of Computer-Mediated Communication (CMC) facilitates peer interactions and the reflective consideration of changing instruction. 123.

(15) Integrating e-learning into Direct-instruction Model. 199. practices. Rossman (1999) also suggested that with CMC, cooperative learning extends beyond a classroom to potentially all classrooms and encourages thoughtful reflection and increasingly complex responses as participants have adequate time to consider prior to posting their own responses. These arguments echo the findings in this study, in which 92% of the participants responded that the instructional design contributed to their cultivation of positive attitudes toward reflection and self-improvement and that such benefits were obtained primarily through discussions and observational learning. A reflective mind is one that takes a reasoned thinking process seriously, and is the hallmark of critical thinking (Schroyens 2005). Studies regarding student teachers have suggested that self-awareness and mindfulness contribute to nurturing reflective practices (Collier 1999; Tillema 2000; Titone et al. 1998) and that ‘‘providing feedback’’ is an effective method of increasing self-awareness and mindful learning (Titone et al. 1998). Similarly, Steele (2001) noted that by interacting with others, learners can reflect and exchange ideas, which contributes to mindful learning. The instructional design utilized in this study increases ‘‘peer feedback’’ and ‘‘teacher feedback’’ simultaneously. While peer feedback is mainly provided through online discussions, teacher feedback is provided through classroom interactions. Besides these, this study also provided feedback through test results in an effort to increase participants’ self-awareness and self-reflection on their degree of professional knowledge and on their ability concerning capabilities vis-a`-vis critical-thinking instruction. Guided practice In this study, e-learning was primarily incorporated into the stages of guided practice and independent practice. After first giving an introductory lesson on 5 critical-thinking skills, the teacher provided examples of corresponding test items, and then requested that the participants submit their own test items for online discussion. The online discussion for each critical-thinking skill lasted 1 week and then face-to-face interactions in class followed. The significant results in the quantitative tests and the qualitative responses in the reflection questionnaire attested to the positive effects of such scaffold teaching and guided practices. Participants felt that their improvement with respect to critical-thinking skills and self-confidence in teaching critical thinking were rooted in the many opportunities they were given to practice critical-thinking skills. Thus, on the weight of the evidence here, it seems that mastering critical-thinking skills via guided practice is crucial to enhancing personal teaching efficacy. Personal teaching efficacy is associated with reflective teaching, goal-setting, and the use of analytical strategies. Direct mastery by way of practical experience is a prerequisite for the creation of a strong sense of efficacy (Bandura 1995) and an effective method of obtaining mastery experience is via well-guided practices (Yeh 2006). Therefore, the findings in this study are consistent with claims that scaffolding tools positively affect the ability of students to ask critical questions (Zydney 2005), and that guided practices contribute to achieving mastery experience that powerfully and positively affects teacher efficacy (Albion 2001; Bandura 1997). Learning community Learning communities are environments that encourage mutual exchange among community members and thereby support individual and collaborative learning. Learning. 123.

(16) 200. Y.-C. Yeh. communities benefit learners as they encourage shared ways of knowing, encourage and facilitate active participation, improve achievements, contribute to knowledge creation, and challenge learners’ cognitive abilities (Moller 1998; Ludwig-Hardman and Woolley 2000). Collison et al. (2000) argued that online learning communities, such as small learnerfacilitated discussion groups, should have such characteristics as routine idea-sharing, honest expression of ideas, obvious interaction, cooperation, and reciprocal teaching. On this note, Hann et al. (2000) noted that online discussions, group tasks or projects, cooperative problem solving, and case studies contribute to the formation of online learning communities. This study attempted to encourage the development of a learning community by means of thematic online discussions, cooperative problem-solving, and group tasks. That the group discussion board was visited with high frequency in this study is indicative that an online learning community had been formed.. Conclusions Many researchers (e.g., Burch 2001; Egbert et al. 2001) have urged that instructional design for e-learning must incorporate appropriate principles and conditions for learning that specifically meet learners’ needs. This study therefore attempted to integrate elearning and the Direct-instruction Model to improve preservice teachers’ professional development in critical-thinking instruction. The results of hypothesis testing in this study along with the participants’ responses in the reflection questionnaire suggest that integrating e-learning into the Direct-instruction Model can improve preservice teachers’ critical-thinking skills, professional knowledge, and personal teaching efficacy, and this is chiefly achieved via the mechanisms of discussion and sharing, observational learning, self-reflection, guided practices, and participation in the learning community. In this era of technology and knowledge economics, critical thinking is essential for individuals to be able to create new knowledge and to adapt to rapid changes in society. As the society asks students to think critically, it is assumed that teachers can think critically and know how to teach critical thinking. Such an assumption is, regrettably, often incorrect. Teacher education must provide appropriate courses that ensure that preservice teachers are effective in teaching critical thinking. Further research should focus on exploring how the mechanisms identified in this study influence preservice teachers in learning how to teach critical-thinking skills with the integration of e-learning. Acknowledgment The author would like to thank the National Science Council of the Republic of China, Taiwan, for financially supporting this research under Project No. NSC93–2520-S-004–002.. References Albion, P. (2001). Some factors in the development of self-efficacy beliefs for computer use among teacher education students. Journal of Technology and Teacher Education, 9(3), 321–347. Bailin, S., Case, R., Coombs, J. R., & Daniels, L. B. (1999). Common misconceptions of critical thinking. Journal of Curriculum Studies, 31(3), 269–283. Bandura, A. (1995). Exercise of personal and collective efficacy in changing societies. In A. Bandura (Ed.), Self-efficacy in changing societies (pp. 1–45). New York: Cambridge University Press. Bandura, A. (1997). Self-efficacy: The exercise of control. New York: Freeman.. 123.

(17) Integrating e-learning into Direct-instruction Model. 201. Bastiaens, T. J., & Martens, R. L. (2000). Conditions for web-based learning with real events. In B. Abbey (Ed.), Instructional and cognitive impacts of web-based education (pp. 1–31). London: Idea Group Publishing. Bliesener, T. (2006). Training synchronous collaborative e-learning. International Journal on E-Learning, 5(2), 185–196. Browne, M. N., & Meuti, M. D. (1999). Teaching how to teach critical thinking. College Student Journal, 33(2), 162–170. Brush, T., & Saye, J. (2001). The use of embedded scaffolds with hypermedia-supported student-centered learning. Journal of Educational Multimedia and Hypermedia, 10(4), 333–356. Burch, R. (2001). Effective web design and core communication issues: The missing components in webbased distance education. Journal of Educational Multimedia and Hypermedia, 10(4), 357–367. Carmen, C., & Kurubacak, G. (2002). The use of the internet to teach critical thinking. In C. Crawford, D. A. Willis, R. Carlsen, I. Gibson, K. McFerrin, J. Price, & R. Weber (Eds.), Proceedings of society for information technology and teacher education international conference 2002 (pp. 2504–2505). Chesapeake: AACE. Chen, Y. M. (2001). Teacher training for the employment of teaching strategies in critical-thinking via a computer simulation: An example in gender equality issues. Unpublished thesis, National Sun Yat-sen University, Kaohsiung, Taiwan. Collier, S. T. (1999). Characteristics of reflective thought during student teaching experience. Journal of Teacher Education, 50(3), 173–181. Collison, G., Elbaum, B., Haavind, S., & Tinker, R. (2000). Facilitating online learning: Effective strategies for moderators. Madison: Atwood Publishing. Cooper, H., Lindsay, J., Nye, B., & Greathouse, S. (1998). Relationships among attitudes about homework, amount of homework assigned and completed, and student achievement. Journal of Educational Psychology, 90(1), 70–83. Eastmond, D. V. (1995). Alone but together: Adult distance study through computer conferencing. Cresskill: Hampton Press. Egbert, J., & Thomas, M. (2001). The new frontier: A case study in applying instructional design for distance teacher education. Journal of Technology and Teacher Education, 9(3), 391–405. Eggen, P. D., & Kauchak, D. P. (1996). Strategies for teachers: Teaching content and thinking skills (3rd ed.). Needham Heights: Ally & Bacon. Eggen, P. D., & Kauchak, D. P. (2001). Educational psychology: Windows on classrooms (5th ed.). Upper Saddle River: Merrill Prentice Hall. Elder, L. (1997). Critical thinking: The key to emotional intelligence. Journal of Developmental Education, 21(1), 40–41. Ennis, R. H., Millman, J., & Tomko, T. N. (1985). Cornell critical thinking tests, level X & level Z- manual (3rd ed.). Pacific Grove: Midwest Publications. Facione, N. C., & Facione, P. A. (1994). The ‘‘california critical thinking skills test’’ and the national league for nursing accreditation requirement in critical thinking. Millbrae: California Academic Press. Gadzella, B. M., & Masten, W. G. (1998). Critical thinking and learning processes for students in two major fields. Journal of Instructional Psychology, 25(4), 256–261. Giancarlo, C. A., & Facione, P. A (2001). A look across four years at the disposition toward critical thinking among undergraduate students. Journal of General Education, 50(1), 29–55. Halpern, D. F. (1998). Teaching critical thinking for transfer across domains dispositions, skills, structure training, and metacognitive monitoring. American Psychologist, 53(4), 449–455. Halpern, D. F. (2003). Knowledge and thought: An introduction to critical thinking (4th ed.). Hillsdale: Lawrence Erlbaum. Hann, D., Glowacki-Dudka, M., & Conceicao-Runlee, S. (2000). 147 practical tips for teaching online groups: Essentials of web-based education. Madison: Atwood Publishing. Harris, J. C., & Eleser, C. (1997). Developmental critical thinking: Melding two imperatives. Journal of Developmental Education, 21(1), 12–19. Hittner, J. B. (1999). Fostering critical thinking in personality psychology: The trait paper assignment. Journal of Instructional Psychology, 26(2), 92–97. Kumta, S., Tsang, P., Hung, L., & Cheng, J. (2003). Fostering critical thinking skills through a web-based tutorial programme for final year medical students—a randomized controlled study. Journal of Educational Multimedia and Hypermedia, 12(3), 267–273. Lawson, T. J. (1999). Assessing psychological critical thinking as a learning outcome for psychology majors. Teaching of Psychology, 26(3), 207–209. Leader, L., & Middleton, J. (2003). From ability to action: Technology-integrated instruction for criticalthinking dispositions. In C. C. Crawford, D. A. Willis, R. Carlsen, I. Gibson, K. McFerrin, J. Price, &. 123.

(18) 202. Y.-C. Yeh. R. Weber, (Eds.), Proceedings of society for information technology and teacher education international conference 2003 (pp. 1360–1367). Chesapeake: AACE. Lin, M., Park, K., & Michko, G. (2003). An investigation of critical thinking skills in computer-based educational software using content analysis. In C. C. Crawford, D. A. Willis, R. Carlsen, I. Gibson, K. McFerrin, J. Price, & R. Weber (Eds.), Proceedings of society for information technology and teacher education international conference 2003 (pp. 755–756). Chesapeake: AACE. Loving, G. (2000). Role modeling critical thinking in an online course for nurse educators. In C. Crawford, D. A. Willis, R. Carlsen, I. Gibson, K. McFerrin, J. Price, & R. Weber (Eds.), Proceedings of society for information technology and teacher education international conference 2000 (pp. 120–123). Chesapeake: AACE. Ludwig-Hardman, S., & Woolley, S. (2000). Online learning communities: Vehicles for collaboration and learning in online learning environments. In Proceedings of world conference on educational multimedia, hypermedia and telecommunication 2000 (pp. 1556–1558). Chesapeake: AACE. Maher, M., & Jacob, E. (2006). Peer computer conferencing to support teachers’ reflection during action research. Journal of Technology and Teacher Education, 14(1), 127–150. MaKinster, J., Barab, S., Harwood, W., & Andersen, H. (2006). The effect of social context on the reflective practice of preservice science teachers: Incorporating a web-supported community of teachers. Journal of Technology and Teacher Education, 14(3), 543–579. McCarthy-Tucker, S. N. (2000). Teaching style, philosophical orientation and the transmission of critical thinking skills in the US public schools. Korean Journal of Thinking and Problem Solving, 10(1), 69– 77. Mevarech, Z. (1999). Effects of cognitive training embedded in cooperative settings on mathematical problem solving. Journal of Educational Research, 92(4), 195–205. Moller, L. (1998). Designing communities of learners for synchronous distance education. ETR & D, 46(4), 115–122. Nelson, T., & Oliver, W. (2004). Maximizing critical thinking skills with technology. In P. Kommers & G. Richards (Eds.), Proceedings of world conference on educational multimedia, hypermedia and telecommunications 2004 (pp. 3982–3986). Chesapeake: AACE. Norris, S. P., & Ennis, R. H. (1989). Evaluating critical thinking. Pacific Grove: Midwest Publications. Paul, R., & Elder, L. (2001). Critical thinking: Tools for taking charge of your learning and your life. Upper Saddle River: Prentice Hall. Rossman, M. H. (1999). Successful online teaching using an asynchronous learner discussion forum. Journal of Asynchronous Learning Networks 3(2). Retrieved June 22, 2006 from the http://www.aln.org/publications/jaln/v3n2/v3n2_rossman.asp. Pressley, M., & McCormick, C. B. (1995). Advanced educational psychology for educators and policy makers. New York: Harper Collins College. Schroyens, W. (2005). Knowledge and thought: An introduction to critical thinking. Experimental Psychology, 52(2), 163–164. Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1), 1–21. Steele, D. F. (2001). Using sociocultural theory to teach mathematics: A Vygotskian perspective. School Science and Mathematics, 101(8), 404–415. Stein M., & Carnine, D. (1999). Designing and delivering effective mathematics instruction. In R. Stevens (Ed.), Teaching in American schools (pp. 245–270). Upper Saddle River: Merrill/Prentice Hall. Sweller, J., van Merrienboer, J., & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10, 196–251. Tillema, H. H. (2000). Belief change towards self-directed learning in student teachers: Immersion in practice or reflection on action. Teaching and Teacher Education, 16(5–6), 575–591. Titone, C., Sherman, S., & Palmer, R. (1998). Cultivating student teachers’ dispositions and ability to construct knowledge. Action in Teacher Education, 19(4), 76–87. Torff, B. (2005). Developmental changes in teachers’ beliefs about critical-thinking activities. Journal of Educational Psychology, 97(1), 13–22. Waltonen-Moore, S., Stuart, D., Newton, E., Oswald, R., & Varonis, E. (2006). From virtual strangers to a cohesive learning community: The evolution of online group development in a professional development course. Journal of Technology and Teacher Education, 14(2), 287–311. Watson, G., & Glaser, E. M. (1980). Watson-Glaser critical thinking appraisal. San Antonio: Psychological Corp. Yeh, Y. C. (1999). A study of substitute teachers’ professional knowledge, personal teaching efficacy, and teaching behavior in critical-thinking instruction. Journal of Chengchi University, 78, 55–84. Yeh, Y. C. (2003). The manual of critical thinking test, level I. Taipei: Psychological Publishing.. 123.

(19) Integrating e-learning into Direct-instruction Model. 203. Yeh, Y. C. (2004). Nurturing reflective teaching during critical-thinking instruction in a computer simulation program. Computers & Education, 42(2), 181–194. Yeh, Y. C. (2005). Integratimg e-learning into teacher education curriculum: Its effects on improving preservice teachers’ ability to teach critical thinking. Taipei: National Council of Science. (NSC93– 2520-S-004–002). Yeh, Y. C. (2006). The interactive effects of personal traits and guided practices on preservice teachers’ changes in personal teaching efficacy. British Journal of Educational Technology, 37(4), 513–526. Zydney, J. (2005). Eighth-grade students defining complex problems: The effectiveness of scaffolding in a multimedia program. Journal of Educational Multimedia and Hypermedia, 14(1), 61–90.. 123.

(20)