Introduction
Facing with the fast development of science and technology, it has been widely recognized that students should engage in real-world tasks from an interdisciplinary curriculum, combining theory and practice, and develop their skills in creative problem solving. In response to the needs, Science, Technology, Engineering and Mathematics (STEM) curriculum model was proposed and one of the most significant and distinguishing curriculum characteristics of STEM compared to other discipline domains is hands-on skill development.
Hands-on skill (practical skill) development can bridge the gap between theory and practice as well as enhance problem solving skills. Dufresne, Gerace and Leonard (1997) have proposed a model for the problem solving process, identifying three key knowledge essentials of the process: conceptual knowledge (CK), operational/procedural knowledge (PK), and problem-state knowledge (PSK).Hands-on skill (practical skill), which includes concept knowledge (CK) and procedural knowledge (PK), has been acknowledged as the core competence in technology education and is defined as the capability to utilize the skills, techniques, and engineering tools for technology practice (Barlex, 2007; McLaren, 2007). Hands-on science could promote learning transference because it echoes with the concrete-to-abstract nature of cog-nitive development, providing additional sources of brain activation via kin-aesthetic involvement and increasing students’ motivation and engagement (Flick, 1993; Klahr, Triona & Williams, 2007). Clough and Kauffman (1999) had encouraged students to make repetitive “connections” between concepts in various of contexts and applications, to enhance their problem-solving skills.
with ComputeR-assisted
Remedial leaRning sYstem
suppoRt FoR voCational
high sChool students
Shu-Hsuan Chang, Ai-Chiao ku,
li-Chih Yu,
Tsung-Chih wu,
Bor-Chen kuo
Shu-Hsuan Chang, Ai-Chiao ku National Changhua University of
Education, Taiwan li-Chih Yu National Chengchi University, Taiwan Tsung-Chih wu National Changhua University of Education, Taiwan
Bor-Chen kuo National Taichung University of Education, Taiwan Abstract. One of the significant and distinguishing curriculum characteris-tics of STEM compared to other subjects domains is hands-on skill development. Hands-on training enables the enhance-ment of learning because it parallels with the concrete-to-abstract nature of cognitive development, providing additional sources of brain activation via kinaesthetic involve-ment and elevating students’ motiva-tion and engagement. To bridge the gap between theory and practice, and advance unprepared students’ hands-on skills, this research proposed and implemented an innovative STEM course with the computer-assisted remedial learning system (CARLS) in the vocational high school experience in Taiwan. The effects of STEM course were examined through an experiment with learning performances hypotheses. A total of 32 students in a vocational high school in Taiwan voluntarily participated in this research and a one-group pre-test and post-test pre-experimental design was adopted. The results of this experimental course demonstrated that CARLS is effective and contributed in enhancing students’ knowledge, achievement and hands-on skill performance in this STEM course. Suggestions and implications for STEM education is also made for practitioners and educators.
key words: hands-on activities, STEM, Computer-Assisted Remedial Learning System, vocational high school.
Flick (1993) pointed out that hands-on activities usually emphasize students’ capacities of logical, mathematical, linguistic, and spatial intelligences. Unal (2008) revealed that hands-on activities did have a significant positive effect to help students to replace their misconceptions with correct ones. Furthermore, the hands-on activity, trial-and-error experiential learning helped students enhance their creative skills and problem-solving abilities as well as realize the value of collaboration (Shieh & Chang, 2014). Undergraduate students’ participation in hands-on activities is widely believed to encourage them to pursuit of advanced degrees and careers in STEM fields (Russell, Hancock & McCullough, 2007). Moreover, internship and mentoring programs in STEM fields can foster student interest and provide valuable hands-on experience (Christie et al., 2008).
It is true that students need to acquire CK and PK that are utilized during their design and/or problem solving tasks. In addition, CK and PK have to be taught in an active manner and be modelled by teachers, experts or fellows (Sidawi, 2009). However, the connection between theory and practice for technology education needs to be established firstly because CK brings up the use of PK (McCormick, 1997). In exploring how novices and experts solved mechanical problems, Johnson (1997) found out that “The better performance of the experts was
attributed to their deeper conceptual understanding of physics principles” (p. 164). Therefore, if education ignores
students’ comprehension of CK, it is highly probable that students cannot solve problems independently and the Einstellung Effect (Set Effect) might occur (Luchins, 1942). Failures in delivering CK result in the inflexibility of the students to deal with novel problems, which is harmful to skill transference (Gagne, Yekovich & Yekovich, 1993). Similar results were reported in the studies upon troubleshooting in the electrical and mechanical systems (Case & McKeough, 1990; Means & Gott, 1988). Many studies have indicated that students’ conceptual understanding is significantly associated with their problem-solving ability (Dym, Agogino, Eris, Frey & Leifer, 2005; Gagne et al., 1993; Luchins, 1942; Lutgens & Mulder, 2002). As a result, the level of CK understanding as well as the interaction of CK and PK enables to effect the successful transfer of knowledge.
In response to the STEM trend, the Ministry of Education (2003) in Taiwan announced a nine-year curriculum guideline for 1-9 grade schools in which “nature and living technology” and “mathematics” emerged as the imple- and “mathematics” emerged as the imple-“mathematics” emerged as the imple-mentation of STEM course, to cultivate the students’ science and technology literacy. The content of the nature and living technology curriculum includes knowledge and abilities related to materials, energy, life science, the planet, environmental science, wildlife and nature preservation, the use of information technology, science and scientific studies, and respect for life. The mathematics curriculum includes the basic concepts of numbers, patterns, and quantities; calculation ability; reasoning and critical thinking ability; and the ability to discuss mathematics with others (Ministry of Education, 2003). However, since them, STEM teaching and learning which emphasized students’ position as the center of learning activities (Lou, Shih, Diez & Tseng, 2011), were still in their cradle stage in Taiwan (Tseng, Chang, Lou & Chen, 2013). Two reasons will be addressed as follows.
(1) Lack of the chance to put theory into practice
In Taiwan, some studies emerged on integrating STEM into experiments in teaching (Chang, 2009; Chang, Chen, Kuo & Shen, 2011; Chang, Wu, Kuo & You, 2012; Tseng et al., 2013; Chang, Yu, Kuo, Mai & Chen, 2015) have found that most students show positive attitudes toward STEM course, but they had no chance to understand the interrelationship of STEM or to put theory into practice, though they acquired more technology-related knowledge at school. Moreover, mathematics was found to be the least popular subject within STEM.
The vocational education of hands-on skill is creating educational structures of CK and PK to deliver 50-minute lectures, and helping students acquire skill in practicum. Students’ achievements of hands-on skills are limited because the teaching paradigm and students’ individual difference, though remedial learning was developed to solve these problems. However, remedial education often carries a stigma because of its association with under-prepared students.
(2) Overemphasize PK teaching and operation training
The mission of Technological and Vocational Education (TVE) in Taiwan seeks to cultivate technical manpower for the country; hence, vocational education programs emphasize students’ hands-on skills as well as problem solving abilities (Ministry of Education, 2006). Senior vocational high school in Taiwan acts in concert with the economical policy of government-providing skilled labor for the small and medium enterprises in 1960s, offering the quantity and quality of labor for the capital and technology-intensive industries in 1970s, and supplying higher levels of technological and business personnel for internationalization and the open market in 1980s (Ministry of Education, 2015). Therefore, senior vocational high school in Taiwan is determined to equip students with
obtain professional certificates and credits to enhance their competitiveness by participating in skill competitions (contests) for entering a higher education. Consequently, some technology teachers seem to overemphasize PK teaching and operation training, ignoring students’ comprehension of CK and the connection between cognition and metacognition, which might become the obstructions for students’ hands-on skill acquisition (Dym et al., 2005; Lutgens & Mulder, 2002).
Therefore, how to generate effective hands-on skill learning system for STEM courses for senior vocational high students by enhancing the interaction of CK and PK to clarify the misconception of students is an important issue.
Due to the robust development of Information Communication and Technology (ICT), the development of computer-assisted learning systems (CALS), which adapts users’ features such as knowledge structure, background, preference and interest to meet the individual needs of the students, becomes promising and feasible (Lazarinis, Green & Pearson, 2010; Lin, Liu & Yuan, 2001; Liu, Lin, Chiu & Yuan, 2001). Meanwhile, CALS not only alleviates the burdens of STEM faculties, but also offers sufficient resources for students. Additionally, it is important to develop a CALS to diagnose student’s individual learning difficulties and misconception; the benefits of which also include providing educational supports and guidance/remedial learning services based on the diagnosis (Chen, 2011; Hsiao et al., in press; Hwang, Panjaburee, Triampo & Shih, 2013). The comparison of expert and student knowledge structures could serve as a key reference for efficient process to diagnose student’s individual learning difficulties and misconception (Appleby, Samules & Treasure-Jones, 1997; Brown & Burton, 1978; Chang, Liu & Chen, 1998; Shih, Kuo & Liu, 2012; Wenger, 1987; VanLehn, 1988; Wu, Kuo & Yang, 2012).
Although people generally consider constructing a computer-assisted learning environment to be expensive, complicated and time-consuming, more evidences are still to convince educators of educational efficiency of CALS. With the attempt to shorten the gap between theory and practice and tackle the problem regarding lack of an educational Paradigm in STEM field. Therefore, this research proposed and implemented an innovative STEM course with CARLS support to help students clarify their misconceptions and enhance the interaction of CK and PK in the STEM course. By applying the Knowledge Structure-based Adaptive Testing algorithm (Shih et al., 2012; Wu et al., 2012), this proposed CARLS has two distinguishing features: (1) CARLS, significantly reducing the number of test items compared with traditional paper and pencil test, can provide different test items according to indi-vidual differences in prior knowledge of students. (2) Test items of CARLS focusing on the interaction of CK and PK different from those of traditional paper and pencil test enhance students’ hands-on ability. From an educational viewpoint, students do not really acquire mastery of hands-on skill until they are able to utilize the interaction of CK and PK to solve problems successfully. Accordingly, the proposed STEM course with CARLS support would enhance the interaction of CK and PK to strengthen students’ hands-on ability. The current research addressed the following issues:
The effectiveness of the proposed STEM course with
1. CARLS support on the knowledge achievement;
The effectiveness of the proposed STEM course with
2. CARLS support in enhancing hands-on skill
per-formance;
The students’ technology acceptance attitude toward
3. CARLS.
Computer-assisted Remedial learning System
Traditional Science and Technology Education (STE) is concentrating on gaining appropriate professional knowledge and skills. Pedagogical process usually is based on memorization of important selected knowledge and learning formal algorithms for corresponding use of them in practice (Broks, 2011; 2014). Developing an effective remedial learning system to inspire scientific thinking as a backbone of new STE is an issue.
With the vigorous development of computer and internet technology, development of teaching/learning system refocuses on adapting users’ features such as goals/tasks, knowledge, background, hyperspace experi-es on adapting users’ features such as goals/tasks, knowledge, background, hyperspace experi- on adapting users’ features such as goals/tasks, knowledge, background, hyperspace experi-ence, preferexperi-ence, and interests to meet the individual needs of students (Lazarinis et al., 2010; Lin et al., 2001; Liu et al., 2001). It is important to develop a CALS wherein testing items and procedures could be personalized according to student’s individual performance, prior knowledge, goals and preferences (Chen, 2011; Hsiao et al., in press; Hwang et al., 2013). Moreover, the key issue is how to diagnose student’s individual learning difficulties and misconceptions as well as to provide educational supports and guidance/remedial education based on the above diagnosis. The comparison of expert and student knowledge structures is an efficient process to diagnose students’ individual learning difficulties and misconceptions (Appleby et al., 1997; Brown & Burton, 1978; Chang, Liu
& Chen, 1998; VanLehn, 1988; Wenger, 1987; Shih et al., 2012; Wu et al., 2012). Therefore, a computerized adaptive testing system (Shih et al., 2012; Wu et al., 2012) for this proposed STEM course was constructed to help students detect misconception, remedy it, and, hopefully, enhance their hands-on skills.
In traditional paper and pencil test item for Conceptual Knowledge, students are asked questions, e.g. If the block (open/short) is short between the node E and F in this circuit (Figure 1), what is current I3, to show their understanding of knowledge. However, CARLS provides students with test item in form of the interaction of CK and PK. With the same question, CARLS would not only ask students to calculate the I3, but also measure it with multimeter operation. Therefore, the question will be revised to be: How do students apply multimeter to measure current I3 (Figure 1)? Firstly, students are able to apply the principle to calculate I3, and decide the block (open or short) is open or short between the node E and F. Finally, students use the multimeter to measure I3, and ensure the calculation value.
AC
Rectifier
Circuit
Capacitor
Filter
Open
or
Short
10K
40K
A
B
C
D
E
F
10K
15K
5k
20K
I
1
I
2
I
3
Figure 1: Calculation of the current.
In the research, 20 industrial electronics experts were invited to construct the expert knowledge structure as well as to modify the item bank of CARLS from Industrial electronics C Technician Skills Certification Test (Skill Evaluation Center, 2013). Figure 2 illustrates the expert knowledge structure of “electronics meter operation and measurement skill”. The lower nodes represent the fundamental concepts; the upper nodes, which are more com-plicated concepts than lower ones, are the integrated/interaction concepts of lower ones.
Instrument operation and measurement
1 Measurement of
multimeter (ACV) multimeter (DCV) 2 Measurement of multimeter (DCmA) 3 Measurement of
1-2 M ea su rem en t o f m ult im ete r (A C V ) 2-1 R ec tifi er c irc uit 2-3 M ea su rem en t o f m ult i-m ete r (D C V ) 3-2 In tro du ctio n o f m ult im ete r (D C m A ) 3-1 O hm ’s L aw 1-1 M ea su rem en t o f m ult im ete r (O hm ) 2-2 F ilte r c irc uit
…
2-1 -1 D isti ng uis h A C an d D C sig na l 1-1 -1 D isti ng uis h e lec tro nic ele m en ts 1-1 -2 In tro du ctio n o f m ult im ete r 1-2 -1 R ea liz e A C sig na l 2-3 -1 R ea liz e vo lta ge div ide rru le 2-3 -2 In tro du ctio n o f m ult im ete r(D
C
V
)
1-2 -2 R ea liz e vo lta ge div ide rru le 1-2 -3 In tro du ctio n of m ult im ete r(ACV )Figure 2: Expert knowledge structure of industrial electronics meter operation and measurement.
The student knowledge structure was constructed according to the 215 students’ responses to the item bank via CARLS in accordance with the expert knowledge structure. The relation of the sequence in each level of the students’ knowledge structure was computed by the item structure algorithm. For example, the students’ structured knowledge was built up by the algorithm to diagnose their misconceptions and personalized the remedy paths. The reliability and validity of the item bank were analysed; meanwhile, the parameters of Item Response Theory (IRT) such as difficulty, discrimination, and estimation, were calculated for the reference of item selection and revision (Chang, 2009). The Cronbach α for the whole test was 0.772, which met the minimal requirement criteria (over 0.7) for internal consistency of test (DeVellis, 1991; Nunnally, 1978).
According to the expert knowledge structure, to achieve the learning objective of “1 Measurement of multi-meter (ACV)”, students have to complete the learning unit in the following sequence, starting with “1-1” and then “1-2”. Similarly, to accomplish the learning objective of “1-1”, students need to complete the sub-skill learning units in the following sequence, “1-1-1” and then “1-1-2”. Therefore, the sequence of test item in traditional assessment from the upper to lower concepts is, 1→1-1→1-2→1-1-1→1-1-2→1-2-1→1-2-2→1-2-3.
CARLS adopted the student knowledge structure to diagnose student’s misconception and personalized the
answer, the lower items affiliating with the upper one; if he/she makes an error on the upper item, he/she should have some misconceptions for the lower items affiliating with the upper one. Accordingly, the items and the sequence of item testing provided by CARLS were created. Figure 3 showed students A’s sequence of test items and remedial learning path calculated by CARLS according to the A’s responses. Student A had the misconception of “2-3-1”, which led to the upper node error. Therefore, student A’s remedial learning path should be 2-3-1→2-3→2.
2-1 R ec tifi er c irc u Ⴟ it 2-3 M ea su rem en t o f m ult i-m ete r (D C V ) X 2-1 -1 D isti ng uis h A C an d D C sig na l Ⴟ
2-3 -1 R ea liz e v olt ag e d ivi de r ru le � 2-2 F ilte r c irc uit Ⴟ 2-3 -2 In tro du ctio n o f m ult im ete r (DCV )X 2 Measurement of multimeter (DCV) XFigure 3: Student A’s remedial learning path.
The design concept of the proposed CARLS is listed as follows.
By challenging the students, starting with single-concept problems and gradually progressing to 1)
multi-concept problems, and by making repetitive connections between the different concepts, students can be trained to apply concepts learned in different places and times to solve problems in new contexts.
Compared with traditional paper and pencil test,
2) CARLS were designed to reduce the number of
test items and provide different test items according to individual differences in prior knowledge of students.
Test items of
3) CARLS were designed to focus on the interaction of CK and PK to enhance students’
Methodology of Research
Participants and the Experimental Setting
To evaluate the effectiveness of the proposed STEM course with CARLS support, a pre-experimental design of one-group pre-test/post-test was employed in this research due to the limitation of students’ enrollment in the proposed course. A total of 32 students (11th grade) from the Department of Electronics of a vocational high
school in central Taiwan voluntarily participated in this research. All students of the class completed all the steps and procedures in the school timetable, and the sessions were implemented in 10 weeks.
According to the one-group pre-test/post-test design (Campbell & Stanley, 1963), no control group or baseline was compared with and thus the fi ndings of the research should be carefully interpreted. However, Yang (2007) ad-thus the fi ndings of the research should be carefully interpreted. However, Yang (2007) ad-the fi ndings of the research should be carefully interpreted. However, Yang (2007) ad-e findings of the research should be carefully interpreted. However, Yang (2007) ad- research should be carefully interpreted. However, Yang (2007) ad-be carefully interpreted. However, Yang (2007) ad-. However, Yang (2007) ad- However, Yang (2007) ad-However, Yang (2007) ad-Yang (2007) ad- ad- ad-opted the pre-experimental design of one-group pre-test/ post-test to reveal the effect of promoting students’ critical thinking skills; Sak and Oz (2010) also adopted this design to investigate the students’ creative thinking activities.
Procedure and Treatment
Table 1 showed the experimental procedure. In the first two weeks, the teacher explained the objectives and went through the syllabus of the proposed course, as well as the fundamental of the electronics (CK) including the symbols of electrical component and basic equations. In the next two weeks, students were taught how to operate multimeters for measuring voltages and currents in electrical devices (PK). Figure 4 showed that students had to master the operation of the multimeter for diff erent purpose such as measuring DC or AC voltage, DC cur-d to master the operation of the multimeter for diff erent purpose such as measuring DC or AC voltage, DC cur- to master the operation of the multimeter for different purpose such as measuring DC or AC voltage, DC cur-rent, resistance, diode, etc. In week 5-6, the teacher taught the interaction of CK and PK for students how to design the circuit layout with the right electronic units and calculate the right data. In the first 6 weeks, students were required to finish the instruction material (films) in advance before the class and teacher help students to solve their difficulties in the class.
After the six-week sessions, to explore the students’ prior knowledge and skills, a pre-test exam was administered. Then, after introducing the functions of CARLS to students, the teacher acted as a facilitator, to help students to proceed with the remedial learning through CARLS. Finally, a post-test was conducted in the 10th week.
Table 1. The experimental procedure and school time table.
Procedure Description School time table (week)
Instruction 1 The teacher taught the CK 1-2
Instruction 2 The teacher taught the PK 3-4
Instruction 3 The teacher taught the interaction of CK and PK 5-6 Pre-test Students took a CK and PK test
7-10 Remedial learning Students proceeded the remedial learning through CARLS
Post-test Students took a CK and PK test through CARLS
Data Analysis
As shown in table 2, the research measured students’ knowledge, achievement, hands-on skill achievement and attitude by three evaluation tools, the tests for the interaction of CK and PK, the operational test of hands-on skill, and the technology acceptance attitude of students toward CARLS, respectively. The collected data were analysed by descriptive statistics, t-test, and ANOVA.
The pre-test/post-test for the interaction of CK and PK were selected from the Industrial electronics C Techni--test/post-test for the interaction of CK and PK were selected from the Industrial electronics C Techni-/post-test for the interaction of CK and PK were selected from the Industrial electronics C Techni--test for the interaction of CK and PK were selected from the Industrial electronics C Techni-test for the interaction of CK and PK were selected from the Industrial electronics C Techni-cian Skills Certification Test (Skill Evaluation Center, 2013) and the operational test of hands-on skill was selected from the “C test of the Industrial Electronics Meter Operation and Measurement Skill” (Skill Evaluation Center, 2013). The tests for the interaction of CK and PK as well as the operational test for hands-on skill were modified according to a strict procedure by 20 industrial electronics experts, so the expert validities are reasonably fair.
Table 2. Description of research measurement.
Objective of evaluation Evaluation instruments Description of the instruments Knowledge Industrial electronics C Technician Skills
Certifica-tion Test Examine the students’ interaction of CK and PK Skills C test of the Industrial Electronics Meter Operation
and Measurement Skill Evaluate the operation and measurement skills by teacher rating scores Attitude Technology Acceptance Model (TAM) toward
CARLS questionnaire and in-depth interview Examine students’ attitudes toward CARLS Results of Research
Three parts in research results are to be discussed concerning the evaluation of the proposed STEM course with CARLS support: (1) the effectiveness of the proposed course with CARLS support upon the enhancement of the knowledge, achievement; (2) the effectiveness of the proposed course with CARLS support upon the enhancement of the hands-on skill performance; and (3-1) the technology acceptance attitude of students toward CARLS; (3-2) the differences of technology acceptance attitude toward CARLS between the high score and the low score groups.
The Effectiveness of STEM Course with CARLS Support Upon the Enhancement of the Knowledge Achievement
Mean scores of the pre and post test as well as the result of t-test is shown in table 3. Mean scores of the pre-test and post-pre-test were 76.56, and 84.75, respectively. Result from t-pre-test as revealed in table 3, demonstrated that effectiveness of the proposed course with CARLS support upon the enhancement of the knowledge achievement is significant (t=7.96, p<.000). Furthermore, this research adopted, the effect size d to measure the strength of the proposed course with CARLS support upon the enhancement of the knowledge achievement (Cohen, 1988). In Cohen’s definition, 0.2<d<0.3 means a small effect size, around 0.5 means a medium effect size, and d>0.8 means a large effect size. In table 3, the Cohen’s d value of 1 indicates a large effect size, suggesting a great help from the proposed course with CARLS support.
Table 3. t-test of pre-test/post-test of students.
N Mean SD df t value p Cohen’s d
Pre-test 32 76.56 8.84
31 7.96 .0001*** 1
Post-test 32 84.75 7.78
A one-way ANOVA was conducted to compare the mean difference of the hands-on skill performance among three different groups, high, medium and low scores, which were assigned according to the scores of post-test. Table 4 illustrated that difference of means among high, medium and low scores groups was significant (F=4. 963, p<0.05). In addition, effect size (η2 = 0.255) was also examined and showed approximately large (η2 > 0.26) effect
size (Cohen, 1992). Nevertheless, pairwise comparison (Scheffé) revealed that there was no significant effectiveness between high and medium score groups (p=.994). The reason could be that students in high and medium score groups might already have achieved well interaction of CK and PK, so they have better hands-on skill performance after the proposed course. These results partially supported that the proposed course with CARLS support could assist in the enhancement of students’ hands-on skill performance.
Table 4. ANOVA for hands-on skill performance of the high, medium and low score groups.
Groups N Mean SD F p Post–Hoc (Scheffé) η2
High score 9 96 0
4.963 .014* 1>3; 2>3 0.255
Medium score 14 94.86 2.905
Low score 9 64 48
Technology Acceptance Attitude of Students toward CARLS
The mean score of Perceived Usefulness of CARLS on the Technology Acceptance Survey was 3.80, which is greater than the median of 3. Table 5 revealed that students regarded CARLS as a beneficial channel to enhance their learning. In particular, high mean scores of some items such as Q1 (mean=4.06), Q4 (mean=4.09), and Q7 (mean=4.06) demonstrated that students considered CARLS an efficient method to examine (or evaluate) their levels of understanding and a helpful skill acquisition system. On the other hand, items with low mean scores such as Q3 (mean=3.25) and Q5 (mean=3.41) implied lower effectiveness of CARLS in enhancing their learning motivation and presenting the detailed explanations of their misconceptions.
Table 5. Perceived usefulness of CARLS technology acceptance.
Dimension Item Content of the Questionnaire Item mean (SD)
Perceived Useful-ness
Q1 CARLS is helpful for my learning. 4.06 (.504)
Q2 CARLS enhanced my learning effectiveness. 3.81 (.592)
Q3 CARLS facilitated my learning motivation. 3.25 (.803)
Q4 CARLS detected my comprehension status of the interactions of CK and PK. 4.09 (.641)
Q5 CARLS provided detailed explanations of my misconceptions. 3.41 (1.073)
Q6 CARLS corrected my misconceptions. 3.91 (.893)
Q7 CARLS is positive for overall learning. 4.06 (.619)
Overall 3.80 (0.81)
As to Perceived Ease-of-Use of CARLS in the Technology Acceptance Survey, the mean of the total items was 3.69, greater than the median of 3. Table 7 showed that students regarded CARLS as a system easy to apply based on the response in Q1 (mean=4.06) and Q7 (mean=4.06). Contrarily, the lowest mean score of Q2 (mean=3.16) implied that students expressed less agreement upon the convenience of CARLS while compared with its paper-and-pencil counterpart. As for Q6 (mean=3.56), leaner’s responses demonstrated less agreement on CARLS as an easy task.
Table 7. Perceived ease-of-use of CARLS technology acceptance.
Dimension Item Content of the Questionnaire Item mean (SD)
Perceived Ease-of-Use
Q1 It is very easy for me to operate CARLS 4.06 (.878)
Q2 It is more convenient to use CARLS than using the traditional paper and pencil test. 3.16 (1.110)
Q3 CARLS is a very easy testing system to use. 3.78 (.870)
Q4 CARLS is quite clear and comprehensible to use. 3.63 (.751)
Q5 It is easy to use CARLS for remedial learning. 3.56 (.669)
Q6 It is very convenient to use CARLS for testing and remedial learning. 3.56 (.716)
Q7 It is efficient to use CARLS. 4.06 (.759)
Overall 3.69 (.87)
The Difference in Students’ Technology Acceptance Survey of CARLS between High and Low Score Groups
A t test was utilized to examine the differences in Technology Acceptance Survey of CARLS between high and low score groups. The result showed no significant difference in both Perceived Usefulness and Perceived Ease-of-Use of CARLS as illustrated in Table 8.
Table 8. t-test of perceived usefulness and ease-of-use of CARLS between two groups.
Item Group N M S.D. t value p
Perceived Usefulness High score 9 3.94 .417 .846 .410
Low score 9 3.76 .457
Perceived Ease-of-Use High score 9 3.79 .429 .852 .407
Low score 9 3.59 .587
Discussion
This research employed a STEM course with CARLS support for vocational high school students in Taiwan to enhance their knowledge achievement, and hands-on skill performance. After statistical analyses, including single group t test and ANOVA, results indicated that CARLS is effective and contributed in enhancing students’ knowledge achievement and hands-on skill performance. Moreover, while examining participants’ attitude toward CARLS by using the Technology Acceptance Survey, the current research found out that over 50% of the students considered
CARLS easy to operate and useful in clarifying their misconceptions. Students who learned through the proposed
STEM course with CARLS support generally had a positive learning experience, perceived their use of CARLS as satisfying, useful and easy and also expressed a continuing intention of using CARLS. Some advantages conveyed by the participants through interviewing the teacher and students after using CARLS are listed as below.
(1) CARLS could benefit educator in the following ways.
(a) CARLS helped the educator comprehend the individual differences of students through the learning portfolios.
(b) CARLS assisted in evaluating whether students have achieved the mastery level of the interaction of CK and PK as well as locating what their learning difficulties are in skill performance through students’ learning portfolios.
(c) CARLS had the merit of time-saving in educator test preparation so that he/she may pay more attention to individual learning progress as well as to provide extra training materials for unprepared students. (2) CARLS could benefit students in the following ways.
in implementing hands-on practice; whereas it guided low mastery level students to do their individual remedial education without pressure.
(c) It promoted students’ self-efficacy along their cognitive stage to consolidate skill acquisition.
In terms of the Technology Acceptance of CARLS, over 50% of the students previewed CARLS easy to operate and useful in clarifying their misconceptions. Nevertheless, only a small part of the participants (25%) indicated that CARLS escalated their motivation to utilize it. Hence, some improvements on CARLS are suggested based on rest (75%) of the students.
(1) Facilitating student motivation
Researchers pointed out the learning effectiveness may be affected by student motivation (Hamjah, Ismail, Rasit & Rozali, 2011; Lim, 2004). Since the design of CARLS took its root in the diagnostic and remedial learning system, which is often labelled as “the system designed for the low academic achievement learner”, it probably eliminated students’ motivation. Therefore, futures studies might focus on developing CARLS that facilitate the students’ motivation to encourage a willingness to learn.
(2) Providing adaptive remedial materials for low achieving students
Students’ interview indicated that CARLS failed to effectively provide satisfying explanations of misconcep-tions, especially for low achieving students. Those students frequently experienced learning frustramisconcep-tions, which resulted in their low motivation. However, besides remedial materials, they probably needed more assistance and guidance to adjust their learning strategies to self-regulation. Therefore, CARLS should be modified to provide appropriate teaching strategies and more adaptive materials for them.
Regarding to evaluation of Perceived Usefulness and Perceived Ease-of-Use of CARLS, the results indicated no significant difference between the high and low score groups. This research inspiringly revealed that students per-research inspiringly revealed that students per-inspiringly revealed that students per-formed CARLS equally well on the knowledge, achievement to apply hands-on skill on problem-solving regardless of what group they were in. The results mirrored with findings from previous research comparing the effectiveness of online and paper-and-pencil tests in learning performance (Anakwe, 2008; Bugbee Jr, 1996; Clariana & Wallace, 2002; Zandvliet & Farragher, 1997). However, slightly over one third of the students (37%) remained their custom in using paper and pencil tests.
Conclusions
This research proposes a STEM course with CARLS support for vocational high school students in Taiwan to enhance their knowledge achievement, and hands-on skill performance. The results of the research revealed that: (1) students with better understanding of how CK and PK interact will benefit in clarifying their misconceptions, which probably improve their knowledge achievement on CK and/or PK further; (2) only after successfully devel- only after successfully devel-nly after successfully devel-oping an understanding of, CK, PK, and how CK and PK interact was the students able to acquire hands-on skill in the proposed STEM course with CARLS support; (3) The reasons causing learning difficulties of low achievement students are complicated and require teacher’s counseling and help individually. This research has provided pre-research has provided pre-has provided pre-liminary empirical evidence to support the effectiveness of a STEM course with CARLS support in terms of facilitat-ing students’ understandfacilitat-ing of interaction of CK and PK to acquire hands-on skill for problem-solvfacilitat-ing in the real world. Furthermore, the proposed STEM course with CARLS support does not only help the educator comprehend the individual differences and misconceptions of the students through their learning portfolios, but also saves the educator’s time for paying more attention to the individual’s learning difficulties as well as to providing extra training materials for unprepared students.
Accordingly, some suggestions are made based on the findings of the current research:
(1) The high score group used CARLS more frequently than its low score counterpart; whereas additional cognitive labour might be present for the low score group students since they had to concentrate not only on knowledge acquisition but also on mastery themselves of the operation of CARLS. To avoid cognitive overload of the low score group, this research recommends that low score students better focus on CK acquisition first, and then proceed to CARLS once they managed to learn CK.
(2) More scaffolding examples for conception clarification, e.g. Dual Situated Learning Model (DSLM) (She, 2002; 2003; 2004a, b) should be added to CARLS for knowledge consolidation instead of knowledge over-generalization. Furthermore, the practicing example, in CARLS should be embedded and scheduled into course design in the first place to help students achieve hands-on skill training.
Some limitations of this research should be presented. In this research, a pre-experimental design of one-group pre-test/post-test was conducted, without baseline or comparison with the control group, and thus the research findings should be carefully interpreted (Campbell & Stanley, 1963). Future studies might address the following issues. First, the findings and implications are obtained from merely one particular learning system; moreover, representativeness of the studied sample is limited since the sample size was not big. Second, individual diff er- not big. Second, individual diff er-not big. Second, individual differ-ence variables such as personality and learning style, could be incorporated in future studies. Third, the students in Taiwan have unique value and behavioural patterns, such as they had diverse definition of learning achievement and remedial learning.
Acknowledgement
This research is partially supported by the National Science Council (Ministry of Science and Technology), Taiwan, R.O.C., under two three-year project grants NSC 98-2511-S-018 -006-MY3 and NSC 101-2511-S-018-013-MY3.
Reference
Anakwe, B. (2008). Comparison of student performance in paper-based versus computer based testing. Journal of Education for
Business, 84 (1), 13-17.
Appleby, J., Samules, P., & Treasure-Jones, T. (1997). Diagnosis: A knowledge-based diagnostic test of basic mathematical skills.
Computers & Education, 28, 113-131.
Barlex, D. (2007). Assessing capability in design and technology: The case for a minimally invasive approach. Design and
Technol-ogy Education: An International Journal, 12 (2), 49-56.
Broks, A. (2011). Philosophical and psychological basis of general science education. Retrieved from University of Latvia website: http://blogi.lu.lv/broks/files/2010/01/A.Broks-REPORT_IOSTE-2.12.11.pdf.
Broks, A. (2014). Scientific thinking - the backbone of modern science and technology education. Journal of Baltic Science
Educa-tion, 13 (6), 764-766.
Brown, J. S., & Burton, R. R. (1978). Diagnostic models for procedural bugs in basic mathematical skills. Cognitive Science, 1, 155-191.
Bugbee Jr., A. C. (1996). The equivalence of paper-and-pencil and computer based Testing. Journal of Research on Computing in
Education, 28 (3), 282-299.
Campbell, D., & Stanley, J. (1963). Experimental and quasi-experimental designs for research. Boston: Houghton Mifflin.
Case, R., & McKeough, A. (1990). Schooling and the development of central conceptual knowledge structure: An example from the domain of children’s narrative. International Journal of Educational Psychology, 13, 835-856.
Chang, K. E., Liu, S. H., & Chen, S. W. (1998). A testing system for diagnosing misconceptions in DC electric circuits. Computers &
Education, 31, 195-210.
Chang, S. H, Chen, M. L., Kuo, Y. K., & Shen, Y. C. (2011). A simulation-based LED device design project in photonics instruction based on industry-university collaboration. IEEE Transactions on Education, 54 (4), 582-589.
Chang, S. H. (2009). Construction and evaluation of the adaptive learning system for problem-solving skills: A case study of electronic
circuit course in vocational high school. National Science council research project report (NSC 98-2511-S-018 -006 -MY3).
Chang, S. H., Wu, T. C., Kuo, Y. K., & You, L. C. (2012). Project-based learning with an online peer assessment system in a Photonics instruction for enhancing LED design skills. The Turkish Online Journal of Educational Technology, 11 (4), 236-246.
Chang, S. H., Yu, L. C., Kuo, Y. K., Mai, Y. T., Chen, J. D. (2015). Applying online peer assessment with total quality management to elevate project-based learning performance. Journal of Baltic Science Education, 14 (3), 379-390.
Chen, L. H. (2011). Enhancement of student learning performance using personalized diagnosis and remedial learning sys-tem. Computers & Education, 56 (1), 289-299.
Christie, K., Bushaw, W. J., Gallup, A. M., Finn, C. E., Darling-Hammond, L., Tucker, M., ... & Merrow, J. (2008). Middle and high schoolers get hands-on STEM experiences. Phi Delta Kappan, 5-19.
Clariana, R., & Wallace, P. (2002). Paper-based versus computer based assessment: Key factors associated with the test mode effect. British Journal of Educational Technology, 33 (5), 593-602.
Clough, M. P., & Kauffman, K. J. (1999). Improving Engineering Education: A Research-Based Framework For Teaching. Journal of
Engineering Education, 88 (4), 527-534.
Cohen, J. (1988). Statistical power analysis for the behavioural sciences (2nd Ed.). Hillsdale, NJ: Lawrence Earlbaum Associates. Cohen, J. (1992). A power primer. Psychological Bulletin, 112 (1), 155-159.
DeVellis, R. F. (1991). Scale development: Theory and applications. Sage publications.
Dufresne, R. J., Gerace, W. J., & Leonard, W. J. (1997). Solving physics problems with multiple representations. Physics Teacher, 35, 270-275.
Flick, L. B. (1993). The meanings of hands-on science. Journal of Science Teacher Education, 4 (1), 1-8.
Gagne, E. D., Yekovich, C. W., & Yekovich, F. R. (1993). The cognitive psychology of school learning (2nd Ed). New York: HarperCollins College Publishers.
Hamjah, H. S., Ismail, Z., Rasit, R. M., & Rozali, E. A. (2011). Methods of increasing learning motivation among students. Procedia
Social and Behavioral Sciences, 18, 138-147.
Hsiao, H. S., Chang, C. S., Lin, C. Y., Chen, B., Wu, C. H., & Lin, C. Y. (in press). The development and evaluation of listening and speaking diagnosis and remedial teaching system. British Journal of Educational Technology. DOI: 10.1111/bjet.12237.
Hwang, G. J., Panjaburee, P., Triampo, W., & Shih, B. Y. (2013). A group decision approach to developing concept–effect models for diagnosing student learning problems in mathematics. British Journal of Educational Technology, 44 (3), 453-468.
Johnson, S. (1997). Learning technological concepts and developing intellectual skills. International Journal of Technology and
Design Education, 7, 161-180.
Klahr, D., Triona, L. M., & Williams, C. (2007). Hands on what? The relative effectiveness of physical versus virtual materials in an engineering design project by middle school children. Journal of Research in Science Teaching, 44 (1), 183-203.
Lazarinis, F., Green, S., & Pearson, E. (2010). Creating personalized assessments based on learner knowledge and objectives in a hypermedia Web testing application. Computers & Education, 55 (4), 1732-1743.
Lim, D. H. (2004). Cross cultural differences in online leaning motivation. Educational Media International, 41 (2), 163-173. Lin, S. S. J., Liu, E. Z. F., & Yuan, S. M. (2001). Web-based peer assessment: Feedback for students with various thinking-styles.
Journal of Computer-Assisted Learning, 17 (4), 420-432.
Liu, E. Z. F., Lin, S. S. J., Chiu, C. H., & Yuan, S. M. (2001). Web-based peer review: An effective web-learning strategy with the learner as both adapter and reviewer. IEEE Transactions on Education, 44 (3), 246-251.
Lou, S. J., Shih, R. C., Diez, C. R. & Tseng, K. H. (2011). The impact of problem-based learning strategies on STEM knowledge inte-gration and attitudes: an exploratory study among female Taiwanese senior high school students. International Journal of
Technology and Design Education, 21 (2), 195-215.
Luchins, A. S. (1942). Mechanization in problem solving: The effect of Einstellung. Psychological Monographs, 54 (6), i-95. Lutgens, G., & Mulder, M. (2002). Bridging the gap between theory and practice in Dutch vocational education. Vocational
Train-ing European Journal, 25, 34-38.
McCormick, R. (1997). Conceptual and procedural knowledge. The International Journal of Technology and Design Education, 7, 141-159.
McLaren, S. V. (2007). An international overview of assessment issues in technology education: Disentangling the influences, confusion and complexities. Design and Technology Education: An International Journal, 12 (2), 10-15.
Means, B., & Gott, S. P. (1988). Cognitive task analysis as a basis for tutor development: Articulating abstract knowledge repre-sentations. In J. Psotka, LD Massey, & SA Mutter (Eds.), Intelligent tutoring systems: Lessons learned (pp. 59-83). Hillsdale, NJ: Lawrence Erlbaum Associates.
Ministry of Education (2003). Curricula guidelines for grade 1-9. Retrieved August 17, 2014, from http://teach.eje.edu. tw/9CC2/9cc_92.php.
Ministry of Education (2006). The curricula temporary principals of division groups of vocational school. Retrieved August 17, 2014, from http://language.tncvs.tn.edu.tw/newnet/3/3-1.pdf.
Ministry of Education (2015). Department of Technological and Vocational Education. Retrieved August 1, 2015, from http:// english.moe.gov.tw/content.asp?CuItem=15156.
Nunnally, J. C. (1978). Psychometric Theory (2nd Ed.) NJ: McGraw-Hill.
Russell, S. H., Hancock, M. P., & McCullough, J. (2007). Benefits of undergraduate research experiences. Science (Washington),
316 (5824), 548-549.
Sak, U., & Oz, O. (2010). The effectiveness of the Creative Reversal Act (CREACT) on students’ creative thinking. Thinking Skills
and Creativity, 5 (1), 33-39.
She, H. C. (2002). Concepts of higher hierarchical level required more dual situational learning events for conceptual change: A study of students’ conceptual changes on air pressure and buoyancy. International Journal of Science Education, 24 (9), 981-996.
She, H. C. (2003). DSLM instructional approach to conceptual change involving thermal expansion. Research in Science and
Technological Education, 21 (1), 43-54.
She, H. C. (2004a). Facilitating changes in ninth grade students’ understanding of dissolution and diffusion through DSLM in-struction. Research in Science Education, 34 (4), 503-526.
She, H. C. (2004b). Fostering “Radical” conceptual change through Dual Situated Learning Model. Journal of Research in Science
Teaching, 41 (2), 142-164.
Shieh, R. S., & Chang, W. (2014). Fostering student’s creative and problem-solving skills through a hands-on activity. Journal of
Baltic Science Education, 13 (5), 650-661.
Shih, S. C., Kuo, B. C., & Liu, Y. L. (2012). Adaptively ubiquitous learning in campus math path. Educational Technology & Society, 15 (2), 298-308.
Sidawi, M. M. (2009). Teaching science through designing technology. International Journal of Technology and Design Education,
19 (3), 269-287.
Skill Evaluation Center (2013). Industrial electronics C Technician Skills Certification Test. Retrieved August 1, 2014, from http:// www.ytvs.tnc.edu.tw/my1/TeachingPlatform/ud51/02/028003-1.pdf.
Tseng, K. H., Chang, C. C., Lou, S. J., & Chen, W. P. (2013). Attitudes towards science, technology, engineering and mathematics (STEM) in a project-based learning (PjBL) environment. International Journal of Technology and Design Education, 23 (1), 87-102.
Unal, S. (2008). Changing students’ misconceptions of floating and sinking using hands-on activities. Journal of Baltic Science Education, 7 (3), 134-146.
VanLehn, K. (1988). Student modeling. In M. C. Polson and J. J. Richardson (Eds.), Intelligent tutoring systems. New Jersey: Lawrence Erlbaum Associates Publishers.
Wenger, E. (1987). Artificial intelligence and tutoring systems: Computational and cognitive approaches to the communication of
knowledge. California: Morgan Kaufmann Publishers.
Wu, H. M., Kuo, B. C., & Yang, J. M. (2012). Evaluating Knowledge Structure-based Adaptive Testing Algorithms and System De-velopment. Educational Technology & Society, 15 (2), 73-88.
Yang, S. C. (2007). E-critical/thematic doing history project: Integrating the critical thinking approach with computer-mediated history learning. Computers in Human Behavior, 23 (5), 2095-2112.
Zandvliet, D., & Farragher, P. (1997). A comparison of computer-administered and written tests. Journal of Research in Computing
Education, 29, 423-439.
Received: August 07, 2015 Accepted: October 16, 2015
Shu-Hsuan Chang Ph. D., Professor, National Changhua University of Education, Department
of Industrial Education and Technology, Taiwan. E-mail: shc@cc.ncue.edu.tw
Ai-Chiao Ku Ph. D. Candidate, National Changhua University of Education,
Department of Industrial Education and Technology E-mail: g1h2c3@gmail.com
Li-Chih Yu Ph. D. Candidate, National Chengchi University, Department of
Management Information Systems E-mail: 100356502@nccu.edu.tw
Tsung-Chih Wu Ph. D., Professor, National Changhua University of Education, Department
of Industrial Education and Technology, Taiwan. E-mail: tcwu@cc.ncue.edu.tw
![TraditionalMLCalgorithmsmainlytacklethebatchMLCproblem,wheretheinputdataarepresentedinabatch[24,28].Nevertheless,inmanyMLCapplicationssuchase-mailcategorization[22],multi-labelexamplesarriveasastream.Onlineanalysisistherefore dimensionreducermotivatedbyma](data:image/gif;base64,R0lGODlhAQABAIAAAP///wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==)