• 沒有找到結果。

Results and discussion of improving conceptual self-awareness

Chapter 4. Breaking concept boundaries to enhance creative potential

4.8. Results and discussion of improving conceptual self-awareness

1. Level of conceptual self-awareness (student/expert score) = student’s self assessment – expert’s assessment

2. Change in level of conceptual self-awareness = student/expertfirst – student/expertsecond

4.8. Results and discussion of improving conceptual self-awareness

4.8.1. Does the ICMSys promote conceptual self-awareness?

As shown in Table 13, the first student/expert score (M = 5.84, SD = 3.61) represents the level of conceptual self-awareness for the first concept map and the second (M = 4.38, SD

= 2.96) represents the level for the revised map. Results from a paired t-test using the two scores indicate a statistically significant improvement in conceptual self-awareness (t = 2.31, p < 0.05), suggesting that the students were more capable of assessing their map quality without overestimation. Results from paired-sample t-tests for measuring improvement in conceptual self-awareness in specific concept map criteria are presented in Table 14. They indicate statistically significant improvements in examples (t = 2.52, p < 0.05) and

relationships (t = 2.18, p < 0.05) but not in hierarchies (t = 1.05, ns) or cross-links (t = 1.67,

ns). A possible explanation is that the students found it easy to identify differences in the first two areas using the ICMSys, but the above-mentioned hierarchy issue made it more difficult for students to find differences in the hierarchy criterion. These results find support in Novak and Gowin’s (1984) observation that students find it difficult to construct and understand the real meaning of cross-links.

Table 13. Statistics for the student, expert, and student/expert conceptual structure scores.

First map Revised map t Significance Assessment

source M SD M SD

Student 28.72 3.26 29.69 3.11

Expert 22.88 4.65 25.31 4.90

Student/expert 5.84 3.61 4.38 2.96 2.31 p < 0.05

Table 14. Improvement in conceptual self-awareness in terms of the four criteria.

Student/expert score

First map Revised map t Significance Criterion

M SD M SD

Examples 1.84 1.42 1.18 1.03 2.52 p < 0.05 Relationships 1.87 1.64 1.37 0.97 2.18 p < 0.05

Hierarchies 1.53 0.80 1.31 0.98 1.05 ns

Cross-links 0.60 0.53 0.52 0.31 1.67 ns

4.8.2. Does the ICMSys help learners make positive conceptual changes in their revised maps?

At issue here is the possibility that students could make negative conceptual changes even though their conceptual self-awareness had improved. To address this question, the experts examined the revised maps in terms of quality. A Kendall’s coefficient of

concordance was performed to measure inter-rater reliability. Agreement rates for both original and revised maps were statistically significant (W = 0.82, p < 0.01 and W = 0.73, p <

0.01, respectively). I therefore combined and averaged the ratings to provide a composite expert assessment figure for each concept map; t-tests were used to determine improvement in the quality of student concept maps as judged by the three experts as well as improvements in specific criteria. As shown in Table 15, the students made statistically significant

improvements in examples (t = 3.22, p < 0.01), relationships (t = 2.35, p < 0.05), and cross-links (t = 2.10, p < 0.05). In other words, they regularly assimilated propositions, cross-links, or new concepts that they found to be meaningful into their conceptual structures with a few changes in existing hierarchies. This suggests that the study participants made significant and positive conceptual changes by breaking conceptual boundaries while their conceptual self-awareness levels improved.

Even though the increase in the hierarchy scale was not statistically significant, increased scores were observed (from M = 6.64, SD = 1.43 to M = 6.95, SD = 1.50) (Table 15). This suggests that the participants made the necessary adjustments to concept hierarchies to better organize their ideas whenever they found major mistakes in their concept maps or irreconcilable differences between their maps and those of other students. One possible explanation for their limited improvement in the hierarchy scale may be the nature of the concept mapping technique—that is, more general concepts are situated in higher map positions and more specific concepts in lower positions. Some students adhered to this model while others did not, causing inconsistency in their hierarchy presentations. To encourage greater flexibility in hierarchy integration, the ICMSys allows students to manually adjust ICMap hierarchies.

Table 15. Concept map quality as assessed by experts in terms of the four criteria.

Experts (average from three)

First map Revised map t Significance Criterion

M SD M SD

Examples 5.50 1.40 6.41 1.63 3.22 p < 0.01 Relationships 9.53 2.05 10.48 2.20 2.35 p < 0.05 Hierarchies 6.64 1.43 6.95 1.50 1.47 ns Cross-links 1.23 0.62 1.45 0.76 2.10 p < 0.05

4.8.3. Does ICMap viewing frequency affect conceptual self-awareness level?

According to the three experts, the participating students tended to select complete concept maps with lots of examples during the viewing process, perhaps because they felt they could make more worthwhile extensions and revisions based on those maps.

The participants were divided into two groups of 16 students each according to ICMap viewing frequency (group 1 = high and group 2 = low). The t-test results shown in Table 16 indicate a statistically significant difference between the first (M = 5.31, SD = 3.07) and second (M = 3.13, SD = 2.45) student/expert scores for group 1 (t = 2.95, p < 0.05) but not for group 2, meaning that group 1 students made a larger contribution to the overall improvement in conceptual self-awareness. The Table 16 data also indicate a significantly smaller (t = -2.52, p < 0.05) student/expert score for revised maps among group 1 students (M

= 3.13, SD = 2.45) compared to group 2 students (M = 5.63, SD = 3.12), suggesting that group 1 students had better conceptual self-awareness than group 2 students, as reflected in the revised concept maps.

Table 16. Data for Integrated Concept Map (ICMap) viewing frequency. Group 1 = high, Group 2 = low.

Group 1 (N = 16) Group 2 (N = 16) t Significance Student/expert score

M SD M SD

First map 5.31 3.07 6.38 4.11 -0.83 ns

Revised map 3.13 2.45 5.63 3.12 -2.52 p < 0.05

t 2.95 0.70

Significance p<0.05 ns

4.8.4. Is there a correlation between conceptual self-awareness level in the revised map and conceptual improvements?

A significant Pearson correlation was found between level of student conceptual self-awareness in revised concept maps and actual conceptual changes as measured by the three experts (r = 0.38, p < 0.05). Specifically, the students did not overestimate or

underestimate their concept maps after viewing many of their peers’ maps. They used other maps as models, located their concept boundaries, understood the relative quality of their own concept maps, and were more self-aware of those boundaries when revising their maps.

Furthermore, the students’ concept maps significantly improved in terms of overall quality.

Again, a possible explanation is that the social comparison process helped students learn previously unknown concepts and incorporate them into their revised maps.

4.8.5. ICMSys questionnaire responses

Data on student perceptions of the ICMSys are shown in Table 11. In the “practicality for comprehension” category, the responses indicate that the majority of students found the ICMSys to be a convenient method for helping them observe (item 1, 78%) and comprehend

(item 2, 91%) major concepts and to understand the target material (item 3, 63%). This suggests that the students’ ideas are not only externalized, but can also be selectively accommodated in representations considered practical for concept comprehension. Under

“capability for conceptual awareness,” the majority of students found the ICMSys to be helpful in terms of comparing their maps with their peers’ maps (item 4, 84%), and therefore helpful in terms of finding concept boundaries (item 5, 88%) and adding extensions or making revisions to their own maps (item 7, 91%). These responses suggest that the ICMSys can assist students in conceptual reflective thinking, as well as in identifying and perhaps breaking through their existing concept boundaries.

Only 25% agreed that the ICMSys helped them find conceptual faults (item 6). The students admitted their limitations in presenting thorough/comprehensive concept maps, yet they asserted that the ideas they presented in their maps were almost correct. A possible explanation is that the students could not recognize their faults; this can be addressed by including expert concept maps as comparison sources or asking teachers to help correct misconceptions in the revised maps. Next, 72% felt that the ICMSys interface was easy to use (item 8), but only 47% stated an interest in using similar systems in the future (item 9). The vast majority of participants made changes to their original concept maps (item 10, 94%).

When asked to identify factors that encouraged them to make revisions, they replied (a) some extensions could be added to make their concept maps more complete and thorough, (b) some previously unknown concepts were essential for inclusion in their revisions, or (c) their concept maps were inferior to their peers’.