The research primarily aims to adopt two stage method to do the cluster analysis on students’ IMPROVE pre-tests. In the first phase, Ward’s least variation in the ordering cluster analysis is adopted to determine the suitable numbers of group dividing. Through the coefficient resulted from population- concentrated, we figure out that when the divided groups
The numbers are the average of factor scores(standardized values) *P<0.05 **P<0.01 ***P<0.001
number three, it is most reasonable. Thus, this research distributes the level of self-awareness into three groups. In the second phase, we have adopted K-means method, which is more commonly used in non-ordering sub-cluster analysis, to discuss the efficacy of the variations.
In order to judge the effect of cluster analysis and confirm the grades of the three self-awareness groups, we use three more clusters as the independent variables to respectively undertake MANOVA, under the four aspects of IMPROVE .(Table.1)
After Duncan’s multiple test, we’ve found that the four aspects in three clusters are all distinctive. Among them, cluster one gets the highest average value in all aspects, which means cluster one behaves the best in all aspects among the three. We therefore name cluster one as the highly-aware group; on the contrary, cluster two gets the lowest average value, thus is named the low-aware group. Cluster three gets the neutral average value, and becomes the medium-aware group.
Table1：cluster analysis on levels of self-awareness
MANOVA
comprehension 1.136 -1.024 0.076 61.169 .000＊＊＊ （1,3,2）
Connection 1.360 -1.047 0.038 94.183 .000＊＊＊ （1,3,2）
Strategic 1.164 -1.092 0.149 75.152 .000＊＊＊ （1,3,2）
Reflection 1.265 -1.027 0.034 74.024 .000＊＊＊ （1,3,2）
name of cluster highly-aw
are low-aware medium-a ware
*p＜.001
1. The influence of peer-feedback’s quality on self-awareness ability
The research takes the artificially assessed qualitative to analyze the relationship between feedback quality and self-awareness. There will be three specialists assessing the quality of peer-judgment and peer-feedback, then classifying all into high-quality feedback and poor-quality feedback. Before the analysis, we took a consistent inspection on the three specialists’ assessment result. Adopting Kappa analysis, we can observe that all specialists possess significance(Table2). Thus, we know that these three specialists’ grading possess consistency, so we can pick out the majority of the same grades to do analysis.
Table2：Results from Kappa analyses of the three experts’ evaluations
Kappa significance
Expert1×Expert 2 .641 .000***
Expert 2×Expert3 .827 .000***
Expert 3×Expert1 .403 .000***
.
After obtaining specialists’ consistent gradings, we do the analysis according to the assessed quality. First of all, t-test is to compare the differences of the feedback quality between IMPROVE pre-test and post-test. From Table.3 we may clearly observe that in high-quality feedback, there is distinctive unlikeness between IMPROVE pre-test and post-test. Of the four aspects, post-test gets higher average grades than pre-test. Thus in this research, we may declare that, obtaining high-quality feedback can help boost the self-awareness ability. As for Table.4, the poor-quality part, only the aspect “Reflection” is distinctive among the four. What’s more, in the “Comprehension” aspect, the average grades of post-test are lower than those of pre-test. Thus we may also say that, getting poor-quality
*p<.05 **p<.01 ***p<.001
*p<.05 **p<.01 ***p<.001
feedback doesn’t do much to boost one’s self-awareness.
Table3：IMPROVE pre-test and post-test on receiving high-quality feedback
high-quality feedback IMPROVE
pre-test
IMPROVE post-test Test items
M SD M SD
t
Comprehension 3.108 0.567 3.415 0.517 -3.327***
Connection 3.101 0.622 3.318 0.605 -3.024***
Strategic 3.087 0.611 3.232 0.574 -2.315**
Reflection 3.297 0.501 3.506 0.528 -3.012***
Table4：IMPROVE pre-test and post-test on receiving poor-quality feedback
poor-quality feedback IMPROVE
pre-test
IMPROVE post-test
M SD M SD t
Comprehension 3.261 0.567 3.015 0.517 0.076 Connection 3.015 0.328 3.019 0.705 -1.375
Strategic 3.608 0.462 3.621 0.771 -1.464
Reflection 3.263 0.511 3.461 0.534 - 2.315**
Besides, according to the result of cluster analysis, we calculate the numbers of people on the judgment and feedback quality that they gave and received. By this we discovered that what the high-aware people give or receive are mostly of high quality. Therefore we may conclude that the highly-aware usually give their peers high-quality feedback, and vice versa.
The low-aware people only get a bit higher quality on receiving peer judgment. As for giving peer judgment and receiving peer feedback, they are of poorer quality. Therefore we may conclude that the low-aware usually give their peers low-quality feedback, and vice versa. As to the medium-aware group, they get more high-quality than poor-quality, but not much higher( Figure.3).
3. How the level of valuing peer-feedback influences the level of self-awareness
The research takes the artificially assessed qualitative to analyze the relationship between the level of valuing peer-feedback and self-awareness. There will be three specialists assessing the quality of peer-feedback survey, then classifying all into “value” and “unvalued”.
Before the analysis, we took a consistent inspection on the three specialists’ assessment result.
Adopting Kappa analysis, we can observe that all specialists possess significance(Table.5).
Thus, we know that these three specialists’ grading possess consistency, so we can pick out
Figure3.statistics of people’s number on different levels of feedback quality and self-awareness
*p<.05 **p<.01 ***p<.001
the majority of the same grades to do analysis.
First of all, t-test is to compare the differences of the level of valuing peer-feedback between IMPROVE pre-test and post-test. From table five we may clearly observe that in the
“value” group, there is distinctive unlikeness between IMPROVE pre-test and post-test. Of the four aspects, post-test gets higher average grades than pre-test. Thus in this research, we may declare that, valuing peer-feedback can help elevate the self-awareness ability. As for table six, the “unvalue” part, all the four aspects in IMPROVE didn’t reach significance, both on pre-tests and post-tests. Thus we may say that, unvaluing peer-feedback doesn’t do much to boost one’s self-awareness.
Table5：IMPROVE pre-test and post-test on value feedback
value feedback IMPROVE
pre-test
IMPROVE post-test
M SD M SD
t
Comprehension 3.432 0.496 3.803 0.503 -2.787***
Connection 3.213 0.487 3.406 0.509 -2.625***
Strategic 3.359 0.623 3.662 0.582 -3.215**
Reflection 3.176 0.548 3.363 0.563 -3.059***
*p<.05 **p<.01 ***p<.001
Table6：IMPROVE pre-test and post-test on unvalue feedback
unvalue feedback IMPROVE
pre-test
IMPROVE post-test
M SD M SD t
Comprehension 3.202 0.357 3.215 0.507 -1.548 Connection 3.215 0.486 3.087 0.853 -1.473
Strategic 3.503 0.362 3.517 0.687 -1.589
Reflection 3.532 0.332 3.541 0.734 - 1.342
According to the result of cluster analysis, we research on the relationship between degree of valuing feedback and self-awareness. We discovered that among the highly-aware people, there are 72.38% of people valuing their peer-feedback, which is much more than the unvaluing 27.62%. On the contrary, only 34.73% of the low-aware people value peer-feedback, much fewer than the unvaluing 65.27%(Figure.4). So we may infer that most highly-aware people do value the received peer-feedback, while the low-aware people mostly unvalue them. The medium-aware group diverges more averagely on “value” and “unvalue”.
*：p<.05 **：p<.01 ***：p<.001
Figure.4 degree of valuing the feedback
53.16%
3.Concept map modeling and inter-assessment can boost self-awareness.
We use t-test to analyze the IMPROVE tables to compare each student’s pre-test and post-test. From the table below we can observe that the students’ scores on comprehension, connection, strategic, and reflection all reach significance. Also, the averages in the post-tests are all higher than those in the pre-tests. So we may say in this research that, receiving feedback through concept map modeling and inter-assessment can truly boost the ability of self-awareness, especially improve the aspect of reflection(Table.7).
Table.7： Results from statistical tests on scores on pre-test and post-test questionnaires
IMPROVE Pre-test IMPROVE Post-test
Test items M SD M SD
t
comprehension 3.108 0.567 3.215 0.517 -2.925**
connection 3.010 0.622 3.178 0.605 -5.004***
strategic 3.487 0.611 3.632 0.574 -3.324***
reflection 3.397 0.501 3.506 0.528 -4.278***
.
Table8:statistics of people’s number on the first choice of interconnection on level of awareness
Table9: statistics of people’s numberon the second choice on level of awareness
Finally, we focus on interconnection and the survey of systematic users to do the analysis. According to the first interconnection result, the highly-aware people’s choices are often highly-aware people, too, while the low-aware people do not incline to any group of people as their choices, and the medium-aware people’s choices are mostly of higher or medium awareness (Table.8). Furthermore, the two choices of highly-aware and medium-aware group are usually the same, while the low-aware group do not necessarily choose the same people(Table.9). As for the analysis on systematic users survey, 76.23 of people think it helpful to fill the concept deficiency and integrate these concepts through concept map systematic modeling and interconnection , and 23.17% unhelpful (Figure.5).
the first choice level of awareness
highly-aware low-aware medium-aware
highly-aware 15 4 6
low-aware 3 9 4
medium-aware 10 9 12
the second choice level of awareness
the same as the first choice different from the first choice
highly-aware 21 4
low-aware 5 11
medium-aware 23 8
Figuer.5 systematic users suver percentage of people
helpful, 76.23%
unhelpful, 23.17%
helpful unhelpful