A Collaborative Reading Annotation System with Formative Assessment and Feedback Mechanisms to Promote Digital Reading Performance
24
0
0
全文
(2) . 1. Introduction With the rapid development of ICT in recent years, digital reading has been gradually emphasized in the digital age as well as a lot of digital reading tools have successfully been developed to more effectively support digital reading (Chen & Chen, 2014; Chang, Chen, & Lin, 2018). Writing down reading annotations on a printed material is a traditional, effective, and common reading strategy used in printed material reading. However, compared to making reading annotations on a printed material, making reading annotations by means of a collaborative reading annotation system on a digital material can more easily store, share and discuss the contents of the reading annotations each other. Many previous studies have paid attention to the development of CRAS, such as CoCoAJ system (Communicative Collection Assisting System)(Ogata, Hada, & Yano, 2000), Annotea annotation system (Kahan, Koivunen, Prud'Hommeaux, & Swick, 2002), EDUCOSM collaborative reading annotation system (Nokelainen, Miettinen, Kurhila, Floréen, &. 政 治 大 2006), and AnnotatEd system (Farzan & Brusilovsky, 2008). Most studies show that using a CRAS 立 to assist digital reading can effectively improve learners’ reading comprehension performance as. Tirri, 2005), u-Annotate webpage annotation system (Chatti, Sodhi, Specht, Klamma, & Klemke,. ‧. ‧ 國. 學. well as help the learners enhance the breadth and depth of reading. However, many learners are generally unwilling to actively contribute their reading annotation knowledge or discuss with others when using the reading annotation system for collaborative reading, thus reducing the reading comprehension from reading annotations (Chen, Li, & Chen, 2018). Namely, many learners are the lurkers who like to read other people’s reading annotations in collaborative reading activities in order to expand their reading knowledge, but they are unwilling to actively contribute reading annotations. Obviously, because of the essential problem from collaborative reading annotation activities, learner’s knowledge share and discussion interaction thus are significantly reduced, thus affecting the effectiveness of each learner’s reading comprehension. In the past studies, formative assessment was usually applied to the individual learner’s learning environment, but it was rarely applied to the collaborative learning environment. The social network analysis applied to the interactive web-based collaborative learning has become more and more important in recent years. For example, Crespo and Antune (2013) found that there was a significant positive correlation between the PageRank score and the academic performance of the group in the circumstance of web-based collaborative learning. Also, Saqr, Fors and Nouri’s study (2018) showed a consistent moderate to strong positive correlation between learning performance, interaction parameters, and students’ centrality measures across all the studied courses in online collaborative learning, regardless of the subject matter. These studies inspired us to develop a CRAS-FAFM based on the features from social network measures, including PageRank, degree centrality, closeness centrality, and betweenness centrality, generated through collaborative digital reading activity with the support of collaborative reading annotation system. This study hopes the CRAS-FAFM can assist teachers to find out the learners who have low reading. n. er. io. sit. y. Nat. al. Ch. engchi. 2 . i n U. v.
(3) . comprehension and to provide immediate feedback for them based on the social interaction factors influencing the reading comprehension, so that the reading comprehension and interactive discussion of the learners with low reading comprehension can be improved. Namely, this study aims to examine whether the CRAS-FAFM can facilitate learners’ reading comprehension and interactive discussion in a collaborative reading annotation activity.. 2. Literature Review 2.1 Formative assessment Some scholars have different opinions on assessment. Bloom, Madaus, and Hastings (1981) believed that assessment not only provides feedback and correction to learners but also a method used to determine learners’ learning levels and teaching effectiveness. The assessment can be divided into summative assessment and formative assessment. The summative assessment refers to assessing the learners’ learning performance after they have completed several units or after they. 立. 政 治 大. ‧. ‧ 國. 學. have completed whole curriculums by examination, and divides the test results into multiple rankings or gives grades to the learners. However, the summative assessment may distort the meaning of the scores, and students may reckon learning as responding to rewards and punishments from external (Abrams, Pedulla, & Madaus, 2003; Shepard, 2000). Bransford, Brown, and Cocking (1999) pointed out that formative assessment emphasizes the evaluation in the learning process and provides immediate diagnosis and feedback based on the assessment results. Because of this, formative assessment is a more important way of learning performance assessment. Compared with the traditional summative assessment, the educators using formative assessment can make assessment by observing students’ learning behaviors and performance during the learning process, and provide immediate feedback based on the assessment results, which can improve teacher’s instruction or give remedy learning according to the learners’ insufficiency (Kulasegaram & Rangachari, 2018). Namely, the main purpose of the usage of formative assessment is to encourage the learners to reflect on the current learning situation through continuous assessment and feedback provided by instructors or computer supported learning systems, and according to which that learners adjust the learning strategy, and enhance learning performance. Currently, few formative assessment mechanisms that can correctly assess learner’s learning performance and provide feedback signals based on individual learner’s learning behaviors in real time to individual learners with low learning performance for facilitating their self-reflection on adjusting learning strategies were proposed in collaborative learning environments, particularly in collaborative reading with the support of CRAS. To reduce effectively the reading anxiety of learners while reading English articles with the support of CRAS, Chen, Wang, Chen and Wu’s (2016) study used a C4.5 decision tree, a widely used machine learning technique, to develop a personalized reading anxiety prediction model (PRAPM) with prediction accuracy as high as 70%. n. er. io. sit. y. Nat. al. Ch. engchi. 3 . i n U. v.
(4) . based on formative assessment of individual learners’ reading annotation behaviors in a CRAS. The analytical results of the study showed that the collaborative annotation learning activity with online instructor’s help for reducing reading anxiety by the proposed PRAPM support indeed helps learners reduce reading anxiety, particularly for the male learners, showing that gender difference exists. Moreover, to reduce the effectiveness of collaborative annotations in promoting reading comprehension due to a large amount of poor quality annotations leading to cognitive load, Jan, Chen and Huang’s study (2016) developed a web-based collaborative reading annotation system with two quality annotation filtering mechanisms (WCRAS-TQAFM)—high-grade and master annotation filters—to promote the reading performance of learners based on formative assessment of readers’ collective annotation behaviors. The analytical results of the study indicate that digital reading performance is significantly better in readers who use the high-grade annotation filter compared to those who read all annotations. Moreover, the high-grade annotation filter can enhance the reading comprehension of learners in all considered question types (i.e., recall, main idea, inference, and application). Obviously, the study on developing a CRAS-FAFM to promote learners’. 立. 政 治 大. ‧. ‧ 國. 學. reading comprehension performance is still lacked. Therefore, this study develops a CRAS-FAFM based on the four considered social network indicators from individual learners’ collaborative reading annotation behaviors that can provide timely feedback to learners with low reading comprehension performance to enhance their reading comprehension performance and interactive discussion with their peers.. sit. y. Nat. 2.2 Social networks analysis. n. al. er. io. The concept of social network was first proposed by Barnes (1954) after studying the social structure of the Brernnes diocese in Norwegian fishing village, and then Bott (1957) utilized social network as analysis tools to establish the difference model of the relationship between husbands and wives；Befu (1963) adopted the concept of social network in Japanese villages to divide the. Ch. engchi. i n U. v. relationship into relatives, friendships, and neighbors. Social network was an important and intangible asset that affects interpersonal relationships and authorities (Bolino, Turnley, & Bloodgood, 2002). Researchers could utilize experiments or questionnaires to find out the structure of relationships and actions between people, and present these results through sociogram (Moreno, 1934; Scott, 2002). Hanneman and Riddle (2005) pointed out that social networks mainly contain three elements, namely actors, relationships, and linkages. The actors were the main subjects in the social network and represented different roles, which can be individuals, organizations, countries, or events. The relationships represent the interdependent modes between actors, including the existence of relationships and the types of relationships. However, the existence of different relationships could affect the interaction between actors. The linkages represent that actors establish their relationships through the path directly or indirectly. In social networks, actors have their own positions, and the positions in the structure will affect 4 .
(5) . how they control the resources. If the actors are located in the center of the network, they will control more resources and benefit more (Ibarra, 1993). Freeman (1979) pointed out that the centrality could examine how actors control and acquire the resources. The centrality is one of the most commonly used social network indicators (Hanneman, & Riddle, 2005; Knoke, & Yang, 2008; Wasserman, 1994). The centrality includes degree centrality, closeness centrality, and betweenness centrality. Degree centrality aims to measure this actor’s degree of centrality in the network by the number of links between actors so as to observe how they held the internet initiative. In cyberspace, the actor with the higher degree centrality has the more connections with other actors and possesses more power of influence and more authority (Brass, & Burkhardt, 1992; Krackhardt, 1993). Closeness centrality aims to measure the distance between actors which represent whether the actors will receive and transmit information easily. Actors with higher closeness centrality could obtain information faster. Betweenness centrality aims to inspect whether an actor is on the contact route between one and another. Actors with higher betweenness centrality could have more opportunities to guide the information flow which means they occupy crucial positions in. 立. 政 治 大. ‧. ‧ 國. 學. controlling receiving and interchange information. (Burt, 1982), and they held more abundant information (Granovetter, 1973). The PageRank algorithm is the most outstanding one out of the sorting algorithms in web structure mining. PageRank was proposed Brin and Page (1998), which mainly refers to the concept of citing academic reference corresponding to the numbers of web pages being cited or being linked, so as to rank the web pages to demonstrate the importance of them. A web page may be important while being cited or linked for multiple times, in the meanwhile another web page may also be important even though it is not being cited or linked for many times, but it is being cited or linked by important web site. When PageRank is applied to social network analysis, the higher the PageRank index of an actor, the more important this actor is being in this network, and the greater influence that the actor has. Therefore, this study considers four social network indicators－ degree centrality, closeness. n. er. io. sit. y. Nat. al. Ch. engchi. i n U. v. centrality, betweenness centrality, and PageRank to develop the formative assessment and feedback mechanism for the CRAS. This study uses these social network indicators as the features which may influence the reading comprehension performance to develop the formative assessment and feedback mechanisms for enhancing learners’ reading comprehension performance on CRAS through providing appropriate feedback based on the formative assessment results. The most important reason of using the four social networks indicators as the features to develop a formative assessment and feedback mechanisms on CRAS for promoting reading comprehension performance is that the learners having lower degree centrality, closeness centrality, betweenness centrality, and PageRank in collaborative learning social networks definitely have poor interaction with their peers who play important roles in a collaborative learning group, thus leading to poor reading comprehension performance. 5 .
(6) . 2.3 Collaborative reading annotation system In the historical development of the CRAS, the single-annotation system was primarily used in the beginning (Kahan, Koivunen, Prud'Hommeaux, & Swick, 2002; LeeTiernan, & Grudin, 2001, Mason, & Woit, 1999; Ogata, Hada, & Yano, 2000). The single-annotation system allows users to mark reading annotations in digital texts, but they did not allow multiple users to discuss and share with one another. Compared with individual learning and competitive learning, collaborative learning has been confirmed its advantages in promoting learners’ learning motivation (Hiltz, Coppola, Rotter, Toroff, & Benbunan-Fich, 2000; Slavin, 1994), learning performance (Johnson, & Johnson, 1991), learning interests (Duren & Cherrington, 1992), communication skills, and improve social interaction (Webb, 1985). Therefore, the CRAS has become the current mainstream of development to support digital reading. There have been many CRASs successfully developed to assist learners’ digital reading so that learners can collaboratively contribute their reading annotations in digital texts and share their reading annotations with their peers, such as the CoCoAJ. 立. 政 治 大 System) (Ogata, Hada,. ‧. ‧ 國. 學. (Communicative Collection Assisting & Yano (2000), EDUCOSM (Nokelainen, Miettinen, Kurhila, Floréen, & Tirri, 2005), and u-Annotate webpage annotation system (Chatti, Sodhi, Specht, Klamma, & Klemke (2006). However, to the best of our knowledge, the study on developing a CRAS-FAFM to promote learners’ reading comprehension performance is still lacked. Therefore, this study uses the C4.5 decision tree to develop formative assessment and feedback mechanisms for promoting reading comprehension performance based on the social network indicators from the interaction and reading annotation behaviors of learners in the CRAS. This study aims at examining whether the developed CRAS-FAFM provides benefits in promoting learners’ reading comprehension performance and interactive discussion levels in digital reading activities.. er. io. sit. y. Nat. al. n. v i n C h Methodology 3. Research engchi U. 3.1 Research participants A total of 81 students (37 females and 44 males) were randomly sampled from three classes of Grade 5 at a primary school in Taoyuan City, Taiwan to participate in the instructional experiment. Of the 81 participants, 26 students (11 females and 15 males) were invited to develop the CRAS-FAFM based on the social network indicators from their collaborative reading annotation behaviors and interactive discussion in the CRAS. The remaining 55 students were assigned randomly into a control group (n=27; 13 females and 14 males) and experimental group (n=28; 13 females and 15 males). The control group used CRAS-NFAFM to assist digital reading activity while the experimental group used CRAS-FAFM.. 3.2 Research design 6 .
(7) . Firstly, this study applied C4.5 decision tree to develop a formative assessment and feedback mechanism based on the four considered social network measures－PageRank, degree centrality, closeness centrality, and betweenness centrality for CRAS, which has good enough prediction accuracy rate that can identify the learners with low reading comprehension and recommend the appropriate learning peers who are determined with high reading comprehension and infrequent interaction with them. Secondly, this study used the quasi-experimental research method to design an instructional experiment and examined whether there are significant differences between the experimental group with CRAS-FAFM support and the control group with CRAS-NFAFM support in reading comprehension and interactive discussion. The learners in both the groups have 20 minutes’ training time to learn the operation of CRAS before the experiment. The following 30 minutes were used to perform a collaborative reading annotation activity for the learners in both the groups. Furthermore, the learners in the experimental group with CRAS-FAFM support will be automatically recommended a list with several appropriate learning peers who are identified with high reading comprehension and infrequent interaction if they are identified with low reading comprehension. Namely, the CRAS-FAFM encourages the learners to discuss and interact more with these learners with high reading comprehension and infrequent interaction to enhance their reading comprehension performance. Over the 30~40 minutes, learners in both the groups were requested to conduct the posttest of reading comprehension after finishing the whole reading learning activities, which is regarded as the assessment basis of reading comprehension. 立. ‧ 國. ‧. sit. y. Nat. 3.3 Research Tool. 學. performance.. 政 治 大. n. al. er. io. 3.3.1 Collaborative reading annotation system (CRAS). i n U. v. The CRAS was developed in this study to support digital reading by allowing learners to read articles and make annotations collaboratively. In addition, learners can give responses to others’ annotations to share and discuss their ideas. To fulfill these two purposes, two kinds of annotations are provided in the CRAS: reading annotation and response annotation. Reading annotations allow students to annotate and share their own ideas about the article and response annotations allow students to give feedback to other learners’ annotations. In addition, reading and response annotation scaffolds are provided to help learners make appropriate annotations and guide them to read and discuss the article. The descriptions of the reading and response annotation types are listed in Table 1.. Ch. engchi. 7 .
(8) . Table 1. Descriptions of reading and response annotation types Annotation type Reading Annotation Response Annotation Give answers to the questions Provide understanding or known raised in other learners’ I know facts of an annotated text annotations New knowledge. Identify new knowledge learned Identify new knowledge learned from an annotated text from other learners’ annotations. Don’t understand. Indicate an annotated text that do Indicate other learners’ not understand annotations that do not understand. Different ideas. Indicate the text that are different Indicate other learners’ from what I think, and give annotations that are different from reasons what I think, and give reasons. Additional Information. Provide supplementary information for an annotated text by using online search tool in CRAS. I want to say. Give comments to an annotated Respond to other learners’ text and invite other students to comments or discussion of an discuss their ideas annotated text. 政 治 大. Remind other learners to correct their problematic annotations or inappropriate use of the annotation types or wordings. ‧. ‧ 國. 學. Correction. 立. Provide supplementary information for other learners’ annotations by using online search tool in CRAS. ---. io. sit. y. Nat. er. To make a reading annotation, a learner has to first select target texts, and then chooses an. al. n. v i n C chosen annotation type (Fig. 1(a)) to helphlearners i U annotation types. For the annotation e n guse c hproper annotation type and writes down annotation contents. A brief description is shown on the side of the. contents, learners can type text, insert pictures or videos, provide webpage links, or use embedded Google search tool to find additional information for the annotated text (Fig. 1(b)). Response annotations are used when learners want to give feedbacks to other learners’ reading or response annotations. After learners choosing an annotation that they would like to reply, the way to make a response annotation is the same as making a reading annotation (Fig. 2). Besides response annotations, learners can also click the “heart” icon to show their agreement or favor toward a reading or response annotation made by other learners. When learners move mouse cursor over the text with annotations, they can see how many annotations have been made for the text and click to read the contents of those reading and response annotations.. 8 .
(9) . (a) Choose the text and select an annotation type. 立. 政 治 大. ‧. ‧ 國. 學 y. Nat. io. sit. (b) Add annotation content. n. al. er. Figure 1. An example of making a reading annotation in the CRAS. Ch. engchi. i n U. v. Figure 2. An example of making a response annotation in the CRAS 9 .
(10) . 3.3.2 Developing a prediction model in the collaborative reading annotation system for identifying learners with low reading comprehension performance When establishing a formative assessment and feedback mechanism for promoting learners’ reading comprehension, the required information includes annotation interaction records, annotation types, and social network indicators. According to Crespo and Antune (2013), the higher group’s academic performance will be gotten in the context of online collaborative learning while the higher Pagerank score is gotten in the social networks of online collaborative learning environment. Moreover, Saqr, Fors and Nouri’s study (2018) showed a consistent moderate to strong positive correlation between learning performance, interaction parameters, and students’ centrality measures across all the studied courses in online collaborative learning. Therefore, there are four main social network indicators considered in the study to establish a formative assessment and feedback mechanism in the CRAS for promoting reading comprehension. They are degree centrality, betweenness centrality, closeness centrality, and PageRank, respectively. This study employs the C4.5 decision tree proposed by Quinlan (1993) as the algorithm to establish the formation assessment prediction and feedback mechanism of reading comprehension. The C4.5 decision tree algorithm is an extension of the ID3 decision tree algorithm. The classification rules are easy to be understood and have high accuracy (Quinlan, 1993), so this study used it. The social network indicators that come from learners’ interaction when annotating a text or replying a text with annotation on the CRAS are used as the features of establishing the formation assessment prediction model based on the C4.5 decision tree. Based on the data collected from a pilot experiment, this study used the above four social network indicators and gender as the features of the C4.5 decision tree provided in Weka to establish a formative assessment prediction model for forecasting a learner’s reading comprehension as high or low level. In the case of different data pre-processing methods, a total of three decision trees are generated. The forecasting accuracy rate of the first decision tree is 63%, while the social network indicators that mainly affect the effectiveness of reading comprehension are betweenness centrality, degree centrality, and PageRank (as shown in Fig. 3). This study found that the learners with relatively low betweenness centrality, low PageRank, and high degree centrality will lead to low reading comprehension from the rules of the first decision tree. The forecasting accuracy rate of the second decision tree is 60%, while the social network indicators that mainly affect the effectiveness of reading comprehension are in-degree centrality, out-degree centrality, and PageRank (as shown in Fig. 4). This study found that the learners with relatively low out-degree centrality and low PageRank will lead to low reading comprehension from the rules of the second decision tree. The forecasting accuracy rate of the third decision tree is up to 82%, while the social network indicators that mainly affect the effectiveness of reading comprehension are betweenness centrality, closeness centrality, and PageRank (as shown in Fig. 5). This study found that the learners with relatively high in-closeness under relatively low betweenness centrality will lead to low reading. 立. 政 治 大. ‧. ‧ 國. 學. n. er. io. sit. y. Nat. al. Ch. engchi. 10 . i n U. v.
(11) . comprehension from the rules of the third decision tree. More importantly, compared to the four considered social network indicators, gender was confirmed as the most discriminated feature in the developed reading comprehension prediction model because it was selected as the root node by the first and second decision trees. The result shows that the gender difference between learners’ reading annotation behaviors affecting reading comprehension exits. Encouragingly, the average prediction accuracy rate of the developed CRAS-FAFM in identifying the learners with low reading performance is as high as 68.33%.. Gender. = girl. 立. <= 0.576923. <= 0.024469. Low reading comprehension. Low reading comprehension. > 0.024469 High reading comprehension. y. sit. > 0.035547 High reading comprehension. n. al. er. io. Betweenness. Nat. <= 0.035547. > 0.576923. ‧. Pagerank. 政 治 大Betweenness. 學. ‧ 國. Degree. = boy. Ch. <= 0.02174. > 0.02174. Low reading comprehension. High reading comprehension. engchi. i n U. v. Figure 3. The first decision tree for identifying learners with low reading comprehension. 11 .
(12) Gender. = girl. = boy. In-Degree. Out-Degree. > 0.333333. <= 0.333333. High reading comprehension. Pagerank. <= 0.041894. > 0.041894. Low reading comprehension. High reading comprehension. ‧ 國. 學. > 0.481481 Low reading comprehension. ‧. High reading comprehension. > 0.333333. 政 治 大 In-Degree 立. Low reading comprehension. <= 0.481481. <= 0.333333. n. al. er. io. sit. y. Nat. Figure 4. The second decision tree for identifying learners with low reading comprehension. Ch. engchi. 12 . i n U. v.
(13) Betweenness. <= 0.067702. > 0.067702. High reading comprehension. In-Closeness. > 0.586957. <= 0.586957. Low reading comprehension. In-Closeness. 政 治 大. <= 0.5625. > 0.5625. PageRank. High reading comprehension. > 0.015601. High reading comprehension. Low reading comprehension. ‧. <= 0.015601. 學. ‧ 國. 立. sit. y. Nat. Figure 5. The third decision tree for identifying learners with low reading comprehension. n. al. er. io. Interestingly, from the three decision trees, this study found that most of the low reading comprehension effectiveness results are related to the low social network indicators, which means too little interaction was taken place with others. In order to enhance the prediction accuracy rate, this study utilized the aggregating concept to determine the prediction results based on the voting result of the three decision trees. That is, the final prediction result for an individual learner is determined by the voting mechanism. For example, if the prediction results of a learner by the three decision trees are respectively low, low, and high reading comprehension, then the final prediction result will be low reading comprehension.. Ch. engchi. i n U. v. 3.3.3 Developing formative assessment and feedback mechanisms in the collaborative reading annotation system for learners with low reading comprehension performance Chen, Wang and Chen (2014) pointed out that the usage of reading annotations will affect the reading comprehension. Therefore, this study assumes that learners with higher reading comprehension will take more time to make rich annotations and respond their peers’ annotations in a collaborative reading annotation activity. Therefore, this study logically employs social network indicators to develop a formative assessment mechanism of reading comprehension. After finishing 13 .
(14) . the development of the formative assessment mechanism of reading comprehension, the mechanism was integrated with the CRAS to further develop the feedback mechanisms for guiding the learners with low reading comprehension to interact with the learners with high reading comprehension and infrequent interaction. Namely, this study establishes CRAS-FAFM to help learners improve reading comprehension and interactive discussion. The CRAS-FAFM can provide a list of learners who are predicted as the low reading comprehension for individual learners. The user interface of identifying the learners with low reading comprehension is shown as Fig. 6. Furthermore, learners can also find out the possible reasons causing them to make low reading comprehension based on the rules provided by the C4.5 decision trees. At the same time, CRAS-FAFM also lists the learners who are worthy to interact, shown as Fig. 7. Learners can interact with the recommended learners to view and absorb better quality annotations contents. When the CRAS-FAFM recommends more than 3 learners, it will automatically randomly pick out 3 learners as recommended candidates to reduce the learner’s choice difficulties and information anxiety.. 立. 政 治 大. ‧. ‧ 國. 學 er. io. sit. y. Nat. al. n. v i n C hlearners with lowUreading comprehension performance Figure 6. User interface of showing engchi. Figure 7. User interface of recommended lists 14 .
(15) . 3.3.4 The code list of interactive discussion gradation The research adopts the coding scheme of interactive discussion level proposed by Hou, Chang, and Sung (2008), which is used to quantify the interactive discussion contents between both the groups. The code scheme can classify the discussion contents into five levels, as shown in Table 2. Table 2. Coding scheme for problem solving discussion contents Description Propose problem or clarify the definition of the problem Provide information or propose solutions to the problem (provide information for partial or full solution). Provide solutions or information for possible answers. P2. Others. such as the sudden drop in temperature at the time, which may lead to the ice age. Summarizing the opinions and information of my classmates before, my knot is: the outward expansion of the orbit may lead to the ice age, because in this state, the earth will look far to the sun, light and heat are reduced, and the temperature is lowered. Due to the increased inclination angle of the earth’s axis, the ice age is formed.. ‧. P5. Organize proposed solutions or comments and form conclusions for solutions. Nat. P4. Organize and form conclusions. I found the following information: For the ice age, the current explanation is due to the outward expansion of the earth’s orbit.. 學. ‧ 國. 立. How was the process of the ice age formed?. I do not think that the previous glare of the ice age was completely explained. Why the 治 政 outward 大 expansion of the orbit will lead to the ice age. I think there should be more reasons,. Analyze, compare, and comment on others’ opinions, solutions, or collected information. Compare, discuss, and analyze. P3. Discussion example. y. P1. Phase Propose, define, and clarify problem. Messages not related to the subject of discussions. I feel so cold during the ice age in the winter.. er. io. sit. Code. n. a lExperimental Resultsi v 4.. n U engchi 4.1 Difference analysis of reading comprehension of both groups. Ch. This study took the Mandarin grades in mid-term test as the pretest to examine the initial reading comprehension abilities of the learners in both the groups. An independent samples t-test was carried out on the pretest scores to determine the differences between reading comprehension of both the groups before the instructional experiment. The results show that the reading comprehension of both the groups do not differ significantly (t = .627, p = .533 > .05), indicating that both the groups have the same initial reading comprehension ability. Table 3 shows the results of the independent samples t-test, indicating that both the groups have significant difference in the posttest score (t = -2.025, p = .048 < .05), and the experimental group is better than the control group. This result confirms that the reading comprehension of the experimental group learners using CRAS-FAFM to assist reading learning is significantly better than that of the control group learners using CRAS-NFAFM. 15 .
(16) . Table 3. Independent samples t-test of reading comprehension performance of learners in both groups Number of Standard Significance Group Mean t deviation learners (two-tailed) Control 27 9.89 3.662 group Posttest -2.025* .048 Experimental 28 11.54 2.219 group *indicates p<.05 Next, this study conducts the difference analysis of reading comprehension of learners with high and low prior knowledge in both the groups based on independent samples t-test. Tables 4 and 5 show the results, respectively. Analytical results show that the learners with high prior knowledge in both the groups do not have significant difference in reading comprehension (t = .233, p = .818. 政 治 大 > .05), whereas the learners with low prior knowledge in both the groups have significant difference 立 in reading comprehension (t =-3.390, p = .002 < .05) and the learners with low prior knowledge in. ‧ 國. 學. the experimental group is superior to the learners with low prior knowledge in the control group.. ‧. Table 4. Independent samples t-test of reading comprehension performance of learners with high prior knowledge in both groups Number of Standard Significance Group Mean t learners deviation (two-tailed) Control 16 11.93 2.113 group Posttest .233 .818 Experimental 12 11.75 1.603 group. n. er. io. sit. y. Nat. al. Ch. engchi. i n U. v. Table 5. Independent samples t-test of reading comprehension performance of learners with low prior knowledge in both groups Number of Standard Significance Group Mean t learners deviation (two-tailed) Control 11 7.18 3.816 group Posttest -3.390** .002 Experimental 16 11.38 2.630 group **indicates p<.01. 4.2 Difference analysis of interactive discussion levels of both groups To examine whether a significant difference exists in the interactive discussion of both the groups, analysis of independent samples t-test was used and shown in Table 6. The results show 16 .
(17) . that both the groups have significant difference in “Level P3” (i.e. compare, discuss, and analyze)(t = -2.458, p = .019 < .05), and the experimental group is superior to the control group. Besides, the remaining levels of interactive discussion have no significant difference. Table 6. Independent samples t-test of different interactive discussion levels of learners in both groups Discussion Number of Standard Significance Group Mean t deviation level learners (two-tailed) Control group 27 3.81 3.013 Level P1 -.361 .719 Experimental 28 4.11 2.986 group Control group 27 10.48 6.930 Level P2 1.681 .099 Experimental 28 7.61 5.711 group Control group 27 1.96 1.931 Level P3 -2.458* .019 Experimental 28 4.11 4.175 group Control group 27 .59 1.047 Level P4 1.571 .125 Experimental 28 .25 .441 group Control group 27 1.89 2.375 Level P5 .639 .525 Experimental 28 1.50 2.134 group *indicates p<.05. 立. 政 治 大. ‧. ‧ 國. 學. sit. y. Nat. n. al. er. io. Next, the independent samples t-test result was used to examine the difference of interactive discussion levels of learners with high and low prior knowledge in both the groups. Table 7 shows the independent samples t-test result of different interactive discussion levels of learners with high prior knowledge in both the groups. The result shows that the learners with high prior knowledge in both the groups have no significant differences in all the five interactive discussion levels. In. Ch. engchi. i n U. v. contrast, Table 8 shows the independent samples t-test result of different interactive discussion levels of learners with low prior knowledge in both the groups. The result shows that the learners with low prior knowledge in both groups have significant differences in the interactive discussion level P3 (i.e. compare, discuss, and analyze)(t = -2.041, p = .046 < .05), whereas the other four interactive levels have no significant differences.. 17 .
(18) . Table 7. Independent samples t-test result of different interactive discussion levels of learners with high prior knowledge in both groups Discussion Number of Standard Significance Group Mean t deviation level learners (two-tailed) Control group 16 4.06 3.356 Level P1 -.393 .698 Experimental group 12 4.58 3.630. Level P5. 12.25. 7.611. Experimental group. 12. 8.08. 7.012. Control group. 16. 2.13. 2.125. Experimental group. 12. 4.17. 4.324. Control group. 16. 0.81. 1.109. Experimental group. 12. 0.17. 0.389. Control group Experimental group. 立. 16 2.00 政12 治 1.50大. ‧ 國. Level P5. 3.75. 2.463. 11. 7.91. 5.069. Experimental group. 16. 7.25. 4.726. 11. 1.73. 16. 5.06. al. Control group. n. Level P4. 3.45. Standard deviation 2.544. Mean. Control group. io. Level P3. Experimental group Control group. Ch. -1.505. .153. 1.923. .065. .551. .587. interactive discussion levels of learners with. ‧. Nat. Level P2. 1.931. .150. 學. Table 8. Independent samples t-test result of different low prior knowledge in both groups Discussion Number of Group level learners Control group 11 Level P1 Experimental group 16. 2.658. 1.482. i U e11n g c h0.27. t. Significance (two-tailed). -.302. .765. .346. .732. -2.041*. .046. -.149. .883. .262. .795. y. Level P4. 16. sit. Level P3. Control group. er. Level P2. 1.679. v n i4.203 0.905. Experimental group. 16. 0.31. 0.479. Control group. 11. 1.73. 2.005. Experimental group. 16. 1.50. 2.338. *indicates p<.05. 5. Discussion Most of the learning assessment methods in the traditional teaching environments are based on summative assessment. As a result, it is difficult for teachers to instantly get the learning effectiveness of learners in the learning process. This study thus considers using four social network indicators－ PageRank, degree centrality, closeness centrality, and betweenness centrality to develop the CRAS-FAFM based on C4.5 decision tree, which can identify learners with low reading 18 .
(19) . comprehension and recommend appropriate learning peers for them to promote their reading comprehension and interactive discussion in a digital reading activity with a CRAS support. This study first examines the forecasting accuracy rate of the developed formative assessment mechanism based on the four considered social network indicators for identifying the learners with low reading comprehension performance as well as providing feedback for them. The results show that the average forecasting accuracy rate of identifying the learners with low reading comprehension performance is approximately 68.33%, indicating that using the four considered social network indicators to predict the reading comprehension performance of a learner as high or low level in a digital reading activity with a CRAS support is practicable. The result echoes Chen, Wang, Chen and Wu’s (2016) study. Their study also used a C4.5 decision tree to develop a personalized reading anxiety prediction model (PRAPM) with prediction accuracy rate as high as 70% based on formative assessment of individual learners’ reading annotation behaviors in a CRAS. In addition, according to the rules of three decision trees, our results also confirm that a significant positive correlation between the PageRank score and the reading comprehension performance of a learner in the digital reading activity with the CARS support exists. Namely, the learners with relatively high PageRank score will lead to high reading comprehension because they play. 立. 政 治 大. ‧ 國. 學. ‧. relatively important roles in the collaborative digital reading activity. Our results echo Crespo and Antune’s study (2013), indicating that there was a significant positive correlation between the PageRank score and the academic performance of the group in the circumstance of web-based collaborative learning. Also, our results also confirm that a significant correlation between the centrality score and the reading comprehension performance of a learner in the digital reading activity with CARS support exists. This study found that most of the learners with low reading comprehension derive from the low centrality indicators, including low out-degree centrality and betweenness centrality, which means that too little interaction was taken place with others in the collaborative digital reading activity. Our results echo Saqr, Fors and Nouri’s study (2018), indicating that a consistent moderate to strong positive correlation between learning performance and students’ centrality measures across all the studied courses exits in online collaborative learning, regardless of the subject matter. Interestingly, compared to the four considered social network indicators, gender was confirmed as the most discriminated feature in the developed reading comprehension prediction model because it was simultaneously selected as the root node by the first and second decision trees. The result shows that the gender difference between learners’ reading annotation behaviors affecting reading comprehension exits. The result is consistent with Chen, Wang, Chen and Wu’s (2016) study, indicating that gender is an important feature of the developed personalized reading anxiety prediction model (PRAPM) in a collaborative reading annotation system.. n. er. io. sit. y. Nat. al. Ch. engchi. i n U. v. The experimental results show that the reading comprehension performance of the learners in the experimental group using CRAS-FAFM to assist reading learning is significantly better than 19 .
(20) . that of the learners in the control group using CRAS-NFAFM, particularly for the learners with low prior knowledge. The result is consistent with several studies (Chen, Wang, & Lin, 2017; Liu, Andre, & Greenbowe, 2008), indicating that computer supported learning systems generally provide more benefits in terms of promoting learning performance for the learners with low prior knowledge in comparison with the learners with high prior knowledge. For example, Chen, Wang and Lin (2017) presents a computer supported learning system called attention-based diagnosing and review mechanism (ADRM) based on brainwave detection to help learners identify the passages with low attention level in a lesson as review targets in order to perform efficiently and accurately review processes while reading paper-based English texts with digital pen support in autonomous learning environments. Their results confirmed that the proposed ADAM exhibited better review performance for the learners with low-ability compared to those in the control group using autonomous review. Also, Liu, Andre and Greenbowe (2008) investigated how college students’ prior chemistry knowledge level affected their interaction with peers and their approach to solving problems with the use of computer simulations that were designed to learn. 立. 政 治 大. ‧ 國. 學. electrochemistry. Their research findings indicated that students with a low level of prior chemistry knowledge more relied on the computer simulations as the main resources to accomplish their tasks than students with a high level of prior chemistry knowledge.. ‧. Furthermore, the interactive discussion performance of the learners in the experimental group in the discussion level of “P3 comparison, discussion and analysis” is significantly higher than that of the learners in the control group, particularly for the learners with low prior knowledge, while no significant difference was found in the other the discussion levels of the learners in both the groups. Remarkably, using CRAS-FAFM to support digital reading can effectively encourage learners to discuss more deeply with appropriate peers by selecting from the recommended lists, and also increase the opportunities to interact with learners having high reading comprehension, thus enhancing learners’ discussion effectiveness in the discussion level of “P3 comparison, discussion and analysis.”. n. er. io. sit. y. Nat. al. Ch. engchi. i n U. v. 6. Conclusions and Future Works To allow learners to effectively proceed digital reading on digital texts and to make interactive discussion on reading annotation contents to improve reading comprehension performance, this study uses C4.5 decision tree to develop a novel CRAS-FAFM having a forecasting accuracy rate as high as 68.33% based on four considered social network measures, which could forecast the learners with low reading comprehension and suggest them to interact with their learning peers who are predicted with high reading comprehension and infrequently interact in the digital reading activity. This study confirms that the reading comprehension and discussion effectiveness level of P3 of the learners in the experimental group using CRAS-FAFM to assist reading learning are significantly better than those of the learners in the control group learners using CRAS-NFAFM. 20 .
(21) . Moreover, the proposed CRAS-FAFM provides more benefits in promoting reading comprehension and discussion effectiveness level of P3 for the learners with low prior knowledge than the learners with high prior knowledge. This study brings the research of CRAS in supporting digital reading activity into a new ground. This study suggests several future research directions. Firstly, the forecasting model of reading comprehension performance is mainly based on primary school pupils’ annotation behaviors, so the prediction results may be inaccurate for other aged learners, such as junior high school, senior high school, and university students. Therefore, in the future, collecting the reading annotation behaviors of other aged learners for developing a formative assessment prediction model of reading comprehension should be considered, and other factors that may affect prediction results, such as the personal background, the usage time of the system, and the length of discussion time, should be considered, so as to establish a forecasting model with higher prediction accuracy rate in identifying learners with low reading comprehension. Secondly, this study is unable to conduct long-term. 政 治 大 experiments due to time constraints. However, it is definitely needed that takes much more time to 立 train the learners’ reading abilities and habits in a digital reading activity. Therefore, it is necessary. ‧ 國. ‧. References. 學. to extend the experimental time to further examine the effects of CRAS-FAFM on learners’ reading comprehension performance and interactive discussion effectiveness.. sit. y. Nat. Abrams, L.M., Pedulla, J.J., & Madaus, G.F. (2003). Views from the classroom： Teachers’ opinions of statewide testing programs. Theory Into Practice, 42(1), 18-29.. n. al. er. io. Barnes, J. A. (1954). Class and committees in a Norwegian Island Parish. HumanRelations, 7(1), 39-58.. Ch. i n U. v. Befu, H. (1963). Patrilineal descent and personal kindred in Japan. American Anthropologist, 65(6), 1328-1341.. engchi. Bloom, B. S., Madaus, G. F., & Hastings, J. T. (1981). Evaluation to improve learning. New York: McGraw-Hill. Bolino, M. C., Turnley, W. H., & Bloodgood, J. M. (2002). Citizenship behavior and the creation of social capital in organizations. Academy of management review, 27(4), 505-522. Bott, E. (1957). Family and social network: Roles, norms and external relationships in ordinary urban families. London: Tavistock Press. Bransford, J. D., Brown, A., & Cocking, R. (1999). How people learn: Mind, brain, experience, and school. Washington, DC: National Research Council. Brass D. J. & Burkhardt, E. (1992). Centrality and power in organizations. In N. Nohria & R.G. Eccles (Eds), Networks and organizations: Structure, form, and action. Boston, MA: Harvard Business School Press, pp. 191-215. Brin, S., & Page, L. (1998). The anatomy of a large-scale hypertextual Web search engine. 21 .
(22) . Computer networks and ISDN systems, 30(1), 107-117. Burt, R. S. (1982). Toward a structural theory of action. New York: Academic Press. Chen, C. M., Li, M. C., & Chen, T. C. (2018). A collaborative reading annotation system with gamification mechanisms to improve reading performance. International Conference on Learning Technologies and Learning Environments. Chen, C. M. & Chen, F. Y. (2014). Enhancing digital reading performance with a collaborative reading annotation system. Computers & Education, 77, 67-81. Chen, C. M., Wang, J. Y., Chen, Y. T., & Wu, J. H. (2016). Forecasting reading anxiety for promoting English-language reading performance based on reading annotation behavior, Interactive Learning Environments, 24(4), 681-705. Chen, C. M., Wang, J. Y., & Lin, M. (2019). Enhancement of English learning performance by using an attention-based diagnosing and review mechanism in paper-based learning context with digital pen support. Universal Access in the Information Society, 18, 141-153. https://doi.org/10.1007/s10209-017-0576-2. 政 治 大 Chang, C. S., Chen, C. M., & Lin, Y. C. (2018). A visual interactive reading system based on eye 立 digital reading performance. International Conference on tracking technology to improve. ‧ 國. 學. Learning Technologies and Learning Environments.. ‧. Chatti, M. A., Sodhi, T., Specht, M., Klamma, R., & Klemke, R. (2006). u-Annotate: an application for user-driven freeform digital ink annotation of e-learning content. In Proceedings of 6th International Conference on Advanced Learning Technologies (pp. 1039-1043). Los Alamitos: IEEE Press.. Nat. y. sit. al. er. io. Chen, C. M., Wang, J. Y., & Chen, Y. C. (2014). Facilitating English-language reading performance by a digital reading annotation system with self-regulated learning mechanisms. Educational Technology & Society, 17(1), 102-114.. v. n. Crespo, P. T., & Antunes, C. (2013). Predicting teamwork results from social network analysis. Expert Systems, 32(2), 312-325. DOI: 10.1111/exsy.12038.. Ch. engchi. i n U. Duren, P. E., & Cherrington, A. (1992). The effects of cooperative group work versus independent practice on the learning of some problemsolving strategies. School Science and Mathematics, 92(2), 80-83. Farzan, R., & Brusilovsky, P. (2008). AnnotatEd: A social navigation and annotation service for web-based educational resources. New Review of Hypermedia and Multimedia, 14(1), 3-32. Freeman, L. C. (1979). Centrality in social networks conceptual clarification. Social networks, 1(3), 215-239. Granovetter, M. S. (1973). The strength of weak ties. American journal of sociology, 78(6), 1360-1380. Hanneman, R. A., & Riddle, M. (2005). Introduction to social network methods. Riverside, CA: University of California, Riverside. Hiltz, S. R., Coppola, N., Rotter, N., Toroff, M., & Benbunan-Fich, R. (2000). Measuring the importance of collaborative learning for the effectiveness of ALN: A multi-measure, 22 .
(23) . multi-method approach. Journal of Asynchronous Learning Networks, 103-125. Hou, H. T., Chang, K. E., & Sung, Y. T. (2008). Analysis of problem-solving-based online asynchronous discussion pattern. Educational Technology & Society, 11(1), 17-28. Ibarra, H. (1993). Network centrality, power, and innovation involvement: Determinants of technical and administrative roles. Academy of Management Journal, 36(3), 471-501. Johnson, D. W., & Johnson, F. P. (1991). Joining together: Group theory and group skills. New Jersey: Prentice-Hall. Jan, J. C., Chen, C. M., & Huang, P. H. (2016). Enhancement of digital reading performance by using a novel web-based collaborative reading annotation system with two quality annotation filtering mechanisms. International Journal of Human-Computer Studies, 86, 81-93. Kahan, J., Koivunen, M. R., Prud'Hommeaux, E., & Swick, R. R. (2002). Annotea: an open RDF infrastructure for shared Web annotations. Computer Networks, 39(5), 589-608.. 政 治 大. Kulasegaram, K. & Rangachari, P. K. (2018). Beyond “formative”: assessments to enrich student learning. Advances in Physiology Education, 42(1), 5-14.. 立. Knoke, D., & Yang, S. (2008). Social network analysis. London: Sage.. ‧ 國. 學. Krackhardt, D. (1993). MRQAP: Analytic versus permutation solutions. (Working paper), Carnegie Mellon University, Pittsburgh.. ‧. Liu, H. C., Andre, T. & Greenbowe, T. J. (2008). The impact of learner’s prior knowledge on their use of chemistry computer simulations: A case study. Journal of Science Education and Technology, 17(5), 466-482.. y. Nat. er. io. sit. LeeTiernan, S., & Grudin, J. (2001). Fostering engagement in asynchronous learning through collaborative multimedia annotation. Proceedings of Interact, 472-479 Mason, D. V., & Woit, D. M. (1999). Providing mark-up and feedback to students with online marking. SIGCSE Bulletin, 31(1), 3-6.. al. n. v i n C Washington, DC: Nervous Moreno, J. L. (1934). Who shall survive?.h e n g c h i U and Mental Disease.. Nokelainen, P., Miettinen, M., Kurhila, J., Floréen, P., & Tirri, H. (2005). A shared documentbased annotation tool to support learnercentred collaborative learning. British Journal of Educational Technology, 36(5), 757-770. Ogata, H., Hada, Y., & Yano, Y. (2000). CoCoAJ: Supporting online correction of hypermedia documents. CALL. ICCE 2000, 1, 323-329. Quinlan, J. R. (1993). C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, California. Scott, J. (2002). Social network analysis: critical concepts in sociology. New York, Routledge. Shepard, L. A. (2000). The role of assessment in a learning culture. Educational researcher, 29(7), 4-14. Slavin, R. E. (1994). A practical guide to cooperative learning. London: Macmillan College. 23 .
(24) . Saqr, M., Fors, U., & Nouri, J. (2018). Using social network analysis to understand online problem-based learning and predict performance. PLoS ONE, 13(9): e0203590. https://doi.org/10.1371/journal.pone.0203590 Wasserman, S. (1994). Social network analysis: Methods and applications. Cambridge, England: Cambridge university press. Webb, N. M. (1985). Verbal interaction and learning in peerdirected groups. Theory into Practice, 24(1), 32-39.. 立. 政 治 大. ‧. ‧ 國. 學. n. er. io. sit. y. Nat. al. Ch. engchi. 24 . i n U. v.
(25)
相關文件
To provide suggestions on how to implement Reading and Writing across the Curriculum (R/WaC) in the school English Language curriculum; and.. To share experiences on implementing
Through arranging various reading activities such as online reading, book recommendation and extended reading materials, schools help students connect reading to
Encourage students to ‘retell’ the water cycle afterwards – speaking and writing (individual and/or group work)... In nature, water keeps changing between liquid water and
Planning and using digital multimodal texts to facilitate KS2 students’ reading skills development and enrich their reading experiences?.
Small temporary groups are formed, based on book choice.. Different groups read
Literature Strategically into English Learning Inside and Outside the
• School-based curriculum is enriched to allow for value addedness in the reading and writing performance of the students. • Students have a positive attitude and are interested and
find a bosom friend (Analects: Selected Reading (II): Methods of choosing friends”) and guides students to carry out a class discussion to understand the methods
