CHAPTER 2: LITERATURE REVIEW
2.5 Related Studies on Learning Outcomes
國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
There are abundant research findings showing the effectiveness of pedagogical design and implementation based on KBPs. The three categories of learning outcomes in KB studies are presented in the next section.
2.5 Related Studies on Learning Outcomes 2.5.1 Cognitive Domain Outcomes
Cesareni, Cacciamani, and Fujita (2016) discovered significant differences between role takers and non-role takers on writing and reading activities. Results showed that role takers perform better in writing and reading compared to non-role takers in post-test. In other words, these results indicated that taking a role was contributed to writing and reading performances.
In addition, Cesareni et al. also discovered that students with a synthesizer role were the most active in the writing activities. Students with a social tutor role were the most active in the reading activities. Both writing and reading activities showed less participation among students with the concept mapper role.
In the study of exploring college students’ epistemic views during their KB activities, Hong, Chen, and Chai (2016) found that there were statistical positive correlation between KB activities and college students’ epistemic views. In other words, college students engaging in group online inquiry activities had more highly developed epistemic views. In addition, Hong et al. also found that students fully involved in group KB activities often changed their epistemic view. The findings indicate that students’ World 2 epistemic view remained the same, while the World 3 epistemic view improved during group KB activities.
Thus, the students proved that higher online interaction is beneficial for advancing knowledge.
Hong, Chai, and Tsai (2015) examined the effect of collective story writing on students’
constructivist-oriented epistemic view of scientific theories. Results showed that there were
‧
significant differences between students’ views of theory as discovered as opposed to
invented during the pre- and post-tests. In other words, students’ views of theory tended to be more in favor of it being discovered rather than being invented. Hong et al. provided
evidence to support the idea that students tend to view theory as something that already exists in nature. Hence, students were passively thought that all the existing theories viewed as true.
However, students’ view of theory as an invented construct showed significant differences between the pre- and post-tests. Hong et al. explained that students accepted that scientific theories were invented because of other scientists proving those theories wrong. This condition was related to a deeper understanding of students’ views of theory.
Sun, Zhang, and Scardamalia (2010) examined the differences between writing and reading notes from the same participants across four semesters, which spanned from Grade 3 to Grade 4. Results showed significant differences in the number of writing notes over these four semesters. The overall number of writing notes increased from each student. However, results showed no significant difference in the percentage of reading notes over these four semesters. Sun et al. had further investigated students’ epistemic complexity level with writing domain-specific words and academic words. Eventually, they discovered a significant positive relationship between students’ epistemic complexity level and writing with
domain-specific words. They also found a significant positive relationship between students’
epistemic complexity level and use of academic words for optical inquiry.
Pelletier, Reeve, and Halewood (2006) investigated the application of KF toward 4- to 5-year-old children’s KB and literacy development. Pelletier et al. divided these children into two groups. One was an experimental group that used electronic photo journals in KF, while another was a comparison group that used a paper format of photo journals. Results revealed that both reading and writing measurements in the experimental group showed significant changes compared to comparison group. Children who employed electronic photo journals
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
had read others’ notes and posted their ideas through KF. Children who employed the paper format of photo journals showed less motivation to read others’ notes on the photos. Pelletier et al. also found there were significant gender differences between the electronic format and paper format of photo journals. Girls tended to adopt the paper format for writing rather than their photo journals. Boys tended to adopt the electronic format for writing rather than their photo journals.
2.5.2 Metacognitive Domain Outcomes
Cesareni et al. (2016) found the differences between role-taker students and students without roles in the “metacognition on process and organization” and “comments” categories.
Role-taker students had reflected more than their counterparts during the process and
organization of metacognition activities. In contrast, those students without roles contributed more comments compared to their counterparts.
One of the studies investigated the effect of KB on reflective assessment among students with low academic achievement. Yang et al. (2016) found that low achieving students were more capable of generating their ideas in solving authentic problems and shared their ideas with community members. Through using Knowledge Connection Analyzer (KCA) in reflective assessment, Yang et al. also discovered significant results in fostering students’
community orientation, promoting synthesis, rise above, and improvement of ideas. Thus, KCA was a powerful tool for students to promote critical thinking during their online discourse.
Hong et al. (2015) found a positive relationship between reflective learning outcomes and online activities. Through contributing notes and reading others’ notes online, students showed awareness of collective knowledge improvement. Additionally, students also showed reflection on knowledge exchange as they read others’ notes. However, the results showed no
‧
significant relationship between students’ individual knowledge growth and online activities.
Among these reflective learning outcomes, collective knowledge improvement and knowledge exchange were reflected directly through online activities.
Seitamaa-Hakkarainen, Raunio, Raami, Muukkonen, and Hakkarainen (2001) discovered five categories of design thinking that stem from the design statements in the database. Most of the design statements were represented by new information, followed by working ideas, metacomments related to design task, statements that defined design context, and general comments from other teams. Seitamaa-Hakkarainen et al. analyzed students’
design thinking with three phases of design activities. Results showed that most of the design thinking appeared in problem-structuring activities, followed by problem solving activities, and decision making activities. Thus, the findings indicated that students’ design thinking was active during problem-structuring activities.
2.5.3 Social Domain Outcomes
Cesareni et al. (2016) discovered that there were significant differences between groups with a higher level of participation and a lower level of participation. The former groups contributed more “maintain relationship” and “reflecting” messages while the latter groups contributed more “introducing” messages. In addition, Cesareni et al. also found that students with a social tutor role often intervened in supporting participation in a group. Motivating students to participate in a group could empower them in a way that is consistent with democratizing knowledge.
Lin and Reigeluth (2016) found that Wiki-supported scaffolding for two different collaborative learning designs showed different outcomes. One of the collaborative learning designs was called small-group project-based and was another called whole-class
collaborative KB. The former design provided students with opportunities for different ways
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
of learning when the scaffold had faded. However, the latter design suggested that there was failure in achieving the goal when the scaffold was removed. Findings indicated that
whole-class collaborative KB was obstructed when students became autonomous learners in
the learning community.
Lai and Law (2006) examined patterns of interaction between students in Hong Kong and Canada during the second stage of learning. Findings indicated that students from Hong Kong tended to contribute more notes and link to others’ notes when collaborating with Canadian students. In this stage, Canadian students shared more views compared to students from Hong Kong. The depth of KB discourse was changed when Canada students joined in the discussion. Hong Kong students tried to focus on the discovery of dissonance and negotiate meanings rather than share information with the Canadian students.
Seitamaa-Hakkarainen et al. (2001) examined the degree of participation among three different design teams. Results showed a significant degree of participation among these design teams, who contributed 454 design statements to the database. To further explore the patterns of collaboration among these design teams, Seitamaa-Hakkarainen et al. analyzed students’ collaborative designing processes in the online learning platform. Students in Team A worked in a relatively intensive collaboration by focusing on a shared design object in producing sketches. Students in Team B worked in a relatively close collaboration without the appearance of a shared design object in the designing process, but with a shared design object appearing during the manufacturing process. Students in Team C worked in a
relatively less collaborative environment compared to their counterparts. Students in Team C worked without a shared design object during the whole process. Hence, they produced different types of clothes. Thus, the findings indicated the nature of collaboration patterns among these design teams.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
25
CHAPTER 3 METHODOLOGY
This study focuses on a review of KB studies from a worldwide perspective. Following the research questions stated in the introductory chapter, all KB studies articles from the Scopus database were processed by following these steps: (a) selection of KB studies using both inclusive and exclusive criteria (see below for details), (b) coding of related information for analysis, (c) analysis of the contribution patterns trends in KB studies.
3.1 Conceptual Factors
Five conceptual factors were examined in this study, including demographic factors, methodologies, online learning platforms, usage of KBPs, and learning outcomes in KB studies.
3.2 Selection of Knowledge Building (KB) Studies
Figure 3-1 illustrates the four steps used for selecting KB studies articles for the
systematic review. First, the Scopus database was selected as the favored searching engine in this study. The main reason was that the Scopus database is identified as a comprehensive database compared to Web of Science and Google Scholar. All results from the Web of Science and Google Scholar can be found in the Scopus database. Hence, the Scopus database was selected for searching articles related to studies of KB pedagogy. The Scopus database was searched with particular keywords and publication types for the initial search periods, which fell on November 23, 2016. This present study mainly focused on articles that were searched with the keyword of knowledge building (KB), as showed in Figure 3-2. The search resulted in 843 articles with the above-noted keyword appearing in the title, abstract, or keywords.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 3-1 Search Strategies Procedure
The next step was to filter irrelevant articles manually. By reviewing the abstracts of these articles, 529 articles were excluded since they defined knowledge building (KB) using a different definition from the one used in this study. The filtered process resulted in 314
articles which were further assessed using the following steps and which were required to meet some inclusive and exclusive criteria for further review. The searched paper/study was included if (a) it was an article written between 2001 and 2016, (b) it was an empirical study, (c) it involved some pedagogical design in KB, and (d) it involved the use of an online learning platform. However, the selected study was excluded if (a) it was a review paper, an unpublished dissertation, or a survey, or (b) it involved gifted students or students with learning disabilities as the participants. Eventually, 56 KB studies (see Appendix A) were selected for this review.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
27
Figure 3-2 Screenshot of Scopus Database Keyword Searching
A total of 22 journals were listed among these articles: (a) Asia Pacific Education Review, (b) Asia-Pacific Education Researcher, (c) Australasian Journal of Educational Technology, (d) British Journal of Educational Technology, (e) Canadian Journal of Science, Mathematics and Technology Education, (f) Computers and Education, (g) Early Education and Development, (h) Educational Technology and Society, (i) Educational Technology Research and Development, (j) IFIP International Federation for Information Processing, (k) Informatics in Education, (l) Instructional Science, (m) International Journal of
Computer-Supported Collaborative Learning, (n) International Journal of Continuing
Engineering Education and Life-Long Learning, (o) International Journal of Technology and Design Education, (p) Journal of Educational Computing Research, (q) Journal of
Mathematics Teacher Education, (r) Journal of Science Education and Technology, (s) Journal of the Learning Sciences, (t) Learning Environments Research, (u) Teachers College Record, (v) Turkish Online Journal of Educational Technology.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
After identifying the selected articles, a coding system was designed to perform further analysis. The coding details are provided in the following section.
3.3 Coding System
3.3.1 Bibliographic Coding
Two bibliographic categories were identified in coding the 56 selected articles including (a) Study ID number, (b) Publication year. These bibliographic categories were derived from coding manual examples of meta-analysis (Lipsey & Wilson, 2001). The Study ID number was coded from the initial search that yielded 843 results. Hence, the selected article ID numbers were between ID1 and ID843. There were 17 articles that included the letters a, b, or c behind the Study ID number with the conditions that these articles contain (a) at least two studies with participants of different education levels, or (b) at least two progressive studies with participants of the same education level. The publication year was directly coded with four digits, for instance 2016.
3.3.2 Demographic Coding
Table 3-1 shows the demographic coding included in this study. There were four items that included the countries where KB studies were conducted, education levels of participants, identities of participants, and subject domains researched in the selected KB studies. Twelve countries were identified in this study: (a) Australia, (b) Canada, (c) Finland, (d) Hong Kong, (e) Italy, (f) Japan, (g) Mainland China, (h) Mexico, (i) Singapore, (j) Taiwan, (k) Turkey, and (l) United States. Among these countries, two categories were identified in the coding: (a) Eastern countries, (b) Western countries. During the coding of different countries, for this study we employed two important criteria for categories. One was related to regions and another one was related to culture. Among these countries, Turkey was found to be difficult
‧
to categorize as either an Eastern country or a Western country, due to its complexity. From the different sources identified in Table 3-2, this study determined that Turkey was grouped in Eastern country based on culture perspective. Turkey was majority of Muslim and steeped in Middle Eastern traditions, which was closer to Eastern country (Mirabella, 2016).
Education level categories were derived and modified from the International Standard Classification of Education (ISCED, 2011). Eight categories of education levels were identified in the coding of this study: (a) Preschool, (b) Grade 1-6, (c) Grade 7-9, (d) Grade 10-12, (e) Undergraduate, (f) Graduate, (g) Postgraduate, (h) Not elsewhere classified.
Three categories of identities of participants were identified in the coding: (a)
Pre-service teacher, (b) In-service teacher, or (c) Learner. All of the participants’ categories were derived from the selected articles.
Twelve categories of subject domains were identified in the coding: (a) Chinese, (b) Design, (c) English, (d) Geography, (e) History, (f) Language, (g) Mathematics, (h) Pedagogy, (i) Science, (j) Technology, (k) Visual Art, or (l) Not specified. All the subject domain
categories were derived from the selected articles.
Table 3-1
Demographic Coding
Item Category
Country 1 Eastern country 2 Western country
Education level 1 Preschool 2 Grade 1-6 3 Grade 7-9 4 Grade 10-12 5 Undergraduate 6 Graduate 7 Postgraduate 9 Not elsewhere
classified Identities of
participants
1 Pre-service teacher 2 In-service teacher 3 Learner
Subject domains 1 Chinese 2 Design 3 English
4 Geography 5 History 6 Language
7 Mathematics 8 Pedagogy 9 Science 10 Technology 11 Visual Art 12 Not specified
‧
Turkey as Eastern Country or Western Country Supported by Different Sources Turkey is marked as country of
Eastern Western Remarked
The IUCN Red List partly in Asia and Europe
Worldatlas (n.d.) X Asia
Mirabella (2016) X X Geographic of
Turkey was closer to Europe, but culture
3.3.3 Research Design Coding
Table 3-3 shows the research design coding included in this study. There were two items that included research designs and analysis methods among these selected KB studies. A total of seven categories of research designs were identified in the coding of this study, which include (a) Case study, (b) Design experiment, (c) Ethnography, (d) True experiment design,
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
31
(e) Quasi-experiment design, (f) Exploratory study, and (g) Longitudinal study. All the research design categories were derived from the selected articles. Hence, each selected article research design was coded from the description of methodology section. However, there were 22 articles that did not clarify the research design employed. To solve the coding problem on these research designs, some solutions were proposed to be discussed with the inter-coder through face-to-face contact.
In addition, three categories of analysis methods approaches were identified in the coding: (a) Quantitative analysis, (b) Qualitative analysis, and (c) Mixed methods analysis.
These categories of analysis method approaches were derived from the selected articles.
Quantitative analysis was conducted by adopting statistical methods in analyzing data, for example pre- and post-tests, correlation analysis, analysis of variance (ANOVA), and so forth.
Qualitative analysis was conducted by analyzing data from interviews, classroom
observations, written productions or sketches, and so forth. Mixed methods analysis was conducted by applying both quantitative analysis methods and qualitative analysis methods in analyzing data. For instance, ID27b involved adopting qualitative content analysis in
analyzing content of discussion of learners and, ID27b also used a process of adopting t-tests for comparing two groups from the level of participation. Qualitative findings supported by quantitative results were found in most of the KB studies that adopted mixed methods
analysis. During the coding process of analysis methods, each article was validated according to the description of the data analysis section.
‧
Research designs 1 Case study 2 Design experiment 3 Ethnography 4 Exploratory study 5 Longitudinal study
6 Quasi-experiment design
7 True experiment design
Analysis methods 1 Quantitative analysis
2 Qualitative analysis
3 Mixed methods analysis
3.3.4 Online Learning Platform Coding
Table 3-4 shows the online learning platform coding included in this study. Two
categories of online learning platforms were identified in the coding: (a) KF and (b) Non-KF.
All the online learning platform categories were derived from the selected articles. During the coding process on online learning platforms, each article was validated according to the description of the online learning environment.
Table 3-4
Online Learning Platforms Coding
Item Category
Online learning platforms 1 KF 2 Non-KF
3.3.5 Knowledge Building Principles (KBPs) Coding
Table 3-5 shows the KBPs coding included in this study. The KBP categories were derived from Scardamalia and Bereiter (2010). Twelve categories of KBPs were identified in the coding: (a) Real ideas, authentic problems, (b) Improvable ideas, (c) Idea diversity, (d) Rise above, (e) Epistemic agency, (f) Community knowledge, collective responsibility, (g) Democratizing knowledge, (h) Symmetric knowledge advancement, (i) Pervasive KB, (j) Constructive uses of authoritative sources, (k) KB discourse, (l) Concurrent, embedded, and transformative assessment. During the coding process on KBPs, each article was validated
‧
according to the description in the following condition: (a) Literature review section, (b) Online learning platforms and KB tools, and (c) Pedagogical designs. All the ambiguous coding problems were solved via inter-coder agreement through face-to-face discussions.
Table 3-5 KBPs Coding
Item Category
KBPs 1 Real ideas, authentic problems
2 Improvable ideas 3 Idea diversity
4 Rise above 5 Epistemic agency 6 Community knowledge, collective responsibility
10 Constructive uses of authoritative sources
11 KB discourse 12 Concurrent, embedded and transformative assessment
3.3.6 Learning Outcomes Coding
The learning outcomes categories were derived from the selected articles and compared with the 21st century skills competencies (National Research Council, 2012).The National Research Council is an authoritative non-profit institution that publishes valuable articles. For this reason, this study chose the data source above as a main reference for comparing learning outcomes categories for this study. Three categories of learning outcomes were identified in the coding: (a) Cognitive domain, (b) Metacognitive domain, and (c) Social domain. Each category consisted of two to three subcategories which were discussed as coding categories with an inter-coder. These subcategories consist of basic and higher-order learning outcomes.
Basic cognitive learning outcomes include basic literacy and domain-specific literacy, while epistemic literacy was marked as a higher-order cognitive learning outcome. Basic
metacognitive learning outcomes involved reflection, while self-regulation and problem
‧
solving were parts of higher-order, metacognitive learning outcomes. A basic social learning outcome included interaction or participation, while metadiscourse was marked as a
higher-order social learning outcome. Table 3-6 illustrates the descriptions of the subcategories of learning outcomes coded in this study.
Table 3-6
Subcategories Learning Outcomes
Category Subcategory Description
Cognitive
Cognitive