CHAPTER 3: METHODOLOGY
3.3 Coding System
國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
After identifying the selected articles, a coding system was designed to perform further analysis. The coding details are provided in the following section.
3.3 Coding System
3.3.1 Bibliographic Coding
Two bibliographic categories were identified in coding the 56 selected articles including (a) Study ID number, (b) Publication year. These bibliographic categories were derived from coding manual examples of meta-analysis (Lipsey & Wilson, 2001). The Study ID number was coded from the initial search that yielded 843 results. Hence, the selected article ID numbers were between ID1 and ID843. There were 17 articles that included the letters a, b, or c behind the Study ID number with the conditions that these articles contain (a) at least two studies with participants of different education levels, or (b) at least two progressive studies with participants of the same education level. The publication year was directly coded with four digits, for instance 2016.
3.3.2 Demographic Coding
Table 3-1 shows the demographic coding included in this study. There were four items that included the countries where KB studies were conducted, education levels of participants, identities of participants, and subject domains researched in the selected KB studies. Twelve countries were identified in this study: (a) Australia, (b) Canada, (c) Finland, (d) Hong Kong, (e) Italy, (f) Japan, (g) Mainland China, (h) Mexico, (i) Singapore, (j) Taiwan, (k) Turkey, and (l) United States. Among these countries, two categories were identified in the coding: (a) Eastern countries, (b) Western countries. During the coding of different countries, for this study we employed two important criteria for categories. One was related to regions and another one was related to culture. Among these countries, Turkey was found to be difficult
‧
to categorize as either an Eastern country or a Western country, due to its complexity. From the different sources identified in Table 3-2, this study determined that Turkey was grouped in Eastern country based on culture perspective. Turkey was majority of Muslim and steeped in Middle Eastern traditions, which was closer to Eastern country (Mirabella, 2016).
Education level categories were derived and modified from the International Standard Classification of Education (ISCED, 2011). Eight categories of education levels were identified in the coding of this study: (a) Preschool, (b) Grade 1-6, (c) Grade 7-9, (d) Grade 10-12, (e) Undergraduate, (f) Graduate, (g) Postgraduate, (h) Not elsewhere classified.
Three categories of identities of participants were identified in the coding: (a)
Pre-service teacher, (b) In-service teacher, or (c) Learner. All of the participants’ categories were derived from the selected articles.
Twelve categories of subject domains were identified in the coding: (a) Chinese, (b) Design, (c) English, (d) Geography, (e) History, (f) Language, (g) Mathematics, (h) Pedagogy, (i) Science, (j) Technology, (k) Visual Art, or (l) Not specified. All the subject domain
categories were derived from the selected articles.
Table 3-1
Demographic Coding
Item Category
Country 1 Eastern country 2 Western country
Education level 1 Preschool 2 Grade 1-6 3 Grade 7-9 4 Grade 10-12 5 Undergraduate 6 Graduate 7 Postgraduate 9 Not elsewhere
classified Identities of
participants
1 Pre-service teacher 2 In-service teacher 3 Learner
Subject domains 1 Chinese 2 Design 3 English
4 Geography 5 History 6 Language
7 Mathematics 8 Pedagogy 9 Science 10 Technology 11 Visual Art 12 Not specified
‧
Turkey as Eastern Country or Western Country Supported by Different Sources Turkey is marked as country of
Eastern Western Remarked
The IUCN Red List partly in Asia and Europe
Worldatlas (n.d.) X Asia
Mirabella (2016) X X Geographic of
Turkey was closer to Europe, but culture
3.3.3 Research Design Coding
Table 3-3 shows the research design coding included in this study. There were two items that included research designs and analysis methods among these selected KB studies. A total of seven categories of research designs were identified in the coding of this study, which include (a) Case study, (b) Design experiment, (c) Ethnography, (d) True experiment design,
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
31
(e) Quasi-experiment design, (f) Exploratory study, and (g) Longitudinal study. All the research design categories were derived from the selected articles. Hence, each selected article research design was coded from the description of methodology section. However, there were 22 articles that did not clarify the research design employed. To solve the coding problem on these research designs, some solutions were proposed to be discussed with the inter-coder through face-to-face contact.
In addition, three categories of analysis methods approaches were identified in the coding: (a) Quantitative analysis, (b) Qualitative analysis, and (c) Mixed methods analysis.
These categories of analysis method approaches were derived from the selected articles.
Quantitative analysis was conducted by adopting statistical methods in analyzing data, for example pre- and post-tests, correlation analysis, analysis of variance (ANOVA), and so forth.
Qualitative analysis was conducted by analyzing data from interviews, classroom
observations, written productions or sketches, and so forth. Mixed methods analysis was conducted by applying both quantitative analysis methods and qualitative analysis methods in analyzing data. For instance, ID27b involved adopting qualitative content analysis in
analyzing content of discussion of learners and, ID27b also used a process of adopting t-tests for comparing two groups from the level of participation. Qualitative findings supported by quantitative results were found in most of the KB studies that adopted mixed methods
analysis. During the coding process of analysis methods, each article was validated according to the description of the data analysis section.
‧
Research designs 1 Case study 2 Design experiment 3 Ethnography 4 Exploratory study 5 Longitudinal study
6 Quasi-experiment design
7 True experiment design
Analysis methods 1 Quantitative analysis
2 Qualitative analysis
3 Mixed methods analysis
3.3.4 Online Learning Platform Coding
Table 3-4 shows the online learning platform coding included in this study. Two
categories of online learning platforms were identified in the coding: (a) KF and (b) Non-KF.
All the online learning platform categories were derived from the selected articles. During the coding process on online learning platforms, each article was validated according to the description of the online learning environment.
Table 3-4
Online Learning Platforms Coding
Item Category
Online learning platforms 1 KF 2 Non-KF
3.3.5 Knowledge Building Principles (KBPs) Coding
Table 3-5 shows the KBPs coding included in this study. The KBP categories were derived from Scardamalia and Bereiter (2010). Twelve categories of KBPs were identified in the coding: (a) Real ideas, authentic problems, (b) Improvable ideas, (c) Idea diversity, (d) Rise above, (e) Epistemic agency, (f) Community knowledge, collective responsibility, (g) Democratizing knowledge, (h) Symmetric knowledge advancement, (i) Pervasive KB, (j) Constructive uses of authoritative sources, (k) KB discourse, (l) Concurrent, embedded, and transformative assessment. During the coding process on KBPs, each article was validated
‧
according to the description in the following condition: (a) Literature review section, (b) Online learning platforms and KB tools, and (c) Pedagogical designs. All the ambiguous coding problems were solved via inter-coder agreement through face-to-face discussions.
Table 3-5 KBPs Coding
Item Category
KBPs 1 Real ideas, authentic problems
2 Improvable ideas 3 Idea diversity
4 Rise above 5 Epistemic agency 6 Community knowledge, collective responsibility
10 Constructive uses of authoritative sources
11 KB discourse 12 Concurrent, embedded and transformative assessment
3.3.6 Learning Outcomes Coding
The learning outcomes categories were derived from the selected articles and compared with the 21st century skills competencies (National Research Council, 2012).The National Research Council is an authoritative non-profit institution that publishes valuable articles. For this reason, this study chose the data source above as a main reference for comparing learning outcomes categories for this study. Three categories of learning outcomes were identified in the coding: (a) Cognitive domain, (b) Metacognitive domain, and (c) Social domain. Each category consisted of two to three subcategories which were discussed as coding categories with an inter-coder. These subcategories consist of basic and higher-order learning outcomes.
Basic cognitive learning outcomes include basic literacy and domain-specific literacy, while epistemic literacy was marked as a higher-order cognitive learning outcome. Basic
metacognitive learning outcomes involved reflection, while self-regulation and problem
‧
solving were parts of higher-order, metacognitive learning outcomes. A basic social learning outcome included interaction or participation, while metadiscourse was marked as a
higher-order social learning outcome. Table 3-6 illustrates the descriptions of the subcategories of learning outcomes coded in this study.
Table 3-6
Subcategories Learning Outcomes
Category Subcategory Description
Cognitive learning outcome
Basic Basic literacy Reading and writing competencies which identify the learning of new vocabularies.
Domain specific literacy
Subject domains competencies,
especially in Science and Mathematics.
Higher-order Epistemic literacy
Design thinking competencies which have a depth understanding of how knowledge is created.
Metacognitive learning outcome
Basic Reflection Ability to become aware through feedback from others as well as one’s self.
Higher-order Self-regulation Ability to sustain self-directed learning.
Problem solving Ability to use metacognition questions and statements to solve authentic problems.
Building on notes and reading others’
notes in KF. This category is
considered as basic interaction during the KB process.
Higher-order Metadiscourse KB discourse to sustain improvable ideas. This category focuses on higher-order interaction with others through building on rise above notes.