CHAPTER 3: METHODOLOGY
3.6 Data Analysis
After calculating Cohen’s kappa coefficient through the five steps (referred to in Appendix B) provided by Andale, Cohen’s kappa coefficient needed to follow the
interpretation of Cohen’s kappa in Table 3-7. Eventually, Cohen’s kappa coefficient k = 0.8 indicated substantial agreement among the coders.
Table 3-7
Interpretation of Cohen’s Kappa
Kappa Agreement
< 0 Less than chance agreement 0.01 – 0.20 Slight agreement
0.21 – 0.40 Fair agreement 0.41 – 0.60 Moderate agreement 0.61 – 0.80 Substantial agreement 0.81 – 0.99 Almost perfect agreement
Note. Adapted from “Understanding interobserver agreement: The kappa statistic,” by A. J.
Viera and J. M. Garrett, 2005, Family Medicine, 37, p. 362.
3.6 Data Analysis
Table 3-8 illustrates the analysis used according to the research questions. All the coding was handled in Microsoft Excel. Overall, the data analysis in this study adopted a descriptive analysis to reveal findings in the following chapter.
Table 3-8
Data Analysis with RQs
RQ Data Analysis
RQ1: What is the overall trend of KB studies over the past 16 years? Descriptive analysis RQ2: What common methodology is employed in KB studies? Descriptive analysis RQ3: What kind of online learning platform is commonly employed
in KB studies?
Descriptive analysis
RQ4: What KBPs are commonly used in KB studies? Descriptive analysis RQ5: What learning outcomes are commonly examined in KB
studies?
Descriptive analysis
‧
This chapter presents the findings collected from 56 selected KB studies that met the selection criteria mentioned earlier in this study. Descriptive analysis was adopted to illustrate findings with research questions respectively.
4.1 Current Trend of KB Studies
4.1.1 Patterns of Countries Where KB Studies Were Conducted
Table 4-1 shows that all 12 countries that conducted KB studies maintained active contribution to the journals over the past 16 years. Canada, Hong Kong, Singapore, and Taiwan were among the top four countries, while Italy, Japan, Australia, Finland, Turkey, Mainland China, Mexico, and the United States showed an emergent interest in KB pedagogy.
Based on the data collected, study ID559a was conducted with a cohort of graduate students from Canada and Hong Kong in 2007. Hence, the total of KB studies countries equaled n =57.
Among these countries, 53% were Eastern countries (n = 30) and 47% were Western countries (n = 27), which are shown in Figure 4-1.
Table 4-1
Ranking of Countries and Distribution Frequencies in Three Time Periods Time period
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 4-1 KB Studies Countries
Figure 4-2 shows that there was an upward trend for both Eastern countries and Western countries in the number of KB studies conducted over the three time periods. However, the number of KB studies conducted in Eastern countries is higher than Western countries between 2006 and 2016.
Figure 4-2 Trend of KB Studies Between Eastern and Western Countries in Three Time Periods
To explore the findings above, this study analyzed the trend of KB studies among 12 countries over three time periods. Figure 4-3 shows that with the exception of Australia,
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
39
Canada, Finland, Hong Kong, Japan, Singapore, Turkey, and United States, all the other countries showed an upward trend in the number of KB studies between 2001 and 2016.
Among these countries, Italy showed a transition from 0 to 5 in number of KB studies after the year 2011. Mainland China and Mexico also showed a transition from 0 to 1 in number of KB studies after 2011. Taiwan presented a rapid growth from 1 to 6 in the number of KB studies after 2011.
On the other hand, Figure 4-3 shows a surprising finding that Canada (n = 10) has the highest number of KB studies among all countries between 2006 and 2010. Although Canada shows a downward trend in the number of KB studies after the year 2011, the country that conducted the most KB studies was Canada (n = 16) over the three time periods. Another finding revealed that Australia, Hong Kong, Singapore, and Turkey had sustained the number of KB studies between 2006 and 2016.
Figure 4-3 Trend of KB Studies Among Countries Over Three Time Periods
4.1.2 Education Levels of KB Studies Participants
As the data shows, study ID339 was conducted using Grade 7-9 and Grade 10-12 participants. Hence, the total participants of KB studies equaled n = 57. Table 4-2 illustrates
‧
that 36.84% of KB studies engaged Grade 1-6 participants (n = 21), followed by 28.07%
involving undergraduate (n = 16) and 10.53% involving Grade 10-12 participants (n = 6). In contrast to the three educational levels stated above, 3.51% of KB studies engaged preschool (n = 2), Grade 7-9 (n = 2), graduate (n = 2) and postgraduate participants (n = 2) respectively.
Table 4-2
Percentage of Education Levels of Participants and Distribution Frequencies in Three Time Periods
Time period
Education level 2001-
2005
To explore the above findings, we had analyzed the trend of KB studies among eight educational levels over three time periods. Figure 4-4 shows that the number of KB studies that engaged Grade 7-9 and undergraduate participants showed an upward trend between 2006 and 2016. Grade 7-9 participants showed a transition from 0 to 2 in number of KB studies after the year 2011. Undergraduate participants presented a rapid growth from 3 to 12 in the number of KB studies after 2011.
On the other hand, Figure 4-4 shows a finding that the number of KB studies that
engaged Grade 1-6 participants (n = 13) was the highest between 2006 and 2010. Despite that the number of KB studies that engaged Grade 1-6 participants showed a downward trend after the year 2011, the most KB studies involved engaging in Grade 1-6 participants (n = 21) over three time periods. Another finding revealed that number of KB studies that involved preschool, Grade 10-12, and postgraduate participants was sustained between 2006 and 2016.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
41
Figure 4-4 Trend of KB Studies Among Educational Levels Over Three Time Periods
Figure 4-5 shows that most KB studies conducted in Eastern countries engaged Grade 10-12 and undergraduates as participants. In contrast, most of the KB studies conducted in Western countries engaged Grade 1-6 students as participants.
Figure 4-5 Comparison of Eastern Countries and Western Countries in Educational Levels
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
To look into participants’ identities in KB studies, Figure 4-6 shows that 67% were learners (n =37) and 33% were teachers (n =18). To explore the finding above, we analyzed the trend of KB studies among participants’ identities over three time periods. Figure 4-7 showed an upward trend for both learners and teachers over the three time periods. However, between 2006 and 2016 the number of KB studies that engaged learners was higher than those with teachers.
Figure 4-6 Participants’ Identities Engaged in KB Studies
Figure 4-7 Trend of KB Studies Among Participants’ Identities Over Three Time Periods
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
43
Table 4-3 illustrates that 78% of KB studies in Eastern countries engaged teachers (n = 14), while only 22% of KB studies in Western countries engaged teachers (n = 4). To explore this finding, this study analyzed the trend of KB studies that engaged teachers from Eastern countries and from Western countries over three time periods. Figure 4-8 shows that the number of KB studies that engaged both teachers from Eastern countries and teachers from Western countries was sustained between 2006 and 2016. However, the number of KB
studies that engaged teachers in Eastern countries was higher than those involving teachers in Western countries between 2006 and 2016.
Table 4-3
Percentage of Participants’ Identities of Teachers in Eastern and Western Countries and Distribution Frequencies in Three Time Periods
Time period
Country 2001-
2005
2006- 2010
2011- 2016
Total (%)
Eastern 0 7 7 14 (78.00)
Western 0 2 2 4 (22.00)
Figure 4-8 Trend of KB Studies Between Eastern Countries Teachers and Western Countries Teachers Over Three Time Periods
‧
4.1.3 Subject Domains of KB Studies
The results in Table 4-4 indicate a preference by KB studies researchers. 30.14% of KB studies were related to Science (n = 22), followed by 17.81% that were related to Technology (n = 13), 16.44% were related to Pedagogy (n = 12), 12.33% were related to History (n = 9) and 5.48% were related to Mathematics (n = 4).
Table 4-4
Percentage of Subject Domains and Distribution Frequencies in Three Time Periods Time period
Subject domain 2001-
2005
To explore the above findings, this study analyzed the trend of KB studies related to 12 subject domains over three time periods. Figure 4-9 shows that except for the subjects of Design, History, Language, Science, and Technology, the number of KB studies for the other subject domains showed an upward trend between 2006 and 2016. Mathematics showed a transition from 1 to 3 in number of KB studies after the year 2011. Pedagogy showed a transition from 4 to 7 in the number of KB studies after the year 2011. English and
Geography showed a transition from 1 to 2 in the number of KB studies after 2011. Chinese and Visual Art showed a transition from 0 to 1 in the number of KB studies after 2011.
On the other hand, Figure 4-9 shows a surprising finding that the number of KB studies that related to Science (n = 12) was the highest between 2006 and 2010. Although the number
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
45
of KB studies in Science showed a downward trend after the year 2011, most of the KB studies were conducted in Science (n = 22) over three time periods. The same condition was found in the number of KB studies using Technology. Another finding revealed that the number of KB studies with Design was sustained between 2006and 2016.
Figure 4-9 Trend of KB Studies Among Subject Domains Over Three Time Periods
Figure 4-10 shows that most of the KB studies conducted in Eastern countries were related to Technology. In contrast to the Eastern countries, most of the KB studies conducted in Western countries were related to History and Science.
Figure 4-10 Comparison of Eastern Countries and Western Countries in Subject Domains
‧
In summary, the number of KB studies conducted in the Eastern countries was slightly higher than that those in the Western countries over the past 16 years. However, the top country for conducting KB studies was Canada, which provided an important discussion clue for the next chapter. Another finding revealed that most of the KB studies engaged Grade 1-6 participants in the subject domain of Science. Further explanations will be discussed in the next chapter.
4.2 Patterns of Research Designs and Analysis Methods 4.2.1 KB Studies Research Designs
Table 4-5 shows the trend of research designs in KB studies over the past 16 years.
There are seven types: 55.36% of research design in KB studies involved case study (n = 31), followed by 17.86% using design experiment (n = 10), 8.93% involved exploratory study (n
= 5), 7.14% used a true experimental design (n = 4), 5.36% used quasi-experimental design (n = 3), 3.57% consisted of a longitudinal study (n = 2) and 1.79% involved ethnography (n = 1).
Table 4-5
Percentage of KB Studies Research Designs and Distribution Frequencies in Three Time Periods
Time period
Research design 2001-
2005
To explore the above findings, this study analyzed the trend of KB studies related to seven research designs over three time periods. Figure 4-11 showed that with the exception of
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
47
design experiment, ethnography, longitudinal study, and true experimental design, all other research designs in the number of KB studies showed an upward trend between 2006 and 2016. Case study showed a transition from 11 to 18 in number of KB studies after the year 2011. Exploratory study showed a transition from 1 to 4 in the number of KB studies after the year 2011. Quasi-experimental design showed a transition from 1 to 2 in the number of KB studies after the year of 2011. In addition, Figure 4-11 reveals that true experimental design in the number of KB studies did not change between 2006 and 2016.
Figure 4-11 Trend of KB Studies Among Research Designs Over Three Time Periods
4.2.2 KB Studies Analysis Methods
Figure 4-12 showed that 56% of analysis methods in KB studies used mixed methods (n
= 31); this is followed by 24% of qualitative analysis (n = 13) and 20% of quantitative analysis (n = 11).
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 4-12 Analysis Methods of KB Studies
To explore the above findings, this study analyzed the trend of KB studies among analysis methods over three time periods. Figure 4-13 showed that with the exception of quantitative analysis, all other analysis methods in the number of KB studies showed an upward trend between 2006 and 2016. Mixed methods analysis showed a transition from 11 to 20 in number of KB studies after the year 2011. Qualitative analysis showed a transition from 5 to 6 in the number of KB studies after the year 2011.
Figure 4-13 Trend of KB Studies Among Analysis Methods Over Three Time Periods
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
49
To summarize, case study was the preferred research design in KB studies over the past 16 years. Instead of quantitative analysis, KB studies researchers adopted qualitative analysis and mixed methods analysis in KB studies between 2006 and 2016.
4.3 Patterns of Online Learning Platforms
Figure 4-14 showed that 86% of the online learning platforms in KB studies was KF (n
= 48) and 14% were non-KF (n = 8). From the data collected, those non-KF examples included Moodle (n = 2), Synergeia (n = 2), Group Scribbles (n = 1), Plone (n = 1), Construe (n = 1), and FLE-Tools (n = 1).
Figure 4-14 Online Learning Platforms in KB Studies
To explore the above findings, this study analyzed the trend of KB studies among online learning platforms over three time periods. Figure 4-15 shows that non-KF platforms
employed in KB studies showed an upward trend between 2006 and 2016. However, KF employed in KB studies was higher than non-KF between 2006 and 2016.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 4-15 Trend of KB Studies Among Online Learning Platforms Over Three Time Periods
Overall, most of the researchers used KF as the online learning platform in KB studies over the past 16 years. All the KF tools aimed to achieve the goals of KB, such as built notes, annotated notes, linked notes, and so forth.
4.4 Patterns of Applying KBPs
4.4.1 Overall Trend of KBPs Employed
Table 4-6 showed the trend of 12 KBPs employed in KB studies over the past 16 years.
17.58% of KBP employed in KB studies involved community knowledge, collective responsibility (n = 48), followed by 15.75% as improvable ideas (n = 43). 12.09% for both epistemic agency (n = 33) and KB discourse (n = 33) were employed in KB studies
respectively. 10.62% for both real ideas, authentic problems (n = 29) and rise above (n = 29) were employed in KB studies respectively. The remaining KBPs included 6.96% of idea diversity (n = 19), 4.76% of constructive uses of authoritative sources (n = 13), 3.66% of democratizing knowledge (n = 10), 3.30% of concurrent, embedded, and transformative assessment (n = 9), 2.20% of symmetric knowledge advancement (n = 6) and 0.37% of pervasive KB (n = 1) was employed in KB studies.
‧
To explore the above findings, this study analyzed the trend of 12 KBPs employed in KB studies over three time periods. Figure 4-16 shows that with the exception of rise above, pervasive KB, and KB discourse, all other KBPs in the number of KB studies showed an upward trend between 2006 and 2016. Real ideas, authentic problems showed a transition from 12 to 15 in number of KB studies after the year of 2011. Idea diversity showed an increase from 5 to 13 in number of KB studies after the year of 2011. Epistemic agency showed a transition from 13 to 20 in the number of KB studies after the year 2011.
Democratizing knowledge showed a transition from two to eight in the number of KB studies after the year 2011. Constructive uses of authoritative sources showed a transition from four to nine in the number of KB studies after the year 2011. In addition, Figure 4-16 reveals that the rise above figures in the number of KB studies was sustained between 2006 and 2016.
Table 4-6
Percentage of KBPs in KB Studies and Distribution Frequencies in Three Time Periods Time period
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 4-16 Trend of KB Studies Among KBPs Over Three Time Periods
To analyze KBPs employed in KB studies, Table 4-7 shows the detail of KBPs
employed among 12 countries that conducted KB studies countries. The highest number of KBPs employed in Australia (n = 2) and Finland (n = 2) were KB discourse. The highest number of KBPs employed in Hong Kong (n = 8) and Singapore (n = 8) were improvable ideas. Epistemic agency was the highest number of KBP employed in Italy (n = 4) and Turkey (n = 2). Community knowledge, collective responsibility was the highest number of KBP employed in Canada (n = 16), Hong Kong (n = 8), Japan (n = 4), Taiwan (n = 7), and Turkey (n = 2). Among these countries, Canada (n = 90) employed KBPs the most, followed by Hong Kong (n= 47), Singapore (n= 41), and Taiwan (n = 28).
‧
Frequencies of KBPs Employed Among Conducting KB Studies Countries
KBP AU CA FI HK IT JP CN MX SG TW TR US
Note. AU is Australia, CA is Canada, FI is Finland, HK is Hong Kong, IT is Italy, JP is Japan, CN is Mainland China, MX is Mexico, SG is Singapore, TW is Taiwan, TR is Turkey and US is United States.
4.4.2 Trend of Three Categories of KBPs Employed
Figure 4-17 shows that 44% of KBPs in the KB studies were idea-related KBPs (n = 120), followed by 32% that were agent-related KBPs (n = 88) and 24% that were
practice-related KBPs (n = 65).
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 4-17 Categories of KBPs in KB Studies
To explore the above findings, this study analyzed the trend of KBPs categories
employed in KB studies over three time periods. Figure 4-18 showed an upward trend for all three categories of KBPs over three time periods. However, the number of KB studies that employed idea-related KBPs (n = 116) was higher than agent-related KBPs (n = 86) and practice-related KBPs (n = 64) between 2006 and 2016.
Figure 4-18 Trend of KB Studies Among Three Categories of KBPs Over Three Time Periods
To summarize, community knowledge, collective responsibility, and improvable ideas were frequently employed in KB studies, which met the core ideas of KB (Scardamalia, 2002;
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
55
Scardamalia & Bereiter, 2010). Another finding indicated that Canada was the country that employed KBPs the most, followed by Hong Kong, Singapore, and Taiwan. Idea-related KBPs were discovered to be the most likely to be employed in KB studies.
4.5 Trend of Learning Outcomes in KB Studies
Figure 4-19 shows that 42% of learning outcomes in KB studies were social learning outcomes (n = 47), followed by 37% of cognitive learning outcomes (n = 42) and 21% of metacognitive learning outcomes (n = 24).
Figure 4-19 Categories of Learning Outcomes in KB Studies
To explore the above findings, this study analyzed the trend of learning outcomes categories in KB studies over three time periods. Figure 4-20 showed an upward trend for all three categories of learning outcomes over three time periods. However, the number of KB studies that examined social learning outcomes (n = 45) was higher than those using
cognitive learning outcomes (n = 41) and metacognitive learning outcomes (n = 23) between 2006 and 2016.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 4-20 Trend of KB Studies Among Three Categories of Learning Outcomes Over Three Time Periods
To further explore findings above, Figure 4-21 showed that 65% of basic learning outcomes (n = 73) were examined in KB studies and 35% of higher-order learning outcomes (n = 40) were examined in KB studies.
Figure 4-21 Basic Learning Outcomes and Higher-Order Learning Outcomes in KB Studies
Table 4-8 illustrates the basic learning outcomes and higher-order learning outcomes findings in detail. Basic cognitive learning outcome (n = 25) was higher than higher-order cognitive learning outcome (n = 17). Basic metacognitive learning outcome (n = 18) was
‧
higher than higher-order metacognitive learning outcome (n = 6). Basic social learning outcome (n = 30) was higher than higher-order social learning outcome (n = 17).
Table 4-8
Percentage of Learning Outcomes in KB Studies and Distribution Frequencies in Three Time Periods
To explore the above findings, this study analyzed the trend of learning outcomes subcategories in KB studies over three time periods. Figure 4-22 showed that with the exception of higher-order cognitive learning outcome and higher-order metacognitive learning outcomes, all other subcategories of learning outcomes showed an upward trend in KB studies over the three time periods. Basic cognitive learning outcome showed a transition from 10 to 15 in number of KB studies after the year 2011. Basic metacognitive learning outcome showed an increase from three to 15 in number of KB studies after the year 2011.
Basic social learning outcome showed a transition from 12 to 16 in number of KB studies after 2011. Higher-order social learning outcome showed a transition from six to 11 in number of KB studies after the year of 2011. In addition, higher-order cognitive learning outcome was sustained between 2006 and 2016.
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Figure 4-22 Trend of KB Studies Among Subcategories of Learning Outcomes Over Three Time Periods
Overall, social learning outcomes were examined the most in the KB studies over the past 16 years. Among the subcategories of learning outcomes, all basic learning outcomes were higher than higher-order learning outcomes. Furthermore, basic social learning outcomes were examined the most in the KB studies. The rationale of these learning outcomes findings will be discussed in the following chapter.