Experiment 2

Methods

Another teaching activity was arranged to investigate student’s reflective journal.

Each student has a blog to make a reflective journal entry. The topic to be studied on the student’s reflective journal is about their thinking level. There are some researches that dealt with the definition of the thinking level. In this thesis, Bloom’s taxonomy is adopted. Bloom (1956) stated the taxonomy in which cognitive domain can be classified into the following levels in order: knowledge, comprehension, application, analysis, synthesis and evaluation according the development procedure of cognitive ability and complexity of learning. Based on Bloom’s model, thinking levels are classified into low and high level thinking. The knowledge, comprehension and application belong to low level thinking, while analysis, synthesis and evaluation belong to high level thinking. To help students catch the core of Bloom’s taxonomy, an example was given and explained to students to make sure they do understand the real meaning of each thinking level in Bloom’s taxonomy. The purpose is to teach students how to make an accurate reflection. The following problems are to be checked:

z Does high level thinking content exist on learning reflection blog?

z Does high/low of thinking level have a correlation with learning attitude?

z Does high/low of thinking level have a correlation with learning performance?

z Does positive/negative learning attitude have a correlation with learning performance?

The thirty-four senior students that took part in a course were computer and information science majors at a technological college in Taiwan. Two of them are

female and the rest are male. The course was a three credit course called “Artificial Intelligence”. The lecturing period was fourteen weeks. At the beginning of the course, a blog was allocated for each student. Students were asked to replace the outlook (user interface) with templates provided in the system or selected from outside world.

This was a mandatory task to ensure each student knew where his/her blog was and how to manage it. Students were requested to fill out a learning attitude inventory on Artificial Intelligence (H=0.83) which were scored on a 5-point Likert Scale, ranging from “most positive attitude” (5 points) to “most negative attitude” (1 point) at the first week. During this lecturing period, students were asked to express their learning reflection on a blog about their learning per week. Teacher encouraged students to make high level thinking on their learning. Student’s reflection was recorded and analyzed to see which thinking level it belonged to. At the end of the course, a final exam was given and the students were requested to fill out the same attitude inventory as when the course began. In addition, an anonymous user satisfaction survey as well as an open problem questionnaire were conducted to understand students’ comments about this system and activity.

Results and discussions

Does high level thinking content exist on learning reflection blog?

First of all, we checked whether the student’s reflective journal contains high level thinking. Based on Bloom’s taxonomy, the content of student’s learning reflection is investigated. The results showed that high level thinking contents existed on the student’s learning reflection. Students not only summarized the learning content in their learning reflection, but also provided critical thinking, made extend learning, and provided viewpoints different from textbook about learning subject. Blog provides

comment and Trackback functions, many students gave comments and viewpoints on classmates’ reflection with these functions. Traditionally, students might keep these ideas or viewpoints in mind, or discuss with the teacher or a few of their classmates.

With blogs, students can make these ideas or viewpoints public with ease, and discuss with their teacher and as many as their classmates.

Does high/low thinking level have correlation with learning attitude?

We want to see whether students have a high level thinking on reflection is related to whether they are toward the positive learning attitude. That is, are students with high level thinking likely to be toward the positive learning attitude? The student’s learning attitude is classified as toward the positive and negative learning attitude. The criterion used to make classification is a mean value of learning attitude survey (M=3.45, SD=0.43 at the beginning; M=3.44, SD=0.49 at the end). The transition of the student learning attitude from beginning to the end of course is checked. If a student’s learning attitude remains positive or changes from negative to positive, then the learning attitude is toward the positive. If a student’s learning attitude remains negative or changes from positive to negative, then the learning attitude is toward the negative. In addition, thinking levels are divided into three groups:

z High level: a blog contains high level thinking reflections

z Low level: a blog does not contain high level thinking reflection but only low level ones.

z Not available: a blog does not contain any meaningful or course relative reflections. It may contain some entries other than course-related subjects.

A 2x3 contingency table was used to represent the relationship between thinking

levels and learning attitude. With SPSS 10.0, the result of a Pearson chi-square test of independence achieved level of significance (χ²(2)=6.795, p=.033<.05). Therefore, these two variables are associated. That is, whether students are toward the positive learning attitude depends on whether they present high level thinking on their reflection.

Does positive/negative learning attitude have correlation with learning performance?

To check the effectiveness of learning attitude on learning performance, an independent samples t test was used. The learning performance was score on a student’s final term exam; and students were divided into two groups based on learning attitude. The Levene's test for equality of variances was not significant (F=.503, p=.483>.05), the two variances are not significantly different. The result of the t test was significant (t(32)=12.69, p=.0 <.05). Therefore, there is a significant difference between the learning attitude and learning performance. Students who are toward the positive learning attitude have significantly higher learning performance than those who are toward the negative learning attitude.

Does high/low of thinking level have correlation with learning performance?

A one-way ANOVA with independent samples was tested to understand the difference of learning performance between three groups of thinking levels. As previous, the learning performance was based on the score of a student’s final term exam; and students were divided into three groups based on their thinking levels. The Levene’s test of homogeneity was not significant, the homogeneity assumption was held. The result of “between groups test” achieved level of significance (F(2,31)=9.275, p<0.001).

It means different thinking levels on a blog affects learning performance. With

Tukey’s HSD Post-hoc Tests, the score of high level thinking group (M=68.3) is significantly higher than low level thinking group (M=48.5) and not available group (M=31.3). It implied that students who made high level thinking on their reflection have better learning performance. In summarizing the previous analysis, we can conclude that learning reflections and learning performance have a correlation. That is, a student expresses high level thinking on reflection has better learning performance.

This conclusion induces an issue about the causal relation between thinking levels and learning performance. That is, whether students made high level thinking on the blog first, then the learning motivation and attitude was strengthened. In turn, it contributes to the better learning performance, or vise versa. We will use user satisfaction survey to verify this.

User satisfaction survey

Upon the completion of the course, an anonymous user satisfaction survey which was arranged with a 6 point Likert scale was sent to students to understand the attitude about this system. An open problem questionnaire was also given to students.

Students gave comments and ideas with their name borne on the open problem questionnaire when they return it.

Table 6-4: user satisfaction survey

Items 6* 5* 4* 3* 2* 1* mean

1 I feel that it is a good experience for expressing learning reflection on a blog.

14% 50% 35% 0% 0% 0% 4.8

2 I feel that learning reflection has a positive influence on my learning.

18% 32% 43% 7% 0% 0% 4.6

3 I anticipate receiving classmates’ 4% 50% 36% 7% 3% 0% 4.2

comments or feedback on my reflection.

4 I anticipate receiving teacher’s comments or feedback on my reflection.

4% 43% 46% 4% 33% 0% 4.4

5 It doesn’t matter if learning reflection is a part of a formal assessment. It does not affect my attitude about blogs.

7% 32% 25% 15% 18% 3% 3.8

6 I feel that it has a positive influence on my learning to view classmates’ learning reflections teacher see the feedback or comments I give to classmates.

11% 35% 44% 7% 0% 3% 4.4

9 I feel that this blog is easy to use. 7% 29% 39% 18% 7% 0% 4.1

*6:strong agree??5:agree??4:little agree??3little disagree??2: disagree??1: strong disagree

By looking into the anonymous survey, almost 88% of students thought that learning reflection has a positive impact on their learning. It can be used to explain the causal relation between the learning reflection and learning performance, in which high learning performance is induced by a good learning reflection. Another finding on this survey is that most of students express highly positive comments on this learning blog;

and students eagerly anticipate responses to their reflection on blogs from their teacher and classmates.

There were six students that failed this course, four of them never expressed reflection on their blog, and the rest only made a couple of reflection entries. A questionnaire was given to these students to collect their comments; the result is shown in table 2.

These students’ attitudes were not good after viewing this table. Another investigation was done to check these students’ learning attitude. These students’ value of learning attitude inventory were 3, 2.85, 2.8 and 3.4 (M=3.44, SD=0.49 globally). It is obvious that these students’ learning attitude was toward the negative and they were not willing to reflect on their learning, it resulted in their poor learning performance.

Table 6-5: survey result of students who were failing on this course Question Under what condition, will

you take learning

When it does really need.

I do not know.

* Department allocates a dedicated Notebook to each student since sophomore year, and campus-wide wireless access is available. It is an excuse for their behavior.

In addition, an analysis was made on the top ten students. The result showed that the score of learning attitude inventory of these ten students (M=3.6, SD=0.4) is higher than the mean score that means their learning attitude towards to positive (M=3.44, SD=0.49). Taking the number of high level thinking entries as measure, the quantitative score of these students on blogs is (M=5.2 SD=2.9) higher than global (M=2.3, SD=2.87). It showed that these students actively participated in blog activity.

Another open ended questionnaire was sent to these top ten students to understand their comment on blog activity; several positive comments are summarized as follows:

blogs help active learning

what I am going to learn….

blogs help me to review the course

…it lets me understand my own learning situation, share my learning experience, understand how others learn, and find out the way to improve my learning….

blogs help cooperative learning

…. Students share their learning experience, raise problems, or discuss on the blog….

…. It always has some additional reward by reading others’ reflection, such as missing lecturing content on class or different viewpoint or thinking. Others’ thinking sometimes inspires me…..

blogs help me to think about learning

…in addition to think about some things, it helps me clarify what I do not understand.

I sometimes objectively thought that I have understood a concept taught in class, but actually have not. With this platform, I can let teacher knows what I do not know, my viewpoint, or thinking, then the teacher may give a further explanation to me…

By viewing the user satisfaction survey, students express positive attitude on this kind of open learning reflection. They not only enjoy suggestions about their learning reflections from classmates, but also comment and response from their teacher and classmates. For such a requirement, it is difficult to realize with traditional paper and pencil style learning reflection.

6.5 Summary

Many students may not make learning reflections frequently; therefore properly

introducing a learning reflection activity into a course with convenient tools (such as blogs) will give positive impact on a student’s learning. Assisting student conduct learning reflection with ICT technology not only lets student makes a learning reflection, but also facilitates peer collaborative learning. In this thesis, high level thinking contents are found on student’ reflection; and an association between the difference of thinking levels on reflection and learning attitude is found. That is, whether students are toward the positive learning attitude depends on if they present high level thinking on reflection. Consequently, students’ learning reflection and learning performance have causal relation in which high learning performance is induced by a good learning reflection; and there is an association between students’

learning attitude and learning performance. In short, students who have a better learning attitude on the Artificial Intelligence subject show better learning performance. We also found that learning performance is induced by learning attitude with the analysis of anonymous and open questionnaire.

In summary, it has been shown that learning performance can be promoted with learning reflection at Artificial Intelligence course for senior students. Students make their learning reflection with a convenient blog environment; most of the students showed positive attitude on open and anonymous questionnaires. Students who have better learning performance show more positive attitude than the average, and the quantitative score of learning reflection on blogs is much better than the average value.

Chapter 7 Conclusions

In this thesis, a portfolio centric learning platform was introduced and discussed. This learning portfolio serves as a central repository and information manipulator for other learning modules. Several learning modules at this platform are classroom response system module, learning status report module, mastery learning module, and learning reflection module.

Learning status report on the basis of learning portfolio provides students a valuable reference and information about their learning situation, which is sometimes ignored until too late to adjust. This information is served as a reference before class. A hybrid type classroom response system is a way to leverage student learning comprehension and promote interactivity in class. The results of experiment on the course entitle

“programming language and practices” showed that it did promote learning comprehension which was also conclusion of many similar researches (Duncan, 2004;

Roschelle, 2003). In addition, it provides a more flexible way, which combines SMS service and web-based approaches, to adopt the learning situation. When students notify that their poor learning situation before class, they may take time to review the taught material and preview the material to be taught. Learning reflection and mastery learning are two approaches for the learning after class. The experiment result shown that learning reflection has positive effect on student learning outcome and learning attitude. In addition, mastery learning provides an effective way to allow students get mastery on the learning material after class.

The learning paradigm shifts from traditional learning, E-learning, M-learning, and currently U-learning. Come with this movement, the learning and/or teaching process

or activity should be transformed or modified to make learner gains better learning outcomes. U-learning paradigm opens a lot of opportunities and challenges for educators and researchers. One of these challenges is how to keep learning portfolio up to date and make portfolio more intelligent, an agent mechanism is a candidate to overcome this problem. With such a mechanism, no matter what the front-end interface learner faces is and what environment learner stays is the learning process and activity can be recorded automatically and efficiently. In addition, the content of learning portfolio not only can serve as a learning record, but also as guideline or recommendation for learner’s further learning process. At the assistant of smart portfolio mechanism, the learning activity of ubiquitous learning will have better learning performance and outcome.

References

Anderson, T. (2002). An updated and theoretical rationale for interaction, IT Forum, retrieved from http://it.coe.uga.edu/itforum/paper63/paper63.htm, date:

5/7/2007.

ATutor. (2001). ATutor Learning Content Management System, retrieved from http://www.atutor.ca/ dated 3/1/2007.

Awwad, Y., Lin, C. F., Lin, H. T., & Yuan, S. M. (2006). The Construction of Information Gathering with SMS Messages, WSEAS Transactions on Communication, 5(6), 1208-1213.

Bandura, A. (1977). Social learning theory. Englewood Cliffs, NJ:Prentice Hall.

Bannan-Ritland, B. (2002), Computer-mediated communication, elearning, and interactivity: A review of the research, Quarterly Review of Distance education, 3(2), 161-179.

Barrett, H. (2005). Researching Electronic Portfolios, retrieved from http://electronicportfolios.org/blog/2005_01_01_eportfolios_archive.html, date:

5/7/2007.

Barrett, H. (2003). Electronic portfolios. In A. Kovalchick and K. Dawson (Ed.), Educational technology: an encyclopedia, Santa Barbara: ABC-CLIO: 171-176.

Block, J. H., Efthim, H. E., & Burns, R.B. (1989). Building Effective Mastery Learning Schools, New York: Longman.

Blood, R. (2004). How Blogging Software Reshapes The Online Community, Communications of the ACM, 47(12), 53-55.

Bloom, B.S. (1981). All Our Children Learning, New York: McGraw-Hill.

Bloom, B.S. (1968). Learning for mastery, Evaluation Comment, 1 (2), 1-5.

Bloom, B.S., Engelhart, M.D., Furst, E.J., Hill, W.H., & Krathwohl, D.R. (Eds.) (1956). Taxonomy of Educational Objectives. The Classification of Educational Goals, Handbook I: Cognitive Domain, New York: David McKay Company.

Bollen, L., Eimler, S., & Hoppe, H. U. (2004). SMS-based discussions: technology enhanced collaboration for a literature course. 2nd IEEE international workshop on wireless and mobile technology in education, Taiwan.

Carroll, J.B. (1989). The Carroll model: A 25 year retrospective and prospective view, Educational Researcher, 18(1), 26-31.

Carroll, J. B. (1963). A model of school learning, Teachers College Record, 64, 723-733.

Cayzer, S. (2004) Semantic Blogging and Decentralized Knowledge Management.

Communications of the ACM, 47(12), 47-52.

Chamberlain, R., & Vrasidas, C. (2001). Creating engaging online instruction, Proceedings of the 17^th Annual Conference on Distance Teaching and Learning, Madison, WI: University of Wisconsin System, 79-83.

Chang, C. C. (2001). Construction and Evaluation of a Web-Based Learning Portfolio System: An Electronic Assessment Tool. Innovations in Education and Teaching International, 38(2), 144-155.

Chen, T. S. (2007). Reviewing the trend of higher education in PRC from teaching evaluation, 2007 Conference on Professional, Quality, and Excellent Development of Education, Yuanpei University, Hsinchu, Taiwan

Chang, M., & Lo, M. (2000). Portfolio Assessment and the Professional Growth of Teaching Interns. Chinese Journal of Science Education, 8(3), 225-249.

Chen, G. D., Liu, C. C., Ou, K. L., & Lin, M. S. (2003). Web Learning Portfolios: A Tool for Supporting Performance Awareness. Innovations in Education and Teaching International, 38(1), 19-30.

Chou, C. (2003) Interactivity and interactive functions in web-based learning systems:

A technical framework for designers, British Journal of Educational Technology, 34(3), 265-279.

Conrad, R. M., & Donaldson, J. A. (2004) Engaging the Online Learner, CA: Jossey Bass.

Cox, W.F. & Dunn, T. G. (1979). Mastery learning: A psychological trap?,

DGBAS (2007). Education Situation: Monthly Bulletin of Statistics, Directorate -General of Budget, Accounting, and Statistics, Executive Yuan, ROC retrived from http://eng.dgbas.gov.tw/public/data/dgbas03/bs7/bulletin_eng/D-1.xls date:

7/25/2007

De Fina A. (1992). Portfolio Assessment: Getting Started, NY: Scholastic Professional Books.

Driver, R. (1995). Constructivist approaches to science teaching. On Steffe, L. & Gale J. (Eds) Constructivism in Education, NJ: Lawrence Erlbaum Associates, 385-400

Du, H. S. & Wanger, C. (2007). Learning with Weblogs: Enhancing Cognitive and Social Knowledge Construction, IEEE Transactions on Professional Communication, 50(1): 1-16.

Duncan, D. (2004). Clickers in the Classroom, San Francisco, CA: Pearson Education Electronic Portfolio White Paper (2003). Retrieved from http://eportconsortium.org/

date: 5/7/2007.

Elixus. (2003) BLOG: The Implementation of On-Line Publishing, and Weblog, Taipei:

Grandtech C. G. Systems Inc.

Erickson, J., & Siau, K. (2003), E-ducation, Communication of ACM, 46(3), 134-140 Fogg, B. J. (2003) Persuasive Technology: Using Computers to Change What We

Think and Do, CA: Morgan Kaufmann.

Garelli, S. (2003). Competitiveness of nations: the fundamentals, IMD World

Competitiveness Yearbook 2003, retrieved from http://www.imd.ch/documents/wcy/content/Fundamentals.pdf dated 6/20/2007

Garner, I, Francis, J and Wales, K. (2002). An Evaluation of the Implementation of a Short Message System (SMS) to Support Undergraduate Student Learning, Proceedings of European Workshop on Mobile and Contextual Learning (MLearn 2002), Birmingham.

Gordon, S. (2006). Rise of the BLOG, IEE Review, March: 32-36.

Guskey, T. R. (1997). Implementing Mastery Learning, Wadeswoth Publishing.

Hadjerrouit, S. (2005). Learner-Centered Web-Based Instruction in Software

Hadjerrouit, S. (2005). Learner-Centered Web-Based Instruction in Software