http://journals.cambridge.org/REC Additional services for
ReCALL:
Email alerts: Click hereSubscriptions: Click here Commercial reprints: Click here Terms of use : Click here
When technology speaks language: an evaluation of
course management systems used in a language learning
context
Wen-Kai Yu, Yu-Chih Sun and Yu-Jung Chang
ReCALL / Volume 22 / Issue 03 / September 2010, pp 332 - 355
DOI: 10.1017/S0958344010000194, Published online: 02 September 2010
Link to this article: http://journals.cambridge.org/abstract_S0958344010000194
How to cite this article:
Wen-Kai Yu, Yu-Chih Sun and Yu-Jung Chang (2010). When technology speaks language: an evaluation of course management systems used in a language learning context. ReCALL, 22, pp 332-355 doi:10.1017/S0958344010000194
Request Permissions : Click here
When technology speaks language:
an evaluation of course management systems
used in a language learning context
WEN-KAI YU, YU-CHIH SUN AND YU-JUNG CHANG
National Chiao Tung University, Institute of Teaching English to Speakers of OtherLanguages, 1001 University Rd., Hsinchu, Taiwan 300 (emails: revolutiontmr@yahoo.com.tw, sunyc@mail.nctu.edu.tw,
yujung@u.washington.edu)
Abstract
In light of the growing popularity of the use of computer management systems (CMSs) in higher education today, this study critically evaluates CMS adoption through a content-specific lens. By employing a mixed-method approach, the study examines college teachers’ and students’ experiences and perceptions of CMS adoption for language learning and teaching purposes. The findings show that despite the users’ perceived advantages of using CMSs in language courses, the systems’ lack of content-area specificity undermines many of the potential benefits. The study calls for better-rounded professional training to assist language teachers in integrating CMS functions strategically into their disciplinary pedagogy and incorporating multimedia language resources selectively to maximize the benefit of CMSs. Furthermore, the findings demonstrate the importance of developing content-specific CMSs with functions tailored toward pedagogical needs in different contexts.
Keywords: Application in subject areas, post-secondary education, computer management systems, computer mediated communication, professional development, language learning/ teaching
1 Introduction
The use of internet and computer technology to create web courses in higher education has provided new possibilities in language teaching and learning (Felix, 2005; Hubbard, 2005). Among computer technology advances, course management systems (CMSs) – with their wide range of functions in assessment, communication, and class adminis-tration – have become the most prominent and promising educational innovation since their development in the 1990s (Cappel & Hayen, 2004; Cloete, 2001; Leahy, 2004).
1.1 CMSs in higher education: Advantages and user attitudes
Today, as CMSs are well-received instructional media in higher education institutions, research abounds about different aspects of using CMSs to supplement traditional classroom instruction. Many have looked at the advantages of CMSs, including the
ways they assisted teachers in redesigning, presenting, and evaluating courses and how they enhanced teaching and learning performance by providing a convenient commu-nicative and collaborative virtual environment (Bongey, Cizadlo & Kalnbach, 2005; Chan & Robbins, 2006; Rosenberg, 2001). Specifically, Driscoll (2008) and Bongey, Cizadlo and Kalnbach (2005) pointed out that CMSs provided user-friendly platforms that saved instructors’ time by sparing them from the task of learning programming languages and setting up the programs. Perkins and Pfaffman (2006) found that the implementation of open source CMSs in a high school not only allowed teachers, parents, and administrators to stay up-to-date on the students’ learning, but also improved communication among students, parents, and school personnel at a low cost. Moreover, the social sciences instructor and pre-service foreign language teachers in Auyeung’s (2004) and van Olphen’s (2007) studies were found to hold positive views towards the use of CMSs because they saw an increase in learners’ confidence and collaboration as a result of using CMSs.
Despite the many advantages of CMSs, end-users’ satisfaction with the technology, as well as their intention to adopt it, largely depend on their attitudes towards the systems. West, Waddoups and Graham (2007) argued that what determined whether instructors move from initially using only one or two functions of CMSs to a more extensive adoption was the intuitive usability of the system – that is, whether the users were able to work with the system without having to devote a great amount of time to it. Liao and Lu’s (2008) examination of users’ intention to adopt and their continued usage of CMSs also showed a clear correlation between the users’ perceptions of CMSs’ relative advantages and compatibility with the pedagogy of the discipline. In a cross-cultural comparison, Liaw (2007) reported that, when it comes to learners’ autonomy in using CMSs, Taiwanese students cared more about the systems’ usefulness while students from the United Kingdom were more concerned with their perceived enjoyment when using the system. Thus, there is no guarantee that a CMS environment that satisfies the needs of one group of users would seem equally useful or effective to others. Never-theless, what has been found was that instructors’commitment and enthusiasm in using technology could exert a positive influence not only on their teaching efficiency, but also on their students’ satisfaction in working with CMSs, and could further alleviate lear-ners’ learning anxiety (Sun, Tsai, Finger, Chen & Yeh, 2008).
1.2 Challenges and selective adoption of CMSs
Although CMSs have been a celebrated addition to college curricula, it is not pro-blem-free. Studies have found some frustrated CMS users challenged by their lack of technical competence or technical classroom management strategies (Nijhuis & Collis, 2003; van Olphen, 2007; West, Waddoups & Graham, 2007). ‘‘Instructors will vary in terms of their handiness with any form of technology, even after several years of experience’’, noted Nijhuis and Collis (2003: 200), who cautioned that even with time devoted to utilizing the ready-made CMSs, there was no guarantee of efficient and optimal teaching and learning. Even when instructors are proficient with CMS features, the experience of trying to integrate the features into their teaching prac-tices ‘‘may mean reconsidering their own sense of what is good pedagogy, or even what the best methods are for class management and what their responsibilities
should be as teachers’’ (West, Waddoups & Graham, 2007: 18). If instructors cannot provide a well-organized learning environment, whether virtual or actual, students may also fail to use available resources and participate in learning activities (Nijhuis & Collis, 2003).
Despite instructors’ efforts to create a conducive virtual environment for learning, the instructors often did not try to adopt all aspects of CMSs in their teaching. Malikowski et al.’s (2006) and Woods, Baker and Hopper’s (2004) investigation pointed out that instructors usually used CMSs to transmit information and evaluate students, especially at the initial stage of adoption, and they chose CMS features that reflected traditions in their own disciplines (Malikowski et al., 2006). In fact, rather than relying only on an ‘all-in-one’ CMS package, instructors often opted to incorporate other supplementary resources, such as commercial grading programs, to suit their individual teaching needs. Thus, the adoption of CMSs involves a decision making process through which the instructors ‘‘weigh the advantages and disadvantages of using individual features of the tool’’ (West, Waddoups & Graham, 2007: 11).
1.3 CMSs and language learning: A gap to be filled
There has been previous investigation into the use of CMSs in different academic contexts, such as in a lecture-based undergraduate anatomy and physiology course (Bongey, Cizadlo & Kalnbach, 2005), a hotel and tourism management department (Williams, 2003), high school science (Perkins & Pfaffman, 2006), and college psy-chology and engineering courses (Sivo & Pan, 2005). In the area of language teaching, practitioners have shared how functions and features of CMSs such as Moodle and Blackboard create an interactive and community-based virtual learning environment that supplements traditional classroom-based language instruction (Brandl, 2005; Chen, Belkada & Okamoto, 2004; Priyanto, 2010; Robb, 2004; Su, 2006). However, the lim-itations of generic off-the-shelf CMSs for language learning have also been acknowl-edged by many language instructors. For instance, language teaching practitioners using Moodle, one of the most rapidly growing and commonly used CMSs around the world (Moodle Statistics, 2010), have pushed for the availability of more convenient built-in functions targeted at language instruction (i.e. learner corpora, concordancer, glossary, and oral/audio components) that do not require additional expense and/or major system server modification (Moodle Forum, 2009).
While many language teachers in ESL/EFL contexts have included CMSs in their language-teaching courses, empirical research conducted on issues related to the adoption of CMSs in language courses has been scarce. That is, other than language practitioners’ fragmented experiences related on discussion forums for CMSs, and papers that provide general system overviews and evaluation, we know little about how CMSs are actually received by language teachers and learners in classrooms. Moreover, as studies of CMSs and other e-learning technologies have been largely quantitative-based, the results are limited to numerical representations of users’ responses. According to Nunan (1992), quantitative data collection methods allow researchers to extract participants’ opinions from a comparatively larger pool of samples at a time. However, such numerical descriptions of quantitative data provide less elaborate accounts of human perception (Brown, 2001). Thus, in an effort to
provide a more comprehensive and in-depth account of how CMSs assist or limit users in their language learning and teaching efforts, as well as how users might work with the CMS functions to negotiate the best strategy of working with the systems, it is deemed important to incorporate a qualitative analysis to enrich the descriptive statistics provided by previous quantitative analysis.
2 Research goals and research questions
This study is an attempt to address the gaps and enrich our understanding of college teachers’ and students’ use of CMSs for the purpose of language teaching and learning. With a mixed method approach – a ‘‘natural complement to traditional qualitative and quantitative research’’ (Johnson & Onwuegbuzie, 2004: 14) – the study utilizes a col-lection of data from both questionnaires and face-to-face interviews with CMS users (teachers and students). The analysis focused on the student and teacher participants’ accounts of their attitudes toward and experiences with the use of CMSs in college language courses. The research questions for this study are as follows:
1. How are CMSs and other computer programs adopted in language courses? 2. What motivates the use of CMSs in language courses?
3. What are the perceived limitations of and desired technical and professional support for using CMSs in language courses?
3 Research methods 3.1 Participants
For the quantitative part of the study, a total of 53 college English teachers (Male 5 12, Female 5 41) participated. Their average age was 42.92 (SD 5 0.74), with the majority of them (53%) between 41 and 50. Regarding their teaching experience, 25% of them had 1–5 years, 21% had 6–10 years, 17% had 11–15 years, 21% had 16–20 years and 16% of them had over 21 years of teaching experience.
In terms of student participants, a total of 241 college students (Male 5 159, Female 5 82) participated in the study. The student respondents were recruited from six different English courses in two universities in Taiwan. All the student respon-dents were native speakers of Mandarin and they came from a variety of disciplines. The CMSs used by the participants in this study were Blackboard, Moodle, and E-campus 3 (E3). Blackboard is a commercial CMS that provides a ready-made template for a web course and expansion programs, offering multiple interactive functions such as e-mail, assessments, gradebook, and discussion groups. Moodle (Modular Object-Oriented Dynamic Learning Environment) is a global development project designed to promote a social constructionist pedagogy, which includes components to support collaboration among students, activity-based learning, and critical reflection. Unlike Blackboard, Moodle is free, open-source software. Therefore, in addition to offering built-in pedagogical functions and activity modules similar to those provided by Blackboard, its extendable platform allows users to edit its code, which changes the format to fit their teaching needs. Moodle also offers the
function of real-time communication among users in addition to asynchronous means of communication such as e-mail, Wiki-posting and editing, and forums. It is not only used by many instructors to supplement face-to face classes, but is also widely used in distance education. Developed by 3 Probe Technologies based in Taiwan, E-Campus 3 is used as the major web-based course management system in National Chiao Tung University along with a few other Taiwanese universities and institutes. It provides administrative and interactive functions similar to Blackboard, but additional plug-ins or extension programs are not available.
Among the teacher participants, eleven teachers were using E3, fifteen were using Moodle, and 27 were using Blackboard. The student participants came from six classes taught by three of the participating teachers (two classes from each teacher). Among the students, 80 were using Moodle and 184 were using both E3 and Blackboard. To make sure that the students were familiar with the use of CMSs in their English courses, questionnaires were administered to students after they had used one of the CMSs for several months. The survey questionnaires were dis-tributed both through email and in person. The format and focus of the ques-tionnaires for students and teachers were largely similar except that the teachers’ version asks about their use of CMSs in teaching whereas students are asked about their use of CMSs in learning (see Appendices A and B). The e-mail surveys were sent out to 287 teachers of whom 36 responded; paper surveys were distributed in person to 21 teachers of whom seventeen filled out the surveys. That is, in total, 308 surveys were sent out and 53 were completed, a return rate of 17.2%. Although not all participants were using the same CMS at the time of the study, they were considered as a single pool in the data analysis because their use of the systems in teaching/ learning English involved similar administrative, communicative, and interactive functions offered by the three CMSs.
Among the questionnaire respondents, seven out of 53 teachers agreed to participate in the post-hoc interview. These teachers all had at least two years of experience in incorporating CMSs into English courses. For students, twelve out of 241 students initially agreed to participate in the post-hoc interview. However, five of them were excluded because they were freshmen and did not have much experience in using CMSs at the time of the study. The seven students who participated in the interviews had at least two years of experience in using CMSs and they came from a variety of academic disciplines including science, engineering, and languages and literature.
3.2 Instruments
3.2.1 Questionnaire. The questionnaire in this study was comprised of three major sections: (1) background information, (2) CMSs and CALL usage, and (3) attitudes toward using CMSs in language learning/teaching. The background information section included participants’ gender, the teachers’ age and years of teaching experience or the students’ year of study, and the types of CMS used on campus. In the second section, the participants were asked to indicate their prior experience in using CALL as a teaching or learning tool and assess the frequency of use on a five point Likert scale (15never, 25seldom, 35sometimes, 45often, 55very frequently). The last section focused on the teachers’ and students’ attitudes toward the use of
CMSs in language courses. The 15 items in this section were embedded with five variables adopted from Liaw (2008): (a) perceived self-efficacy, (b) perceived enjoyment, (c) perceived usefulness, (d) behavioral intention, and (e) perceived quality of the CMSs used. The participants were asked to rate their attitudes toward CMSs based on a five-point Likert scale (1 5 strongly disagree, 2 5 disagree, 3 5 neutral, 4 5 agree, 5 5 strongly agree).
Three sets of open-ended questions were incorporated into the questionnaire to investigate the types of CMS functions used in language courses, the teachers’ teaching approaches and beliefs when using CMSs, the students’ likes and dislikes about using CMSs in language learning, and the users’ perception about the advantages of using CMSs for language learning/teaching purposes.
3.2.2 Interviews. Face-to-face interviews were conducted with the fourteen parti-cipants. Each was interviewed once and each interview took about 45 minutes. All interviews were audio-recorded and transcribed. Mandarin Chinese, the common language used in Taiwan, was used throughout the interviews. Interview protocols were prepared in advance and the questions centered on four themes: (1) definitions of and attitudes toward CMSs, (2) teaching/learning approaches and personal beliefs when using CMSs, (3) perceived strengths and weaknesses of using CMSs in English courses, and (4) creative ideas and useful suggestions for using CMSs. The interviews were semi-structured and follow-up questions and discussion were proposed based on the responses provided by each interviewee.
3.3 Data analysis
The analysis of the quantitative data was conducted with the Statistical Package for the Social Sciences (SPSS). The Cronbach’s a is 0.94 for teachers and 0.91 for students. The reliability of the questionnaire content was high. The analysis of the qualitative data started with examining the participants’ responses gathered from the open-ended questions in the questionnaire and the post-hoc interviews. Since our goal is to understand the personal experiences of students and teachers using CMSs for language learning purposes, the analysis focused on the shared themes among the participants’ responses. Moreover, since the interview questions shared related themes with and acted as an in-depth exploration of the open-ended questions in all questionnaires, the qualitative data were coded together with the research questions in mind. Qualitative analysis software was used to help organize and sort the data sets and extract salient thematic patterns.
In the presentation of the results, if the quote is from open-ended question data, it will be noted as ‘‘OT’’ or ‘‘OS’’, indicating open-ended question data from teachers and open-ended question data from students, respectively.
4 Results
The results of the data analysis are presented in three parts, corresponding to the three research questions raised at the beginning of the study. In each part, we present the findings from both the quantitative and qualitative analysis.
4.1 How are CMSs and other computer programs adopted in language courses? Based on the responses gathered from the questionnaire, Table 1 summarizes the teachers’ and students’ use of the different functions of CMSs. Among the different types of CMS functions, announcing news, uploading/downloading files, assigning/ submitting/receiving homework assignments, discussing in the forums, and assign-ing/participating in group work were the most commonly used. Teachers also used CMSs for some administrative functions, such as keeping attendance records and managing grades, or importing/exporting course design formats. On the other hand, students had more experience in using the activity functions such as writing on the message boards and taking quizzes and examinations.
Despite apparent varied usage, a closer look at the data showed that of the 25 different CMS functions listed in the questionnaire, 15 (60%) were actually used by less than 10% of the teacher respondents and 16 (64%) were used by less than 20%
Table 1 Commonly used functions on CMS
Commonly used functions
Teachers Students
N % Rank N % Rank
Course administration
Keep attendance records 16 30.1 5 23 9.5 16
Manage grades 15 28.3 6 45 18.7 9
Give grades 12 22.6 7 53 22.0 7
Import/export course design formats 11 20.8 10 25 10.4 14
Edit/design personal courses 5 9.4 11 19 7.9 19
Group students online 5 9.4 11 16 6.6 20
Change user interface 3 5.7 15 7 2.9 23
Track learning progress 1 1.9 18 21 8.7 17
Use calendar 1 1.9 18 33 13.7 12
Activities
Upload homework assignments 22 41.5 3 132 54.8 4
Discuss in the forum 21 39.6 4 142 58.9 3
(Assign/Participate in)group works 12 22.6 7 60 24.9 6
Correct/comment on assignments 12 22.6 7 42 17.4 10
Administer questionnaires/survey 5 9.4 11 34 14.1 11
Leave comment on message board 3 5.7 15 71 29.5 5
Write blog 1 1.9 18 25 10.4 14
Chat in the text chat room 1 1.9 18 33 13.7 12
Hold online conference 0 0.0 23 5 2.1 25
Give/Take quizzes 5 9.4 11 53 22.0 7
Vote for issues/topics 0 0.0 23 14 5.8 21
Chat with instant message 3 5.7 15 11 4.6 22
Resources
Announce news 34 64.2 1 216 89.6 1
Upload/download document files 34 64.2 1 200 83.0 2
Make interview appointments 0 0.0 23 20 8.3 18
of the student respondents. That is, a significant portion of the functionality pro-vided by a CMS was rarely used in these English courses. In fact, rather than solely relying on CMSs to provide comprehensive functions, teachers and students were drawing on additional computer programs and online resources to help them teach and learn English. Table 2 presents their use of other computer programs and online resources in English courses.
The data from the questionnaire showed that the use of PowerPoint in English courses was the most common among teachers, followed by learning websites, YouTube, and online dictionaries. For the student respondents, the use of an online dictionary in English courses was the most common, followed by YouTube, learning websites, and Power-Point. The qualitative data also showed teacher respondents’ efforts to incorporate other resources into CMSs to strengthen their capability in language education. As one tea-cher put it when answering the open-ended questions in the questionnaire, ‘‘Basically, the CMS acts as a base from which I create links and extensions to other resources’’ (OT). Teacher interviewees reported using exam creators (e.g. ExamView, AuthorPlus), online recording programs, as well as adding links to online photo albums and online language games to complement the use of CMSs in their language courses. Many also prepared lectures in Word documents and PowerPoint slides so that the digitized lecture notes could be uploaded to CMSs for students to access and review after class.
The findings on how computer programs, other online resources, and CMSs were integrated in language courses illustrate that teachers and students were flexible and creative in utilizing technology in their learning and teaching efforts. Echoing West, Waddoups, and Graham’s (2007) findings, some teachers would adopt CMSs selectively according to their teaching objectives rather than using all the available tools provided in the package. As one of the teacher interviewees commented,
‘‘For me, I pick only the useful functions for language teaching [on CMSs], the ones useful to help me teach English. I don’t think I need to incorporate all the functions’’ (T2).
Table 2 The use of additional computer programs and online resources in language courses (From 1 ‘‘never’’ to 5 ‘‘very frequently’’)
Items Participants M SD
PowerPoint Teachers 3.51 1.22
Students 2.55 1.04
Instant messenger Teachers 1.70 1.03
Students 1.46 0.80
Blogs Teachers 1.85 1.13
Students 1.53 0.74
YouTube Teachers 2.38 1.26
Students 2.92 1.00
Online dictionaries Teachers 2.20 1.21
Students 3.39 1.09
Language Learning websites Teachers 2.81 1.27
Other teachers, as previous studies indicated, looked for additional programs and online resources to supplement their use of CMSs so that technology could be better integrated into their specific teaching contexts (Bongey, Cizadlo & Kalnbach, 2005; Malikowski, Thompson & Theis, 2006; West, Waddoups & Graham, 2007).
Finally, in terms of gender, an issue often discussed in previous CALL research (Grace, 1998; Liaw, 2007; Liu, Moore, Graham & Lee, 2003; Meunier, 2006), the results of the analysis showed that the mean of the 6 variables from male teachers was 2.39, while that of the female teachers was 2.45. That is, female teachers showed higher familiarity with using computer technology for language learning/teaching purposes than male teachers. As for the students, the mean of the 6 variables from male students was 2.45, while that of the female students was also 2.45, showing no initial difference between the male and female students’ familiarity with computer technologies.
4.2 What motivates the use of CMSs in language courses?
In looking at the language teacher and student respondents’ motivation to use CMSs in English courses, the questionnaire surveyed the respondents’ perceived quality, self-efficacy, enjoyment, usefulness, and behavioral intention of using CMSs in English courses. Table 3 summarizes the findings. The statistical analysis showed that the respondents, especially instructors, found CMSs to be useful in teaching/learning English and they had a strong intention of adopting CMSs in their courses. Moreover, the teachers and students were also confident about their ability in using CMSs for language teaching/learning purposes (self-efficacy). While the students generally held a slightly more positive view towards the quality of the CMSs used in their English courses, they did not seem to enjoy using CMSs as much as the instructors.
The results of the quantitative analysis showed a clear consensus among ques-tionnaire respondents’ positive attitudes toward the use of CMSs in their English curriculum. The qualitative data, on the other hand, generated further insights into the different reasons behind such an attitude.
First, as computers are becoming essential tools for teaching and learning in today’s digital world, some teachers were eager to ride the technology wave.
Table 3 Respondents’ perception and behavior intention toward using CMS in language courses (from 1 ‘‘strongly disagree’’ to 5 ‘‘strongly agree’’)
Variables Participants M
Perceived quality of CMS Teachers 3.24
Students 3.49
Perceived self-efficacy in using CMS Teachers 3.65
Students 3.48
Perceived enjoyment in using CMS Teachers 3.41
Students 3.13
Perceived usefulness in using CMS Teachers 4.14
Students 3.80
Behavioral intention in using CMS Teachers 3.83
They faced the immediate need to learn new technologies and incorporate systems like CMSs to communicate with their students whose ‘‘lives are closely tied to the internet’’ (OT).
‘‘Youngsters nowadays grow up in the digital world. They are born as digital natives while teachers are digital immigrants. We have to communicate with students in their language. Therefore, if you keep in touch with the latest computer technology, you can share the same communication platform with the students’’ (T3).
Echoing the teachers’ point of view, many student interviewees held positive attitudes towards using CMSs because they were ‘‘used to working on computers and confident in operating the online learning system’’ (S5). Even novices to CMSs were able to learn to navigate CMSs quickly and start enjoying the benefit of learning English via such a system.
Second, many language teachers pointed out that the virtual community created by CMSs could help them facilitate the use of teaching approaches that were par-ticularly important for language teaching and learning purposes. Some examples of approaches mentioned by the teachers included communicative language learning, collaborative learning, constructivism, developing learning autonomy, and creating a learner-centered and task-based learning environment:
‘‘Learning is situated. On CMSs, all the activities are conducted in English. In this case, the English texts are not only for students to learn, but serve as authentic text materials [for communication and interaction]. Students have to understand the English text [on CMSs], including what the teachers announce and other classmate’s words before they can carry out a task’’ (T2).
‘‘When you use CMSs as a medium of interaction, it scaffolds students’ lan-guage learning. A good use of the activities can enhance students’ learning motivation y and [provide opportunities for] collaboration y which would in turn give them more opportunities to use the [target] language’’ (T6).
Moreover, the opportunities for hands-on practice (especially in writing) and interaction outside of the time and space restrictions of the actual classroom encouraged teachers and students to adopt CMSs in their language courses:
‘‘It allows me to communicate with my students and allows students to com-municate amongst themselves after class y . As long as the teacher is willing to use the CMSs, they can provide students with more opportunities to practice and learn and expand the amount of language input and output’’ (T7).
‘‘With easy access to read other classmates’ papers, I can learn from those who receive good grades y [because I can see] what their papers look like y . We can learn from each other and inspire each other y and achieve the effect of peer learning’’ (S4).
For many students who responded to the open-ended questions in the ques-tionnaire, CMSs were a positive addition to their study of English because the many built-in activities in CMSs made ‘‘the curriculum richer and more diverse’’, and made ‘‘learning more entertaining’’ and ‘‘fun’’.
Aside from that, a couple of junior teachers mentioned that monetary incentives from the educational institutions were what motivated their initial use of CMSs in teaching. As T4 pointed out:
‘‘When I first started as a lecturer, the university provided a small amount of grant money for those who were interested in using Blackboard. I started using the system because of that. Even though there are no more rewards now, since I am already used to using the system, my motivation remained high’’ (T4). According to the teacher participants, the grant money allowed them to hire additional teaching assistants or purchase computer accessories needed to make CMS adoption more efficient.
Past studies have noted that the users’ perceived usefulness, perceived enjoyment, and perceived self-efficacy are the keys to determine end-users’ behavioral intentions (Compeau, Higgins & Huff, 1999; Liaw & Huang, 2003; Moon & Kim, 2001; Szajna, 1996; Taylor & Todd, 1995). The results of the current study, however, further indicated that in teaching or learning English, the teachers’ and students’ intention to use CMSs is influenced by not only the general functionality of the system, but also by the capability of CMSs to specifically support language learning and teaching (creating a communicative, interactive, and autonomous learning environment that maximizes language input and output) as well as by outside incentives such as the availability of funding.
4.3 What are the perceived limitations of and desired technical and professional support for using CMSs in language courses?
While the teachers and students acknowledged many advantages of using CMSs in teaching and learning a language, they were also aware of the limitations when trying to adopt a fundamentally discipline-neutral program into their courses. In terms of the functionality of CMSs, many pointed out their lack of support for the speaking components of language training. For instance, students like S1 looked forward to a built-in online recorder that could provide additional speaking practice after class: ‘‘I hope there will be a way to record and upload our voices online so we don’t have to wait until in-class presentations [to receive feedback]’’. Instructors also mentioned the convenience of having a universal recording device that automatically rendered audio files in the same format:
‘‘[Without a common recording device on CMSs], [the students] will struggle with recording and uploading files. We [teachers] will end up working with recordings of different formats and levels of quality’’ (T2).
Some students in this study also complained about the interfaces and functions of CMSs being non-user-friendly and unintuitive. For instance, S2 said she was not really motivated to use CMSs because the systems were rigid and the functions used were rather limited so she did not ‘‘think it is of much help to English learning’’ (S2). S7 also commented that CMSs remain probably ‘‘more of management tools rather than learning platforms.’’ In addition to the shortcomings of the systems, some students gave pedagogical suggestions for language instructors using CMSs, such as
adding interesting audio and video clips, making more frequent use of the chatroom function, and providing links to useful web resources (i.e. adding more outside reading materials). Additionally, some students pointed out that teachers had to be better-versed in using the technology so that time would not be wasted on troubleshooting.
As CMSs are not tailored for language instruction, their successful implementation may not rely solely on their users’ knowledge of CMS functions but also on their ability to formulate a creative plan that accommodates the existing CMS functions. Williams (2003) indicated that the development of courses on CMSs needed to ‘‘emphasize the pedagogical aspects as much as the ‘nuts-and-bolts’ aspects’’ (op. cit.: 65). As one student put it, whether CMSs can work to benefit a language course and motivate its users depended on ‘‘the way instructors make use of[the CMSs], their attitudes towards the systems, and their expectations of their students and their curriculum design’’ (S3). Teachers also cautioned that one
‘‘can’t just use CMSs for the sake of using them; the important thing is to know how to work with the system and take students’ four skills (listening, speaking, reading, and writing) into account when designing language courses on CMSs’’ (T4).
After all, without a thoughtful and well-organized learning environment, students’ learning results and motivation might be undermined (Chan & Robbins, 2006; Nijhuis & Collis, 2003).
Nevertheless, thoughtful planning and dedication takes time and, as a teacher pointed out: ‘‘If you want the system to look rich and interesting, teachers have to spend time working on it – teachers have to be the gardeners’’ (T3). As course creators, the extra time needed to put materials and activities together on CMSs created an additional burden:
‘‘[Teachers] will need to spend time setting up the courses and utilizing the different functions [of each system/program]. The time needed will double or triple depending on the number of courses you have y it could be troublesome to reset [the course materials] each semester’’ (T1).
However, according to previous research, the dedication of time is not all it takes for successful CMS adoption, and that a good quality of system integration is not readily available (Nijhuis & Collis, 2003). As a matter of fact, to ensure quality use of CMSs in language instruction, the participants’ discussion in this study centered on the need for better institutional support and a content-based CMS.
First, the teachers voiced the need for better professional training and financial support to help them become effective and efficient users of CMSs and other com-puter programs:
‘‘Schools can invite teachers who are experienced and creative users of CMSs to lead workshops and demonstrate how they integrate CMSs into English courses y. or hold panel discussion and have teachers, or even students, to join the discussion and brainstorm together’’ (T1).
Second, many participants in this study also looked forward to the development of CMSs designed for language learning and teaching purposes. Specifically, some pointed out that the inclusion of real-time video conferencing and instant messaging
could enable convenient communication among students within as well as outside of the same class; it could enhance students’ learning motivation, and increase the amount of language input and output (S1, T2 & T6). Others envisioned CMSs that incorporate resources or functions that are text-based (i.e. writing consultation, spell-check, an online dictionary), media-based (podcasting, YouTube links, an online audio/video recorder), and communicative-based (links to social networking websites such as Facebook, MySpace, and Twitter) (T2, T4, T6 & S6). T2 further commented that ideal CMSs should be compatible with tools in web 2.0 so that students’ work (including voice, video, and writing) could be published online and thereby accessed in different parts of the world via search engines.
In short, the findings of the analysis showed that despite the many valuable fea-tures of CMSs for language learning and teaching, users could be better supported and CMSs could be further modified to benefit the efforts of language teachers and learners at college level. The key to bringing about such benefits and minimizing the disadvantages of CMSs does not lie solely in the users’ hands but also in a more well-rounded support system provided by academic institutions, as well as the develop-ment of CMSs designed with language teaching and learning purposes in mind.
5 Conclusions and implications
This study employed mixed methods to provide a more comprehensive under-standing of the end-users’ – both teachers and students – experience and perceptions of using CMSs in college language courses. The findings indicate that the partici-pants in this study generally held positive attitudes towards the use of CMSs, which were believed to encourage collaborative learning, provide convenient access to and distribution of course materials and language resources, and help the teachers carry out their teaching philosophy. Thus, many welcomed the addition of CMSs to aid their teaching and learning efforts.
At the same time, however, the findings show that successful implementation of CMSs in language teaching does not always come naturally. Despite the many potential advantages of CMSs, they are not systems specifically designed for lan-guage learning and teaching purposes. So, how and to what extent one may be able to enjoy the benefit of CMSs largely depends on the course creators’ methods of incorporating CMSs into their curriculum. In this study, some teachers used CMSs primarily for uploading news and announcements, some utilized the CMSs’ built-in functions more comprehensively, and many mentioned the need to bring in addi-tional internet resources and computer programs to supplement the existing func-tions of the CMSs. Students, while expressing appreciation for the extra learning opportunities provided by their instructors’ effective and creative use of technology, were also aware that it was the well-designed and carefully planned online curricula that enabled CMSs to function as a learning platform rather than merely a man-agement system.
Echoing previous research that highlighted instructors’ varied technical and ped-agogical proficiency (Nijhuis & Collis, 2003), the findings of this study also illumi-nate the need to provide better support – technical, pedagogical, and financial – to assist teachers in enjoying the benefits of using CMSs. In other words, school
administrators cannot expect English teachers to become CMS-proficient without offering technical assistance, professional training, and additional human and ped-agogical resources. In terms of technical support, before course-wide or depart-mental-wide CMS adoption, a needs analysis of each course must be conducted with students and instructors who are going to be at the frontline of using, administering, and creating materials on CMSs. Then, teachers and students should be provided with the opportunity to test and compare the functionalities of different CMSs to ensure that the users’ demands can be properly met. Once a system is implemented, the university or educational institution should maintain and update the system periodically so that teachers and students can enjoy the benefits of CMSs without being overwhelmed by technical difficulties (Curtin & Shinall, 1987; Decoo & Colpaert, 1999).
As technology itself does not provide better education, the findings of this study also highlight the critical need to assist teachers in developing a strategy for their use of CMSs to enhance their pedagogy in the disciplinary. For instance, since CMSs are specifically powerful when used to provide additional language input and interaction outside of the classroom setting, language course designers have to learn to develop teaching plans that maximize the functionality of CMSs in this area. To this purpose, orientations, professional workshops and teaching demonstrations should be con-ducted regularly to allow course creators and instructors to share and learn from each other’s teaching experiences. Moreover, peer support from within the dis-cipline’s teaching community is also critical to teachers’ professional development. As there might never be a system that offers a total solution to satisfy every instructor’s needs, the emphasis of effective CMS use should be placed on training users to selectively adopt CMSs to suit their language teaching objectives rather than accommodating course content to the existing CMS functions. Furthermore, as CMS adoption takes time and computer-related resources, providing enough man-power and equipment could help shorten the hours of preparation and thus boost the teachers’ motivation for using CMSs in their courses. Therefore, in addition to offering technical assistance and professional training, additional resources as such should also be provided to further encourage teachers to experiment with innovative pedagogy.
Since their first creation as media of instruction, CMSs have come a long way towards becoming a well-received addition to many college classrooms. However, a long way is still ahead for CMSs to become indispensable to instructors and stu-dents. While their many built-in functions seem applicable to courses in different subject areas, the fact that they are not ready-made for courses in any specific subject often makes their pedagogical use less than convenient and intuitive. In the case of language courses, CMSs fall short as learning platforms because they lack com-prehensive functionalities geared towards the development of language skills. Thus, it is time for CMS developers to shift their focus from providing ‘‘please-all’’ information management and dissemination systems geared toward different dis-ciplines and pedagogical needs. Learning platforms tailored for specific content areas would not only free users from the trouble of fumbling with different programs to participate in a single online activity (such as audio recording), but would also help them concentrate on the actual teaching and learning of the course content.
A limitation of the current study is that it only examined college teachers’ and students’ experiences of using CMSs in English courses in two universities in Taiwan. In addition, those English courses using CMSs in the study were face-to-face rather than online courses. Future research, employing a wider range of participants and with a focus on online language courses using CMSs, is called for.
Finally, there is more to be discovered about the efficacy of CMSs in language teaching and learning. While the participants might be speaking from their own perspectives when talking about their perceived advantage of CMSs to the teaching and learning of languages, because of the study’s one point-in-time snapshot, it was not easy to tell whether the use of CMSs did actually enhance students’ progress in learning English. Thus, it would be important for future research to examine the long-term effect of CMS users’ learning progress. Do those enrolled in English courses that make good use of CMSs in fact progress faster than those who parti-cipate only in courses that do not employ CMSs? The findings of an empirical investigation could help university administrators and policy makers re-examine the justification for the prevalent adoption of CMSs in higher education institutions. In addition, studies examining the effects of enhanced efficiency (i.e. time-saving factors) in teaching and learning with the support of CMSs would also be valuable.
References
Auyeung, L. H. (2004) Building a collaborative online learning community: a case study in Hong Kong. Journal of Educational Computing Research, 31(2): 119–136.
Bongey, S. B., Cizadlo, G. and Kalnbach, L. (2005) Using a course management system (CMS) to meet the challenges of large lecture classes. Campus-Wide Information Systems, 22(5): 252–262.
Brandl, K. (2005) Are you ready to Moodle? Language Learning & Technology, 9(2): 16–23. Brown, J. D. (2001) Using surveys in language programs. Cambridge: Cambridge University
Press.
Cappel, J. J. and Hayen, R. L. (2004) Evaluating e-learning: A Case Study. Journal of Computer Information Systems, 44(4): 49–56.
Chan, C. H. and Robbins, L. I. (2006) E-learning systems: promises and pitfalls. Academic Psychiatry, 30(6): 491–497.
Chen, J., Belkada, S. and Okamoto, T. (2004) How a Web-based Course Facilitates Acqui-sition of English for Academic Purposes. Language Learning & Technology, 8(2): 33–49. Cloete, E. (2001) Electronic education system model. Computers & Education, 36(2): 171–182. Compeau, D. R., Higgins, C. A. and Huff, S. (1999) Social cognitive theory and individual reactions to computing technology: a longitudinal study. MIS Quarterly, 23(2): 145–158. Curtin, C. O. and Shinall, S. L. (1987) Teacher training for CALL and its implications.
In: Smith, W. F. (ed.), Modern technology in foreign language education: applications and projects. National Textbook Company: Lincolnwood, 255–285.
Decoo, W. and Colpaert, J. (1999) User-driven development and content-driven research. In: Cameron, K. (ed.), Computer assisted language learning (CALL): media, design and appli-cations. Lisse: Swets & Zeitlinger Publishers, 165–181.
Driscoll, M. (2008) Hype versus reality in the boardroom: why e-learning hasn’t lived up to its initial projections for penetrating the corporate environment. In: Carliner, S. and Shank, P. (eds.), The e-learning handbook: past promises, present challenges. San Francisco, CA: Pfeiffer, 29–54.
Felix, U. (2005) Analysing recent CALL effectiveness research – towards a common agenda. Computer Assisted Language Learning, 18(1–2): 1–32.
Grace, C. (1998) Personality Type, Lexical Ambiguity, and Vocabulary Retention in CALL. CALICO Journal, 15: 19–45.
Hubbard, P. (2005) A review of subject characteristics in CALL research. Computer Assisted Language Learning, 18(5): 351–368.
Johnson, R. B. and Onwuegbuzie, A. J. (2004) Mixed methods research: a research paradigm whose time has come. Educational Researcher, 33(7): 14–26.
Leahy, C. (2004) Researching language learning processes in open CALL settings for advanced learners. Computer Assisted Language Learning, 17(3–4): 289–313.
Liao, H.-L. and Lu, H.-P. (2008) The role of experience and innovation characteristics in the adoption and continued use of e-learning websites. Computers & Education, 51(4): 1405–1416.
Liaw, S.-S. (2007) Sex differences and learners’ autonomy toward e-learning based on surveys in UK and Taiwan. Psychological Reports, 100(3): 949–954.
Liaw, S.-S. (2008) Investigating students’ perceived satisfaction, behavioral intention, and effectiveness of e-learning: a case study of the Blackboard system. Computers & Education, 51(2): 864–873.
Liaw, S. S. and Huang, H. M. (2003) An investigation of users’ attitudes toward search engines as an information retrieval tool. Computers in Human Behavior, 19(6): 751–765. Liu, M., Moore, Z., Graham, L. and Lee, S. (2003) A look at the research on computer-based
technology use in second language learning: A review of the literature from 1990–2000. Journal of Research on Technology in Education, 34(3): 250–273.
Malikowski, S. R., Thompson, M. E. and Theis, J. G. (2006) External factors associated with adopting a CMS in resident college courses. The Internet and Higher Education, 9(3): 163–174.
Meunier, L. E. (2006) Human Factors in a computer assisted foreign language environment: The effects of gender, personality and keyboard control. CALICO Journal, 13(2 & 3): 47–72. Moodle Forum (2009) New Proposals for Development.
http://moodle.org/mod/forum/dis-cuss.php?d5121996&mode51
Moodle Statistics (2010) Moodle Statistics. http://moodle.org/stats/
Moon, J. W. and Kim, Y. G. (2001) Extending the TAM for a World-Wide-Web context. Information and Management, 38: 217–230.
Nijhuis, G. G. and Collis, B. (2003) Using a web-based course-management system: an eva-luation of management tasks and time implications for the instructor. Evaeva-luation and Program Planning, 26(2): 193–201.
Nunan, D. (1992) Research methods in language learning. Cambridge: Cambridge University Press. Perkins, M. and Pfaffman, J. (2006) Using a course management system to improve classroom
communication. Science Teacher, 73(7): 33–37.
Priyanto, A. D. (2010) Facilitating Language Learning with LMS: (A Brief Review on Blackboard and Moodle). http://agusdepe.staff.uns.ac.id/2009/04/06/facilitating-language-learning-with-lms-a-brief-review-on-blackboard-and-moodle/
Robb, T. N. (2004) Moodle: A Virtual Learning Environment for the Rest of Us. TESL-EJ, 8(2). http://tesl-ej.org/ej30/m2.html
Rosenberg, M. J. (2001) E-learning: strategies for delivering knowledge in the digital age. New York: McGraw-Hill.
Sivo, S. A. and Pan, C. C. (2005) Undergraduate engineering and psychology students’ use of a course management system: a factorial invariance study of user characteristics and atti-tudes. Journal of Technology Studies, 31(2): 94–103.
Su, C. (2006) Moodle for teachers. The proceedings of 2006 International Conference and Workshop on TEFL & Applied Linguistics,321–330. http://www.opensource.idv.tw/paper/ Moodle/Moodle-for-English-Teachers.doc
Sun, P.-C., Tsai, R. J., Finger, G., Chen, Y.-Y. and Yeh, D. (2008) What drives a successful e-Learning? An empirical investigation of the critical factors influencing learner satisfaction. Computers & Education, 50(4): 1183–1202.
Szajna, B. (1996) Empirical evaluation of the revised Technology Acceptance Model. Man-agement Science, 42(1): 85–92.
Taylor, S. and Todd, P. A. (1995) Understanding information technology usage: a test of competing models. Information Systems Research, 6(2): 144–176.
van Olphen, M. (2007) Perspectives of foreign language pre-service teachers on the use of a web-based instructional environment in a methods course. CALICO Journal, 25(1): 91–109. West, R. E., Waddoups, G. and Graham, C. R. (2007) Understanding the experiences of instructors as they adopt a course management system. Educational Technology Research and Development, 55(1): 1–26.
Williams, G. (2003) Implementation of a course management system: experiences and stu-dents’ thoughts. Journal of Teaching in Travel & Tourism, 3(2): 59–69.
Woods, R., Baker, J. D. and Hopper, D. (2004) Hybrid structures: faculty use and perception of Web-based courseware as a supplement to face-to-face instruction. Internet and Higher Education, 7(4): 281–297.
Appendix A Questionnaire for teachers
Appendix B Questionnaire for Students
