Chapter Five
Suggestions and Conclusion
This final chapter summarizes the evaluation and application of Junior High School CD-ROMs. The purpose of this study, research questions, CALL literature, evaluation criteria, research design and methodology, findings of the target CD-ROM evaluation and their implementation in English classes will be discussed.
5.1 Summary of the Study
Language learners and even their teachers are not aware of how to choose the software that fit learners’ language levels and learning needs, nor are they familiar with the most efficient way of navigating different programs. The purpose of this study is to propose a set of criteria to evaluate Junior High School CD-ROMs and to identify their strengths and weakness. In addition, in helping teachers effectively use these programs, their application in class is discussed according to the sections that the programs are composed as well as teachers’ teaching mode.
Most of the target CD-ROMs are complimentary to the teachers along with their teachers’ guides in order to promote textbooks. Computer-based supplements can be used to diversify classroom activities, but teachers seldom use them in class. This situation aroused my interest to examine the reasons why teachers do not take
advantage of the CD-ROMs. Three research questions are brought up to explore the major concern of this study:
1. What are the criteria for evaluating instructional CD-ROM titles?
2. What are the characteristics of instructional junior high school CD-ROM titles?
What are their strengths and limitations?
3. How does the instructional CD-ROM apply to language teaching in school settings?
To find out the appropriate evaluation criteria in answering the first research question, theories of SLA and CALL have been reviewed as the fundamental bases of evaluation criteria. For instance, Chapelle (2001) elaborates the context for
computer-assisted SLA and principles for CALL evaluation. The Interactionists’
theory of input, output, and interaction are reviewed to examine the user’s cognitive process in CALL. On the other side of human-computer interaction cycle, the user interface provides the accessible route for the user to read the language materials on the screen, do the exercise, attend the activities, and receive feedback from the program.
The main concern of the evaluation is to discover the potential software and identify its strengths and weakness. This kind of evaluation belongs to summative judgment since it is used for assessing the software after publication. Also in section 2.2, the method employed in this study is judgmental analysis, as many software reviews on TESL-EJ, Language Learning & Technology, CALICO, and The Educational Software Selector (TESS), a database containing reviews of diverse published educational software for teachers and users.
Theoretical criteria as well as many practical guidelines have been collected and analyzed in order to choose an appropriate evaluation tool for busy teachers. Among them are for general subject areas such as Dallas College’s CD-ROM evaluation criteria in section 2.2.3. There are also criteria and checklists designed especially for ESL/EFL CD-ROM evaluation, for example, the GSL520 CALL ESL software
evaluation form.
The original article by Ioannou-Georgiou was adapted and transformed into a checklist style. This set of checklists is considered the most appropriate assessment tool for the reasons of the summative evaluation, criteria for evaluating instructional language learning CD-ROMs, equal emphasis on computer ability, program content
and pedagogical value, practical checklists, and explicit instances. In addition, this practical set of checklists is comprised of three aspects, namely, technical,
methodological, and pedagogical, each with eight to fourteen items, and a total of 122 questions, which provide adequate criteria for identifying the strengths and limitations of instructional CD-ROMs.
With regard to the evaluation method, the judgmental analysis is adopted because of the difficulties of evaluating CALL mentioned in section 2.2.3 and the practical usage of choosing potential software for language teachers. The evaluation objects are three Junior High School CD-ROMs, Book One, for the reason of their good selling records. Ioannou-Georgiou’s checklists are served as the instrument of this study.
The evaluation procedure begins with data collection and analysis, in which literature of second language learning and CALL are studied, evaluation criteria and checklists are reviewed and examined, and the target CD-ROMs are chosen. The second step is performing a pilot study with another evaluator to test the inter-rater reliability and to reduce the influence of subjective opinion to the minimum. The processes of pilot testing include:
1. Discuss the checklists until a consensus about each evaluation question is made.
2. Assess Nani Junior High School CD-ROM Book One separately according to Ioannou-Georgiou’s checklists.
3. Compare and discuss the result. Among the 122 questions, there are only four discrepancies between the evaluators related to the compatibility of methodology.
The evaluators finally agreed that the program is mainly drill and practice, lacking in communicative function.
4. Revise the evaluation form. Seventeen supplementary questions are added in corresponding to junior high school setting in Taiwan and recent technology
developments.
In the third step, I run every interface of the target CD-ROMs and on the basis of the chosen instrument, make notes to see whether they fit the criteria, identify the strength and weakness of each program. In addition, for practical use of these programs, their implementation is considered according to teachers’ daily instructional procedure.
With respect to research question 2, the target CD-ROM titles run smoothly and intuitively. The multimedia design of the presentation of texts and various activities are novel and attractive to junior high school students. Nani’s CD-ROM is unique in its simplicity. Joy’s program is famous for a series of rewritten stories, ‘Tom N Huck’, and the saving system for the user’s reference. Kang Hsuan’s is the most sophisticated design among the three in terms of the games for group work, the most efficient navigation system, and the most diverse activities and practice.
However, the three CD-ROMs still need improvement because of the following insufficiency:
1. Lack of authoring ability to include the teacher’s own data to meet their students’
needs.
2. Lack of the design of hyperlinking to background knowledge and other information.
3. Lack of the design of dictionaries to enhance vocabulary learning.
4. Feedback offers only correct answers without other possible authentic replies.
5. Lack of the design of Speech Recognition Technology (SRT) to practice speaking.
6. Lack of the design of downloadable texts, worksheet, graphics, or practice.
7. Lack of user’s tailor ability to change colors, sound, and animation.
In answering research question 3, the three CD-ROM titles present a strong support to their correspondent textbooks through their authentic pronunciation, word
illustration and examples, embedded pre-recording vocabulary and reading, and a variety of exercises. It is quite different from their written versions arranged in a page-by-page order; they provide hyper links to pronunciation, words, grammar, practices, chants, songs, and games. Such designs support learning for the user’s satisfaction of immediate need of answers in a friendly learning environment.
Nonetheless, when applying these programs to language classes, their weaknesses are as the following:
1. The choices of activities are limited.
2. The presentation of learning materials is similar to those on the textbooks without many changes except for the sound clips and animation.
3. The exercises are mainly drill practice for individual use.
4. The help options in vocabulary are limited.
5. There is no design for practicing speaking and writing.
5.2 The Limitations of the Study
There are some limitations that confine this study to a more thorough and
complete study. First, only CD-ROMs of Book One are under examination. Due to the limited scope of this study and the tight publication schedule, the target
CD-ROMs are only Book One. The CD-ROMs of Book Five and Book Six have not been published at the moment of this study. The textbook stores have only recently published their textbooks of Book Five, and the corresponding CD-ROMs will be published around August 2004. As mentioned in section 3.4.1, every piece of CD-ROM possesses similar format and overall structure. Yet in order to evaluate their effectiveness, all six copies of the CD-ROMs should be included.
Second, an empirical study should be conducted in examining the effectiveness of these programs. The evaluation methodology used in this study is mainly based on
my assessment of the target CD-ROMs. In compensation for my subjective
judgments, another evaluator has joined the pilot study to test the inter-rater reliability and to reduce bias or prejudice. Nonetheless, this study still lacks experimental research on the implementation of the three CD-ROMs in the language classroom.
As mentioned in section 2.2.2, empirical and judgmental analyses serve as complementary information. Therefore, the experiment of effectiveness of the software implementation should also be examined. In addition, students’ attitude or opinion about using these instructional CD-ROMs in class or as a self-learning aid at home should be studied and analyzed.
Third, the rationale or the theoretical background in the construction of these
CD-ROMs should be inquired. The program designers or course developers need to be interviewed to explore the overall structure, navigation system, content and the reasons why some activities are designed in a certain way. With my personal review and judgment alone, the study would be too subjective or incomplete.
5.3 Suggestions
Based on the above summary, some key issues deserve more attention.
Language teachers, school administrations, and courseware developers have their
crucial roles in helping learners to acquire language in the current CALL environment.
The following are some pedagogical suggestions for them.
5.3.1 For Language Teachers
1. Integrating CD-ROM activities into language classes.
If a teacher can combine the instructional multimedia programs into his or her syllabus, it will arouse students’ interests and provide the motivation to keeping on learning.
2. Possessing the ability to evaluate multimedia software.
The question of whether the material is clearly presented should be carefully examined as well as the reliability of the published company or products. When using multimedia design in the classroom, it is important for the teacher to identify whether the multimedia designs are attractive or distractive to students’ learning?
What catches students’ attention and afterwards gets the retention, multimedia designs or language elements, and to what extent? These questions deserve teachers’ fullest attention.
In general, the use of CALL needs to be determined by the teacher, not by the software, or the software designer. Only teachers can decide how much
multimedia should be used in the courses, and the role it should play.
3. Being aware of individual differences.
In spite of the beneficial factors of computer usage, individual differences should be examined carefully. Teachers can arrange or modify their lesson plans flexibly according to their instructional needs and students’ differences in terms of personality, computer skills, interest in English and language levels. Shy or quiet learners may need the teacher’s support most. Without the encouragement of the teacher, the lack of such students’ participation and interaction with other students or the computer could lead to students’ lackluster performance.
5.3.2 For Educational or School Administration
1. Provide adequate computer equipment and teachers’ support. The schools and educational administrations should offer enough basic computer facilities for teachers and students.
2. Enhance language teachers’ computer literacy. A research conducted by Beaudin and Gregg demonstrates that successful computer integration into classroom
practice is highly associated with teachers’ computer literacy. The inadequacy of computer literacy of teachers and students might influence the effect of CALL application. School administrations and educators need to help teachers gain some computer technologies by weekend workshops, cross-school seminars or in-service teaching training. The teachers’ increasing ability of computer usage will
eventually translate into classroom practice.
3. Construct a self-learning center. It would be beneficial for school administrations to establish a self-access center-- a language lab, or a computer classroom installed with a variety of language learning software for students inside or outside of class.
In addition to providing the materials, the center should clearly instruct students on using the program. Secondly, a short and clear description of how to make the best use of the program should be easily accessible. Third, students would be highly motivated if the teacher introduces or uses some of the programs in class.
Finally, the survey of learners’ feedback after they are using the programs should be collected and analyzed for further improvement.
5.3.3 For Software Developers
Based on the summary above, textbook publishers or software developers may add a few of the following designs to their programs:
1. More Classroom Activities. Games in Kang Hsuan software are a good example, which bring variation and interest into the classroom.
2. Authoring Ability. Teachers would dream of the function which can include their own data in the program or make some changes to meet their students’ needs.
3. Hyperlinking and Branching Systems. Hyperlingking to other related texts, background information, or dictionaries would be attractive to learners who want to learn more.
4. Tracking System. The programs that can record the user’s performance would offer useful information for further improvement or for the teacher’s reference.
5. Downloadable System. The downloadable or printable graphics, sentence structures, or worksheets are practical and useful to teachers and learners.
6. Dictionary. There are two kinds of recently developed computer technology:
Dictionary and SRT. Walter (2002) contends that the application of CD-ROM dictionary certainly makes it easier for students and teachers in terms of the multi-functions of the thesaurus, which lists words in the same topic group or the same semantic field. A specialty in the Cambridge Learner’s Dictionary
CD-ROM (2002) is that the user is able to hear the word being pronounced and recorded by the user himself or herself, and then compared to the voice on the CD-ROM to see the difference in order to make necessary changes. Other characteristic features include filters, copying and pasting, spelling, helping, and exercise. Filters are a powerful searching tool that can perform many kinds of search on areas such as grammar or registers.
6. SRT. Chen (2001) emphasizes the use of Speech Recognition Technologies (SRT) to improve learners’ speaking ability. An example of Syracue English
Comprehensive Learning Series is a PC-based learning resource for developing speaking skills, in which more than 360 interactive activities and tools are included.
In a role-play activity, the user chooses a role to play and reads the lines out.
Afterwards, his or her lines of conversation will be recorded and evaluated in terms of his or her pronunciation and intonation. If they are unacceptable, the user will have to try again, or after a few unsuccessful tries, the intervention of authentic voice will reappear. Another interesting task is Virtual Challenge, in which the user moves around and talks to the simulated characters in a 3D virtual
environment. This activity increases the opportunities for the user to interact with
virtual characters, performing many authentic activities. It is quite attractive for the beginners to be able to engage in speaking with computers. It is also a powerful tool for the users to hear their own pronunciation compared to the voice on the program, recognize the difference, and make adjustments accordingly. To be able to talk to the on-screen robot helps to rise learners’ interests in using English and decrease the anxiety of face-to-face talk with others due to time pressure, limited vocabulary, and low-level language proficiency.
5.4 Suggestions for Further Studies
This study evaluates three instructional junior high school CD-ROMs and ways to implement these programs into language classrooms. In order to make the evaluation more objective and complete, the following are suggestions for future studies.
First, the evaluation of Junior High school CD-ROMs in this study is limited to Book One. The others, Book Two to Book Six, should all be included. Second, some of the commercial CD-ROMs suitable for junior high school students, such as Studio Classroom, My ET and Live ABC, may be reviewed together in comparison with their content, media integration, navigation, hyperlinking and branching. Third, in order to compare the judgmental evaluation with the data collected from users, future studies may include some empirical research including students’ responses and performance as well as teachers’ reflection and suggestions.
5.5 Conclusion
The highly developed computer technology and worldwide communication have dramatically changed language-learning styles as well as teachers’ instructional modes. In addition, due to the background of non-native English teachers, culture
difference, and English only as a foreign language, it is hard to apply authentic Communicative Approach in language classroom (Brown, 1994). Recently, Liou (2001) promotes teacher’s action research in the field of CALL development in our country. It encourages teachers’ involvement in integrating computer technology into language teaching. These factors urge teachers today to use multimedia materials to teach English.
The benefits of integrating technology into classroom have been documented.
With multimedia presentation, even a sentence pattern drill could become much more fun and easier to understand and remember. Furthermore, using these CD-ROMs in class is not difficult at all, all teachers have to do is to run them first, see which sections can be integrated into their curricula, design worksheets if necessary, and bring students to computer labs. If there are not enough computer classrooms for teachers to use, teachers can simply bring their laptop with a transmitted connection wire to TV, or a projector, enabling students to see projected images.
Technology is a helpful and sometimes even transformative addition to the L2 classroom (Taylor et al, 1996). CALL is motivational, but it is not a panacea or cure-all method that could solve all the existing problems. The instructional CD-ROMs stimulate language learning in group activities and serve as reviewing materials at home. However, whether the computer or the multimedia can serve as entertainment or ‘edutainment’ depends on the teachers’ choice and wisdom in using resources.
