國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
CHAPTER THREE
METHODOLOGY
This chapter describes the materials, the instruments for the study and the process of analyzing and evaluating the materials. The present study primarily aims to answer the research questions about the connection between the TVEE and the goals and objectives of the curriculum guideline, and the construct validity of the dialogue section in the English test of the TVEE. In order to answer the two research questions, the study analyzed two kinds of materials: one was the Vocational High School Curriculum
Guideline, and the other was the Technological and Vocational Educational Examination.
Other than the materials, the study also involved two specially designed checklists as instruments, the first checklist was established based on the outline of the curriculum guideline and the second checklist included speaking constructs used to examine the validity of the dialogue section of the test.
Materials
The materials used in this study consisted of (1) the Vocational High School Curriculum Guideline (VHSCG), and (2) the English tests of Technological and Vocational Educational Examination (TVEE). The following section explains the structure of the VHSCG and the TVEE in detail respectively.
The Vocational High School Curriculum Guideline Background Information
The latest version of VHSCG was updated in 2010, and it is available to public on the website of Course Department Groups under the administration of Technological and Vocational Department, Ministry of Education (MOE). MOE divides the departments of vocational high education into fifteen groups according to their professional fields, and
27
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
each vocational group has its own curriculum guideline. The VHSCG of English is categorized in the Language Field of General Subjects.
General Introduction to the English Guideline
English is a required subject throughout the entire six semesters in vocational high schools. Two credits are the minimum requirement for each semester, which means students have at least two course sessions, approximately two hours, in a week. The title of the course is called English I to VI.
Course Objectives
Four main course objectives are set for the first semester (English I), which are also the basic requirements of the entire six courses through the vocational high school education: (1) to train students to use the lexis and the syntax for daily life
communication; (2) to foster students to have the ability to learn English effectively; (3) to guide students to acquire and understand the cultural differences between the Eastern and the Western; and (4) to guide students to think independently.
The curriculum guideline keeps the four basic requirements and adds more objectives to enrich the content and expand the dimensions of the courses each year. To be more specific, in the second semester (English II), the curriculum guideline modifies the description of the course objectives by adding the cultivation of the sound learning attitude and the emphasis on knowing the technological knowledge. Four new targets are implemented to enrich the basic objectives in the second year (English III and IV): to develop the interest on learning English, to build up the capacity for humanity education, international affairs and life education. Several new dimensions are added to enrich the teaching content of English V and VI. In the last year of vocational education, students need to know how to apply English into their future jobs, strengthen their self-study ability as a basis of life-long learning, and construct their own value systems.
28
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
From the course objectives, we can see that the objectives throughout the three years are interrelated with one another, and both the complexity and the dimension increase and expand along with the development of the English courses. On the other hand, it is evident that the focus of EFL in vocational high school aims to cultivate the communicative ability for daily life, prepare learners for the future job market, and establish the ability of thinking independently by implementing knowledge from different aspects.
Teaching Guideline
The VHSCG states four principles that should be taken care of throughout the three-year curriculum. First, it is stated that the length and the topic of the reading material should be comprehensive with particular focus on students’ interest, the texts’
usefulness and informativeness. Secondly, the teaching pedagogy and approaches applied in the language classroom should stress the communicative function of English.
Authentic tasks are, thus, preferable since they allow students to have the opportunity to apply the language in real-life use. Third, the evaluation of the course should combine formative and summative assessment, and make a proper use of portfolio assessment. It also stresses that the use of language is the focus of evaluation, while the knowledge of language is the minor purpose. Both the fluency and accuracy of language use are important when it comes to evaluation. Fourth, the curriculum guideline encourages the instructors to apply teaching aids and multimedia computer-assisted softwares. It also requires teachers to list the extracurriculum material references relating to the topic for students to learn more extensively on their own after school.
29
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Teaching Material Topics.
The suggested topics of the material are one of the most important components of the curriculum guideline. Although most of the topics are included throughout the three academic years, the last two topics are implemented only in the last two semesters. Ten topics are given for English I to English IV: interpersonal relationships, leisure activities and entertainment, daily life, modern technology, culture and customs, literatures and arts, language, industrial, commercial and agricultural knowledge, environmental education, and employment. In English V and VI, two more topics are added: (1) travel,
transportation and traffic; (2) global view. Put together, there are twelve unit topics in the material outline. The VHSCG itself does not explain the reason why the last two topics are not introduced until the third academic year, it might be that these two topics are more advanced and complicated for vocational high school students. Since these two topics are less invested, the amount of their coverage in the TVEE is expected to be lower than other topics.
Linguistic knowledge.
The VHSCG also explains the stages of developing three kinds of linguistics knowledge in the footnote. The linguistics knowledge includes phonology, vocabulary and syntax. The knowledge is repeated in the six semesters while the complexity increases as learners move on along the program, which helps them deepen their understanding toward these ideas. Below is the description about how linguistic knowledge is addressed in three years.
For phonological knowledge, teachers should review phonetic symbols, phonics and pronunciation and improve learners’ accuracy of prosodic feature, such as stress, intonation and rhythm in the first year. In the second and the third year, teachers should
30
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
review the prosodic feature in word, chuck, and sentential levels, while facilitates students’ use of phonological knowledge, such as pause, speed, reduction and linking.
As for vocabulary, the vocabulary used in the material should be chosen from the 4000 frequently-used words in English I and II, and new vocabulary in a textbook should not be more than 300 words. In English III and IV, the vocabulary size increases to the 4500 frequently-used words. The number of new words in a textbook should be limited within 350 words. In the last two semesters, English V and VI, the word bank expands as big as the 5000 frequently-used words; also, the word bank expands to 400 words per textbook.
As for syntax and grammar, teachers are not suggested to introduce rarely-used sentence patterns. All of the English courses require teachers to provide students with sufficient practice to enhance their understanding of structure at sentential level. To compare and synthesize the linguistic knowledge addressed in the VHSCG, the researcher summarizes the content of the three-year curriculum in vocational high school in Table 3.1.
31
‧
Table 3.1 The Content of the Three-year Curriculum of the Vocational High School Years
Contents
The first year The second year The third year
Course title English I, II English III, IV English V, VI
Word bank 4000 frequently-used words
1. Review stress, intonation, rhythm and read-aloud skills.
2. Improve the phonological concept of pause, speed, linking and reduction.
Topic
Interpersonal relationships; leisure activities and entertainment; daily life; modern
technology; culture and traditions; literature and arts; language; industrial, commercial and agricultural knowledge; environmental
education; employment education.
All topics listed in the column on the
1. To cultivate the ability to communicate and express ideas.
2. To train students to have the language ability to conduct general daily life communication.
Syntax and grammar
1. Avoid difficult rarely-used sentence pattern.
2. Use charts and graphs for content-based instructions and situational learning.
3. Provide chances to practice.
32
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Benchmark.
Benchmark listed in the curriculum guideline describe the expected progress of the four skills for the six English courses. Since the present study aims to find out whether the dialogue section in the TVEE correspond to the constructs of speaking, the researcher focuses on the benchmark of speaking ability. 3
The benchmark specifically state six abilities for speaking skills for the entire three years. First, students should be able to use the classroom English. Second, students are expected to be capable of asking questions and providing answers to questions on reading texts in textbooks. Third, students need to be able to participate the
conversational practice in the classroom. Fourth, students should be able to conduct simple daily life communication. In addition to that, the fifth descriptor states that
students should be able to read aloud the texts, short essays and stories accurately. Lastly, learners should be able to describe things in daily lives. The above speaking abilities can be generally divided into two kinds: the first kind refers to the language use in the classroom that is more controlled and often guided by instructors in a classroom setting, for example, the participation of the speaking practice and the read-aloud activity. The second part contains unrestrictive tasks that can be conducted in various settings, such as daily life conversation and the description of events.
Based on the content of the guideline, the researcher designed a checklist that incorporated all the major points of the guideline. It will be presented on the instrument section on page 36. The checklist is used to see to what extend and how the VHSCG corresponds to the dialogue section of the TVEE.
3Although conversational ability involves both listening and speaking ability, the dialogue section of the TVEE does not examine test takers’ listening skills. That is the reason why the present study does not include the benchmark and the constructs of listening ability during the analysis.
33
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
The Technological and Vocational Educational Examination
For the analysis of the TVEE, the researcher collected the test items from the website of the Testing Center of Technological and Vocational Education. Since the VHSCG was renewed in 2010, the original texts of the TVEE from 2010 to 2015 were analyzed. The following will present a general introduction to the test.
The English test of the TVEE consisted of four test types, 50 single-answered multiple choices in total, from 2010 to 2014. In 2015, a writing section was incorporated into the test. The following paragraphs introduce each test type briefly in order to build a holistic view of how the examination was constructed.
From 2010 to 2014, the vocabulary section of the TVEE was composed of two kinds of tests in 15 multiple choices, 7 blank-fillings and 8 synonym-selections. In 2015, the vocabulary section reduces to 10 questions, with 6 blank-fillings and 4 synonym-selections. The words used in this section are mostly from the frequent-used 4000 words.
It examined test takers’ ability to recognize the options and identify the contextually appropriate option.
The second section of the TVEE is the dialogue, which is the targeted section of the present study. It is an indirect speaking test that examines conversational ability in a written form. Many idiomatic and colloquial expressions are applied in this section. It contained 10 questions from 2010 to 2015. As introduced in the last section of literature review, in the dialogue section, a conversation about certain topics between two speakers is presented with one of the turns removed. The candidate has to choose the best answer from the four options to make the dialogue complete.
The third section is the cloze section, which contained 15 questions with three short articles from 2010 to 2014. One article was deleted in 2015, making it 10 questions in total. The cloze section provides a meaningful context to examine learners’ vocabulary
34
‧
size, their knowledge of cohesive ties and grammatical rules. Each article has
approximately four to six blanks, and test takers have to choose one correct answer from four options to fill up the blank.
The last section of multiple choices is the reading comprehension. It has ten comprehension check questions along with two to three articles. The questions include local and global questions, testing the details or overall understanding of the article. The topics of selected articles varies, and they are related to authentic life experience or extracurricula knowledge of vocational high school students.
In 2015, the writing section is added to test the writing ability. It contains two parts. At the first part, two sentences are presented with two blanks in each one. Students are asked to fill the blank according to the Chinese translation of the sentence. For the second part of the writing section, students have to rearrange scrambled sentences into correct order. Punctuation is taken into account to the scoring.
To have a general picture of the changes of test types of TVEE, the test types from 2010 to 2015 are displayed in Table 3.2.
Table 3.2 The Test Types of TVEE from 2010 to 2015
Year Multiple Choices Non-multiple Choices
Vocabulary Dialogue Cloze Reading Comprehension
In order to answer the research questions, the researcher specially designed two checklists to examine the content coverage and the construct validity of the dialogue
35
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
section. In the following section, first, the researcher discusses the process of developing the curriculum checklist. Then, explanations and details on how to build the speaking constructs checklist are provided.
Curriculum Checklist
The Curriculum Checklist contains two parts with the purpose of examining the correspondence between the guideline and the test. The checklist includes two parts: (1) benchmark for speaking skills and (2) the topics of materials. The first part of the
curriculum checklist consists of two columns. The first column lists the abovementioned six speaking abilities given in the benchmark. The first four descriptions are the language abilities that are presented in the classroom, while the last two is the task description of daily life communication. The second column is designated for recording the numbers of the matched test items. Since benchmark for speaking in the VHSCG are task-oriented descriptions, it is proper to apply them to check the degree of correspondence to see whether the test items in the TVEE match the content in the VHSCG.
The second part of the Curriculum Checklist includes two columns. The first column lists the 12 topics suggested by the VHSCG. The second column is used to record numbers of matched test items. The checklist shows the domain coverage of the TVEE after coding the targeted test items from 2010 to 2015. By using this checklist based on the VHSCG, we can see how the content of the dialogue section matches the goals and objectives of the curriculum guideline. The details of the checklist are provided in Table 3.3.1 and Table 3.3.2.
36
‧
Table 3.3.1 The First Part of the Curriculum Checklist
Descriptors for Speaking Numbers of Matched Items
Teacher-guided task description
Being able to use classroom English
Being able to ask and reply questions based on the reading texts
Being able to participate in classroom speaking practice
Being able to read aloud texts, short essays and stories4
Daily life- based task description
Being able to conduct simple daily life conversation (e.g. greetings, small talks, shopping and telephone talk, etc.)
Being able to describe things in everyday lives
Table 3.3.2 The Second Part of the Curriculum Checklist Frequencies Unit Topics
Numbers of Matched Items
Interpersonal relationships
Leisure activities and entertainment Daily life
Modern technology Culture and customs Literatures and arts Language
Industrial, commercial and agricultural knowledge Environmental education
Employment
Travel, transportation and traffic Global view
4 Although reading aloud was categorized as the teacher-guided task description, it is actually a skill used in both classroom and daily life.
37
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Speaking Constructs Checklist
On the other hand, a checklist for the constructs of speaking ability was specially designed to evaluate the construct validity of the test items. For construct-oriented descriptions, the skills of speaking ability listed by Hughes (2003) are a straightforward detailed elaboration. It defines what abilities learners need to perform certain speaking tasks. Therefore the second checklist (Table 3.4) was developed on the basis of Hughes’
three speaking skills: informational skills, interactional skills and skills in managing interaction.5 Each major skill has several sub-skills, and the present study categorized those sub-skills into certain levels according to their complexity.
The informational skills describe the ability to convey the factual information to the interlocutors. The present study divided its sub-skills into three levels according to the complexity of their communicative functions. Twelve language functions were listed in the first level. The speaking skills in this level primarily focus on expressing the speakers themselves, such as providing information, expressing need and describing events. The second informational level consisted of seven language functions that react to or are based on other speakers’ speech, for example, the skills of analyzing, paraphrasing, making comparisons and drawing conclusions. Lastly, eight behaviors that perform
speech acts to achieve communicative purposes in interaction were listed in the third level, for instance, apologizing, justifying opinions and complaining, and so on.
Interactional skills focus more on the communicative effect. They can be further categorized into three levels. The first level was composed of five initiative skills and strategies which can be used to start an interaction. The second levels include responsive skills used to express thoughts and opinions toward the involved topic. Nine language
5 The categorization of the three major skills and their sub-skills may vary and the distinction may not always be clear-cut. The present study consults with an expert to decide on this model and the complexity level for the analysis.
38
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
functions were included in this level, such as the behaviors of expression, modification and elicitation. The third level of interactional skills contained four items, and it
emphasized the ability to respond to the other speakers, such as persuasion and question assertion.
The third column is skills in managing interaction, which are the idiomatic expressions that compose conversation. It consists of seven language functions, and the examples for this category are the turn-taking skill, the language to initiate or end a
The third column is skills in managing interaction, which are the idiomatic expressions that compose conversation. It consists of seven language functions, and the examples for this category are the turn-taking skill, the language to initiate or end a