A total of 183 college students from two school sites in northern Taiwan participated in the experiment. Due to administrative constraints, this study used convenience sampling.

CHAPTER THREE METHODS

In order to answer research questions outlined in Chapter One, three different pre-reading treatments were designed to accompany each of the three reading units. The treatments of (1) Vocabulary List, (2) Self Appraisal, and (3) Case Study were compared to investigate the effects of a traditional cognition-oriented approach (Vocabulary List) versus two alternative attention-arousal strategies (Self Appraisal and Case Study). In this chapter, I will discuss the participants, the selection and adaptation of reading materials, research design, data collection procedure, instrumentation, scoring of variables, and data analysis procedures.

Participants

A total of 183 college students from two school sites in northern Taiwan participated in the experiment. Due to administrative constraints, this study used convenience sampling.

One hundred and eight of these participants were junior and senior

International Trade majors from a private university of science and technology (Site A) where the researcher worked and 75 were college freshmen from the school of management at a national university (Site B) that the researcher had access to. Participants from Site A were 104 females and 4 males enrolled in the course of Business English and those from Site B were 48 females and 27 males enrolled in Freshmen English in the spring semester of 2002. The majority of students in Site A were females while males accounted for about one-third of the student population in Site B. The proportion of females to males sampled in this study adequately reflected the gender proportion of the two respective sites. All participants had received at least three years of formal English education in junior high schools and another three years in

1

For the convenience of comparison, students from Site A are referred to as juniors and seniors here since most

of them were at the same age with juniors and seniors in a regular four-year university. However, they were

enrolled in an advanced two-year program and the two classes of students from Site A were in their first and

second year in the program.

senior high. At the time of the experiment, they were all using English-language textbooks in some of their specialized courses as required by their content-area teachers.

Site A is a university under the Taiwanese vocational and technological education system and students there are generally regarded as lower-achievers in academics. They follow the vocational education track in business since they were in vocational senior high schools. A major part of their learning in vocational senior high is centered on basic business courses such as Bookkeeping. Some of these vocational students graduate into the job market directly after high schools. Some continued their vocational training into a two-year junior college. Still others, like those participated in this study, further pursue an additional two years’ study and eventually earn their bachelor’s degrees.

Site B, on the other hand, is a large university with much more resources and admits high school graduates who rank on the very top academically among their peers. These students, during their high school years, spent relatively more time on courses such as mathematics and English in order to pass competitive unified exams to enter good universities like Site B.

Therefore, students from Site B are generally considered successful learners in their previous academic pursuit.

Site A is a very different university from Site B in terms of students’ educational background, learning styles, motivational inclination, and general academic achievement.

Site B participants were comparatively higher achievers in most schoolwork and Site A

participants had a relatively lower academic performance. However, those from Site A,

when compared with their peers in the vocational education track, demonstrated a stronger

will to persist to their present level of education despite the fact that they did not excel

academically in their previous learning experiences. Regardless of students’ EFL

proficiency in either school site, however, reading authentic English-language textbooks for

content-area courses has been a common practice and an indispensable part of their school

lives.

Selection and Adaptation of Reading Materials for the Experiment

With my past experiences in American business school as an EFL student as well as an instructor teaching the subject Organizational Behavior (OB), I chose reading materials from the field of OB for the following reasons. First, it is a subject that most business majors will encounter in academic studies and issues in OB are practical matters that happen in workplace environments. Second, the study of OB, unlike Finance and Economics, does not usually involve equations and formulas. Learners have to rely on their linguistic ability and business and world knowledge to understand concepts introduced in this course. Therefore, studying for OB is linguistically more challenging and serves better purpose from a language teacher’s perspective. Third, with such an established discipline, there are abundant well-written textbooks and supplementary materials readily available from publishers and bookstores, thus making the design of tasks and activities in the classroom much easier.

Reading materials were adapted from the OB textbook by Robbins, Organizational Behavior – Concepts, Controversies, Applications (1996, 7

edition). OB, a subject that many business and management majors will be required to study, is concerned with individual and group behaviors in various organizations. Large amount of reading is usually an essential part for learners studying this course. Rapid reading comprehension ability, general business knowledge, as well as world knowledge are necessary for an adequate understanding of such texts.

The reason to choose solely from one area and one author was to maintain consistency in general concepts, genres, languages, and styles, following the “narrow reading” concept introduced by Krashen (1982). Besides, the chosen textbook on OB has been very popular among college campuses both in the U.S. and internationally and has been reprinted and revised many times. It is, therefore, considered suitable for this study on EAP reading.

Three topics were selected — Job Satisfaction, Early Motivation Theories, and

Leadership — from the selected OB textbook. Each reading was adapted from the relevant

chapters into a coherent article for learners to read within normal class time. No attempt was made to make the language easier since it was not considered practical in everyday classrooms. All adapted articles were reviewed by a native English-speaking OB professor from Site A to make sure they were coherent and presented concepts in an understandable way for learners to read on their own. Each article was around 1,800-word long and had a Flesch-Kincaid Grade Level of about 12, calculated using Word 2000 readability facility in the tool bar. Necessary graphs, tables, and illustrations were reproduced to aid comprehension. Linguistic signaling such as boldfaces, underlines, and italics were also maintained to help learners acquire the main ideas. A summary of the content of the three reading materials chosen and related readability scales are presented in Table 6.

Table 6. Summary of the Contents of Reading Materials and Readability Indices Articles Summary of Content Word

Counts

Flesch Reading Ease Flesch-Kincaid Grade Level

Job Satisfaction

Measuring job satisfaction; what determines job satisfaction; effect on employee performance; how employees express dissatisfaction.

1853 35.5% 11.3

Early Motivation

Theories

Definition; Early Theories;

Hierarchy of Needs Theory; Theory X and Theory Y; “Motivation Theories are culture bound.”

1805 39.2% 11.9

Leadership

Definition; transition in theories;

trait theories; behavioral theories;

recent approaches to leadership;

attribution theories; charismatic theory.

1710 32.7% 12.0

Note: The most widely used Reading Ease formula developed by Flesch (cited in Zakaluk & Samuels, 1988) is based on the following rules for calculation:

RE = 206.835 - .846 wl – 1.015 sl

RE = Reading Ease, on a scale from 100 (very easy to read) to 0 (very difficult to read) wl = average word length, in syllables

sl = average sentence length, in words

Flesh soon found that there exists a need for a grade level scale to satisfy users. The Flesch-Kincaid index thus indicates the general grade level needed (for native speakers) for reading the articles concerned.

All three articles were linguistically equivalent in terms of length and difficulty on lexis,

grammar, and discourse. However, the researcher was more reserved about the conceptual

difficulties in the three articles after consulting the OB professor. Even though each article presented an independent and complete unit, they cannot be said to be irrelevant of one another. It was decided that the sequence of the three articles’ appearance in typical OB texts be maintained. Regardless of treatment sequences received by the three groups of participants, Job Satisfaction was read first since it is an introduction to an everyday concept and the causes and consequences of such concept. Early Motivation Theories introduces two major theories besides the illustration of psychological processes of motivation and was considered more difficult than Job Satisfaction. In Leadership, not only were more theories introduced, but also there was some discussion on how theory transitions occurred.

Therefore, Leadership was considered a more difficult article conceptually and was scheduled to be read the last.

Research Design

This study employed a 3 x 3 factorial design, three different articles being a three-level within-subject factor, and three different treatments as another three-level within-subject factor. Data collected from Site A and Site B were compared and then analyzed separately since they were believed to represent student populations of different characteristics and in fact, demonstrated significance differences in both pretest and dependent variables, including General Academic Motivation, EFL Learning Motivation, EFL Reading Comprehension, Academic Reading Comprehension in L1, and the three EAP Reading Comprehension scores measured in posttests.

All participants read three articles in the same order (the first article – Job Satisfaction, followed by the second article – Early Motivation Theories, and then the third article – Leadership). Participants were randomly assigned into one of the three groups to receive

treatments in different orders. Group One participants received pre-reading treatments

Vocabulary List (VOC), Self Appraisal (SAP), and Case Study (CAS) for the first, second,

and third article respectively. Group Two participants received SAP for the first, CAS for the second, and VOC for the third article. Group Three participants had CAS for the first article, VOC for the second, and SAP for the third. The above arrangement made it possible for three treatments to be received within each of the three articles. Each participant had the chance to experience each kind of treatment once after the experiment was completed.

Although there were three other possible orders of arranging the treatments (VOC followed by CAS and then SAP, SAP followed by VOC and then CAS, and CAS followed by SAP and VOC), these three orders were not considered because order effect was not considered to have a major influence in this study. First, the principle of maintaining variability as mentioned in Keller’s three design principles for attention has been accounted for through the existing three orders. Second, with weekly interval among three readings and the majority of time spent on the reading task rather than the treatment which was at the very beginning of weekly meetings, there should not be much carry-over effect attributable to treatments from one task to another. Table 7 shows the plan of participant assignment for treatments.

Table 7. Participant Assignments for Articles and Treatments

Participant Groups N Job Satisfaction Early Mot. Theories Leadership

Group One Site A: 36 Site B: 25

Vocabulary List Self Appraisal Case Study

Group Two Site A: 36 Site B: 25

Self Appraisal Case Study Vocabulary List

Group Three Site A: 36 Site B: 25

Case Study Vocabulary List Self Appraisal

Pretests were administered to collect participants’ individual information to be used as

covariates in the ANCOVA analysis. These were each participant’s General Academic

Motivation, Academic Reading Motivation in L1, EFL Learning Motivation, EFL Reading

Comprehension, and L1 Academic Reading Comprehension. These variables, if correlated

significantly with dependent variables, were controlled as covariates in the examination of

treatment effects.

The main independent variable was the treatments, including two pre-reading attention-arousal strategies operationalized in the Self Appraisal treatment and the Case Study treatment, as well as one cognition-oriented approach operationalized in the Vocabulary List treatment. Another independent variable was the three different texts. Article effect was not a concern in this study. However, as explained earlier, although efforts had been put forward to control the linguistic elements across three articles, the possible effect of concepts conveyed in the articles and their interaction with different articles remained to be observed.

Two dependent variables were participants’ situational motivation level measured at two points in time: (i) right after participants finished the treatment and before they started to read the English texts, and (ii) after they finished reading the English texts and before they answered comprehension questions. A third dependent variable was their comprehension on 15-item multiple-choice questions in L1.

Comparison with a Similar Empirical Study

Chang and Lehman (2002) is probably the only research study of this kind that is presently available for public access. In their study, Chang and Lehman investigated the motivational strategies that were designed focusing on the relevance component of the ARCS model and effects of motivational strategies on college EFL students’ situational motivation and comprehension. Therefore, it is necessary to compare Chang and Lehman (2002) with the present study in more detail.

This study is comparable to Chang and Lehman (2002) because of the following

similarities. First, participants from both studies were non-English-major Taiwanese college

students with similar learning background and socio-cultural experiences. More specifically,

participants from one of the two school sites in this study, like those in Chang and Lehman

(2002), were from the same vocational and technical educational system. The other group of

participants in this study, however, has followed a more academic-oriented educational track

and thus provides more information for comparison on different learner populations. Second, both studies based their design of motivational strategies on Keller’s ARCS model. Third, tasks in both studies involved learning of concepts through the medium of English as a foreign language. Finally, both studies concerned learners’ situational motivation and comprehension as two major dependent variables.

There also exist a few major differences between Chang and Lehman (2002) and this

study. First, the learning tasks were different in length and complexity. Chang and Lehman

(2002) used a one-shot experiment lasting for 75-80 minutes, with the length of the video

component of the program around 12 minutes. This study, however, repeated the same

reading task three times with three different articles over a one-month period and each time

participants spent 60 minutes on the tasks. The Chang and Lehman study had the acquisition

of a more general concept – criticism and communicative competence as the main purpose of

the learning program, while in this study the objectives involved reading EAP texts for

comprehension of more complicated academic-purpose theories and concepts. The levels of

task difficulty thus were very different and may produce fundamentally different results on

the effectiveness of motivational strategies. Second, the Chang and Lehman study chose to

focus on the second component of the ARCS model, i.e. relevance, as the main thrust of

motivational design, while this study focused on the first component, attention. Third, this

study took into account not only learners’ trait motivation but also their EFL reading

proficiency as factors that may influence results measured in dependent variables; while in the

Chang and Lehman study, only intrinsic motivation was considered relevant and this measure

was viewed as parallel to motivational design and used as one of the two factors in a 2 by 2

factorial design. Moreover, in order to better control variables that could influence the

results, this study measured learner’s trait motivation and proficiency level by three scales and

two tests. After examining the correlation between relevant dependent variables with these

pretest variables, several pretest variables were selected as covariates for different ANCOVA

analyses.

With reference to the existing study on the effectiveness of motivational strategies for EFL learning, this study, in addition to providing another piece of empirical evidence, helps us examine the related issues more comprehensively for more careful implications.

Data Collection Procedures

The researcher met with the students from both school sites separately during their weekly class hours. There were four meetings, with the first one for administering pretests and the remaining three for participants to complete one article in each meeting. Prior to conducting the official experiment with all participants, all experimental materials were piloted with two student informants from Site A. Due to administrative constraints, Site B students were not able to serve as informants in the pilot stage. The length allotted to each part of the experimental processes was a result of discussions with student informants after they practiced each part. Detailed procedures in the four meetings are presented below.

First Meeting (Pretests)

1. The researcher gave brief introduction to participants about what they were required to do in the whole experimental process.

2. Participants were given two pages of Likert-type questionnaire to tap their General Academic Motivation (10 statements), Academic Reading Motivation in L1 (10 statements), and EFL Learning Motivation (20 statements). They were given 10 minutes to finish this part.

3. Participants were then given 25 minutes for an EFL Reading Comprehension test.

4. Another 30 minutes were given for Academic Reading Comprehension test in

participants’ native language, Mandarin Chinese. In the first 20 minutes, they were to

quickly read the whole article. At the end of the 20 minutes, 15 multiple-choice

questions were distributed. Participants were then given another 10 minutes to answer these 15 questions by referring back to the reading material.

Table 8 presents a summary of the sequence and time allocated in all parts of the pretests.

Table 8. Time Allocated to Parts of Pretest

Parts of Pretests Minutes

3 motivation-related questionnaires 10

EFL reading comprehension test 25

L1 reading without comprehension questions 20

Answering comprehension questions with reference to L1 reading material 10

Total 65 Procedures Identical for the Second, Third, and Fourth Meetings

1. Participants were given 10 minutes to work on the assigned treatments. They then read the guiding questions and were reminded to think about these questions later when they were reading the articles.

2. At the end of the first 10 minutes, participants rated against the 10 Likert-type statements on their situational motivation to indicate their perceived interest for the upcoming reading material at this point.

3. Thirty-five minutes were allotted to the reading of the texts. Participants were encouraged to refer to dictionaries and other resource materials they brought to the classroom. They were also encouraged to mark and note on their texts when they felt the need as they read. It was also stated in the instructions that they should try to pay attention to both iconic and non-iconic signaling in the text for better understanding. In this 35-minute reading period, participants were expected to work individually without any discussion with the researcher or classmates.

4. After the 35-minute reading was finished, participants were required to report on the

same situational motivation questionnaire to indicate their perceived interest at this

second point in time.

5. The 15 comprehension questions were then distributed to participants. They were given 15 minutes to answer the questions and were allowed to refer back to the texts and the notes they made.

Table 9 presents time allocated for each part of the entire reading experiment that is identical across three readings.

Table 9. Identical Procedures in the Three Meetings for the Reading Experiment Identical Parts for the 2

, 3

, and 4

Meetings Minutes Pre-reading treatments + rating of pre-reading situational motivation 10 Reading the articles + rating of post-reading situational motivation 35

Answering comprehension questions in L1 15

Total 60 Wrap-up Preference Check in the Fourth Meeting

At the end of the fourth class meeting when participants completed the whole experiment, written instructions were given for learners to choose two treatments and briefly explain their reasons in writing. Participants were asked “If you are going to read similar articles again and you have the choice among three types of pre-reading materials to receive, which one would you choose to have? Why? Which one would be the least possible choice?

Why?” The original question provided a simulated situation and asked for participant choices based on their actual experiences. For the sake of convenience in reference, the term

“most preferred treatments” and “least preferred treatments” are used throughout the rest of the paper.

Instrumentation

In this section, the procedures of how various instruments were selected and adapted for the purpose of this investigation will be described.

Pretests

Variables that may influence participants’ situational reading motivation and reading

comprehension were controlled so that treatment effects could be analyzed without the interference of participants’ individual motivational orientation and EAP reading related abilities. Therefore, motivational scales and proficiency tests were administered in the pretest stage to collect data on participants’ original motivational orientation and EAP reading ability. Attitude/Motivation Test Battery (AMTB, Gardner, 1985; Gardner, Tremblay, &

Masgoret, 1997) was considered first as an instrument to collect data on learner motivation.

In order to address the aspects of general academic learning and academic reading in L1, which was not covered in AMTB, two more scales were sought and adapted for the purpose of this study. As for EAP reading related abilities, it was decided that reading comprehension in L1 and EFL be tested separately so that information about participants’

academic reading competence in L1 and EFL reading ability could be observed more clearly.

In the following sections, I will explain in detail the origins and adaptation procedures for the five pretests: (1) questionnaire on General Academic Motivation, (2) questionnaire on L1 Academic Reading Motivation, (3) questionnaire on EFL Learning Motivation, (4) EFL Reading Comprehension test, and (5) Academic Reading Comprehension test in L1.

General Academic Motivation

This 10-item pretest, which aimed at finding out participants’ general academic motivation, was adapted from Montgomery’s (1999) Scale of Academic Motivation (SAM).

The original scale, developed in the U.S., was “designed to be a measure of motivation and characteristics of students’ (grades 6 – 12) classrooms and homes that research has shown to be associated with academic motivation” (Montgomery, 1999, p. iv). There were a total of 78 statements for learners to rate on 5-point and 3-point Likert scales. The overall design of the original SAM is shown in Table 10. Since statements in Part II and Part III of SAM are more situation specific and phrased especially for children age 6 to 12, they were not considered applicable for this study.

Ten items from the original 14 in its Part I were selected for the present participant

population. The selected 10 statements were then translated into participants’ native language, Mandarin Chinese, by the researcher. A Taiwanese professor teaching EFL from Site A was invited to double-check the translation to ensure that the meaning of original statements was clearly conveyed and the translated version used language appropriate for the college students in this study. The adapted 10-item Chinese version of General Academic Motivation (GAM) and the original English items are shown in the first part of Appendix A.

Table 10. Components of the Original Scale of Academic Motivation

Sections Description Number of Statements

Part I About Myself 14

What happens in my classroom 27

Part II

What happens at home 10

What motivates me in my classroom 27 Part III

What motivates me at home 10

Academic Reading Motivation

This 10-item questionnaire on college students’ academic reading motivation was developed from Wigfield, Guthrie, and McGough’s (1996) Motivations for Reading Questionnaire (MRQ). The 54-item MRQ assesses 11 possible dimensions of reading motivations for students in late elementary school and middle school. The 11 categories are:

Reading Efficacy (4 statements), Reading Challenge (5 statements), Reading Curiosity (6

statements), Aesthetic Enjoyment of Reading (6 statements), Importance of Reading (2

statements), Compliance (5 statements), Reading Recognition (5 statements), Reading for

Grades (4 statements), Social Reasons for Reading (7 statements), Reading Competition (6

statements), and Reading Work Avoidance (4 statements). In order to adapt the

questionnaire for the college student population and for a concern on academic reading rather

than general reading, the researcher chose 5 items from Reading Challenge and another 5

from Compliance to form the present questionnaire for academic reading motivation. The

same translation and check-up procedure done with general academic motivation was

repeated here for academic reading motivation. The adapted academic reading motivation

questionnaire and the original English items are presented in the second part of Appendix A.

EFL Learning Motivation

This 20-item questionnaire on participants’ EFL learning motivation was adapted from Gardner et al.’s (1997) new version of Attitude/Motivation Test Battery (AMTB). The AMTB contains 100 statements designed under Gardner’s socio-educational model of second language learning attitude and motivation in the French-as-a-Second-Language learning context in Canada. The 1997 revised AMTB has the following categories: Attitudes Toward French Canadians (10 statements, 5 positively and 5 negatively phrased), Attitudes Toward Learning French (5 positive and 5 negative statements), Desire to Learn French (5 positive and 5 negative statements), French Class Anxiety, (5 positive and 5 negative statements), French Use Anxiety, (5 positive and 5 negative statements), Interest in Foreign Languages (5 positive and 5 negative statements), Instrumental Orientation (4 positive statements), Integrative Orientation (4 positive statements), Motivational Intensity (5 positive and 5 negative statements), Self-Confidence (10 positive statements), Self-Confidence (Ability Controlled) (6 positive statements), and Self-Confidence (Given Ability) (3 positive and 3 negative statements). Many categories such as attitudes and integrative orientation are not directly relevant for learners in a foreign language situation. Research done in the Taiwanese EFL context (e.g. Chang, 1999; Warden & Lin, 2000) has shown that instrumental orientation has always been an important factor for the student population in Taiwan. For the purpose of this study, the researcher decided to use items from Desire to Learn and Motivational Intensity in order to distinguish participating students on their EFL learning motivation. The same procedure of translation and check-up for questionnaires on general academic motivation and academic reading motivation was repeated here. The adapted and translated questionnaire on EFL learning motivation is shown in the third part in Appendix A.

These three scales, (a) general academic motivation, (b) academic reading motivation,

and (c) EFL learning motivation were all arranged in the form of statements on a 7-point

Likert scale. They were piloted with 103 students other than the participants in this study at Site A before they were officially administered. The Cronbach alpha calculated with these scales in the pilot stage was .92. Since reliability coefficients were above satisfactory levels, no item was deleted or revised for subsequent studies.

EFL Reading Comprehension Test

The EFL Reading Comprehension pretest contains 4 articles taken from the reading comprehension part of General English Proficiency Test (GEPT) and Test of English as a Foreign Language (TOEFL). The passages were arranged in the order from the shortest to the longest and from the easiest to the most difficult. There were 3, 2, 5, and 10 comprehension questions following each of the passages. In order to make sure participants’

proficiency scores were not contaminated by the possibility that some students might have done the same tests before, a safeguard question was included on the answer sheet to screen out those who had done the same tests. The data from those participants who indicated previous experiences with the content was given up and not included in the analysis.

Participants were given 25 minutes to work on this comprehension test without consulting dictionaries or discussing with teachers and classmates. The readability indices of these 4 passages are presented in Table 11. This 4-page test is shown in Appendix B.

Table 11. Readability Scales of the Four Pretest EFL Reading Passages Readability Scales

Passages

Word Counts Flesch Reading Ease

Flesch-Kincaid Grade Level

1 126 61.7% 7.7

2 141 79.4% 4.4

3 260 45.9% 11.8

4 252 28.8% 12.0

Academic Reading Comprehension Test in L1

The academic reading comprehension pretest was to distinguish learners by their

academic study skills using their native language. After discussions with the

native-speaking American OB professor, I chose from the OB textbook to be used in the experiment an article from the chapter on Foundations of Individual Behavior to form a 6.5-page reading text. With reference to a Chinese translated version of the chosen OB textbook (Li, Li, & Chao, 1994), I adapted the translated text to make it long enough for participants to exercise their various study skills such as skimming and scanning, but short enough to be finished in the allotted time frame. It was made to include abstract concepts arranged in hierarchical order, with several sub-theories introduced and explained, and illustrations, tables, and highlights from the original textbook reproduced.

The participants were expected to demonstrate their experiences and competence in academic reading regardless of their English proficiency. The content of the Chinese text was very relevant to subsequent EAP readings in the experiment except that it was written in these participants’ L1 rather than in their L2. Three safe-guard questions were included in the end of the reading text to screen out anyone who had taken a similar course or read similar articles. The researcher put a short written instruction before the reading and a 15-item multiple-choice comprehension test at the end to examine participants’ ability in understanding, extracting ideas, and inferring information from the text. In addition to gathering information on participants’ academic reading comprehension ability in L1 for covariate to be controlled in ANCOVA analyses, this pretest served two more purposes.

Conceptually, the content was a basis for topics to be read later. Procedurally, this L1 article familiarizes participants with the kind of speedy academic reading required in later experimental procedures. This pretest is presented in Appendix C.

Two student informants from Site A completed both the EFL reading comprehension test

and the academic reading skill test in L1 in their original forms. Due to administrative

constraints, students from Site B did not participate in this pilot stage. Detailed discussions

with these informants then followed to modify the materials and to ensure appropriate

difficulty level of the texts for participants, to clear up trivial problems in the language, and to

make sure the articles have the adequate length for the time given to participants. A summary of the pretest instruments is presented in Table 12.

Treatments

The three treatments (Vocabulary List, Self Appraisal, and Case Study) were positioned in the pre-reading stage to facilitate participants before they started to read. They were short pre-reading activities written in L1 consisting of three parts: a written introduction for the overall pre-reading and reading activity, the treatment part, and four or five open-ended questions that aimed at guiding participants to relate the pre-reading activity to subsequent reading materials.

Table 12. Summary of Pretests Pretest Type Name

of Pretest

Sources Language Used

Time Needed (minutes)

Description

GAM Montgomery, 1999

L1 10 statements

ARM Wigfield, et al., 1996

L1 10 statements

Motivation- Related Pretests

ELM Gardner, 1997 L1

10

20 statements ERC GEPT, TOEFL L2 25 4 passages and

20 questions Ability-

Related

Pretests ARC Robbins, 1996 L1 20+10* a 6.5-page text and 15 questions

Note : GAM= General Academic Motivation; ARM= Academic Reading Motivation;

ELM= EFL Learning Motivation; ARB= Achievement-Related Beliefs; ERC= EFL Reading Comprehension Test; ARC= Academic Reading Comprehension Test in L1.

* Participants were first given 20 minutes to read the text without the comprehension questions.

Then the questions were given together with the text for the next 10 minutes.

For the purpose of this study, I examined in detail all materials available in the chosen

OB text. At the end of each of the eighteen chapters, there are various kinds of

supplementary materials readily available for my purpose. Table 13 shows a summary of

types of supplementary materials that appeared in each chapter.

Table 13. Summary of Supplementary Materials Available in the OB text Types of Supplementary Material Description

OB in the News Point/Counter-point

Learning About Yourself Exercises Working With Others Exercises Ethical Dilemma Exercises Case Incidents

Video Cases

Short news clips inserted in the texts.

Articles presenting two sides of arguments Self appraisal questions for personal traits Tasks by interacting within a group Articles on ethical issues

Real-world cases Cases on video

Based on Keller’s (1987b) guidelines for the attention-getting strategies in his ARCS model as previously illustrated in Table 3 of Chapter Two, I considered Case Incidents and Learning About Yourself Exercises to have the most potential in arousing learning attention and I adapted these two into “Case Study” and “Self Appraisal” for the purpose of this study.

The Case Study treatment met the requirements for Inquiry Arousal and the Self Appraisal treatment met the Perceptual Arousal as indicated in Table 14 below.

Table 14. Keller’s Attention-getting Guidelines and the Corresponding Treatments Subcategories & Process

Questions

Main Supporting Strategies Corresponding Treatments in this Study

Perceptual Arousal

What can I do to capture their interest?

Create curiosity,

wonderment by using novel approaches, injecting personal and/or emotional material.

Self Appraisal

Inquiry Arousal

How can I stimulate an attitude of inquiry?

Increase curiosity by asking questions, creating paradoxes, generating inquiry, and nurturing thinking challenges.

Case Study

Variability

How can I maintain their attention?

Sustain interest with variations in presentation style, concrete analogies, human interest examples, and unexpected events.

Participants encountered different treatments each time.

It was not expected that motivation-oriented treatments would have any positive effects

on reading comprehension, since literature on motivation has generally agreed that motivation

does not lead directly to comprehension (Dörnyei, 2001b). In order to compare their effects

with more traditional language-oriented, instead of motivation-oriented, facilitations for EAP

reading, I designed a third set of materials – Vocabulary Lists. The comparison of these three treatments was the major focus of this study.

Vocabulary Lists for the three articles were prepared by the researcher, following the way most EFL reading materials were annotated with learners’ linguistic needs in mind. Self Appraisals for the three articles were adapted from the “Learning About Yourself Exercises”

appended to the relevant chapters from Robbins (1996). Case Studies were adapted from the

“Case Incidents” supplementary readings that were also appended to the main texts in the relevant chapters. Vocabulary Lists were a traditional approach aiming at facilitating comprehension. In order to serve the attention-arousing purposes intended in the study, I designed the Self Appraisal and the Case Study treatments based on guidelines (Keller &

Burkman, 1993) for the selection and integration of instructional materials, i.e. ease of administration, relevance to the main topic, having practical restrictions considered, and acceptable to students. The following is a detailed description of the three different types of treatment.

Language of Treatment Materials

While the supplementary materials had the potential for stimulating learners to read the

relevant EAP texts, they were seldom used in content classes since they present an additional

linguistic burden for EFL students. Learners are busy enough decoding main texts, the

supplementary materials, however interesting and stimulating, can only make EFL students

more confused lexically and syntactically. The effects they may have on native-speaking

learners would not show on EFL students because of the language. It was hypothesized that

using these materials in learners’ L1 will generate in EFL students similar effects that their

native-speaking counterparts may experience. In order for the attention-getting strategies to

function, translated versions of this OB textbook were sought. There are more than one

version of Chinese translation for the chosen textbook and they are within easy access in the

marketplace. It was decided that the one that has been most popular (Li, Li, & Chao, 1994)

should be used.

The First Treatment – Vocabulary List

Twenty key vocabulary items were chosen from each of the three articles and their Chinese equivalents were provided on separate lists. They were explained so as to best facilitate participants’ understanding of the article they were to read. For example, the word

“voice” in the article Job Satisfaction indicates one of the four kinds of employee attitude when they are dissatisfied with their jobs. Some employees may constructively and actively transfer their suggestions into a kind of “voice” to the organization. A common dictionary explanation on the word “voice” does not help participants understand the important concept and may even confuse them, especially when learners’ comprehension on other parts of the same article is not complete. The L1 equivalents provided in Vocabulary Lists, therefore, were directed to the articles so that better comprehension could be ensured. Twenty key vocabulary items from each of the 3 articles were selected to form 3 similar Vocabulary Lists.

Participants were allowed to refer back to these supporting materials in later reading and when answering comprehension questions.

Previous studies on Taiwanese college students also indicate that “previews that include vocabulary instruction should be particularly facilitative for difficult selections and with less competent and confident readers” (Chen & Graves, 1995, p. 682). Therefore, the effects pre-reading vocabulary treatments in this study have with longer and more complicated expository EAP texts can serve as a good reference and comparison with more innovative motivational teaching materials.

The Second Treatment – Self Appraisal

The second treatment, Self Appraisals, included short exercises that asked participants to

respond to short questionnaires concerning their own personalities, beliefs, and attitudes

related to the topics to be read. The materials were translated into Chinese and adapted by

the researcher in consultation with the English-speaking OB professor. Scoring schemes and

explanations for scores were provided to participants when they finished answering relevant questions. The same guiding questions given at the end of the first treatment Vocabulary Lists also appeared in the second treatment, Self Appraisals. However, all questions were modified toward a consideration of personal relevance. For example, the question “What is the process of motivation?” in the first treatment, Vocabulary List, became “What is the process of motivation? Have you experienced such process?” in the second treatment, Self Appraisals. Participants were also allowed to refer to these pre-reading materials in their later reading to answer comprehension questions. The three Self Appraisal treatments for the three articles were “What do you value? – Personal Job Satisfaction Orientation Test,”

“What motivates you? – Personal Motivational Orientation Test,” and “What’s Your LPC (least-preferred co-worker) Score? – Personal Leadership Style Test.”

The Third Treatment – Case Study

The third treatment, Case Studies, was short case descriptions presenting real-world business problems in organizations that were relevant to the topics to be read. Materials were translated and adapted by the researcher with consultation from the OB professor.

Participants read the translated version and similar guiding questions that were modified toward a concern for the case read. For example, the question on the process of motivation was now written as “What is the process of motivation? Can what is said in the text explain the truck drivers’ behaviors in the case studied?” The 3 Case Studies for the 3 articles Job Satisfaction, Early Motivation Theories, and Leadership were “Benny and Smith –

Productivity Problems in Crayola Company,” “Nissan vs. American Trucks – Employee Turnover Problems,” and “The Case Against ‘Vision’ – Different Leadership Style in Chrysler” respectively.

Posttests

Posttests include pre-reading situational motivation, post-reading situational motivation,

and comprehension.

Pre-reading and Post-reading Situational Motivation

This 10-item Likert-type questionnaire was translated from Schraw, Bruning, and Svoboda’s (1995) Perceived Interest Questionnaire (PIQ) for reading. It measured the overall situational motivation the reader has on the relevant article. Schraw, et al. (1995) claimed that their PIQ contained statements (e.g., “I thought the story was very interesting,”

“I would read this story again if I had the chance.”) that were expected to yield a single situational motivation factor. In order to examine the effect of treatments before it was interfered by the reading material and after it was mediated by the actual reading experiences, I measured participants’ situational motivation for reading at two points in time. The same 10 translated statements were presented to participants twice on a 7-point Likert scale, once right after they finished the pre-reading treatment and before they started to read the English article, and once after they finished reading the entire article and before they answered the comprehension questions. The language in the pre-reading questionnaire was slightly modified to reflect the fact that participants were anticipating their reading experience instead of reflecting on it.

Reading Comprehension Tests

With reference to the Test Item File, a teachers’ resource accompanying Robbins (1996), and general principles for designing reading comprehension questions, the researcher prepared 15 multiple-choice questions in learners’ L1 for each article to measure participants’

comprehension level.

Each of the three complete reading units included three pre-reading treatments (each

participant experienced only one of the three treatments in one single reading unit), the

questionnaire on pre-reading situational motivation, the article, the questionnaire on

post-reading situational motivation, and the comprehension questions. All the materials

were piloted with the two student informants from Site A and were later modified according to their feedbacks prior to the formal experimental procedures. The three complete reading units are shown in Appendices D, E, and F.

Scoring of Variables

The scoring mechanism of variables is presented below:

Pretest Variables

1. Scores for General Academic Motivation had a range from 10 to 70, calculated by summing up the scores given to the 10 statements, with Items 2, 4, 6, 8, and 10 reversely coded.

2. Scores for L1 Academic Reading Motivation had a range from 10 to 70, calculated by summing up the scores given to the 10 statements, with Items 2, 4, 6, 8, 10 reversely coded.

3. Scores for EFL Learning Motivation ranged from 20 to 140, calculated by summing up the scores given to the 20 statements, with Items 2, 3, 6, 7, 9, 12, 14, 16, 17, and 19 reversely coded.

4. Scores for EFL Reading Comprehension ranged from 0 to 20. One point was given to each correct answer to the 20 multiple-choice questions.

5. Scores for Academic Reading Comprehension in L1 ranged from 0 to 15. One point was given to each correct answer from the 15 multiple-choice questions.

Dependent Variables

1. Scores for pre-reading situational motivation had a range from 10 to 70, calculated by summing up the scores given to the 10 statements.

2. Scores for post-reading situational motivation had a range from 10 to 70, calculated by summing up the scores given to the 10 statements.

3. Scores for comprehension ranged from 0 to 15, calculated by giving one point to each

correct answer to the 15 multiple-choice questions.

Data Analysis Procedures

The actual number of data sets collected from the entire experiment by site, pretest, article, and treatment is presented in Table 15. Dropouts, incomplete data, and those screened out were not included for analysis.

Table 15. Number of Data Sets Collected by Site, Pretest, Article, and Treatment

Article 1 Article 2 Article 3

N Pretest

VOC SAP CAS Sum VOC SAP CAS Sum VOC SAP CAS Sum

Site A 104 33 35 33 101 35 33 27 95 24 32 30 86

Site B 71 21 20 23 64 19 20 20 59 23 18 15 56

Total 175 54 55 56 165 54 53 47 154 47 50 45 142

Note: VOC = Vocabulary Lists; SAP = Self-Appraisals; CAS = Case Studies.

Descriptive statistics for all variables were calculated first, including number, means,

and standard deviations. Comparison between two school sites on all pretest and dependent

variables were made by independent-samples t tests to see if participants from two sites differ

significantly with each other. Since the answer was positive, follow-up analyses for

treatment effects and participant preference were done by separating the sites. Correlation

analysis among variables was employed to examine the relationship between variables and to

help determine the selection of covariates for subsequent ANCOVA analyses. Two-way

ANCOVAs were conducted to examine article, treatment, and interactive effect on

pre-reading situational motivation, post-reading situational motivation, and comprehension,

with pretest variables that significantly correlated with the respective dependent variables

used as covariates. Two-way contingency table analyses were performed to test the

homogeneity of proportions for participants’ preference over treatments. The reasons

participants provided for their preference and dislike were also analyzed by grouping similar

reasons together. Finally, MANOVAs were conducted to see if participants with different

preference had significantly different pretest scores. The analyses were carried out using

SPSS 10.0 for Windows. Significance level for all analyses was set at .05.

In this chapter, the scope of the study, including the participants and the selection and

adaptation of materials, the research design, the data collection procedures, instrumentation,

and data analysis procedures were presented. The results of the analyses will be presented in

the following chapter.