• 沒有找到結果。

Integrating portfolios into learning-strategy-based instruction for EFL college students

N/A
N/A
Protected

Academic year: 2021

Share "Integrating portfolios into learning-strategy-based instruction for EFL college students"

Copied!
25
0
0

加載中.... (立即查看全文)

全文

(1)

instruction for EFL college students

NAE-DONG YANG IRAL 41 (2003), 293–317 0019042X/2003/041-0293 c Walter de Gruyter Abstract

What teachers can do to facilitate their students’ learning and develop auton-omy has been one of the major concerns in second language instruction. This article explores the use of portfolios as a tool to facilitate student learning and develop learner autonomy. A pilot study with a class of 42 college students in 2000–2001 and then a formal study with a class of 45 students in 2001–2002 were conducted. Portfolios were integrated into the framework and procedures – i.e., diagnosis, preparation, instruction, and evaluation – for strategy training in a freshman English course at a four-year public university in Taiwan. Stu-dents’ English proficiency was tested via a pretest and a post-test. Information about students’ beliefs and attitudes about using portfolios and their learning style preferences was collected by both open-ended and Likert-scaled question-naires. The study found that portfolios raised students’ awareness of learning strategies, facilitated their learning process, and enhanced their self-directed learning.

1. Introduction

The importance of helping students to become effective and autonomous learn-ers has been recognized by many second language teachlearn-ers and researchlearn-ers in recent years. It is also the goal of language learning strategy instruction (Oxford 1990; Oxford and Leaver 1996; Wenden 1991). Williams and Burden (1997), for example, have pointed out that language teachers should go far beyond the transmission of knowledge and should empower students by assisting them in acquiring the knowledge, skills, and strategies needed to become autonomous learners who can take responsibility for their own learning.

Since little research has been done concerning the use of portfolios with col-lege students in an English as a foreign language (EFL) environment, this study investigated the effectiveness of integrating portfolios into a learning

(2)

strategy-based EFL course. The study also explored the effects of portfolios on helping college students to become effective, autonomous learners.

The organization of this article is as follows. Section 2 presents a review of relevant literature, followed by the significance of the study (Section 3), the research questions (Section 4), and the methodology (Section 5). Section 6 offers results and discussion, followed by caveats snd suggestions (Section 7), and conclusions (Section 8).

2. Research review

2.1. Nature and use of portfolios

As a purposeful collection of student work, portfolios show student effort, progress, achievement, and self-reflection in one or more areas (Paulson and Paulson 1991). Portfolios have been used as an assessment device in composi-tion instruccomposi-tion, in other disciplines, and in a variety of educacomposi-tional programs. As the use of portfolios may encourage students to take more initiative and control of their learning, to become more autonomous learners, and to reflect on their learning over time, portfolios may be a useful tool for learner training. Portfolios are collections of students’ work selected by the students (with the teacher’s guidance) to represent their learning experiences. Just as an artist or a designer gathers paintings and works in a portfolio to show prospective clients, portfolios usually involve students selecting and gathering samples of their second language use (such as compositions and video clips) into a folder to show peers, parents, and others. Paulson, Paulson, and Meyer (1991) de-scribed portfolios as an intersection of instruction and assessment. A portfolio should include information about the activities that produced the portfolio, the process of development (possibly including drafts and revisions), and a nar-rative in which the student reflectively describes the learning that took place. Materials in the portfolio can represent the whole learning content (Gottlieb 1995; Yueh 1999). Portfolios provide a type of personal assessment that is di-rectly related to classroom activities. That is why they are deemed particularly appropriate for assessing language learning processes (Belanoff and Dickson 1991; O’Malley and Pierce 1996).

Portfolios have thus become a popular method of assessment in recent years. Interest in portfolios as assessment devices first emerged in the field of compo-sition instruction, but it evolved almost concurrently in other disciplines in the language arts such as reading, literacy, and oral language development (e.g., Farr and Tone 1994; Hewitt 1994). Portfolio assessment was then adopted in teacher training and various educational programs at different levels (e.g., Mokhtari et al. 1996; Reed 1997; Spence and Theriot 1999). English teachers have also used portfolios to evaluate and enhance student learning (e.g., Fradd and Hudelson 1995; Murphy 1997).

(3)

A number of differences exist between portfolios and traditional assessment, according to existing research (e.g., O’Malley and Pierce 1996; Paulson and Paulson 1991). First, portfolios can assess students’ learning in a wide range of areas, while more traditional assessment can only evaluate students in lim-ited areas. Second, students participate in the process of portfolio evaluation, while more traditional assessment is teacher-centered and usually conducted in a more mechanical way. Third, portfolios take individual differences into consideration, while more traditional assessment uses the same test for every-one. Fourth, while the teacher is the only evaluator in traditional assessment, portfolio evaluation involves everybody in the assessment process, including students, teachers, and peers. Fifth, self-evaluation is one of the learning ob-jectives for portfolios, but not for traditional assessment. Sixth, portfolio eval-uation stresses improvement, effort, and achievement, while traditional assess-ment only focuses on results. Finally, portfolios connect learning, assessassess-ment, and instruction whereas these three aspects are separated in traditional assess-ment. In brief, portfolios are considered a better tool for student learning.

The advantages of using portfolios include offering students a concrete way to value their work, reflect on their performance, enhance their learning and autonomy, alter their views of the teacher’s role and their own role, encourage themselves to take responsibility for their learning, and involve themselves in the assessment process (Brown 1998; Paulson, Paulson, and Meyer 1991; Yueh 1999). Through describing students’ growth, portfolios can help students to think holistically and provide the evaluators (teachers, principals, and others) with records of learning activities.

However, some researchers have also cautioned that certain problems, such as politics, logistics, interpretation, reliability, and validity, could arise, espe-cially if using portfolios as an assessment tool on a large scale (for an overview, see Norris 1996). Terwilliger (1997), in particular, has questioned some claims made by proponents of portfolios. As Terwilliger (1997) has indicated, it is not clear that portfolios are useful assessment devices in all subject matters and at all grade levels, though portfolios may have value as instructional aids. 2.2. Learning strategy-based instruction

Learning strategy-based (LSB) instruction is a learner-focused approach to teaching that emphasizes both explicit and implicit integration of language learning strategies in the language classroom, with the goal of creating greater learner autonomy and increased proficiency (Brown 2001; Cohen 1998). Learning strategy-based instruction helps students to become more aware of available strategies, to understand how to organize and use strategies systemat-ically and effectively, and to learn when and how to transfer strategies to new contexts (Brown 2001; Yang 2002). Researchers have suggested a sequence

(4)

of steps to follow when conducting learning strategy-based instruction (e.g., O’Malley and Chamot 1990; Oxford 1990, 1993; Weinstein and Underwood 1985). These general procedures are summarized as follows:

– Diagnosis: Identifying and assessing students’ learning strategies through observation, interviews, questionnaires, diaries, or think-aloud procedures. – Preparation/Awareness-Raising: Developing students’ awareness of

differ-ent strategies; explaining the concept and importance of learning strategies; developing goals for strategy use and affective control for individuals and the entire class.

– Instruction: Providing direct and informed instruction on learning strategies through explanation, modeling, practice, and integration; providing different practice opportunities with varied learning tasks or content.

– Evaluation: Helping students evaluate their own strategy use; evaluating the whole strategy training and revising the training component if necessary.

3. Linking portfolios to learning strategy-based instruction: Significance of the current study

There has been little research on the relationship of portfolios to learning strat-egy-based instruction, and that is why the current study was conducted. As this study demonstrates, portfolios can serve at least three major functions in learning strategy-based instruction. First, portfolios can document the plan-ning, learplan-ning, monitoring, and evaluation processes. Second, portfolios can help raise students’ awareness about learning strategies and autonomy. Third, through compiling portfolios, learning strategy-based instruction can be rein-forced.

4. Research questions

This study posed three research questions:

1. What do college students think about the use of portfolios in their EFL class?

2. What are the benefits or problems of using portfolios as tools to enhance learning in an EFL course?

3. Which assessment tool – portfolios or traditional pen-and-paper tests – is more effective in facilitating EFL student learning and autonomy?

5. Methodology of this study

5.1. Participants

Participants were students enrolled in a freshman English course (with au-ral/oral training) taught by the researcher. Altogether 42 students participated in the pilot study in 2000, including 27 females and 15 males. Among them,

(5)

21 (50 %) were majoring in law and social sciences, 19 (45 %) were study-ing in the College of Management, and 2 students were majorstudy-ing in electronic engineering.

In the first semester of the 2001–2002 academic year, 45 students partici-pated in the formal study. Among these students, 13 (29 %) were majoring in law and social sciences and 32 (71 %) were studying in the College of Man-agement. There were 36 female and 9 male students.

In the second semester of the 2001–2002 academic year, 36 students, in-cluding 28 females and 8 males, participated in the formal study. Of these 36 students, 11 (31 %) were law or social science majors, and 25 (69 %) were man-agement majors. Basically they are the same group of students as those in the first semester since this course is a two-semester course with normal student attrition and addition. Eleven students dropped out after the first semester and two new students were added in the second semester. There is no significant difference between the two groups.

5.2. Procedures and schedules

The researcher conducted the pilot study in 2000–2001 and the formal study in 2001–2002. In both the pilot study and the formal study, portfolios were inte-grated into the framework and procedures – i.e., diagnosis, preparation, instruc-tion, and evaluation – for learning strategy-based instruction in a freshman En-glish course. The pilot study and the formal study generally followed the same procedures. However, there was one main difference. In the pilot study, when students’ reactions were solicited after the portfolio sharing (in mid-semester) and portfolio presentation (at the end of the first and second semesters), open-ended questions were used. In the formal study, both open-open-ended and five-point, Likert-scaled questions were included in the questionnaires at the times when data were gathered.

The general procedures of integrating portfolios into the learning strategy-based instruction are outlined in Appendix A. They are summarized by semes-ter as follows.

5.2.1. First semester. At the beginning of the first semester, for the purpose of diagnosis, an English learning questionnaire was used to assess students’ beliefs and learning strategy use, and an English proficiency test was given as a pretest. For preparation, the instructor (researcher) introduced the guidelines (Appendix B) to students on how to prepare a portfolio.

During the semester, portfolios were integrated into learning strategy-based instruction. Activities, such as setting goals, making learning plans, monitor-ing progress, and understandmonitor-ing learnmonitor-ing styles were conducted in class. Mid-semester portfolio sharing and students’ self-reflection (about plans, progress

(6)

so far, resources to share, and problems and possible solutions) were imple-mented.

Finally, at the end of the semester, English listening and reading tests were given as a post-test. Portfolio presentation and evaluation (including self-evalu-ation, peer evaluself-evalu-ation, and instructor evaluation) were completed, followed by a course evaluation and portfolio survey.

Below is the first-semester schedule:

a. Specifying the purposes of portfolios, matching the purposes with instruc-tional goals, and integrating portfolios into classroom activities and the stu-dents’ work and assignments; developing syllabus and course materials (1st month).

b. Introducing the guidelines to students for preparing portfolios (2nd month). c. Integrating portfolios into class instruction; recording students’ progress

and performance (3rd–5th month).

d. Arranging for mid-semester portfolio showcase and students’ self-reflection; allowing students to look at each other’s portfolios to help students under-stand what makes a good language portfolio (4th month).

e. Allocating time for end-of-semester portfolio presentation and final peer and self-evaluation. Administering first portfolio survey and learning style survey (6th month).

5.2.2. Second semester. To compare the impact of portfolio assessment and traditional assessment (pen-and-paper tests) on students’ learning, students were evaluated by both language proficiency tests and a portfolio assessment in the second semester of the 2001–2002 year. Information on students’ learn-ing progress and classroom performance was recorded durlearn-ing the semester. Likewise, students were surveyed about their reactions toward these different assessment tools.

The second-semester schedule was as follows:

a. Implementing portfolio assessment and traditional tests according to the course syllabus (7th–10th month).

b. Managing time for end-of-semester portfolio presentation and final peer and self-evaluation.

c. Administering the second portfolio survey on students’ reactions toward both assessment tools (11th month).

5.3. Instruments

The English learning questionnaire, designed by the researcher (Yang 1992), was used to assess students’ beliefs and learning strategy use. It contained three sections. Section 1 contained 35 items from the Beliefs About Language Learning Inventory (BALLI, ESL/EFL version, Horwitz 1987) and one final

(7)

open-ended question for additional beliefs from the students. In this section, the subjects answered on a 5-point scale whether they disagree or agree with the statements. Section 2 contained 49 items of statements adapted from Ox-ford’s Strategy Inventory for Language Learning (SILL, EFL/ESL 7.0 version, Oxford 1989) and one final open-ended question for additional strategy use. This section assessed the frequency with which students used various language learning strategies. The last section intended to obtain students’ background information, such as their gender, age, major, their perceived motivation and proficiency in learning English, and experiences of traveling or living in an English-speaking country.

The questionnaire was translated into Chinese, pilot tested, and modified in previous studies (Yang 1992, 1999). Both the BALLI and SILL were found to have content validity and weak, non-significant correlations with the Marlow-Crowne Social Desirability Scale (Yang 1992). This means that social desir-ability response bias (i.e., answering according to what one thinks the re-searcher wants) is negligible. As assessed earlier, the Chinese BALLI had a Cronbach alpha of 0.69, while the Chinese SILL had a Cronbach alpha of 0.94 for internal-consistency reliability (Yang 1999). In this specific study, internal consistency reliability using Cronbach’s alpha, based on the 45 subjects, was 0.69 for the BALLI and 0.92 for the SILL.

The learning style survey (Table 1) was modified from David Nunan’s docu-ment (Nunan 1995), which focused on students’ favored mode of learning (e.g., reading vs. listening, groups vs. alone) and whether the learner preferred self-directed learning, teacher-self-directed, or teacher-mediated learning. It contained 25 items. The learning style survey had a Cronbach alpha of 0.72 (43 subjects). The first portfolio survey (Table 2) used in the formal study contained 32 statements that were mostly composed from previous student reactions to the open-ended questions from the pilot study. The questions assessed students’ attitudes and reactions toward the use of portfolios in four areas: (1) actual practice in preparing portfolios (e.g., items 5, 9, 15, 25), (2) advantages and disadvantages of portfolios (e.g., items 3, 8, 10, 11, 12, 16, 18, 22, 23, 26, 27, 28, 30, 31, 32), (3) the portfolio sharing activities (e.g., items 2, 6, 14, 21), (4) perception or understanding about the portfolio (e.g., items 1, 4, 7, 13, 17, 19, 20, 24, 29). In addition, the survey had one open-ended question for additional opinions from the students. In this study, the first portfolio survey’s internal-consistency reliability using Cronbach’s alpha on 43 subjects was 0.82.

The second portfolio survey (Table 3) included 19 statements that students rated on a five-point scale from 1 (strongly disagree) to 5 (strongly agree) and three open-ended questions. The questions concerned reactions to the use of portfolios as a learning tool (e.g., items 1, 4, 6, 7, 8, 11, 12; Cronbach’s alpha = 0.82) and as an assessment tool (e.g., items 2, 3, 9, 10, 17; Cronbach’s alpha = 0.74). In this study, Cronbach’s alpha coefficient for all items of the second

(8)

Table 1. Frequencies of response (in %), means and standard deviations for the learning

style survey items

Item descriptiona 1b 2 3 4 5 Mean SD 1. I feel embarrassed when

the teacher asks me to speak in front of other students.

11.6c 14.0 46.5 18.6 9.3 3.00 1.09

2. I like to learn by reading 0 2.3 14.0 48.8 34.9 4.16 0.75 3. I like to learn by

listening to and using cassettes or CDs.

2.3 16.3 30.2 37.2 14.0 3.44 1.01

4. I like to learn through games.

4.7 11.6 30.2 16.3 37.2 3.70 1.23

5. I like to write down what I hear, speak, or read.

7.0 23.3 60.5 9.3 0 2.72 0.73

6. I like the teacher to explain rules of grammar to me.

7.0 14.0 37.2 39.5 2.3 3.16 0.95

7. I like to figure out rules of grammar myself.

9.3 11.6 51.2 25.6 2.3 3.00 0.93

8. I like the teacher to explain the meaning of new words.

2.3 14.0 37.2 39.5 7.0 3.35 0.90

9. I like to figure out the meanings of words by myself.

0 0 25.6 62.8 11.6 3.86 0.60

10. I like the teacher to tell me all my mistakes.

2.3 4.7 34.9 46.5 11.6 3.60 0.85

11. I like the teacher to let me find my own mistakes.

2.3 23.3 46.5 20.9 7.0 3.07 0.91

12. I like to study by myself. 0 11.6 23.3 39.5 25.6 3.79 0.97 13. I like to work in pairs. 4.7 9.3 25.6 39.5 20.9 3.63 1.07 14. I like to work in small

groups.

4.7 4.7 18.6 46.5 25.6 3.84 1.02

15. I like to work with the whole class.

7.0 0 20.9 39.5 32.6 3.91 1.09

16. I like to practice outside of the classroom.

0 2.3 25.6 44.2 27.9 3.98 0.80

17. I like to be part of the decision-making process about what we will learn (that is, the content of the lessons).

(9)

Table 1 (cont.). Frequencies of response (in %), means and standard deviations for the

learning style survey items

Item descriptiona 1b 2 3 4 5 Mean SD 18. I like the teacher to make

all the decisions about what we will learn (that is, the content of the lessons).

4.7 32.6 51.2 7.0 4.7 2.74 0.85

19. I like to be part of the decision-making process about how we will learn (that is, the classroom tasks and activities we will take part in).

2.3 2.3 20.9 41.9 32.6 4.00 0.93

20. I like the teacher to make all the decisions about how we will learn (that is, the classroom tasks and activities we will take part in).

2.3 25.6 53.5 14.0 4.7 2.93 0.83

21. I like to be responsible for my own learning.

0 0 14.0 53.5 32.6 4.19 0.66

22. I like the teacher to explain the objectives of the course to me.

2.3 7.0 16.3 55.8 18.6 3.81 0.91

23. I like the teacher to give reasons for what we are learning and how we are learning.

2.3 9.3 20.9 48.8 18.6 3.72 0.96

24. I like to assess my own progress.

2.3 4.7 30.2 51.2 11.6 3.65 0.84

25. I like the teacher to assess my progress for me.

2.3 20.9 58.1 14.0 4.7 2.98 0.80

Note: The learning style survey’s internal-consistency reliability using Cronbach’s alpha on 43

subjects in this study was 0.72.

a The survey was administered in Chinese and the items presented here were modified from David Nunan’s English version (1995).

b 1 = Strongly disagree, 2 = Disagree, 3 = Neutral, 4 = Agree, and 5 = Strongly agree. c The percentage has been rounded.

(10)

Table 2. Frequencies of response (in %), means and standard deviations for the first

portfolio survey items

Item descriptiona 1b 2 3 4 5 Mean SD 1. Compiling a portfolio

requires a clear organizing concept.

0cc 2.3 7.0 60. 30.23 4.19 0.66

2. The portfolio sharing activities offered me chances to learn good English learning methods from other classmates.

0 0 0 2.3 97.7 4.98 0.15

3. It takes me a lot of time to compile the portfolio.

0 2.3 9.3 58.1 30.2 4.16 0.69

4. I feel the appearance of the portfolio (including its cover and art design) is important.

4.7 9.3 30.2 46.5 9.3 3.47 0.96

5. I reflected on and evaluated my learning in each aspect when compiling the portfolio.

2.3 2.3 14.0 55.8 25.6 4.00 0.85

6. During the portfolio sharing activities, I was able to share useful English learning resources with my classmates.

2.3 0 14.0 60.5 23.3 4.02 0.77

7. I have a clear concept about how to compile a portfolio.

2.3 2.3 58.1 25.6 11.6 3.42 0.82

8. The portfolio helped me organize and arrange my English learning.

2.3 7.0 27.9 48.8 14.0 3.65 0.90

9. I have completed most of the learning goals I set up in my portfolio.

4.7 20.9 44.2 30.2 0 3.00 0.85

10. The portfolio increased the burden of my study.

(11)

Table 2 (cont.). Frequencies of response (in %), means and standard deviations for the

first portfolio survey items

Item descriptiona 1b 2 3 4 5 Mean SD 11. The portfolio allowed

me to choose what I like to read or listen to according to my personal interests.

0 0 4.7 53.5 41.9 4.37 0.58

12. The portfolio increased my willingness to learn actively.

2.3 2.3 39.5 34.9 20.9 3.70 0.91

13. I feel the content of the portfolio (including its organization and details) is important.

4.7 2.3 11.6 39.5 41.9 4.12 1.03

14. Portfolio sharing can increase classmates’ cooperative learning and mutual growth in English.

2.3 2.3 9.3 48.8 37.2 4.16 0.87

15. The goal I originally set up in the portfolio was too difficult and I couldn’t reach it.

2.3 20.9 39.5 27.9 9.3 3.21 0.97

16. The portfolio allowed me to choose how to plan and learn my English according to my learning style.

2.3 0 4.7 62.8 30.2 4.19 0.73

17. Preparing a portfolio requires good computer skills and artistic talent.

4.7 11.6 48.8 23.3 11.6 3.26 0.98

18. The portfolio helped me understand my strengths and weaknesses in English.

2.3 4.7 11.6 69.8 11.6 3.84 0.78

19. I think reflection and self-evaluation is the most important part of the portfolio.

2.3 7.0 14.0 46.5 30.2 3.95 0.97

20. The handout’s introduction to the portfolio was not clear enough.

(12)

Table 2 (cont.). Frequencies of response (in %), means and standard deviations for the

first portfolio survey items

Item descriptiona 1b 2 3 4 5 Mean SD 21. The portfolio sharing

activities were helpful to my understanding of the portfolio.

2.3 2.3 2.3 69.8 23.3 4.09 0.75

22. The portfolio will be of great help to my future independent learning.

2.3 7.0 27.9 48.8 14.0 3.65 0.90

23. I feel some practices (like listening or speaking) could not be easily presented in the portfolio.

0 4.7 11.6 60.5 23.3 4.02 0.74

24. Looking at previous students’ portfolio samples gave me a better idea about portfolios.

2.3 4.7 37.2 37.2 18.6 3.65 0.92

25. I added appropriate explanations to the samples or records, which I included in the portfolio.

2.3 9.3 20.9 51.2 16.3 3.70 0.94

26. I feel the portfolio can present my learning results.

4.7 4.7 30.2 51.2 9.3 3.56 0.91

27. The portfolio made me realize that I can learn anytime and anywhere, not just in the classroom.

0 0 9.3 55.8 34.9 4.26 0.62

28. Portfolios can show my efforts in learning English outside of the classroom.

0 2.3 16.3 51.2 30.2 4.09 0.75

29. I still don’t understand why we needed to compile a portfolio.

32.6 48.9 7.0 9.3 2.3 2.00 1.00

30. When compiling the portfolio, I had a chance to reflect on my English learning of this semester.

2.3 2.3 7.0 62.8 25.6 4.07 0.80

31. I feel the portfolio is not helpful to my learning.

(13)

Table 2 (cont.). Frequencies of response (in %), means and standard deviations for the

first portfolio survey items

Item descriptiona 1b 2 3 4 5 Mean SD 32. The portfolio helps me

to keep the habit of listening to, reading, or writing English regularly.

2.3 18.6 44.2 25.6 9.3 3.21 0.94

Note: The first portfolio survey’s internal-consistency reliability using Cronbach’s alpha on 43

subjects in this study was 0.82.

a The survey was administered in Chinese and the item description presented here was translated from Chinese into English by the researcher.

b 1 = Strongly disagree, 2 = Disagree, 3 = Neutral, 4 = Agree, and 5 = Strongly agree. c The percentage has been rounded.

Table 3. Frequencies of response (in %), means and standard deviationsfor the second

portfolio survey items

Item descriptiona 1b 2 3 4 5 Mean SD

1. The portfolio helped me organize learning in this course.

0c 0 14.3 68.6 17.1 4.03 0.57

2. The portfolio is a good tool to show my learning process.

2.9 0 14.3 71.4 11.4 3.89 0.72

3. The portfolio represents my learning results in this course.

0 5.7 11.4 74.3 8.6 3.86 0.65

4. The portfolio helped me to reflect on my learning in this course.

0 0 8.6 68.6 22.9 4.14 0.55

5. The portfolio increased the interaction between students and the instructor.

0 8.6 34.3 40.0 17.1 3.66 0.87

6. The portfolio is a good tool to help students learn.

0 2.9 28.6 60.0 8.6 3.74 0.66

7. The portfolio provides a multi-dimensional perspective about learning.

(14)

Table 3 (cont.). Frequencies of response (in %), means and standard deviationsfor the

second portfolio survey items

Item descriptiona 1b 2 3 4 5 Mean SD 8. The portfolio will help

my learning in future courses.

0 11.4 42.9 37.1 8.6 3.43 0.81

9. The portfolio is a good tool to evaluate students’ performance.

5.7 2.9 34.3 40.0 17.1 3.60 1.01

10. The portfolio provides a multi-dimensional perspective about evaluation.

0 2.9 8.6 68.6 20.0 4.06 0.64

11. The portfolio helps my future independent learning.

0 5.7 17.1 60.0 17.1 3.89 0.76

12. My experience of learning through a portfolio in this course has been successful.

2.9 0 45.7 42.9 8.6 3.54 0.78

13. It has been successful to use portfolios for teaching and evaluation in this course.

0 2.9 22.9 60.0 11.4 3.82 0.67

14. I will use portfolios in my future learning.

0 14.3 45.7 37.1 2.9 3.29 0.75

15. I like to assess my own progress.

0 8.6 22.9 57.1 11.4 3.71 0.79

16. I like the teacher to assess my progress for me. 0 11.4 45.7 40.0 2.9 3.34 0.73 17. I prefer to be evaluated by portfolios. 0 8.6 40.0 48.6 2.9 3.46 0.70 18. I prefer to be evaluated by paper-and-pencil tests. 5.7 17.1 54.3 22.9 0 2.94 0.80

19. I think we should use multiple evaluation methods

0 0 5.7 60.0 34.3 4.29 0.57

20. According to your experience, what are the advantages of using portfolios?

(15)

Table 3 (cont.). Frequencies of response (in %), means and standard deviationsfor the

second portfolio survey items

Item descriptiona 1b 2 3 4 5 Mean SD 21. According to your

experience, what are the disadvantages of using portfolios?

22. Please compare traditional

pen-and-paper tests (e.g., class quizzes, and GEPT tests) with portfolio assessment (including self-evaluation, peer evaluation, and instructor’s evaluation). In your opinion, which one is better? Why?

Note: The second portfolio survey’s internal-consistency reliability using Cronbach’s

alpha on 35 subjects in this study was 0.81.

a The survey was administered in Chinese and the item description presented here was translated from Chinese into English by the researcher.

b 1 = Strongly disagree, 2 = Disagree, 3 = Neutral, 4 = Agree, and 5 = Strongly agree. c The percentage has been rounded.

portfolio survey computed on 35 subjects was 0.81. As for the first and the sec-ond open-ended questions, students were asked their opinions about the advan-tages and disadvanadvan-tages of using portfolios. The third question asked students to compare which assessment tool – portfolio or traditional pen-and-paper tests – is better and to explain their reasons. This questionnaire was administered to students at the end of the 2001–2002 academic year.

The study used two listening and reading tests from the General English Proficiency Test (high-intermediate level) for the pretests and post-tests. The General English Proficiency Test was developed by the Language Training and Testing Center in Taiwan and has reported high reliability and validity. Though the pre- and post listening test (listening test 1 and 3) were the same, there was a nine-month interval between the two tests. The General English Proficiency Test considers scores above 80 to be a passing score.

(16)

6. Results and discussion

6.1. Results of the English learning questionnaire

According to the questionnaire results, these students reported using compen-sation strategies most frequently and memory strategies least frequently, sim-ilar to the profile of previous students taking the course (Yang 1996). These students believed that English is an important tool for study, important to their future careers and that they could learn English well. They stressed the impor-tance of good pronunciation and frequent practice.

Such information was helpful to the instructor (researcher), since students’ beliefs could be taken into consideration when planning lessons, while stu-dents’ strategy profiles made it easy to adopt materials or mini-lessons devel-oped previously for learning strategy-based instruction in this portfolio study. Besides, students could use this information about their learning strategy use to assist them in setting learning goals and planning their study. During the semester, they could also use portfolios to monitor and record their general learning process as well as specific strategy use.

6.2. Results of the learning style survey

Most of the responses to the learning style survey (see Table 1) were in the neutral range or slightly positive, indicating that there was no single learning style profile for the whole group. However, some items stood out as having clear agreement: S2, “I like to learn by reading” (mean = 4.16 out of a possible 5); S19, “I like to be part of the decision-making process about how we will learn” (mean = 4.00), and S21, “I like to be responsible for my own learning” (mean = 4.19).

6.3. Students’ reactions to the use of portfolios

According to the final portfolio survey in the pilot study of fall 2000, among the 30 students who answered the open-ended survey anonymously, most students felt positive about the experience of compiling portfolios in their freshman En-glish class. They reported that in order to prepare the portfolios, they had to continuously keep contact with the language they were learning by listening and reading in daily life. “A daily language learning habit” was what they got from preparing their portfolios. One student wrote “I got the habit of listening to English every day.” Another wrote “I can make use of possible opportunities to learn English in daily life.” In the opinion of these students, the portfolio experience also helped them to extend their learning approach and explore new learning materials from the Internet, newspaper, or magazines, instead of just limiting themselves to learning in the classroom or from textbooks. They were able to select their favorite topics for in-depth learning or work on their weakest

(17)

areas. From the portfolio preparation, they also learned how to collect, orga-nize, and present information. At the same time, they were able to reflect on their own learning results and check on their progress. One student wrote that the portfolio helped her “know what [she] had done in this semester.” Another commented that “It is a really good way to check learning.” To sum up, 80 % of these students wanted to continue using portfolios the next semester, because they felt that the portfolios offered them direction and chances for learning.

Students also had positive reactions to the use of portfolios in the English class in the formal study of 2001–2002 (see first portfolio survey, Table 2). Ninety-five percent of the students agreed or strongly agreed that the portfolio allowed them to choose what they like to read or listen to according to their personal interests. The portfolios also let them plan how to learn English ac-cording to their learning styles (93 %), and they made them realize that they could learn anytime and anywhere, not just in the classroom (90 %). In addi-tion, the second portfolio survey in 2001–2002 also revealed students’ positive feeling about using portfolios as a learning tool (see Table 3). Over 90 % of the students agreed or strongly agreed that the portfolios provided a multi-dimensional perspective about learning, and that the portfolios helped them reflect on learning (91 %) and organize their learning (86 %). Above all, over 70 % of these students found it successful to use portfolios for instruction and evaluation in this course.

Most students found the portfolio sharing activities very helpful. Over 80 % of the students reported that the portfolio-sharing activities offered them chances to develop useful English learning methods from other classmates, increased classmates’ cooperative learning and mutual growth in English pro-ficiency, let them share useful English learning resources with their classmates, and helped them understand a variety of learning processes. More details are provided in Table 2.

6.4. Advantages and disadvantages of using portfolios as a learning tool

According to students’ answers to the open-ended questions in the pilot study as well as in the second portfolio survey (see Table 3) of the formal study, the advantages of using portfolios included promoting the use of metacognitive strategies (e.g., managing and organizing learning, monitoring and observing one’s learning, reflecting on one’s learning); motivating and enhancing active, multi-dimensional learning; being helpful to future independent learning; facil-itating interaction among the instructor and students; showing learning process and progress; presenting learning results; providing a sense of achievement; and practicing presentation skills.

Students’ comments about disadvantages included “It takes a lot of time”, “It is too much trouble, a burden to prepare”, “Some students may complete it

(18)

in a short time and cheat on some samples or fake the records”, “Some learning results and methods are hard to present”, and “Some learning processes may not be easily recorded”.

When asked what problems or difficulties they had encountered when prepar-ing the portfolio, these students listed several personal or technological diffi-culties. They are categorized as follows: (1) time management (in studying and preparation); (2) learning attitudes (e.g., laziness, procrastination, and lack of determination); (3) problems in record keeping (e.g., some students did not keep good records, some learning could not be recorded easily, or some didn’t know how to record or just forgot to record); (4) information management (e.g., how to select and organize work); and (5) other computer-related and access problems.

6.5. Students’ learning style preferences in relation to portfolio use

Pearson correlations were computed between items on the learning style survey (S1 to S25, Table 1) and items on the first portfolio survey (P1 to P32, Table 2). Numerous significant correlations were found between students’ learning style preferences and portfolios. Among the most interesting are the following. Students who liked to write down what they hear, speak, or read (S5) felt posi-tive that the portfolio helped them organize and arrange their English learning (P8), r = .54, p< .001, and believed that portfolio sharing increased coopera-tive learning and mutual growth in English (P14), r = .48, p< .001. Those who liked to work in pairs (S13) or small groups (S14) were likely to agree that portfolio sharing could increase classmates’ cooperative learning and mutual growth in English (P14), r = .55, p< .001 and r = .49, p< .001, respectively. Students who liked to work in small groups (S14) or with the whole class (S15) tended to agree that they reflected on and evaluated their learning in each as-pect when compiling the portfolios (P5), r = .69, p< .001 and r = .62, respec-tively. That is probably why they also agreed that, by reflection and self-evaluation, portfolios helped them understand their strengths and weaknesses (P18), r = .56 and r = .62, p< .001, respectively.

Students who liked to practice English outside of the classroom (S16) tended to agree that through portfolios they could learn anytime and anywhere (P27), r = .40, p <.01. The portfolios demonstrated their efforts accordingly.

Students who liked to assess their own progress (S24) tended to agree that portfolios helped them learn English according to their learning styles (P16), r = .49; organize and arrange their learning (P8), r = .62; reflect on learning (P30), r = .57; understand their strengths and weaknesses (P18), r = .60; share useful English learning resources with classmates (P6), r =.60;, and keep the habit of reading or listening regularly (P32), r = .49; all p < .001. These same students felt that they had a clear concept of how to compile a portfolio (P7), r=

(19)

.49; that portfolios increased cooperative learning (P14), r= .50; that portfolio sharing was helpful (P21), r = .54; and that reflection and self-evaluation were the most important parts of portfolio use (P19), r = .53; all p < .001.

Even those students who liked the teacher to assess their progress (S25) tended to agree that when compiling the portfolio, they had a chance to reflect on their English learning of this semester (P30), r = .49, p < .001. Students who liked the teacher to explain the objectives of the course (S22) also felt positive about the helpfulness of the portfolio to their learning (P22, P30), r = .50 and r=.55, respectively, both p < .001, and disagreed with the statement that portfolios were not helpful (P31), r =−.50, p < .001.

6.6. Proficiency test results

The results of the students’ pretest and post-test on the General English Pro-ficiency Test are presented in the table in Appendix C. With 80 as a passing score, over 90 % of these students passed the reading pretest, while only 50 % passed the listening pretest. After two semesters of learning strategy-based in-struction, students made progress in their listening to a passing rate of over 80 %. In fact, paired t-tests revealed that these students’ listening post-test re-sults were significantly different from their listening pretest rere-sults (t =−6.64, df = 33, p < 0.0005).

6.7. Students’ reactions to using portfolios for evaluation

When students were asked to compare portfolios as an assessment tool to tradi-tional pen-and-paper tests, over one half (51 %) of the students responded that they would prefer to be evaluated by portfolios, while only 23 % preferred to be evaluated by paper-and-pencil tests. It may be due to the fact that portfolios offered them chances for self-evaluation because over two-thirds (69 %) of the students responded that they liked to assess their own progress. Many (89 %) agreed that the portfolio provided a multi-dimensional perspective to evalua-tion. The majority (94 %) also agreed that multiple evaluations should be used. In summary, this study, similar to prior reports in the literature (e.g., Paulson, Paulson, and Meyer 1991; Yueh 1999), found that portfolios operated very dif-ferently from ordinary paper-and-pencil tests and that students valued portfolio use.

7. Caveats and suggestions

The integration of portfolios into learning strategy-based instruction, student work, and student assessment needs careful planning. In addition, there needs to be time to prepare for evaluation, which may include assembling the finished portfolios, arranging portfolio presentation and peer evaluation, and complet-ing the final evaluation by the instructor.

(20)

To integrate portfolios into English courses, teachers should: (1) introduce the concepts of autonomous learning to students; (2) develop mini-lessons and provide guidelines to help students develop their portfolios; (3) implement fre-quent portfolio sharing and checking; (4) allocate adequate class time for as-sembling the portfolios; (5) encourage students to include more than one type of material in the portfolio; and (6) consider other media instead of paper for portfolios.

8. Conclusions

Although the challenge to integrate portfolios into learning strategy-based in-struction and to facilitate students’ autonomous learning is not an easy one, the benefits for both the teacher and students are great. In this study, students gen-erally had positive reactions toward using the portfolio in the class for learning and assessment. Over one half (51 %) agreed that their experience of learning through portfolios was successful. Fewer than 3 % thought that using portfo-lios was very unsuccessful. Only 5 % felt portfoportfo-lios were not helpful to their learning.

The recent development of Web-based learning provides a promising au-tonomous learning environment in which learning strategy-based instruction may play an important role. Future studies might incorporate learning strategy-based instruction with Web-strategy-based or on-line language learning with portfo-lios as an important learning tool. Some of the problems encountered, such as difficulty in record keeping, might be improved through computer-assisted or Web-based learning. To conclude, portfolios are a good tool that helps raise students’ awareness about learning strategies, facilitate their learning process, and enhance their self-directed learning.

National Taiwan University

(21)

Appendix A

Chart for the general procedure of integrating portfolios into learning strategy-based instruction

Pre−test Setting learninggoals Resources

Plans Progress Problems and resources Self Peer Instructor guidelines

Introducing the Portfolio sharing Portfolio presentation evaluation Portfolio started Getting Preparation Diagnosis Instruction Evaulation Appendix B

Guidelines to preparing an English portfolio

(Handout for students, Fall 2001, adapted from McNamara and Deane 1998) Objectives

– To reflect on the progress you made in an English course. – To assess strengths and weaknesses in learning English. – To establish goals and plans for future independent learning.

General guidelines

– Go over all the work you did during the week or month and decide which activities (either course work or learning experiences outside class) were the most meaningful to you and from which you learned the most. Think about your learning and keep a learning journal every week or every month. – Choose one or two representative samples of your work during the period

that shows your ability to use English in any of the four skill areas: listening, reading, writing, or speaking.

– For each piece of evidence that you include, please prepare a brief written explanation of what the sample shows about your progress and development as a user of English – in other words, how the sample is significant to your

(22)

development in English. This written explanation is the cover sheet that in-troduces the evidence that you are presenting in your portfolio.

– As part of the portfolio, please write a letter to the instructor, describing:

◦ how much you have completed your plans for improving your weakest

area or increasing your skills in English,

◦ the area in which you made the most progress this semester and the reason

you improved so much,

◦ any new area of English in which you feel you need further improvement, ◦ an assessment of your effort in the class this semester,

◦ the grade you would give yourself for the portfolio, and ◦ anything you feel like telling the instructor.

– As a final organizing step, please prepare a table of contents and a cover sheet for each section of your portfolio. Present the portfolio materials in a folder or binder with a clear index system.

– Portfolio sharing: There will be two opportunities for you to share your port-folio with other classmates during the semester. Details will be announced later.

The content of the portfolio

The materials in the portfolio can be direct or indirect evidence of your use of English. Examples of direct evidence are your reading portfolios, vocabu-lary logs you kept, weekly listening logs, weekly learning journals, news story summaries, free journal writings, any reactions you posted on the course web-site, the preparation materials for an oral report, a script of a role play, and a photo of the presentation you participated in. An example of indirect evidence is a short written description of a time you used English (e.g., a telephone call, a conversation with an American friend, an e-mail with a Canadian pen pal).

Assessment criteria for the English portfolio

The portfolios will be read holistically with the three criteria below in mind, and you may use the following questions as references when preparing the portfolio. The portfolio makes up 20 % of your final grade.

1. Have you included all the required materials in the portfolio? That is, (1) learning journals and reflections; (2) representative samples of your work during the semester in listening, reading, writing, speaking, and vocabu-lary; (3) a written explanation (cover sheet) for each sample describing its significance to your progress in learning English; and (4) a letter to the in-structor describing your progress this semester, strong and weak areas, your plan for improving weak areas, and a self-assessment of work during the semester?

(23)

2. Do the representative samples and the written explanations demonstrate your progress in learning English this semester?

3. Does the portfolio show evidence of your thoughtful reflection on the ma-terial and activities of the course and careful assessment of progress and learning in the course?

Your self-evaluation and peer evaluation count 3 % each.

Additional notes

– Learning journals may include the reflective writing you complete at regular intervals (e.g., every week, every two weeks, or every month) and the reflec-tions you write at the end of English classes as required by the instructor or by yourself.

– Samples of reading report sheets, weekly listening logs, news story sum-maries, and vocabulary logs will be provided for reference. You may design the format of your reading report sheet, weekly listening log, news story summary, and vocabulary log based on your needs and purposes.

– Free journal writing focuses on honest and meaningful communication you make with your classmates and the instructor on the course website. The emphasis is on content and interaction, but you should also pay attention to the correctness of your spelling and sentence structures. You write and share about what you think and feel about the reading selection. More details will be posted on the course website.

Appendix C

Results of students’ English pretest and post-test

Pretest: Listening test 1 (2001/9) Pretest: Reading test 1 (2001/9) Listening test 2 (2002/3) Post-test: Reading test 2 (2002/6) Post-test: Listening test 3 (2002/6) Mean 79.91 94.35 77.39 91.89 93.25* SD 19.11 10.37 17.89 11.51 16.49 Passing N (%) 17 (50 %) 31 (91.2 %) 17 (47.2 %) 32 (88.9 %) 29 (80.6%) N 34 34 36 36 36

Note: Listening test 1 and test 3 used the same test.

Note

∗.∗ The research was funded by the National Science Council (NSC90-2411-H-002-045).

(24)

References

Belanoff, Patricia and Marcia Dickson (eds.) (1991). Portfolios: Process and Product. Portsmouth, NH: Boynton/Cook.

Brown, H. Douglas (2001). Teaching by Principles: An Interactive Approach to Language

Peda-gogy. 2nd edition. White Plains, NY: Addison Wesley Longman.

Brown, James Dean (ed.) (1998). New Ways in Classroom Assessment. Alexandra, VA: Teachers of English to Speakers of Other Languages.

Cohen, Andrew. D. (1998). Strategies in Learning and Using a Second Language. White Plains, NY: Addison Wesley Longman.

Farr, Roger and Bruce Tone (1994). Portfolio and Performance Assessment: Helping Students

Evaluate Their Progress as Readers and Writers. Fort Worth, TX: Harcourt Brace College

Publishers.

Fradd, Sandra and Sarah Hudelson (1995). Alternative assessment: A process that promotes col-laboration and reflection. TESOL Journal 5 (1): 5.

Gottlieb, Margo (1995). Nuturing student learning through portfolios. TESOL Journal 5 (1): 12– 14.

Hewitt, Geof (1994). A Portfolio Primer: Teaching, Collecting, and Assessing Student Writing. Portsmouth, NH: Heinemann.

Horwitz, Elaine K. (1987). Surveying student beliefs about language learning. In Learner

Strate-gies in Language Learning, Anita L. Wenden (ed.), 119–129. Englewood Cliffs, NJ: Prentice

Hall.

McNamara, Martha J. and Debra Deane (1998). Self-assessment: Preparing an English portfolio. In New Ways in Classroom Assessment, James Dean Brown (ed.), 15–17. Alexandra, VA: Teachers of English to Speakers of Other Languages.

Mokhtari, Kouider, David Yellin, Kay Bull, and Diane Montgomery (1996). Portfolio assessment in teacher education: Impact on preservice teachers’ knowledge and attitudes. Journal of

Teacher Education 47 (4): 245–252.

Murphy, Sandra M. (1997). Who should taste the soup and when? Designing portfolio assessment programs to enhance learning. Clearing House 71 (2): 81–84.

Norris, J. M. (1996). Performance and Portfolio Assessment (1985-1995): An Extended Annotated

Bibliography of Courses Useful for Language Teachers (Research Note). Honolulu:

Univer-sity of Hawaii, Second Language Teaching and Curriculum Center.

Nunan, David (1995). Likes and dislikes. Learning Matters: A Task-Based Course in Becoming a

Better Learner (draft): 12. Hong Kong: The English Center, University of Hong Kong.

O’Malley, J. Michael and Anna U. Chamot (1990). Learning Strategies in Second Language

Ac-quisition. New York: Cambridge University Press.

O’Malley, J. Michael and L. Valdez Pierce (1996). Authentic Assessment for English Language

Learners: Practical Approaches for Teachers. Reading, MA: Addison-Wesley.

Oxford, Rebecca L. (1989). Strategy Inventory for Language Learning (SILL). Version 7.0 (EFL/ ESL). Alexandria, VA: Oxford Associates.

(1990). Language Learning Strategies: What Every Teacher Should Know. Boston: Heinle and Heinle.

(1993). Research on second language learning strategies. In Annual Review of Applied

Lin-guistics 13, William Grabe (ed.), 175–187. Cambridge: Cambridge University Press.

Oxford, Rebecca L. and Betty L. Leaver (1996). A synthesis of strategy instruction for language learners. In Language Learning Strategies around the World: Crosscultural Perspectives, Re-becca L. Oxford (ed.), 227–246. Manoa: University of Hawaii Press.

Paulson, F. Leon and Peal R. Paulson (1991). Portfolios: Stories of knowing. Paper presented at the Annual Meeting of the Claremont Reading Conference. (ERIC Reproduction Service No. ED377209).

Paulson, F. Leon, Peal R. Paulson, and A. Meyer (1991). What makes a portfolio a portfolio?

(25)

Reed, Candi Mascia (1997). Grammar and student portfolios: Self-evaluation and pride in learning.

Perspectives in Education and Deafness 16 (2): 2–3.

Spence, Sarah L. and Billie Theriot (1999). Portfolios in progress: Reevaluating assessment.

Re-search and Teaching in Developmental Education 15 (2): 27–34.

Terwilliger, James S. (1997). Portfolios and classroom assessment: Some claims and questions. (ERIC Reproduction Service No. ED419000).

Weinstein, Claire E. and Vicki L. Underwood (1985). Learning strategies: The how of learning. In Thinking and Research Skills: Relating Instruction to Research, Judith W. Segal, Susan F. Chipman, and Robert Glaser (eds.), 241–258. Hillsdale, NJ: Erlbaum.

Wenden, Anita (1991). Learner Strategies for Learner Autonomy: Planning and Implementing

Learner Training for Language Learners. Hertfordshire, UK: Prentice Hall.

Williams, Marion and Robert L. Burden (1997). Psychology for Language Teachers: A Social

Constructivist Approach. Cambridge: Cambridge University Press.

Yang, Nae-Dong (1992). Second language learners’ beliefs about language learning and their use of learning strategies: A study of college students of English in Taiwan. Ph.D. diss., University of Texas.

(1996). Effective awareness-raising in language learning strategy instruction. In Language

Learning Strategies around the World: Crosscultural Perspectives, Rebecca L. Oxford (ed.),

205–210. Manoa: University of Hawaii Press.

(1999). The relationship between EFL learners’ beliefs and learning strategy use. System 27: 515–535.

— (2002). Using portfolios as a learning tool. Paper presented at the 13th World Congress of the International Association of Applied Linguistics, Singapore.

Yueh, Hsiu-Ping (1999). Theory and practice of portfolios. In Handbook for Portfolio Workshop, 1–10. Taipei, Taiwan: National Taiwan University.

數據

Table 1. Frequencies of response (in %), means and standard deviations for the learning style survey items
Table 1 (cont.). Frequencies of response (in %), means and standard deviations for the learning style survey items
Table 2. Frequencies of response (in %), means and standard deviations for the first portfolio survey items
Table 2 (cont.). Frequencies of response (in %), means and standard deviations for the first portfolio survey items
+5

參考文獻

相關文件

Writing texts to convey information, ideas, personal experiences and opinions on familiar topics with elaboration. Writing texts to convey information, ideas, personal

Now, nearly all of the current flows through wire S since it has a much lower resistance than the light bulb. The light bulb does not glow because the current flowing through it

‧ After submitting the Enrolment Survey, report promptly the Date of Entry from Mainland for all new students with STRN, such as P1 students allocated to the school,

Based on Cabri 3D and physical manipulatives to study the effect of learning on the spatial rotation concept for second graders..

Writing texts to convey simple information, ideas, personal experiences and opinions on familiar topics with some elaboration. Writing texts to convey information, ideas,

Having regard to the above vision, the potential of IT in education and the barriers, as well as the views of experts, academics, school heads, teachers, students,

DVDs, Podcasts, language teaching software, video games, and even foreign- language music and music videos can provide positive and fun associations with the language for

Seeing other good project videos from classmates may motivate students to prepare better. No excuses for “sudden” technical