行政院國家科學委員會補助專題研究計畫成果報告
※※※※※※※※※※※※※※※※※※※※※※※※※
※
※
※ 學習成果檔案在語言學習上之設計、實施與評量 ※
※
Portfolios in Language Learning: ※
※ Design, Implementation and Evaluation ※
※※※※※※※※※※※※※※※※※※※※※※※※※
計畫類別：v 個別型計畫 □整合型計畫
計畫編號：NSC90－2411－H－009－012
執行期間：90 年 8 月 1 日至 91 年 7 月 31 日
計畫主持人：郭志華
執行單位：國立交通大學語言教學與研究中心
中華民國九十一年十月二十八日
行政院國家科學委員會專題研究計畫成果報告
學習成果檔案在語言學習上之設計、實施與評量
Por tfolios in Language Lear ning:
Design, Implementation and Evaluation
計畫編號：NSC90-2411-H-009-012
執行期限：90 年 8 月 1 日至 91 年 7 月 31 日
主持人：郭志華 國立交通大學語言教學與研究中心
一、 中文摘要 學習成果檔案是學習者學習過程各階 段表現成果之收集，並用以做為有別於傳 統測驗之一種評量方式。它源自美術與建 築設計，後來廣泛應用於教育上，強調評 量應以學習之自然呈現為主。使用學習成 果檔案作為評量可以避免測驗帶給學生的 壓力，了解學生學習的過程和進展，並可 增進學生自我主導學習和自省的能力。 近年來國外亦有許多報導使用學習成 果檔案於語言教學上，並有各類型成果檔 案之設計應用。學習成果檔案常用於寫作 課程，因為從學生長期的寫作作品可以分 析瞭解其寫作能力之進展情形，而且寫作 評量難以測驗方式進行，而適合以作品來 評量。 本研究探討學習成果檔案於英文閱讀 課程之使用，研究包括學習成果檔案之設 計、實施、與評量三階段。設計階段探討 閱讀成果檔案之目標、準則與內容之設 計、與教學內容之互動、以及學生如何編 寫成果檔案等方面之規劃。實施階段則是 依設計好的計畫逐步實施，包括擬定指導 綱領、說明實施成果檔案之學習意義、指 導編寫過程、及收集完成之成果檔案。評 量階段主要為分析並評量學生完成之閱讀 成果檔案及實施意見調查，瞭解實施成效 及學生對編寫成果檔案之看法。 研究結果顯示學生對於閱讀新聞、雜 誌、小說故事、有關運動及電影之報導等 主題較感興趣。從檔案包括之內容顯示大 部分學生仍著重字彙及片語等之學習而較 少省思閱讀過程或有批判性閱讀。此外， 學生意見調查顯示，幾乎所有學生都認為 閱讀成果檔案之編寫有助於英語閱讀之學 習，而大部分的學生也指出編寫閱讀成果 檔案有助於瞭解自己的閱讀興趣及閱讀過 程，閱讀文章之後撰寫摘要及省思有助學 習效果。另一方面，許多學生指出編寫成 果檔案的困難在於尋找適合興趣與程度的 文章及撰寫文章之摘要及省思。 關鍵詞：學習成果檔案、學習評量、學習 過程、學習省思 Abstr actA portfolio is the collection of a stu-dent‘s works over a long period of learning. It is used as an alternative to traditional test-based assessment. The notion of portfo-lio came from art and architecture design, and was later widely applied in education. Portfolio assessment emphasizes evaluation that can really demonstrate students’ achievement in a natural context. It can avoid the pressure caused by test. Furthermore, it reflects students’ improvement over the learning process and facilitates active learn-ing and reflection.
Recently, there have been a number of studies on the use of portfolios in the lan-guage classroom. Many types of portfolios have been designed and successfully imple-mented. Portfolios are often used in writing courses since they help reveal students’ writ-ing process and improved writwrit-ing ability throughout the curriculum. It is also more appropriate to evaluate the strengths and weaknesses of their writing works by portfo-lio than by test.
This study investigated the use of port-folios in a college English reading course. It focused on three important stages of portfolio compilation: design, implementaton, and evaluation. The design stage explored the
design of a portfolio plan, including the ra-tionale, objectives, principles, and contents of the portfolio, interactions between instruc-tion and portfolio, and student commitment to the portfolio. The implementation stage was the practice of the portfolio plan in the classroom, including explanation, content selection, progress monitoring, and collection of portfolios. The stage of evaluation in-cluded analysis and evaluation of portfolios and collection of student feedback.
Results show that students are interested in reading newspapers, magazines, reports about sports or movies, or fiction. Most stu-dents still focused on the learning of vocabu-lary and idioms rather than critical thinking or reflecting over the learning process. On the other hand, almost all of the students indicated that portfolio compilation is helpful to the learning of English reading. A majority of students indicated that portfolio compila-tion helps them find out their reading interest, and understand their reading process, and that writing summaries and reflections after reading a text increases the learning effect. About the problems in portfolio compilation, many students found it difficult and time-consuming to find appropriate reading materials that fit their reading level and in-terest, and to write summaries and reflec-tions.
Keywor ds: portfolio, learning assessment,
learning process, reflection
二、緣由與目的 (Intr oduction)
Traditionally, norm-referenced, forma-tive tests have been used to assess, and even guide students’ learning. Standardized tests gained momentum in the 1920s and 1930s. The establishment of Educational Testing Service (ETS) in the U.S. in 1947 seemed, furthermore, to confirm testing as the ulti-mate tool of assessment. (Cole et al. 2000) In the 80s and 90s, however, educators and teachers began to realize the limitation of test as the sole source of assessment. They seek for more authentic assessment which can really demonstrate students’ learning process and improvement in a natural context.
Dis-tinct from test-based evaluation, portfolio assessment is performance-based, and learn-ing- and process-oriented. The notion of portfolio came from art and architecture de-sign. It refers to a collection of a learner’s art or design works for the purpose of presenting to and requesting for an evaluation by an art or design master. It was then gradually adopted in education as an alternative to test-based assessment.
As Vavrus defined, a portfolio is “a systematic and organized collection of evi-dence used by the teacher and student to monitor growth of the student’s knowledge, skill and attitudes” (1990:48). As a portfolio is the product of long-term effort of a stu-dent, it functions not only as assessment but as a record of learning, progress and achievement. Besides, the student is actively involved in determining the content of the portfolio, it promotes active and motivational learning. Therefore, portfolios are now ac-knowledged as the best tool to combine in-struction with assessment. (Hamp-Lyons 1994)
Recently, portfolios have caught the at-tention of language teachers. Many types of portfolios have been proposed in various learning contexts, such as experience portfo-lio (Sweet 1976), learning portfoportfo-lio (Graves 1992), student portfolio (Calfee and Perfumo 1996), literacy portfolio (Johns 1995), disci-plinary portfolio (Hirvela 1997), and writing portfolio (Yancey 1992; Chen 1999).
Among the studies using portfolios for evaluation, Padilla et al. (1996) analyzed students’ portfolios to assess students’ im-provement in foreign language learning. It was found that many factors contribute to the determination of the objectives and content of portfolios. Many studies have used portfo-lios in writing courses. (Yancey 1992; Hamp-Lyons 1994; Belanoff and Dickson 1991; Chen 1999) For example, Yancey (1992) indicated that the writings students carefully selected to be included in their portfolios revealed both the effort and the ability throughout the curriculum. Chen (1999) used portfolios in EFL university writing instruction to analyze students’ writ-ing process and evaluate the strengths and
weaknesses of their writing.
On the other hand, a number of studies have focused on the design and implementa-tion of portfolios. Herman, Gearhart and Aschbacher (1996) discussed in detail the issues involved in portfolio design and prac-tice, including priority purposes, specifica-tion of contents, specificaspecifica-tion of scoring cri-teria, fair assessment, and validity. Gottlieb (1995) proposed a CRADLE (collections, reflections, assessment, documentation, linkages, and evaluation) approach to portfo-lio development, each signifies a stage of development and has a set of specified crite-ria that shape its function. Burch (1997) of-fered a two-tiered model for portfolio con-struction.
Domestically, portfolios have been used mostly in education. Little has been pub-lished about the application of portfolios in language learning. However, it was thought that language learning emphasizes regular practice and long-term effort. The learning process can reveal how students develop their interest and competence step by step. In addi-tion, portfolios in the language classroom can help the teacher understand individual stu-dent’s reflection upon specific learning points or problems which may otherwise be neglected. Portfolios, therefore, should be a valuable alternative to test-based assessment for language learning. On the other hand, they may be used in not only writing but reading courses. Layzer and Sharkey (1996) reported the use of portfolio in an English reading course at a Japanese college. They found that students’ reflections on language learning included in the portfolios is a con-tributing factor of teaching success. In a reading course, students may often be re-quired to do outside reading to build up their extensive reading ability. It is usually diffi-cult and inappropriate to use test-based evaluation for extensive reading since stu-dents may select very different reading mate-rials, depending upon their interest and read-ing ability. The use of portfolios can not only accommodate individualized learning but also assess learning effects more reasonably.
This study, therefore, investigated the use of portfolios in a college English reading
course. It focused on the three important stages of portfolio compilation: design, im-plementaton, and evaluation. The design stage explored the design of a portfolio plan, including the rationale, objectives, principles, and contents of the portfolio, interactions between instruction and portfolio, and stu-dent commitment to the portfolio. The im-plementation stage was the practice of the portfolio plan in the classroom, including ex-planation, content selection, progress moni-toring, and collection of portfolios. A guide-line was prepared for the students explaining the why, what, and how of portfolio compila-tion, and their role. The evaluation stage in-cluded the analysis and evaluation of stu-dents’ portfolios and a survey of stustu-dents’ opinions about compiling portfolios.
The portfolio was required of students in two reading classes of the same reading course the author was teaching. At the begin-ning of the semester, the author, as the teacher, while explaining the syllabus to the students, initiated the idea of incorporating the portfolio into the course and using it as part of assessment. On the third week when the students were familiar with the regular classroom practice, the author gave students a guideline clearly specifying the principles of portfolio compilation, content require-ments of the portfolio, and criteria of evalua-tion. Students were asked to compile a port-folio, individually or in a group not more than three. The compilation should go along with the classroom instruction; that is, stu-dents read regularly one or two reading texts every week and included them in their port-folios. For each entry in the portfolio, they should follow the basic requirements de-scribed in the guideline, but could include more categories they thought appropriate. The portfolio would account for 30 percent of the total assessment for the semester. Stu-dents turned in their portfolios three weeks before the end of the semester. Then, a ques-tionnaire which contains eight questions about portfolio compilation was used to col-lect students’ feedback. The author evaluated the portfolios according to the criteria speci-fied earlier and returned them to the students. The survey was also analyzed.
三、結果與討論 (Results and Discussions)
Design
Portfolio is regarded as an excellent tool for assessing the learning process and foster-ing self-monitored learnfoster-ing; therefore, the rationale of using it in the reading classroom is to help students discover or develop their reading interest as well as good reading hab-its and to understand their reading process. Furthermore, at the university level, we think it is important for students to summarize the main ideas of a text after reading it and to reflect on the text or the learning process, since these skills not only improve students’ reading ability but also foster their metacog-nition. Then, for language learning, regular practice of vocabulary, sentence structures, and rhetorical patterns should also play a role. These, hence, become the basic requirements for the content of the reading portfolio. With these restrictions, however, each student can decide on the topic or theme he or she likes to work on and then searches for the reading texts he or she thinks interesting and appro-priate to read. In compiling the portfolio, stu-dents must organize all reading texts as well as their summaries and reflections into a self-contained product. This could create a sense of commitment, ownership, and ac-complishment.
Implementation
As most students were not familiar with portfolios, it was necessary for the teacher to explain what a portfolio is, why we want to use as it as part of assessment in a reading course, and how each student can compile a portfolio successfully. Students could choose a topic or theme related to the instruction or of their interest. They then should regularly select and read appropriate materials centered on the topic or theme. Students were told it is essential for them to reflect on the reading process and on what they learn from the ma-terials. Finally, the teacher explained how the portfolio would be evaluated. To provide students with clear and specific principles for the compilation of a reading portfolio and the criteria of evaluation, a guideline was
pre-pared and given to the students.
The teacher emphasized the importance of regular reading practice and of the portfo-lio as representation of their reading per-formance. To ensure that each student or group of students decided on an appropriate topic or theme and started to select and read appropriate materials, the teacher asked each student to turn in a draft outline in two weeks and discussed with those whose outlines were considered not adequate for a rear-rangement of the content.
Evaluation
Three weeks before the end of the se-mester, students’ reading portfolios were collected and a survey was conducted. An analysis of the portfolios showed that stu-dents were interested in topics such as news-papers, magazines, reports about sports or movies, or fiction. Most students just fol-lowed the requirements specified in the guideline without contributing additional categories to the content of the portfolio. Only a few students carefully selected the reading materials which are closely related to one another or can complement one another in terms of the subject content. In some port-folios, the progressively increasing difficulty level of the reading materials can be recog-nized. Most students still focused on the learning of vocabulary and idioms rather than critical thinking. Most could write summaries better than reflections which were often very short and similar to summaries. Evaluation of the portfolios followed the criteria: reading materials 40%, summary 20%, reflections 20%, and other categories (including lan-guage learning, portfolio design, etc.) 20%.
Student Feedback
There are eight questions in the ques-tionnaire. The first one asks students if they knew what a portfolio is before taking this course. Five of them are 4-item questions about whether portfolio compilation helps English reading. Two are open-ended ques-tions about students’ general feelings of compiling portfolios and difficulties encoun-tered in the compiling process. There are 62 usable questionnaires.
The results of the survey show that only 7 students knew a little (6) or a lot (1) about portfolio before they took this course. Almost all of the students (60 out of 62) thought portfolio compilation was helpful to the learning of English reading. Among them, 20 indicated that it helped a lot. For the writing of summaries and reflections, respectively, 16 and 17 students thought they could greatly increase the learning effect. 42 and 35, re-spectively, indicated that they could slightly increase the learning effect. With regards to the usefulness of portfolio compilation to help find out one’s reading interest, 17 and 36 students indicated that it helped a lot and slightly, respectively. About the usefulness of portfolio compilation to help understanding one’s reading process, 20 and 32 students indicated that it helped a lot and slightly, re-spectively.
Answers to the two open-ended ques-tions seem to reveal more concrete and de-tailed opinions and feelings about portfolio compilation. Seventeen students indicated that portfolio compilation, though time-consuming and difficult, improved their English reading ability, including reading speed. Seven students felt happy about dis-covering their own reading interest as they were searching for appropriate materials to read. Seven students indicated that portfolio compilation provided them with incentive to finish reading a lot regularly over a period of time. They said that otherwise they would never be able to read so much. Four students expressed that they learned to self-monitor their learning and enjoyed active learning. Other opinions include realizing their reading problems, obtaining new information and knowledge, learning a lot of vocabulary, im-proving writing ability through the writing of summaries and reflections, feeling great to find and read many texts on a topic one is interested in, and having a sense of accom-plishment.
With regards to the difficulties of piling a portfolio, sixteen students com-plained about the difficulty of writing sum-maries and reflections. Fifteen students also found it difficult to find appropriate reading materials that fit their reading level and
in-terest. Eight students indicated the difficulty in comprehension because of either too many unfamiliar words or complicated sentences. The same number of students complained that portfolio compilation was time-consuming, especially in searching for reading materials. Other difficulties students identified include the lack of experience, dif-ficulty in recognizing the main ideas in read-ing materials, and inability to finish readread-ing so many texts. Students also made sugges-tions such as less restricsugges-tions on the form of the portfolio, more teacher involvement in the process of compilation, and requirement of progress report weekly or bi-weekly.
四、計畫成果自評 (Self-evaluation)
This project investigates the use of portfolios in a reading course. It focuses on the design, implementation, and evaluation of portfolios. It seems that the use of the portfo-lio in this case study is successful in terms of its effect on improving students’ overall reading ability, but not as successful as ex-pected on active learning or students’ reflec-tions on their learning process. The reasons are as follows:
1. The students are not used to the compilation of a portfolio. A num-ber of students complained that the portfolio cost them too much time. Especially, selecting appropriate reading materials is time-consuming. This also shows that students did not have the habit of reading English materials regularly and they were not familiar with the reading sources.
2. A number of students did not select and read texts regularly; instead, they tried to complete the portfolio within the last two or three weeks before it was due, and thus they re-garded the portfolio merely as an-other form of homework and ne-glected the meaning of portfolio as the representation of long-term learning effort.
五、參考文獻 (References)
Belanoff, P. & Dickson, M. (eds.) (1991).
Portfolios: Process and Product.
Ports-mouth, NH: Boynton/Cook.
Burch, C. B. (1997). “Creating a Two-Tiered Portfolio Rubric,”English Journal, 86, 1,
55-58.
Calfee, R. C.& Perfumo, P. (1996). “A Na-tional Survey of Writing Portfolio Prac-tice: What We Learned .and What It Means,” in R. C. Calfee & P. Perfumo (eds.), Writing Portfolios in the Class-room. Mahwah, NJ: Lawrence Erlbaum
Associates, Publishers, 63-81.
Chen, Y. M. (陳玉美) (1999). “A Portfolio Approach to EFL University Writing In-struction,” in Proceedings of 16th Con-ference on English Teaching and Learn-ing in the Republic of China, 313-332.
Cole, Donna J., et al. (2000). Portfolios across the Curriculum and Beyond.
Corwin Press, Inc.
Gottlieb, M. (1995). “Nurturing Student Learning through Portfolios,” TESOL Journal, 5, 1, 12-14.
Graves, D. H. (1992). “Portfolios: Keep a Good Idea Growing,” in D. H. Graves & B. S. Sunstein (eds.) Portfolio Portraits.
Portsmouth, NJ: Heinemann, 1-12.
Hamp-Lyons, L. (1994). “Interweaving As-sessment and Instruction in College ESL Writing Classes,” College ESL, 4, 1,
43-55.
Herman, J. L., Gearhart, M. & Aschbacher, P. R. (1996). “Portfolios for Classroom As-sessment: Design and Implementation Issues,” in R. Calfee & P. Perfumo (eds.),
Writing Portfolios in the Classroom.
Mahwah, NJ: Lawrence Erlbaum Associ-ates, Publishers, 27-59.
Hirvela, A. (1997). “’Disciplinary Portfolios’ and EAP Writing Instruction,” English for Specific Purposes, 16, 2, 83-100.
Johns, A. M. (1995). “An Excellent Match: Literacy Portfolios and ESP,” English Teaching Forum, 33, 16-21.
Layzer, C. & Sharkey, J. (1996). “Proficiency, Portfolios, and Learner Development in the Reading Classroom,” paper pre-sented at the 30th Annual TESOL
Con-vention, Chicago, IL.
Padilla, A. M., et al.. (1996). “Development and Implementation of Student Portfo-lios in Foreign Language Programs,”
Foreign Language Annals, 29, 3,
427-438.
Sweet, J. (1976). “The Experience Portfolio: An Approach to Student Writing,” Eng-lish Journal, 65, 50.
Vavrus, L. (1990). “Putting Portfolios to the Test,”Instructor, 100, 1, 48-63.
Yancey, K. B. (1992). “Portfolios in the Writing Classroom: A Final Reflection,” in K. B. Yancey (ed.) Portfolios in the Writing Classroom: An Introduction.
