110/06/29 1
The Application of
Web-based Peer
Evaluation System
Yu-Hui Tao
National University of
Kaohsiung
ytao@nuk.edu.tw
March 13, 2009
(Your way)
國立台灣師範大學人力資源與發展研究所Motivation
Started teaching in 1997
Initial impression of Taiwanese students: low class participation
despite what I have done
Started adding various activities in class design for increasing
students’ responsibility through their performance evaluation
Often asked students to criticizing others’ work/statements/…
In some courses, students were asked to write down their comments
to the presenters for me to evaluate, and sometimes I cut and past
these comments to the presenters
110/06/29 3
Outline
1.
Concept of Peer Evaluation System (PES)
2.Comparison with Online Learning System
and Instant Response System
3.
PES Use Case Diagram and Process Flow
Screenshots
4.
PES Examples
PES Concept in Class
Assignment
110/06/29 5
Peer Evaluation in Literature
陶幼慧、李兆庭，同儕作業評量之教學策略執行平台，電子商務與數位生活研討會， 3/17 ，台北實踐大學，2007 。
同儕評量可培養學生的自主性及高層次思考技能 (Heywood, 2000; Tsai, et
al., 2001; Tsai, Lin and Yuan, 2002) ，也屬於同儕教學的一種，可發展評估 、證明正當性以使用學科的知識的能力 (Donaldson and Topping, 1996) 。
同儕評量的理論支柱包含 ( 徐雍智、蔡今中及陳明璋， 2005) Vygotsky 的社會建構主義 Piaget 的認知衝突理論 以及從其他同儕輔助學習而來的教育 理論 其影響範圍包含了認知、後設認知、情意、社會與傳達技能、整體受益等方面 向 包括寫作、教學、商務、科學、工程到醫學方面 (Falchikov, 1995; Freeman, 1995; Topping, 1998) 一般而言，形成性評量提供意見，而總結性 (summative) 評量提供分數 (Bostock, 2000) 。
Peer Evaluation in Literature
更有學者推薦書面與口頭同儕回饋的運作，因為學生在書面評量時僅專注於評 論的撰寫，而口頭報告中學生還同時會質詢、解釋及給建議的活動 (Berg, Admi raal and Pilot, 2003) 。
同儕評量可能的好處除了成本效益外，最重要的是教師可管理其程序的進行而 非自己評量大量學生的作品 (Race, 1998) ，並可透過網際網路科技來協助同儕 評量的進行。
儘管如此， Wong and Ng (2005) 的研究指出許多的教學管理系統並不支援正式 的評量 (CELT, 2004) ，但學生很滿意線上的作業繳交與網路同儕評量的使用。
110/06/29 7
Peer Evaluation in Literature
Peer Evaluation in Literature
• 整體而言，在 Topping (1998) 分析的 109 篇同儕評量的研究中得知，具一 定的信度與效度，且其的應用在高等教育許多的領域中已經被證實有正面的學 生滿意度與學習效能 (Searby & Ewer, 1997; Stanier, 1997; Gatfield, 1999; Hafner & Hafner, 2003)
• 雖然實務上學生所作評量的效度與可信度，仍可能是一個潛在基本的問題，這 部份的判斷可調查受評者對得到同儕的評量是否感覺正確與有價值。而這樣的 問題也可透過如明確的條件、雙向匿名與多個評量者來改善。
• 根據劉旨峰、林珊如及袁賢銘 (2001) 整理出目前研究中，具高信度的同儕互評 的應用範圍有散文寫作 (Haaga, 1993; Orpen, 1982) 、多媒體創作
(Rushton, Ramsey & Rada,1993) 、口頭報告 (Freeman, 1995; Hughes & Large, 1993a,1993b) 、選擇題 (Catterall, 1995) 、實用報告 (Hughes, 1995) 與專業技能 (Korman &Stubblefield, 1971; Ramsey et al., 1996) 。
• 低信度的則有散文寫作 (Mowl & Pain, 1995) 及口頭報告 (Taylor,1995; Watson, 1989) 等等。所以，文獻調查結果間仍是有部份意見分歧的現象 • Bostock (2000) 指出管理學生同儕評量的主要困難為「在學生間傳遞產生評
110/06/29 9
Peer Evaluation in Literature
• 整體而言，在 Topping (1998) 分析的 109 篇同儕評量的研究中得知，具一 定的信度與效度，且其的應用在高等教育許多的領域中已經被證實有正面的學 生滿意度與學習效能 (Searby & Ewer, 1997; Stanier, 1997; Gatfield, 1999; Hafner & Hafner, 2003)
• 雖然實務上學生所作評量的效度與可信度，仍可能是一個潛在基本的問題，這 部份的判斷可調查受評者對得到同儕的評量是否感覺正確與有價值。而這樣的 問題也可透過如明確的條件、雙向匿名與多個評量者來改善。
• 根據劉旨峰、林珊如及袁賢銘 (2001) 整理出目前研究中，具高信度的同儕互評 的應用範圍有散文寫作 (Haaga, 1993; Orpen, 1982) 、多媒體創作
(Rushton, Ramsey & Rada,1993) 、口頭報告 (Freeman, 1995; Hughes & Large, 1993a,1993b) 、選擇題 (Catterall, 1995) 、實用報告 (Hughes, 1995) 與專業技能 (Korman &Stubblefield, 1971; Ramsey et al., 1996) 。
• 低信度的則有散文寫作 (Mowl & Pain, 1995) 及口頭報告 (Taylor,1995; Watson, 1989) 等等。所以，文獻調查結果間仍是有部份意見分歧的現象 • Bostock (2000) 指出管理學生同儕評量的主要困難為「在學生間傳遞產生評
量訊息給受評者和評量者，以及儲存這些資料供日後稽核」，而同儕評量的缺 點可透過匿名性、多位評量者及教師調節來避免
葉盈秀 ， Visual Interface Development for e-Learning Management System,
淡江大學教育科技學系碩士班 ， 2006 。
Online Learning System
Comparison
110/06/29 13
Online Learning System
Comparison
Online Learning System
Comparison
110/06/29 15
IRS (Instant Response
System)
Source：http://www.habook.com.tw/software/Detail_soft.asp?BookNo=ME010200&B_key=IRS即時反饋系統&CatalogID =12 Interactiv e white board (IWT) Teacher’s remote Student’s remote IRS System Learning profile database Receive r
IRS Strategies
School Harvard University Taipei Medical University Taipei Municipal University of Education Class Physics course Medical course Information Science course
Applicati on Model Source ： HaBook onine DM 2007/09/21 Preview & Q&A
before class answer Q’s at Preview but
beginning of the class No preview
110/06/29 17
Online Learning System vs. IRS
vs. PES
Level of Synchronizatio n Sync. Async.Online Learning System
IRS PES Level of Interaction (class activities) High Low
PES –
How it
works?
English Manual: http:// tao.nuk.edu.tw/m atsuyama/PESHelp Student.ppt110/06/29 19
PES – Home page
(
http://
pes.nuk.edu.tw)
Class
announcemen ts
PES – Teacher’s first
page
Current pageand path
110/06/29 21
PES –Create a new class
Co-teacher setup Class status setup Advanced features setup Evaluation assignment setup
PES –Advanced features
setup
Mostly default Depends on class design110/06/29 23
PES–Evaluation assignment
setup
For individual assignmen t For group assignmen t Plus self evaluationThree assigning rules: 1.N students before you 2.N students after you
PES–Student account
registration
The first thing students do
110/06/29 25
PES –Student enrolling
classes
Enroll a class
PES –Class list for
enrollement
Available ones110/06/29 27
PES – Teacher’s first
page
Student
PES – Student
list/management
Check the approved
110/06/29 29
PES – Teacher’s assignment
screen
Insert/Edit/Delete an assignment
Assignment Version
PES – Adding a new
assignment
Can override the default setup at class creation Individual Group Test Roll call Assigning discussion Competition (N/A) Free discussion (N/A)Click confirm will go to due date
110/06/29 31
PES –Due dates setup
For every version For setting upload, pasdue, evaluation, and reply dates & times
PES–Student assignment
screen (before due date)
This group’s upload link Other group’s uploaded assignment
110/06/29 33
PES–Student assignment
upload
This group’s upload link Other group’s uploaded assignment Browse theZIP file for upload
PES–How to zip a file?
This group’s upload link Other group’s uploaded assignment110/06/29 35
PES –PES–Student
assignment screen (after
due date)
Past due and disabled
PES – Student assignment
screen
Assignment content Assignment in a new large window110/06/29 37
PES – Student evaluation
Opinion & reply
+/- score Teacher can evaluate this student to other’s evaluations Only shown in teacher’s screen
PES –Student evaluation
can be revised before due
date& time
110/06/29 39
PES – Teacher’s evaluation on
each student’s evaluation of
others
Performance evaluation of this student’s peer evaluation Student’s evaluation of assigned students’ assignmentsPES – Student score
ranking
Rank by scoresScore Ranking Tracing scoring Exporting Scores .xls Exporting evaluation performance .xls
110/06/29 41
PES – Parameters of
assignment activities
1.
Individual vs. group
2.
In-class vs. after-class (vs.
combo)
3.
Formative vs. summative
4.
Competition vs. not-competition
24=16 variationsFormative assignment –
upload the revision based on
peer feedback
Editing this Version listing110/06/29 43
Formative assignment –
creating one more version
create a new version here
PES–Competition func. N/A but
can be done with regular
functions
3 competitors from version 1 Screening 3 finalist from round 1110/06/29
PES can incorporate what class
activities ？
Ind. Sync. Assign. Grp.Syn c. Assign. Ind. Async. Assign. Grp. Async. Assign. Ind. Form. Assign. Grp. Form. Assign. Ind. Comp. Assign. Ind. Comp. Assign. Ind. Assigned discus. Grp. Assigned discus. Ind. Test Upload Grp. Class Disc. upload Ind. Class Rollcall Ind. Free Disc. N/A … …PES – Assigned discussion
with students replies
Teacher ’s posting Student s’ replies
110/06/29 47
PES - Online test with
upload
PES – Class roll call
Student side110/06/29 49
PES - Exporting assignment
with a HTML file linking
PES – Other possibilities?
Online
survey
Blog
Web-based
BBS
IRS functionalities
…
110/06/29 51
PES – What I have learned
Teacher ：
Focusing on managing students’ assignment process instead of
grading them all
With PES skill, class activity design can be agile and flexible
Student ：
Focusing on training higher-order thinking and peer mutual
learning/instructions
Focusing on learning process not the final scores
PES ：
110/06/29 53
