第五章 結論與未來研究
5.2 未來研究
本研究因時間關係有部份研究並未完成，測驗的分析是需要較多時間來驗 證，使得分析結果可以更為準確，在此提出為後續研究可以繼續進行，對於研究 中提出的方式可能有更多的修正及建議。
(1). 取得較多的樣本進行實驗分析，由較多次的測驗結果中來進行分析，這樣正 確性和準確率都能大符提升，並檢視是否有助於學生的學習或是教師在教學 上的改變。
(2). 系統提供教材的存取，目前系統上僅提供線上測驗的功能及簡單的回饋資 訊，並未有完整的教材學習功能及線上閱讀的功能，可以再加上更多的課程 資訊來輔助學生快速查找資訊。
(3). 增加更多測驗題型，目前系統中提供單選型選擇題，尚可以加入複選型選擇 題或是非題題型，只要是能夠讓系統明確判斷正確解答的試題題型，都可以 納入本系統之出題題型。
(4). 與課程做更完善的結合，在課程實施的過程中，配合系統的分析結果給學生 回饋資訊。
(5). 本研究中，所提出兩個自訂公還需要更多的資料驗證其實用性，或可以設計 更為適合學生學習情況的公式，計算出不同的精熟標準給教師或學生作為參 考的依據。
(6). 針對試題的品質可以使用更多的方式去檢驗，本研究所使用 S-P 分析中的試 題注意係數去診斷，可以再加入另外的方式交叉比對，例如基因演算法、計 算試題難度或鑑別度等方式。
參考文獻
[1] 王素賢，淡江大學教育科技研究所在職專班碩士論文，高中數學科補救教學 教材設計之研究，2003 年。
[2] 台灣網路資訊中心，「九十八年度台灣寬頻網路使用狀況調查摘要分析」，
2009 年。
[3] 台灣網路資訊中心，「2008 年一月份台灣寬頻使用調查報告」，2008 年。
[4] 江仲翔，國立中山大學應用數學系研究所碩士論文，高級中學教師自編測驗 評量方式之探討，2003 年。
[5] 余民寧，教育測驗與評量-成就測驗與教學評量 p.301-344，心理出版社，1997 年初版。
[6] 余民寧，國立政治大學教育學系，精熟學習、測驗診斷、與補救教學，2008 年。
[7] 吳婉嫕，利用 S-P 表分析高中生地圖技能以一個班級為個案研究，國立台 灣大學地理環境資源研究所碩士論文，2006 年。
[8] 林重新，心理與教育測驗 p.222-227，揚智文化，2001 年。
[9] 林進材，有效教學-理論與策略 p.117-121，五南圖書出版公司，2000 年初版。
[10] 涂金堂，認知診斷評量探p.76-81，究國立臺南師範學院「南師學報」第37 卷第二期教育類，2003 年。
[11] 徐正穩，教育心理學 p.238-249，中國行為科學社，1991 年。
[12] 黃天、歐陽謙，國民小學課程與教學 p.42，考用出版社，2007 年。
[13] 國科會數位典藏國家型科技計畫，數位產業內容白皮書，經濟部工業司，2003 年。
[14] 游姮茹、林渙祥、黃台珠，高中學生科學創造性問題解決能力評量工具之研 發與初探，第二十一屆科學教育學術研討會，國立彰化師範大學，2005 年。
[15] 陳德懷、黃亮華，邁向數位學習社會，遠流出版公司，2003 年。
[17] 溫明麗、黃乃熒、林建福、黃純敏、黃桂君、陳怡如、梁瑞芸譯，Richard D.
Parsons, Stephanie Lewis Hinson, Deborah Sardo-Brown 著。教育心理學：教育 的行動研究 p.576-578，洪葉文化，2005 年。
[18] 遠見雜誌廣告部，「未來競爭力的關鍵報告-百大企業數位學習大調查」，遠見 雜誌，第 246 期，p.42-43，2006 年。
[19] 劉奕樟，國立交通大學網路學習碩士在職專班碩士論文，網路多媒體評量與 學習診斷系統之建置，2003 年。
[20] 賴慧玲譯，Mary Alice Gunter, Thomas H. Estes, Jan Schwab 著，教學模式 p.113-121，五南圖書，2002 年。
[21] Abdolhossein Sarrafzadeh, Samuel Alexander, Farhad Dadgostar, Chao Fan, Abbas Bigdeli, “How do you know that I don＇t understand? A look at the future of intelligent tutoring systems”, Computers in Human Behavior 24, p.1342–
1363, 2008.
[22] Bloom, B. S., Hasting, J. T. & Madaus, G. F.. “Handbook on formative and summative evaluation of student learning”, New York: McGraw-Hill, 1971.
[23] Bloom, B. S., Hasting, J. T. & Madaus, G.,” Evaluation to Improve Learning, p.238-249”, New York: McGraw-Hill, 1981.
[24] Enis Dogan & Kikumi Tatsuoka, “An international comparison using a diagnostic testing model : Turkish students’ profile of mathematical skills on TIMSS-R”, 2007.
[25] Gunter, M. A., Estes, T.H., & Schwab, J. Instruction: A models Approach.
Needham Heights, MA: Allyn and Bacon, p.99-104,1995.
[26] Keller, F. S., “Good-bye, teacher”. Journal of Applied Behavior Analysis, 1, p.79-89, 1968.
[27] Menucha Blrenbaum, Anthony E.Kelly & Klkumi K. Tatsuoka, “Diagnosing Knowledge States in Algebra Using the Rule Space Model”, 1992.
[28] Menucha Birenbaum, Kikumi K. Tatsuoka and Yaffa Gutvirtz, ”Effects of Response on Diagnostic Assessment of Scholastic Achievement”, 1992.
[29] Rogers, C. R. “Freedom to learn for the 80’s ”, Columnbus Ohio, 1983.
[30] Sato, T, “The S-P chart and caution index”, NEC Educational Information Bulletin, p.80-1, 1980.
[31] Sato, T. “The state of art on S-P analysis activities in Japan.C&S System Reach Labs”, NEC Corp, 1984.
[32] Shute, V. J. & Psotka, J., “Intelligent tutoring systems: Past, present, and future.”, Handbook of research on educational communications and technology,1995.
[33] Tatsuoka, K. K., & Tatsuoka, M. M. Spotting erroneous rules of operation by the individual consistency index. Journal of Educational Measurement, p.221-230, 1983.
[34] Tennyson, R. D., and Park, O., “The teaching of concepts: A review of instructional design literature. Review of Educational Research”, 50, p.55-70, 1980.
[35] Trace A. Urdan, & Cornelia C. Weggen, Corporate E-learning: Exploring a new frontier, WR Hambrecht Co., 2000.
附錄
To analyze assessment using SPC Table
Wen-Chih Chang, Tsung-Pu Lee
Chung Hua University, Hsin-Chu, Taiwan, earnest@chu.edu.tw
Chung Hua University, Hsin-Chu, Taiwan, kfb526@hotmail.com
Abstract
How to provide the information about problem and course to learner? It was the major function that SPC table would solve. Using SPC table made the exam data became the quantification profile. It could provide us much information that came from each student, problem and course. First, viewed students had progressed status in each chapter of posttest. Second, displayed the further data about the problems.
Third, the educator could comprehend the education situation. We used much data to try out the effect about SPC table. In the experiment we added some variable was we give the feedback information to class B. The significant propose was made difference between two classes and viewed the SPC table easily. The result exhibited obvious the information from each exam.
Keywords: SPC table, quantification profile, comprehend
Introduction
Every teacher usually wanted to know the education result which was understood by students. Unfortunately, their information sources were limited. Therefore we got the information which derived form the exam to enable the information quantification.
Our experiment subject was the sophomore. The course name was the business data communication. The course content was about the network. There are two classes in this year, so we want to group into two. One is experiment group and another is control group. In this course, we used four chapters which had pretest and posttest. The four chapters include chapter five to eight. It also means every group had two times exams in the same chapter. The different between experiment group and control group who would get feedback information about exam to experiment group. Feedback information was came from the incorrect answers that each student to respond in pretest. It had the section and book page in feedback information. The main goal was help each student received the correct concepts easily.
In our experiment the exam date between pretest and posttest was a week. The experiment group reviewed the feedback information, whereas control group had nothing in this week.
After that we had to execute the posttest. Finally, it must be compared with two exam used the S-P table.
We selected the forty students as the sample from each class. Their responded would become our major analysis data. Every exam was included twenty questions about a chapter.
So the first exam had twenty questions about the pretest of chapter fifth. The second exam had forty questions that included chapter fifth posttest and chapter eighth pretest. The chapter eighth had similar relate to chapter fifth. The third one was the chapter eighth posttest and chapter sixth pretest. The forth exam was the chapter sixth posttest and chapter seventh pretest. The chapter seventh had no posttest because it’s time to final term. Therefore we used the final exam respond to be the posttest about chapter seven.
Related Work
We found three papers for the comparison sample. First, “A Study on Senior High School Teacher-made Assessments”, it used the S-P table to analyze student response. It compared the different to each school and the grade in two exams. The main goal at understand the reason that students probably made mistake after test. Second, “Using s-p table to evaluate students’ study condition”, found out student's study situation using the S-P table analysis method. It was assisted teacher to grasp the wrong concept about students. They had pretest and posttest. The result of their experiment there are more students improved their concept.
Third, “The Design and Implementation of a Web-based Multimedia Cognitively Diagnostic Test and Learning system”, developed an assessment system about learning. The S-P table analysis was the system main composition. The author was increased two new variables into S-P table. Enabled the system accurately diagnosis student's situation. The suggestion which gave to the teacher was established by the system. Teacher could hand on the situation immediately.
In the research we used the SPC (student-problem-course) table. It could divide into three parts, S-P table, S-C table and C-P table. The table was an examination analysis method which cared about the topic to student did in the examination test question answers the response group whether had the unusual. This analysis method was suitable to take the class and grade as the unit. The students were not very huge analysis data.
The S-P table was created by Takahiro Sato in 1970. It used the figure to response situation about student who was replied for examination. The most important goal was get the learning information from each student. Then it also can provide teacher the effective feedback.
The S-P table has two dimensions. In this table, the row means student number and the column means problem number. If student A has correct answer in first problem, we mark “1”
to express his answer. On the other hand we mark “0” to express the incorrect answer. Then we need to sum up the number of mark “1” that student and problem numbers. Next the two summations must to sort to decrease by degrees. Finally, it should to draw the boldface line to express the correct rate of student and the dotted line to express the correct rate of problem.
The S curve was dividing line between correct and incorrect student answers. Most of the correct answers were in the part of left.
The Figure1 was S-P table. In the table boldface line was the S curve means students and dotted line was the P curve means problem.
6 8 2 3 4 5 10 9 1 7 A 1 1 1 1 1 1 1 1 1 0 9 F 1 1 1 1 1 1 0 0 1 1 8 B 1 1 1 1 1 0 0 1 0 1 7 C 1 0 1 1 1 1 1 1 0 0 7 E 1 1 1 1 0 0 1 0 1 1 7 D 1 1 0 0 1 1 1 0 0 0 5 G 0 1 0 0 0 1 1 1 0 0 4 6 6 5 5 5 5 5 4 3 3
Figure 1. S-P table
S-C table and C-P table also indicated Similarly S-P table. S-C table expressed the student replied in each chapter of problems. The row means chapter number of the course and the column means student number. Expression like the figure 2 shows. In the figure 2 boldface
A F B C E D G 3.2.1 1 1 1 1 1 1 1 7 3.1.1 0 1 1 1 1 1 1 6 3.3.2 1 1 0 1 1 1 0 5 3.1.2 1 1 1 1 0 0 1 5 3.4.1 1 1 1 1 1 0 0 5 3.2.2 1 1 0 1 1 0 0 4 3.4.2 1 0 1 0 1 1 0 4 3.3.3 1 1 1 0 0 1 0 4 3.4.3 1 0 1 1 0 0 1 4 3.3.1 1 1 0 0 1 0 0 3 9 8 7 7 7 5 4
Figure 2. S-C table
And C-P table expressed the problem stems from the chapter part or the number of pages.
The row means chapter number of the course and the column means problem number.
Expression like the figure 3 shows. In the figure 3 boldface line was the C curve means chapters and dotted line was the P curve means problems.
3 1 4 7 2 6 5 10 8 9 3.2.1 0 1 0 1 0 1 0 1 0 0 4 3.1.1 0 1 0 1 0 0 1 0 1 0 4 3.1.2 0 1 0 1 0 0 1 0 1 0 4 3.2.2 0 1 0 0 0 1 0 1 0 0 3 3.4.3 1 0 1 0 0 0 0 0 0 1 3 3.4.1 1 0 1 0 0 0 0 0 0 0 2 3.4.2 0 0 1 0 0 0 0 0 0 1 2 3.3.2 1 0 0 0 1 0 0 0 0 0 2 3.3.1 1 0 0 0 1 0 0 0 0 0 2 3.3.3 0 0 0 0 1 0 0 0 0 0 1
4 4 3 3 3 2 2 2 2 2 Figure 3. C-P table
Experiment
In the research, we gave no feedback to class A students. They could know their test grade but couldn’t know the problem which was wrong. So they should to find the incorrect answers by their own. On the contrary, the class B students could get the information after the pretest. They knew the incorrect answers found it in correct chapters. The answer correct rate in class A and class B of pretest and posttest are shown in Figure 4(a) and Figure 4(b).
Comparing the result from Figure 4(a) and Figure 4(b) that could view the progress had obvious difference. Class B students had progressed in all of the chapters .With our system’s support, class B had better academic performance than class A in all of the chapters.
Figure 4(a). Class A
Figure 4(b). Class B
The standard deviation is the most common measure of statistical dispersion, measuring how widely spread the values in a data set. The standard deviation is small which means many data points are close to the mean. In Table 1, the posttest standard deviation of class B is smaller than class A. In order to observe the standard deviation of two classes, we apply Table 2 which includes the quantity of students who answer correct questions. The data which was finally exam in table 2. It was the assessment included the four chapters represented the final summary enough.
Table 1.The pretest and posttest in class A and B
Class A Class B
standard deviation standard deviation
Chapter Pretest Posttest Pretest Posttest
CH5 2.24 4.15 3.29 2.41
CH6 2.63 2.23 3.22 2.04
CH7 3.68 2.80 4.58 2.81
CH8 3.11 3.24 3.68 2.12
Table 2. Students who answer questions correct in class A and B Test items Class A Class B
0~30 1 0
31~35 2 2
36~40 9 3
41~45 7 6
46~50 5 8
51~54 17 12
55~59 14 9
60 0 0
Figure 5. Students who answer questions correct disparity curves in class A and B It was apparent the class B curve looks like normal distribution than class A in Figure 5.
For this reason, class B academic performance is better than class A.
1 14 4 2 3 15 6 8 9 13 5 10 11 7 12 2 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 14 3 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 14 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 0 13 16 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 13 29 1 1 1 1 1 1 1 0 1 1 1 1 1 0 1 13 30 1 1 1 1 1 1 1 1 0 1 1 1 1 0 1 13 4 1 1 1 1 1 1 1 1 1 0 1 0 1 0 1 12 19 1 1 1 1 1 1 1 1 1 1 0 1 0 0 1 12 21 1 1 1 1 1 1 1 0 1 0 1 1 1 1 0 12 34 1 1 1 1 1 0 1 1 1 1 1 0 0 1 1 12 24 1 1 1 1 1 1 1 0 0 1 1 1 1 0 0 11 10 1 1 1 1 1 1 1 1 1 0 1 0 0 0 0 10 14 1 1 1 1 1 1 1 1 1 0 1 0 0 0 0 10 39 1 1 1 1 1 1 1 0 1 1 1 0 0 0 0 10 12 1 1 1 1 0 1 1 1 1 1 0 0 0 0 0 9 18 1 1 1 1 1 1 0 1 1 0 0 0 1 0 0 9 27 1 1 0 1 1 1 1 1 0 1 1 0 0 0 0 9 31 1 1 1 1 1 1 1 1 0 1 0 0 0 0 0 9 5 1 1 1 1 1 0 1 0 0 0 1 0 1 0 0 8 11 1 0 1 1 1 0 1 1 0 0 1 0 0 1 0 8 13 1 1 1 1 1 1 0 1 0 1 0 0 0 0 0 8 17 1 1 1 1 1 1 0 1 0 1 0 0 0 0 0 8 9 1 0 1 0 0 1 0 0 0 1 1 1 1 0 0 7 23 1 1 0 1 1 1 0 1 0 0 1 0 0 0 0 7 26 1 1 0 1 0 0 0 0 0 0 0 1 1 1 1 7 28 1 1 0 0 0 1 0 1 0 0 0 1 1 1 0 7 38 1 1 1 1 1 0 0 0 1 1 0 0 0 0 0 7 40 1 1 1 0 0 0 1 1 1 0 0 0 0 1 0 7 33 1 0 1 0 1 1 0 1 0 1 0 0 0 0 0 6 37 1 1 1 0 1 0 0 0 1 0 0 0 1 0 0 6 7 1 0 1 0 0 0 1 0 1 0 0 0 0 1 0 5 20 1 1 1 0 0 1 0 0 0 0 0 1 0 0 0 5 25 1 0 0 0 0 1 0 0 1 0 0 0 0 1 1 5 8 1 0 0 0 0 0 0 1 0 0 0 1 1 0 0 4 15 1 1 0 0 0 0 0 1 0 0 0 1 0 0 0 4 22 1 0 0 1 1 0 0 0 0 0 0 0 0 1 0 4
35 1 0 0 0 0 0 1 0 1 1 0 0 0 0 0 4 6 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 3 32 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 36 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
38 29 28 26 26 25 23 23 20 19 18 16 16 10 9
Figure 6. S-P table to Class A
－represented the P curve -- represented the S curve
Figure 6 and 5 shows the S-P table to Class A and B for chapter 5 posttest. We received the detailed information for every student. Most of class B students had done better than class A. In the table, the scope of S-P curve in figure 7 was bigger than figure 6.
15 1 9 14 12 13 4 8 3 6 7 5 2 10 11 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15 23 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15 29 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15 3 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 14 10 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 14 20 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 14 21 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 14 24 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 14 30 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 14 32 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 14 33 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 14 5 1 1 1 0 1 1 1 1 1 1 1 1 1 0 1 13 13 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 13 16 1 1 0 1 1 1 1 0 1 1 1 1 1 1 1 13 19 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 13 25 1 1 1 1 1 1 1 1 1 1 0 1 1 0 1 13 26 1 1 1 1 1 1 1 1 0 1 1 0 1 1 1 13 34 1 1 1 1 1 1 1 1 0 1 1 1 1 0 1 13 40 1 1 1 1 1 0 1 1 1 1 1 1 1 1 0 13 4 1 1 1 1 1 1 1 1 1 0 1 1 0 1 0 12 9 1 1 1 1 1 1 1 1 1 1 0 1 0 0 1 12 15 1 1 1 1 1 1 1 1 1 0 1 0 0 1 1 12 17 1 1 1 1 1 0 1 0 1 1 1 1 1 0 1 12 35 1 1 1 1 1 1 1 1 0 0 1 1 0 1 0 11 6 1 1 1 1 0 1 1 0 1 0 1 1 0 1 0 10 22 1 1 1 1 1 0 1 0 1 1 0 1 1 0 0 10 1 1 1 1 0 0 1 0 0 1 1 1 0 1 1 0 9 8 1 0 1 1 1 1 1 1 0 1 0 0 0 0 1 9 12 1 1 1 1 1 0 0 1 0 0 1 0 1 1 0 9 14 1 1 0 1 1 1 0 1 0 0 1 1 1 0 0 9 18 1 1 1 1 0 1 1 0 1 0 0 0 1 1 0 9 27 1 1 1 1 1 1 0 1 0 0 1 1 0 0 0 9 28 1 1 1 1 1 1 0 1 0 1 0 0 0 0 1 9 31 1 1 1 0 0 1 0 1 1 1 0 1 1 0 0 9 38 1 1 1 1 1 1 0 1 1 1 0 0 0 0 0 9 7 1 1 1 0 0 1 1 0 1 0 1 0 0 1 0 8 11 1 1 1 1 1 1 1 0 0 0 0 0 1 0 0 8
37 1 0 1 0 0 1 0 1 0 1 0 0 0 0 0 5 36 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 3
40 38 35 34 33 33 30 30 29 28 27 26 24 23 23
Figure 7. S-P table to Class B
－represented the P curve -- represented the S curve
Student Response
At the final exam we gave a simple questionary for Class B students. It inquired for the suggestion to students who got the feedback after every exam. The detailed contents to four questions under line:
1. Did you feel the feedback for each student helped review the course after the exam?
2. Did you feel the feedback enhanced the concept for each chapter?
3. Did you feel the feedback helped to know the misunderstand part in each chapter?
4. Did you feel the feedback helped the course very much?
Table 3. Student questionary for feedback item SD(%) D(%) N(%) A(%) SA(%)
1 0 0 19.57 58.70 21.74
2 0 4.35 21.74 50 23.91
3 0 0 8.70 52.17 39.13
4 0 0 19.57 45.65 34.78
(SD: Strongly Disagree D: Disagree N: Neutral A: Agree SA: Strongly Agree)
All of the result appeared in table 3.Most students (80.43%) felt the feedback helped them review the course. Furthermore, 73.91% agreed or strongly agreed with feedback enhanced their concept. 91.30% know their misunderstand part after get the feedback. There were 80.43% students believed the feedback helped they for study the course.
There were some different suggestions for the feedback. It likely some of the students may depend on the information too much. A few of students thought the information no use to them. And too crowded exam would make preparation not enough.
Table 4. Reference compared table