本研究提出一個對學生摘要自動進行評分的系統，針對英文文章，挑選適合 的文意理解問答題，並且對學生答案自動評分。第一種主角問答題，系統將文章 建立出語意關係圖，使用圖形結構分析找出主角，與學生的答案進行比對後，可 以評估學生的理解程度，由實驗的結果可看出系統可以找出文章中的主角。第二 種是文意理解敘述題，系統透過比對文章及學生摘要的語意關係圖擷取特徵，使 用機器學習的方法，建立預測老師評分等第的分類模型，用來對學生的答案進行 語意符合程度自動評分，實驗結果顯示文章中有明確可代表文章重點的字詞時，
可達到不錯的正確率。
本研究透過文章和答案的語意關係圖的比對過程，擷取特徵進行評分。比對 特徵中，大部份是與語意節點相關的特徵值，語意關係的部份較少，因此未來研 究可以加入更多的語意關係的特徵值，以及進一步能夠對於語法結構及文法結構 進行分析比對，擴展預測評分的分類模型，使其可以對於學生摘要的答案進行綜 合性的分數預測。
參考文獻
[1] M. Agarwal and P. Mannen, “Automatic Gap-fill Question Generation from Text Books,” in Proceeding IUNLPBEA '11 Proceedings of the 6th Workshop on Innovative Use of NLP for Building Educational Applications, Pages 56-64, 2011.
[2] L. F. Bachman, N. Carr, G. Kamei, M. Kim, M. J. Pan, C. Salvador and Y. Sawaki,
“A Reliable Approach to Automatic Assessment of Short Answer Free Responses,”
in Proceeding of the 19th international conference on Computational linguistics - Volume 2, Pages 1-4, 2002.
[3] C.-C. Chang and C.-J. Lin, “LIBSVM: A Library for Support Vector Machines,”
National Taiwan University, Taipei, Taiwan, 2013.
[4] S. Curto, A. C. Mendes and L. Coheur, “Exploring linguistically-rich patterns for question generation,” in Proceeding of the UCNLG+Eval: Language Generation and Evaluation Workshop, Pages 33-38, 2011.
[5] R. Mitkov and L. A. Ha, “Computer-Aided Generation of Multiple-Choice Tests,”
in Proceeding Natural Language Processing and Knowledge Engineering, 2003.
[6] M. Mohler, R. Bunescu and R. Mihalcea, “Learning to Grade Short Answer Questions using Semantic SimilarityMeasures and Dependency Graph,” in Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1, Pages 752-762, 2011.
[7] A. J. C. Joseph D. Novak, “The Theory Underlying Concept Maps and How to Construct and Use Them,” Technical Report IHMC CmapTools 2006-01 Rev 01-2008, Florida Institute for Human and Machine Cognition, 2008.
[8] C. P. Ros´e, A. Roque, D. Bhembe and K. VanLehn, “A Hybrid Approach to Content Analysis for Automatic Essay Grading,” in Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology: companion volume of the Proceedings of HLT-NAACL 2003--short papers - Volume 2, Pages 88-90, 2003.
[9] S. G. Pulman and J. Z. Sukkarieh, “Automatic Short Answer Marking,” in Proceedings of the second workshop on Building Educational Applications Using NLP, Pages 9-16, 2005.
[10] E. Sumita, F. Sugaya and S. Yamamoto, “Measuring Non-native Speakers Proficiency of English by Using a Test with Automatically-Generated Fill-in-the-Blank Questions,” in Proceedings of the 2nd Workshop on Building Educational Applications Using NLP, pages 61–68, 2005.
[11] Y.-C. Yang, J.-F. Yang, J.-M. Chang and J. S. Chang, “電腦輔助閱讀測驗自動出 題 Development of a Computer Assisted Reading Comprehension Test,” in International Conference on English Instruction and Assessment, 2006.
[12] K. Zubrinic, D. Kalpic2 and M. Milicevic, “The automatic creation of concept maps from documents written using morphologically rich languages,” Expert Systems with Applications, page 12709-12718, 2012.
[13] R. Ziai, N. Ott and D. Meurers, “Short Answer Assessment: Establishing Links Between Research Strands, ” In Proceedings of the Seventh Workshop on Building Educational Applications Using NLP, page 190-200, 2012.
[14] 張春興, 教育心理學-三化取向的理論與實踐, 台北市: 東華書局, 1989.
[15] 李秀娟, “不同教學策略對國中生學習生物的影響,” 國立台灣師範大學科學 教育研究所碩士論文, 1998.
[16] Stanford Corenlp, http://nlp.stanford.edu/software/corenlp.shtml.
[17] Wordnet, http://wordnet.princeton.edu/.
附錄一 系統自動擷取主角詞彙結果
1-1 無方向性語意關係圖的主角詞彙結果
表 19 主角詞彙(無方向性語意關係圖)
文章 主角詞彙 教師答案 分數值
Reading 101 Sam Sam 1.0
Reading 102 Gary 's father, restaurant Gary’s father, restaurant 1.0
Reading 103 Jimmy Jimmy 1.0
Reading 104 Duke Duke(Jay’s dog) 1.0 Reading 105 baby panda Panda family 0.33
Reading 106 Mrs. Chen 0.0
Reading 107 The author The author (and his family) 1.0 Reading 201 Sam, desk, Sam 's Sam, Sam’s room 0.5 Reading 202 Mandy 's Mandy’s birthday 0.5
Reading 203 Jimmy Jimmy 1.0
Reading 301 school, care School faculty and staff 0.09
Reading 302 Tim Tim, housework 0.5
Reading 303 Jane, list Jane, plane outfit 0.33 Reading 304 school The author school, Mr. Hsu 0.125
Reading 401 worms Worm 1.0
Reading 402 Oscar David, Oscar, baseball game 0.33
Reading 403 carnival Carnival 1.0
Reading 404 Tina AID, Tina 0.5
Reading 405 contest Jimmy, Kelly, school library
contest 0.09
Reading 406 Sally Roller coaster ride, Mark,
Sally 0.33
Reading 407 bacteria bacteria, weather 0.5 Reading 408 Jenny, Brian Jenny, Brian 1.0
Reading 501 O. Henry O. Henry 1.0
Reading 502 tsunami Tsunami, preventing tsunami 0.5 Reading 503 Della, Jim, hair Jim, Della, Christmas present 0.5 Reading 504 Gary, mom, Blue whale 0.0 Reading 505 leaves, eucalyptus, koalas,
zoos eucalyptus leaves, koala 0.5
1-2 雙向語意關係圖的主角詞彙結果
表 20 主角詞彙(雙向語意關係圖)
文章 主角詞彙 教師答案 分數值
Reading 101 Sam Sam 1.0
Reading 102 Gary 's father, restaurant Gary’s father, restaurant 1.0
Reading 103 Jimmy Jimmy 1.0
Reading 104 Duke Duke(Jay’s dog) 1.0 Reading 105 baby panda Panda family 0.33 Reading 106 Mrs. Chen Mrs. Chen 1.0 Reading 107 The author The author (and his family) 1.0 Reading 201 Sam 's desk, desk, Sam Sam, Sam’s room 0.43 Reading 202 Mandy Mandy’s birthday 0.2
Reading 203 Jimmy Jimmy 1.0 Reading 301 school, care School faculty and staff 0.09 Reading 302 Tim, day, week, chart Tim, housework 0.2 Reading 303 Jane, list, plane outfit Jane, plane outfit 0.67
Reading 304
school, Mr. Hsu, students, tribe, classrooms, travelers, experience, world
The author school, Mr. Hsu 0.08
Reading 401 worms, Mr. grylls, time, sick Worm 0.25 Reading 402 David, Oscar, game David, Oscar, baseball game 0.625 Reading 403 carnival, rio de janeiro
carnival, people, show Carnival 0.71 Reading 404 Tina, aid, program, chinese,
students AID, Tina 0.5
Reading 405 Kelly, school, poster contest, Jimmy, contest
Jimmy, Kelly, school library
contest 0.6
Reading 406 Mark, Sally, ride Roller coaster ride, Mark,
Sally 0.63
Reading 407 bacteria, hailstones,
weather, balls, influence bacteria, weather 0.4 Reading 408 Jenny, Brian Jenny, Brian 1.0 Reading 501 O. Henry, stories, Della,
hair, Jim, gifts, watch O. Henry 0.14 Reading 502
tsunami, english girl, tilly, ground, sea water, farmers, guinness garden
Tsunami, preventing tsunami 0.28
Reading 503 christmas, Della, Jim, hair Jim, Della, Christmas present 0.56 Reading 504 mom, whale Gary, mom, Blue whale 0.43 Reading 505 leaves, eucalyptus, koala,
zoos eucalyptus leaves, koala 0.5
1-3 單向語意關係圖的主角詞彙結果
表 21 主角詞彙(單向語意關係圖)
文章 主角詞彙 教師答案 分數值
Reading 101 Sam, Sam 's mom, sam 's
uncle, Sam 's grandma Sam 0.25 Reading 102 Gary 's father Gary’s father, restaurant 0.5
Reading 103 Jimmy Jimmy 1.0
Reading 104 Duke Duke(Jay’s dog) 1.0 Reading 105 baby panda Panda family 0.33 Reading 106 mrs. chen, hair, writer Mrs. Chen 0.33 Reading 107 The author The author (and his family) 1.0 Reading 201 Sam Sam, Sam’s room 0.5 Reading 202 Mandy’s parents, guess Mandy’s birthday 0.29
Reading 203 Jimmy Jimmy 1.0
Reading 301 care School faculty and staff 0.0 Reading 302 Tim, day Tim, housework 0.33 Reading 303 Jane, packing, list Jane, plane outfit 0.25 Reading 304 school, parents, Mr. Hsu,
tribe, travelers The author school, Mr. Hsu 0.24 Reading 401 worms, Mr. Grylls, time Worm 0.33 Reading 402 David, Oscar David, Oscar, baseball game 0.67 Reading 403 carnival, rio de janeiro
carnival, people Carnival 0.33 Reading 404 Tina, aid, program AID, Tina 0.67 Reading 405 Kelly, Jimmy, contest Jimmy, Kelly, school library
contest 0.63
Reading 406 Mark, Sally, ride Roller coaster ride, Mark,
Sally 0.63
Reading 407 bacteria, people, scientist bacteria, weather 0.25 Reading 408 Jenny, Brian Jenny, Brian 1.0
Reading 501
O. Henry, government, O.
Henry 's stories, Jim and Della, christmas eve
O. Henry 0.2
Reading 502
tsunami, damage, tilly, tilly 's family, farmers, coast, guinness garden
Tsunami, preventing tsunami 0.14
Reading 503 Della, Jim, hair Jim, Della, Christmas present 0.5 Reading 504 mom, whales Gary, mom, Blue whale 0.43 Reading 505 leaves, koalas, zoos,
zookeepers eucalyptus leaves, koala 0.5
附錄二 實驗資料(英文文章)
Reading 101
Sam gets his backpack.
Sam gets his shirts.
Sam gets his pants.
Sam gets his socks.
Sam puts on his jacket.
Sam puts on his backpack.
Sam’s mom takes him to the station.
Sam kisses his mom goodbye.
Sam gets on the train.
Sam reads on the train.
Sam gets off the train.
Sam’s uncle waits for him.
Sam’s uncle drives him home.
Sam’s grandma gives Sam a big hug.
Reading 102
Gary’s father is a cook. He is fifty-four years old. He is tall, thin, and handsome.
He doesn’t wear glasses. He runs a Japanese restaurant in Taipei. It opens six days a week, from Tuesday to Sunday. Gary’s mother, Susan, and the other cook, Mr. Chen, also work in the restaurant. They work together from 11:30 a.m. to 09:30 p.m. Many people eat in the restaurant during lunch break. Families and friends also get together to enjoy dinner. It is a popular restaurant in the neighborhood.
Reading 103
Hi, my name is Jimmy. I am twelve years old. I am from England. My father has a new job in Taiwan, so we move here. I have a sister. She is 7 years old. She studies in this school, too. We walk to school together. I am good at sports, music and math. I like the fruit and food in Taiwan. But I don’t like the summer weather here. It’s too hot.
Reading 104
(Ted and Jay are friends. They are in the park now.) Ted: Hey, Jay. What’s that in your bag?
Jay: Oh, it’s my dog.
Ted: Really? Can I see it?
Jay: Sure.
Ted: Wow! It’s so cute! What’s its name?
Jay: His name is Duke. He is my 10-year-old birthday gift.
Ted: How old is he now?
Jay: He is two years old now.
Ted: Can he do any tricks?
Jay: Of course. He can sit. He can kiss. He can jump. And he can stand on a ball!
Ted: That’s great!
Jay: Thank you. Do you want to play with Duke?
Ted: Yes. But, I’m going home now. Maybe Sundy.
Jay: Okay. See you!
Reading 105
Today is Monday. The weather is sunny. The Panda family is going hiking in the mountains this afternoon. Before that, Mama Panda and Baby Panda are going shopping for food in the supermarket.
Baby Panda likes chocolate milk very much. She also likes chocolate cookies.
Mama Panda buys some of those for her. Baby Panda also picks up some apples for her Papa and Mama. Baby Panda is excited and cannot wait to go!
Reading 106
My homeroom teacher, Mrs. Chen, is a very good teacher. She is not tall but looks fit. Her hair is long and black. Her eyes are brown and she doesn’t wear glasses. She likes T-shirts and jeans. Mrs. Chen likes to read English stories to us in class. We usually read funny stories together. Sometimes we play games in class. Sometimes we sing songs. We play ball games in the afternoon, too. I like Mrs. Chen. I like her classes. We all like her.
Reading 107
It’s six o’clock now. My mother is cooking in the kitchen. My father is working in his office. My brother is taking a shower. My grandma is watching TV in the living room.
My grandpa is reading a book and drinking tea. They are all busy. What am I doing now?
Ha! Ha! I’m playing my computer games in my bedroom!
Reading 201
This is Sam’s room. His bed is at the corner. His desk is on the other side. A box of crayons and a pencil case are on the desk. There is a chair in front of the desk. Under the chair is a toy box. All Sam’s toy robots and teddy bears are there. There is a bedside table between his bed and his desk. Sam puts his favorite storybooks on the bedside table.
He likes to read them every night.
Reading 202
Mandy’s birthday is coming. Her parents are shopping for her birthday party. They are at a bakery. They want to buy a cake. They also want to buy some balloons and birthday hats. They need some candy and chocolate in a candy store, too. Now they are at a gift shop. Guess what gift Mandy will get for her 12th birthday.
Reading 203
Hi, my name is Jimmy. I am twelve years old. I am from England. My father has a new job in Taiwan, so my family comes here. I have a sister. She is seven years old.
She goes to this school, too. We get to school by bike. I love PE, music and math. I like the fruit and food here. But I don’t like the summer in Taiwan. It’s too hot.
Reading 301
In school, different people take care of us for our different needs. They are our school caretakers. Principals take care of everyone and everything in schools. Teachers take care of our learning and our feelings in class. Janitors take care of deliveries and supplies for offices and our classrooms. Security guards take care of our school safety.
It’s good to have caretakers around us!
Reading 302
Tim and his classmates were going to celebrate their elementary school graduation with a trip to Dream Mall. He needed money for this plan. He asked his mother for more allowance, but she told him,” If you need more money, you have to earn it on your own.
You can help me with the housework. I will pay you for doing chores.” Tim didn’t like doing housework, but he would do it for his graduation trip.
The next day, Tim’s mom gave him a “Housework Reward Chart.“ She said,
“You‘ll get chore stickers for your help. Every week, I’ll count up your stickers. Each sticker will be worth ten dollars.” Tim was excited because this chart made doing house chores fun. He set the table and fed pets every day. He also collected garbage, collected dirty laundry, and hung out laundry twice a week. He also made breakfast for his family on the weekend. However, he didn’t like to mop floors and clean windows. He only did them once in the week. Tim looked at his chart for the first week! Hmmm… Not bad.
He earned more than 350 dollars on his own.
Reading 303
Jane is a secretary in a big company. She travels from country to country on business. Sometime she visits countries of different weather in a short time. Therefore, wise packing is very important for her.
Jane has a packing list for her travelling. She puts all things on the list into a carry-on bag and calls them “a plane outfit.” This plane outfit means everything necessary for her plane ride. Look at Jane’s outfit list below. Maybe you can use this list for your own travel too.
Reading 304
My school is a very small school in southern Taiwan. It takes two and a half hour to drive from the city to my school. My school is surrounded by beautiful scenery; fresh air, clear sky and special wild flowers. But we don’t have many students. Many parents leave our tribe for work, and their children leave with them, too.
Several years ago, Mr. Hsu came here. He was our new principle. He loved our school and the students. But the school was poor. We, as students, knew little about the world outside our tribe. Then he came up with an idea. He made our school become a travelers’ school. He invited people in our tribe to help paint and clean our extra classrooms. These classrooms became travelers’ bedrooms, restaurants, restrooms and teaching center. If travelers could give students a one-day teaching about their own skills or experience, then they could get free meals and free rooms to stay. “Through their stories and experience, travelers become our students’ window to the world, ” said Mr.
Hsu.
My school is still small, but more and more travelers visit my school. Some told us amazing stories happening in the world; some taught us knowledge about science. One visited our school and then stays for good in our tribe. If you would like to enjoy traveling for free, come here and teach us.
Reading 401
According to the great English explorer Bear Grylls, it is safe to eat worms if you really need to! But, if you really need to eat worms, it will be better if you cook them.
Why? Because the worms move and stay under the ground all day long! The things in their bodies are not clean. According to Mr. Grylls, when he ate worms for the first time, he felt sick and almost couldn’t breathe! However, after trying for several times, he said the taste was fine. In the end, he finally could eat them without feeling sick.
Reading 402
David and Oscar are talking online. (Like MSN)
David: Hey, did you watch the baseball game last night?
Oscar: Yeah, I did. So sad.
David: I agree. I thought we could have won the game.
Oscar: Me too. You know, when the 7th inning was over, we still led by 2 runs in the game. I felt we would definitely win, so I went to take a shower. When I came back, the Japanese team had tied the game 3-3. I was so surprised!
David: I know how you felt. I had the same feelings.
Oscar: Then, I decided not to watch the game because I was tired of watching our team losing.
David: Well, it’s just a game, not the end of the world!
Oscar: You’re right. Maybe I need to learn to deal with my feelings.
David: We’re playing again tomorrow evening. Do you want to watch it with me? We can cheer for our team!
Oscar: Sounds like a great idea!
Reading 403
Carnival is the most important holiday in Brazil. It falls on a different date every year, about 40 days before Easter. Rio de Janeiro Carnival is the funniest and the most famous. It is also called the Greatest Show on Earth. You will be surprised at the number of people for this event. In 2011, about 4.9 million people celebrated Carnival in this city!
Carnival starts with parades. People dance, play music and have fun. You can see people put on amazing masks everywhere. Some mask design is so special that visitors would buy them back and treat them as works of art. People also wear beautiful costumes
Carnival starts with parades. People dance, play music and have fun. You can see people put on amazing masks everywhere. Some mask design is so special that visitors would buy them back and treat them as works of art. People also wear beautiful costumes