E-learning
Chun-Chia Wang1,*, Jason C. Hung1, Timothy K. Shih2, and Hsiau Wen Lin2 1
Department of Information Management
Technology and Science Institute of Northern Taiwan Beitou, Taipei, Taiwan 112
gcwang@mail.tsint.edu.tw
2
Department of Computer Science and Information Engineering
Tamkang University Taipei, Taiwan 251 tshih@cs.tku.edu.tw
Received 18 June 2006; Revised 1 August 2006 ; Accepted 5 September 2006
Abstract. With the rapid development of E-learning, collaborative learning is an important kind of group learning mode. It emphasizes the communication and information sharing among group members. Current collaborative learning systems haven’t reached a high interaction degree between the students and the instructor to gain knowledge. In order to solve this problem, this paper introduces a semantic-based automated question answering (Q&A) system with a learning object repository like a virtual teacher to respond to student questions online. Through the proposed mechanism, the Q&A knowledge base will be enriched in the future process of Q&A. Furthermore, not only the students can get answers to their questions, but also the instructors could know what problems students encountered in learning. This would be a great help for both the instructor and the students in collaborative learning environment.
Keywords: Automated Question Answering System, Natural Language Processing, Collaborative Learning, Word Sense Disambiguation, Repository.
1 Introduction
The concept of collaborative learning has been around for a long time. Collaboration on computer systems is totally different from that in face-to-face environment. It stresses the importance of shared dialogue and inquiry and means students working together to accomplish shared learning goals and to maximize their own and their group members’ achievements. In [7], authors indicate that students learn better when they learn together and foster creative thinking as members in a group generated new ideas, strategies, and solutions more frequently than working individually.
Indeed, the effectiveness of collaborative learning on the World Wide Web has been identified by various studies. It is found that students’ levels of involvement and incentive to learn have increased significantly with a wider and more complete understanding of the subject knowledge [10,11,15]. Knowledge is a product out of interaction. Interaction among learners is fostered as communication over the Internet is simple and convenient when addressing to a single user or multiple users. However, interaction between students and an instructor addresses a problem the instructor cannot be online all the time and it is not possible for the instructor to deal with lots of questions proposed from students in a timely manner. In the communication of collaborative learning, students are encouraged to ask questions. Therefore, there is a need to describe an automated Q&A system to support learning efficiency of collaborative learning.
Started in early 90’s, the Advanced Distributed Learning (ADL) initiative [1,2,3] proposed the SCORM specification. ADL divides an e-Learning environment into three components according to
*
e-learning product functionality. These components are Authoring Tools, Learning Management Systems (LMSs) and Repositories [4]. In the specification, the Sharable Content Object Reference Model (SCORM)[3] is composed of three parts: a Content Aggregation Model to ensure the reusability of learning objects and to pack the learning objects, a Run-Time Environment to deliver contents and to support learning activity, and a Sequence Specification to provide an adaptive learning mechanism. SCORM covers features of Authoring Tools and LMSs. However, the definitions of repository are popular and not unique. Various platforms, schemas, and frameworks [12,13,17] increase the difficulty of interoperability in repository systems, while an indirect drawback of low interoperability is that it reduces the sharability and reusability of learning objects.
In [5] we have developed an online Q&A system for distance learning which adopts the traditional word-based textual retrieval technology. Textual retrieval systems that utilize automatic indexing techniques to create text representatives from natural language for better performance must deal with the problems of polysemy and synonymy. Polysemy, a single word form having more than one meaning, decreases retrieval precision by false matches. While synonymy, multiple words having the same meaning, decreases the recall by missing true conceptual matches. Also, in [6,20], we tried to overcome these problems by indexing textual information by its underlying concepts (using word sense disambiguation technique) rather than the keywords (word forms).
According to our previous works, we examine how to apply online Q&A based on a learning object repository to facilitate a progress of the collaborative learning in this paper. We demonstrate how a Q&A system empowers knowledge gaining, accumulating and sharing between students, instructor and knowledge base in a collaborative learning environment.
The rest of this paper is organized as follows. Section 2 presents a survey of the related work on collaborative learning. In Section 3, we demonstrate how our proposed Q&A system facilitates the collaborative learning. In Section 4, we present the architecture of interactive collaborative learning environment based on automated Q&A system. Several interfaces and the implementation of the metadata wizard that enable the interactions between users and the system are designed in Section 5. Finally, a summary of this paper and several conclusions are enumerated in Section 6.
2 Related Work
Collaborative Learning promotes a type of group learning mode. Several researchers agree that students perform better through group learning than by learning alone [7,14]. In this learning mode, students who are interested in sharing their knowledge form a learning group. Students may learn through the assistance of other group members. Group members communicate experience and viewpoint, discuss all kinds of questions, help each other, and teach each other, etc. Therefore, learning is both a group activity and a social process and thus learning performance is strongly affected by peers [9].
In the development of networks, the learning eliminates the obstacles of time and space. Students can take part in collaborative learning by computer at anyplace, at the same time or different time--synchronous and asynchronous, respectively. Researchers have used activity theory to analyze Computer Supported Collaborative Work (CSCW) systems [8]. Group communication relationship [18] refers to the intra-group relationships determined by the interactions among members. However, how to form a group is a problem in collaborative learning. [16,19] give us two methods to form a learning group. But, in these methods, new learners cannot participate in the learning group after the group has been formed and teachers must all be online. This paper brings forward a new Q&A system to form a group model, in which learners can attach their questions to the group when they want to collaborate with others. The method also considers that learners can communication through Q&A interaction such as discussion forum to provide support for construction and access of information sharing.
ADL has established a Content Aggregation Model based on nine categories of the IEEE 1484.12.1-2002 Learning Object Metadata (LOM) standard. Typically learning objects are separated into different Content Model Components, namely Asset, SCO, Activity, Content Organization, and Content Aggregation. Each SCORM Content Model Component has corresponding requirements for each of metadata element as shown in Table 1. Therefore, the total number of metadata items is 77 in each type.
Table 1. Distributions of metadata items CMC Element type Content Aggregation Content Organization / Activity / SCO Asset
Parent 19 19 19
Mandatory 0 11 8
Optional 58 47 50
Total Number 77
Table 1 reveals that the number of mandatory elements of the Content Aggregation is zero, implying that all metadata elements in this layer are optional. In other words, a learning object without any metadata is also acceptable. Therefore, the learning object creator will not fill in the complete metadata actively and it results in repositories are difficult to search and to be found. This decreases reusability.
Parent elements in the table act as “containers” for other elements. That is, parent elements have no values associated with them. Hence, there are 58 elements need to be filled. Among the Mandatory elements, 1.1.2 Entry and 3.1.2 Entry (see SCORM 2004 specification) [3] are reserved elements with values assigned by a Handle System [6]. After removing the parent items, the optional items, and the reserved elements from the total, each SCO and Asset should at least have nine and six elements, respectively. However, the efficient reusability of learning object and accurate retrieval ability of repository depends on the completeness of the metadata (i.e., 58 elements). As shown in Fig. 1, the more complete the metadata, the higher it’s filling-in cost, but the lower the overall searching costs.
Filling in the 58 elements is merely a time-consuming task for metadata specialists. And even badly, it is a challenge for creators who are not familiar with the metadata definitions. The learning curve of metadata will never be an easy task for general users such as elementary school teachers. This is the main cause of low reusability of learning objects. Therefore, this investigation attempts to generate metadata automatically for each learning object by constructing a Metadata Wizard. The approach reduces the total cost by lowering the cost of filling in metadata. The dotted lines in Fig. 2 denote the fall in Total Costs and Overall Filling-In Metadata Costs. Additionally, ensuring that each learning object has complete metadata helps for creating a practical repository environment.
Fig. 1. Trade off between Searching Costs and Filling in Metadata Cost
Cost Total Costs Overall Filling-In Metadata Costs Overall Searching Costs Metadata Completeness
3 Question Answering Mechanism in Collaborative Learning
Fig. 3. Knowledge gaining, accumulating and sharing cycle formed by the interactions between students, instructor and the system ( repository knowledge base).
An automated Q&A system in collaborative learning operates based on the question answering knowledge base. When a student has an information need, he can raise a question through a designed interface. When a new asked question enters the system, its representation (query) is created. This representation is then compared with the representations (index) of the Q&A sets in the repository knowledge base. A rank is given to any match between the new question and any existing Q&A sets. The matched answers are then presented to the student who asks the question.
If no any answer is provided by the system to a new question, nor the student is satisfied with any of the answers, it means that no proper match with respect to the question is found from the knowledge base. In the circumstance, the question is then presented to the instructor demanding an answer for it. After the instructor manually answers the question, not only the answer is sent to the student, but also a new Q&A set is formed and entered into the question answering knowledge base.
Besides raising questions when meeting some difficulties, even when having no difficulties, a student can still browse the knowledge base to see what problems other students have encountered in learning - now and in the past and see the answers or solutions the instructor offered.
The whole process of question answering in the collaborative learning environment is as shown in Fig. 3. Through student raising questions and instructor providing the answers, the knowledge in the
Fig. 2. The decrement of Total Costs when Metadata Wizard applied
Overall Filling-In Metadata Costs Total Costs Metadata Completeness Overall Searching Costs Cost
database continues accumulating. This accumulated knowledge will then be shared with other students through raising questions or browsing the repository knowledge base. Accordingly, the interaction between the students, instructor and the system (repository knowledge base), the whole process is actually a cycle of knowledge gaining, knowledge accumulating and knowledge sharing. That is what we need for collaborative learning.
4 An Architecture of the Repository-Based Question Answering System
The architecture of the semantic based online automated question answering system is shown in Fig. 4. There are six main components in the system, including the student assistant agent, the Q&A acquirer, the question answering repository knowledge base, the lexical parser, the semantic index module and the answer generator.
Fig. 4. Architecture of the Semantic-based Automated Question Answering
The functions of the components are briefly described as follows.
Student Assistant Agent: The main function of this component is to be the interface between the student and the Q&A system. It provides student an interface to send his questions and receive answers. Based on this agent, student can also provide relevance feedback to the instructor asking for manually answering the question if all the answers returned by the system are not satisfied. Q&A Acquirer: This component temporarily stores unanswered questions from the Student
Assistant Agent, and then presents them to the instructor with an interface for the instructor to manually answer the questions when he gets online. After a question is replied by the instructor, not only the student will be notified, but also a new Q&A set will be formed and saved to the question answering knowledge base. We have this component for a reason that student’s questions might be unanswered by the system due to no suitable or desirable answers for them in the knowledge base. Under this circumstance, the instructor needs to answer the questions by himself; at the same time, the knowledge base would grow through this procedure.
Question Answering Repository Knowledge Base (Metadata Wizard): This is the repository of all Q&A sets and their metadata. A Q&A set is a pair of question with its corresponding answer. It can be collected on a question-by-question basis through the Q&A acquirer, or from a batch file arranged by the instructor. Through the process of student asking questions and instructor
answering question, the knowledge will be accumulated and the knowledge base will be enriched gradually. Not only the system will have more ability to answer the various questions from students, but also the instructor and the students can know what problems students have encountered when they are learning. Table 2 lists the schema of the Q&A set records in the repository knowledge base.
Table 2. Database schema of Q&A set records.
Columns
Q&A_ID asker_ID asked_Date replier_ID replied_Date
keywords semantic_Index viewed_Times category
Besides, the framework of the metadata elements are split into two categories User-Defined and System auto-Filled. The creator sets the values of User-Defined elements depend on subjectivity of creator. System auto-Filled are generated by the Metadata Wizard. Fig. 5 depicts the framework of the metadata wizard which has two kernels, the metadata wizard and the deduction engine. These kernels are described in detail as follows.
The Metadata Wizard obtains information directly from the Environment Agent and the User Profile. The Metadata Wizard can obtain the values of environment and platform dependent elements via the Environment Agent. Considering platform-dependent variables, the Metadata Wizard maps the system default language to 1.3 Language, and the system time zone and current date to 2.3.3 Date. The values of environment-dependent variables, such as 4.1 Format, 4.2 Size and 4.3 Location can be obtained from learning object files. Information about users is stored in User Profiles in the Learning Content Management System , and is provided at login time to assist the Metadata Wizard map to elements with the Vcard data type, such as 2.3.1 Role, 3.2.2 Entity and 8.1 Entity.
If the information cannot be provided from the system, then Metadata Wizard fills in the elements based on system deduced. In other words, the Metadata Wizard obtains the deduced information from the Deduction Engine. The Deduction Engine infers data from three sources, the Question Agent, Mandatory Element and Content Organization.
Deduced from Question Agent:
The Question Agent has some intuitive questions to create interactive dialogue with users. One question might be, “Who is the target learner of this course?” The answer is clearly related to the learner’s context. Therefore, the Metadata Wizard can fill the value in 5.6 Context, 5.7 Typical Age Range according to the results from the Deduction Agent. Deduced from Mandatory Element:
As seen in Table 1, both SCO and Asset have some mandatory elements that must be fed with some values. The Deduction Engine can analyze and process these values when they are obtained from the Metadata Editor of the Authoring Tool. For instance, 1.4 Description is a mandatory element. After obtaining this value from the Metadata Editor, Deduction Engine processes those value using techniques such as natural language analysis, then extracts the
Fig. 5. The Metadata Wizard Framework
Learning Content Management System Authoring Tool Metadata Editor User Profile Mandatory Element Deduction Engine Metadata Wizard Environment Agent Content Organization Question Agent
useful data to fill in 1.5 Keyword. Deduced from Content Organization:
A Content Organization is a map representing the content through structured units of instruction. As mentioned above, each Content Model Component describes itself with its metadata. The Deduction Engine, after obtaining metadata of low-level units of instruction (such as Asset), can then map or operate to corresponding higher-level component of instruction (such as Content Aggregation). The easiest of structural relations deduction approach is to aggregate the lower level units to their higher level units. Moreover, structural relations deduction can be operated by mathematical operations as well as aggregation. Considering 4.2 Size as an example, if an Activity has three Assets, then each Asset Size can be summed and the value assigned to Activity 4.2 Size.
Every rule in the Deduction Engine is either a Copy Function or If-Then Function. Notably, If-Then Functions can only be applied to valid sets of tokens in the SCORM Metadata Standard.
Additionally, to provide a more elastic and powerful deduction engine, the Deduction rules can be designed by professionals and customized to individual needs. That is, the Deduction Engine permits the users to alter all deduction rules.
Lexical Parser: This component accepts sentences from user question or from Q&A sets in the knowledge base, recognizes compound words and phrases, and tags part-of-speech (POS) to them. These words combined with their POS would be used by semantic index module to form semantic index. For further details, please refer to our previous work [5].
Q&A Browsing Component: Besides raising questions when meeting some difficulties, even when having no difficulties, a student can still, through this component, browse the knowledge base to see what problems other students have encountered during learning - now and in the past, and see the answers or solutions the instructor offered. We have designed three kinds of browsing functions: “top viewed”, “recently added” and “browsing by category”.
Semantic Index Module: This module takes the output (text with their POS from an asked question or Q&A set) of lexical parser, extracts significant keywords as the arguments to tag the senses (by carrying out WSD) to these words. Our algorithm for WSD is described in [5]. In Wordnet each synset represents a single distinct sense or concept, and is assigned a unique identification, called senset-id. After we disambiguate words in a text, we will get the corresponding concepts/senses defined in Wordnet for the keywords and we use their senset-ids as the index for that text. The text here is a user question or Q&A sets, but any document written in natural language can be dealt in this way as well.
In order to extract significant keywords from texts, we utilize a log-entropy weighting schema to assign minimum weight to terms equally distributed over documents and maximum weight to those terms concentrated in a few documents. Entropy takes into account the distribution of terms over documents. We eliminate the lower half weighted keywords, and left the upper half (which can be adjusted adapting to condition at that time). The equation 1 is log-entropy formula.
∑
=•
−
=
n j j PjP
T
Entropy
1 2log
1
)
(
··· (1),where T is a term in the collection, Pj is “the frequency of term T in a Q&A set j” divided by “the total number of times term T occurs in the whole collection.”
Answer Generator: This component computes the match rank between the semantic index of a new incoming question and the semantic index of the Q&A sets in the Q&A database. We use the simple Boolean match to retrieve semantically related Q&A sets to the user question, that is, when a user question and a Q&A set have common sense id, then this Q&A set would be judged as semantically related to the user question and then be retrieved. After retrieving all the related Q&A sets, we rank them (give each of these Q&A set a score) by equation 2. For a pair of Boolean matched question q and Q&A set qai, the rank for them is
i i qa C M qa q rank( , )= × ………..……(2),
where M is the total count of instance of any concept (senset id) matches, C is the count of unique concept match, | qai | is the length (number of all words) of the Q&A set qai.
For example, if a user question q is indexed with Wordnet senset id’s as {2, 5, 30}, and there is a matched Q&A set qai, which is indexed as {2, 78, 3, 2, 50, 61, 13, 5, 2}, and the length
(number of words) of qai is 25. then the rank of this Q&A set to the question is
i i qa C M qa q rank( , )= × = 25 2 ) 1 3 ( + ×
, as there are three instances of match of the senset-id “2”,
one instance of match of senset-id “5”, and there are two distinct senset-id’s, “2” and “5” matched.
The answer generator then produces the ranked final answer list and sends it to the question assistant agent.
5 The Implementation of the Repository-Based Q&A System
5.1 User Interactions
To let students and instructor be able to interact with the system, several interfaces have been designed. For the Student Assistant Agent, the interfaces for the student to ask questions and get answers are developed as shown in figure 6. Under the interface in Fig. 6, student can key in an English question in natural language and get answers immediately without waiting for the instructor to get online. A list of answers provided by the answer generator is displayed to the student, the answers are ordered according to their rank. If no satisfactory answer is found, then student can press a button to send the question to the Q&A acquirer, which will then present the answered questions to the instructor demanding for a manual answer whenever he is online.
Besides raising questions when meeting some difficulties, even when having no difficulties, through the interfaces in Fig. 7, a student can still browse the knowledge base to see what problems other students have encountered in learning - now and in the past, and see the answers or solutions the instructor offered. Three kinds of browsing activities are designed – “top viewed Q&A”, “recently added Q&A” and “browsing Q&A by category”. All these provide particular functions to the students.
Fig. 7. Three kinds of the browsing activities
Fig. 8. Instructor interfaces – browse and answer unanswered questions; and manually provide a new Q&A set to the repository knowledge base.
For the Q&A acquirer component, several interfaces for the instructor are also designed. Through the interfaces in Fig. 8, the instructor can browse the unanswered questions stored in the database. The interface shows a list of unanswered student questions to the instructor. The instructor can choose any question to respond with an answer. When the instructor clicks on a question, he can then key in his answer to the question. Not only the student who asked the question will be notified with the answer, but also a new Q&A set will be automatically formed and stored into the question answering repository knowledge base. Day by day, this process is enriching the repository knowledge base.
Besides, we also design another interface to the Q&A acquirer component as shown in Fig. 8 (manually add a new Q&A set). If the instructor himself finds some Q&A worth collecting but has not been asked by any student, through this interface he can manually input the question and its answer to the repository knowledge base, and a new Q&A set will also be formed through this way.
5.2 The implementation of the Metadata Wizard
In the implement, each component in the Metadata Wizard Framework is a component program in DLL form. To perform the implementation, the Metadata Wizard was integrated into the Hard SCORM Authoring tool [21]. The development environment is under Microsoft Visual Studio.NET 2003 with C#.
A creator designs a new course, which may inserts some existing learning objects from the resource pool. The bottom-left corner of Fig. 9 displays a resource pool. Resources with metadata are revealed. On the top-left side of Fig. 9, the window illustrates how to arrange a course structure. The creator, after dragging the learning objects from the resource pool to the content aggregation tree, can fill out all metadata elements for each learning object with the Metadata Editor, or to allow the Metadata Wizard to help. Fig. 10 depicts the relevant Metadata Wizard selections in the “Metadata” drop-down menu.
Fig. 10. The “Metadata” Drop Down Menu
The “User Profile” and “Questions” functions allow users to modify their personal data and their answers to improve the accuracy of deduction. The “Options” function enables the user to disable particular deduction rules. The left window of Fig. 11 illustrates the Question Agent interface, which allows users to design their own interactive questions and corresponding deduction rules. All rules in the Deduction Engine, including the Copy and If-Then Functions, can be modified in the right side window. After design the deduction rules, they can be saved as a rule template for sharing. In addition, the “Metadata” drop-down menu has “Generate” and “Completed generation” functions. The “Generate” function generates individual learning object metadata, and the “Completed generation” function generates all learning objects metadata under the content aggregation. Finally, the “CA metadata editor” can be applied to display and edit the generated Content Aggregation Metadata. After a course is created, the “Completed generation” function is embedded in the “Save” function to regenerate and double-check all metadata.
Table 2 lists the “System Provided” element, which represents information obtained directly from the Environment Agent and User Profile. “System Deduced” means information extracted form the Deduction Engine. After reserved and parent elements are subtracted from the total number of elements, 54 elements have to be filled out. The automatic generation rate is at least 61% when the Metadata Wizard Framework is implemented. The result indicates that the Metadata Wizard decreases the creator loading in absolute terms.
Table 2. Automatic generation of Content Model Component (CMC)
CMC Catalog
Content Aggregation
Content Organization /
Activity / SCO Asset
Reserved 4 4 4
System Provided 15 15 20
System Deduced 20 20 13
Generation Rate 65% 65% 61%
6 Conclusions
In this research, a system for online automatically answering students’ questions in the collaborative learning environment has been designed. The system operated upon the repository-based question answering knowledge base including the metadata wizard. In the knowledge base, pairs of question with its corresponding answer (Q&A sets) were collected through the process of students raising questions and the instructor providing answers to them.
It was very important to have such a system in collaborative learning environment. It benefited both the instructor and the student – the instructor can release from the heavy load of answering student’s questions and the students can look for the answers to their questions without the constraint of time and space. In some way, the proposed system acted like an advanced frequently-asked-questions (FAQ) machine, which not only automatically answered the questions based on its knowledge base, but through this process also enriched its own knowledge.
References
[1] ADL Technical Team, Content Object Repository Discovery and Registration/Resolution Architecture, ADL 1’st International Plugfest, June 2004.
[2] ADL Technical Team, DOD Learning Repositories Use Cases Version 1.0, October 2003.
[3] ADL Technical Team, Sharable Content Object Reference Model (SCORM) 2004 Documentation 1st Edition, Advanced Distributed Learning (ADL), January 30, 2004.
[4] Authoring Tools Application Guidelines - Practical Examples and SCORM Implementation Suggestions Version 1.0, Advanced Distributed Learning (ADL), July 2003.
[5] J. C. Hung, C.-S. Wang, C.-Y. Yang, M.-S. Chiu, and G. Yee, “Applying Word Sense Disambiguation to Question Answering System for e-Learning”, in Proceedings of the IEEE 19th International Conference on Advanced Information Networking and Applications (AINA2005), Tamkang University, Taipei, Taiwan, March 2005, pp. 157-162.
[6] H.-H. Hsu, C.-Y. Yang, and T. K. Shih, “An Automated Question Answering System for e-Learning”, in Proceedings of the International Workshop on Distance Education Technologies (DET'2003), Miami, Florida, USA, September 2003.
[7] Johnson, D.W. and Johnson R.T., Cooperation and competition: Theory and research, Edina. MN: Interaction Book Company, 1999.
[8] K. Kuutti, “The concept of activity as a basic unit of analysis for CSCW research,” in Proceedings of the Second European Conference on Computer-Supported Co-operative Work: EC-CSCW’91 (eds. L.J. Bannon, M. Robinson & K. Schmidt) Kluwer, Dordrecht, 1991, pp. 249-264.
[9] J. Lave and E. Wenger, Situated Learning: Legitimate Peripheral Participation, Cambridge University Press, Cambridge, 1991.
[10] Y. H. Lee and N. S. Chen, “Group composition methods for cooperative learning in web-based instructional systems,” in Proceedings of the 8th International Conference on Computers in Education/International Conference on Computer-Assisted Instruction, 2000, pp. 1535-1538.
[11] M. Nagai, Y. Okabe, J. Nagata, and K. Akahori, “A study on the effectiveness of web-based collaborative learning system on school mathematics: Through a practice of three junior high schools,” in Proceedings of the 8th International Conference on Computers in Education/International Conference on Computer-Assisted Instruction, 2000, pp. 279-283.
[12] N. Pasini, D. Rehak, “A Process Model for Applying Standards in Content Development,” in Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications, June 2003, pp.542-544. [13] S. Hasegawa, A. Kashihara and J. Toyoca, “An e-Learning Library on the Web,” in Proceeding of the
ICCE02, 2002, pp.1281-1282.
[14] R. Slavin, “Research on cooperative learning and achievement: what we know, what we need to know,” Contemporary Educational Pshchology, Vol.21 No.1, 1996, pp. 43-69.
[15] J. Su, W. Chen, F. Chen, and Y. Tsai, “The project-based collaborative learning on Internet—A case study on Geology Education,” in Proceedings of the 8th International Conference on Computers in Education/International Conference on Computer-Assisted Instruction, 2000, pp. 303-308.
[16] S. Thepchai, I. Akiko, I. Mitsuru, J. Toyoda, and M. Riichiro, “Learning Goal Ontology Supported by Learning Theories for Opportunistic Group Formation,” in Proceedings of the 9th International Conference on Artificial Intelligence in Education (AI-ED 99), 1999.
[17] N. Vassiliadis, F. Kokoras, I. Vlahavas, and D. Sampson, “An Intelligent Educational Metadata Repository,” Intelligent Systems: Technology and Applications, Vol. 4, Databases and Learning Systems, CRC Press 2003. [18] P. Watzlawick, Pragmatics of Human Communications: A Study of Interactional Patterns. Pathologies and
Paradoxes, W.W. Norton, New York, 1967.
[19] W. Martin and P. Hans-Rudiger, “Group Formation in Computer-Supported Collaborative Learning,” in Proceedings of the 2001 International ACM SIGGROUP Conference on Supporting Group Work, September 2001.
[20] C.-Y. Yang, M.-S. Chiu, C.-H. Yang, and T. K. Shih, “A Semantic-based Automated Question Answering System for e-Learning,” in Proceedings of the Tenth International Conference on Distributed Multimedia Systems (DMS'2004), San Francisco, September 2004.
[21] T. K. Shih, et. al., “The Hard SCORM LMS: Reading SCORM Courseware on Hardcopy Textbooks,” in Proceedings of the 5th IEEE International Conference on Advanced Learning Technologies, Taiwan, July 2005.
