Wiki-based rapid prototyping for teaching-material design in e-Learning grids

(1)

Wiki-based rapid prototyping for teaching-material

design in e-Learning grids

Wen-Chung Shih

a

, Shian-Shyong Tseng

a,b,*

, Chao-Tung Yang

c

a

Department of Computer Science, National Chiao Tung University, Hsinchu 30010, Taiwan, ROC b

Department of Information Science and Applications, Asia University, Taichung 41354, Taiwan, ROC c

High Performance Computing Laboratory, Department of Computer Science and Information Engineering, Tunghai University, Taichung 40704, Taiwan, ROC

Received 25 March 2007; received in revised form 11 October 2007; accepted 21 October 2007

Abstract

Grid computing environments with abundant resources can support innovative e-Learning applications, and are prom-ising platforms for e-Learning. To support individualized and adaptive learning, teachers are encouraged to develop var-ious teaching materials according to different requirements. However, traditional methodologies for designing teaching materials are time-consuming. To speed up the development process of teaching materials, our idea is to use a rapid pro-totyping approach which is based on automatic draft generation and Wiki-based revision. This paper presents the approach named WARP (Wiki-based Authoring by Rapid Prototyping), which is composed of five phases: (1) requirement verification, (2) query expansion, (3) teaching-material retrieval, (4) draft generation and (5) Wiki-based revision. A pro-totype system was implemented in grid environments. The evaluation was conducted using a two-group t-test design. Experimental results indicate that teaching materials can be rapidly generated with the proposed approach.

Keywords: Teaching material design; Wiki-based authoring; Rapid prototyping; e-Learning grid

1. Introduction

During the past decade, information technologies have advanced at an amazing pace. Web 2.0, character-ized by the techniques of blog, Wiki, RSS, mashup, etc., has been widely discussed and referred to as the sec-ond generation of web-based services (O’Reilly, 2005). Another representative is Grid computing (Foster, 2002; Foster & Kesselman, 1997), which supports resource-sharing and can overcome the limitations in tra-ditional platforms. Therefore, grid computing technologies provide possibilities for supporting innovative applications, such as e-Learning. In fact, more and more eﬀort has gone into the ﬁeld of e-Learning grid, using

* _{Corresponding author.}

E-mail addresses: [email protected] (W.-C. Shih), [email protected], [email protected] (S.-S. Tseng), ctyang@ thu.edu.tw (C.-T. Yang).

Computers & Education 51 (2008) 1037–1057

(2)

grid technologies in the context of e-Learning. Among these, ELeGI (European Learning Grid Infrastructure, 2004–2008) is the most representative project (Gaeta, Ritrovato, & Salerno, 2003).

With the trend of individualized and adaptive learning, there will be a great demand to various teaching-materials. A typical approach to content design is ADDIE (2004), which consists of five stages: Analysis, Design, Develop, Implement, and Evaluate. The primary disadvantage is its time-consuming development process. In addition, it requires expensive human resources. Furthermore, redundant efforts could happen when different sites develop teaching materials for the same course units simultaneously. To solve the problem, a new method is needed for teachers to rapidly develop their own course materials.

Our idea is to design teaching materials by a rapid prototyping approach based on automatic draft gener-ation and Wiki-based revision. Rapid prototyping is the process of quickly building and evaluating a series of prototypes of a system, which has been widely applied to manufacturing, software engineering, etc. (Luqi, 1989). First, a draft is automatically generated by combining relevant teaching materials in e-Learning grid. However, the main challenges result from verification of user intention and finding useful teaching materials from existing ones. To address this issue, we plan to enhance searching performance by using expertise acquired by a powerful knowledge acquisition tool to speed up the development process. Next, we adopt a Wiki-based authoring environment to revise the automatically generated draft. Wiki is an accessible markup language for people to edit a site together (Louridas, 2006). Wikipedia is the most successful Wiki-based pro-ject (Wikipedia, 2004). Our method is to utilize the collaborative intelligence and labor to accelerate the revi-sion process. The primary difficulties are the storage requirements and overheads of maintaining historical revisions. This issue is alleviated in grid computing environments because of its abundant resources of storage and computation.

Based on the aforementioned ideas, we propose an approach named WARP (Wiki-based Authoring by Rapid Prototyping), which is composed of ﬁve phases: (1) requirement veriﬁcation, (2) query expansion, (3) teaching-material retrieval, (4) draft generation and (5) Wiki-based revision. The goal is to reduce the devel-opment time of teaching materials. Firstly, the system attempts to clarify users’ intention by interactive ways, such as asking questions, requesting more query terms, etc. In the second phase, users’ queries are expanded by using domain expertise to retrieve more relevant documents. Next, the system searches for existing teach-ing-materials related to the expanded query in the grid. Then, the retrieved documents are combined into a draft automatically in the fourth phase. Finally, the draft is placed in a Wiki-based authoring environment for collaborative revision.

The advantages of WARP are twofold: time-saving and low-cost, which result from eﬀective sharing and reusing of resources. Meanwhile, our primary contribution is the idea of a rapid prototyping approach to teaching-material design for e-Learning grids. In addition, we deployed a prototype system in a grid environ-ment, implementing each phase of the WARP approach. Twenty four randomly selected teachers from ele-mentary schools participated in an experiment based on a two-group t-test design. Experimental results show that teachers in the experiment group can generate high-quality teaching materials more rapidly than those in the control group.

The rest of this paper is organized as follows. In Section2, we review background knowledge and related work on this research. Then, the problem and the proposed approach are presented in Section3. Next, imple-mentation and experimental results are discussed in Section4. Finally, the concluding remarks are given in Section5.

2. Preliminaries and related work

This section brieﬂy introduces the preliminaries of e-Learning grids, Wiki and rapid prototyping, which are essential to this work. Next, previous researches related to this paper are also described.

2.1. e-Learning grids

Grid computing systems (Foster, 2002; Foster & Kesselman, 1997) are transparent resource-sharing infra-structures, which can overcome the limitations in traditional e-Learning platforms, such as scalability, inter-operability, availability, etc. Grid middleware plays an important role in grid infrastructures, which provides

(3)

upper applications with transparent resources. Many middleware platforms have been developed, such as Glo-bus Toolkits (Foster & Kesselman, 1997) and Condor (Litzkow, Livny, & Mutka, 1988), etc. Currently, the trend of grid technology is toward service-oriented grid architecture, which represents the convergence of grid computing and Web services. The distinguished speciﬁcations include OGSA/OGSI (Foster, Kesselman, & Nick, 2002) and its successor, WSRF (Joseph, Emest, & Fellenstein, 2004).

Conventionally, e-Learning platforms were developed independently. Therefore, learning objects and functions are platform-dependent, and the collaboration between diﬀerent systems becomes diﬃcult. The proposal of learning standards, such as SCORM, has improved the sharing of learning content. However, e-Learning still meets many challenges which address sharing and interoperability of learning resources. Grid computing is an appropriate solution to a resource-sharing e-Learning infrastructure. Also, grid com-puting technologies provide possibilities for supporting innovative applications of e-Learning. For exam-ple, a medical college can provide students with three-dimensional simulation of human body anatomy using high performance grid computing systems, which is beyond the ability of traditional e-Learning platforms.

More and more eﬀort has gone into the ﬁeld of Learning grid, using grid technologies in the context of e-Learning (Li, Yang, Jiang, & Shi, 2006). There have been researches on e-Learning grid, such as the works by LeGE-WG (Dimitrakos, Randal, & Ritrovato, 2002). Among these, ELeGI (European Learning Grid Infra-structure, 2004–2008) is the most representative project with respect to e-Learning Grid (Gaeta et al., 2003). Its goal is to address and advance current e-Learning solutions through use of geographically distributed resources as a single e-Learning environment.

2.2. Wiki technology

The term ‘‘Wiki’’ originates from the Hawaiian ‘‘wee kee wee kee,’’ which means ‘‘quickly.’’ In the domain of computer science, a Wiki is a web-based hypertext system which supports community-oriented authoring, in order to rapidly and collaboratively build the content. The concept of Wiki was proposed by Ward Cunn-ingham in 1995 as the Portland Pattern Repository, to create an environment for co-workers to share speci-ﬁcations and documents for software design.

Wiki is not the ﬁrst technology for collaboration. Other collaborative technologies, such as discussion boards, have also been widely used for years. Nevertheless, the primary reason why Wiki is so attractive can be attributed to the successful application (Wikipedia, 2004). Traditionally, an encyclopedia is built by a num-ber of experts with a tremendous amount of time and money. However, Wikipedia is an innovative project which endeavors to build an online open-source encyclopedia based on Wiki and GNU Free Document License (http://www.gnu.org/licenses/#FDL). This system began in 2001, and the number of English items exceeds 500,000 in 2005. The rapid growth of the Wikipedia system shows that the concept of the Wiki is both viable and feasible. In addition, there are many related projects based on Wiki, such as Meta-Wiki, Wiktionary, Wiki-books, Wikiquotes, to name a few (http://meta.wikimedia.org/wiki/-Complete_list_of_Wikimedia_projects).

The attractive characteristics of Wiki, which favor its use, can be summarized as follows.

Rapidness. The Wiki pages can be rapidly constructed, accessed and modiﬁed, in hypertext form.

Simpleness. A simple markup scheme (usually a simpliﬁed version of HTML) is used to format the Wiki pages, instead of the complicated HTML.

Convenience. Links to other pages, external sites, and images can be conveniently established by keywords. Moreover, the targets of the keywords, links, need not exist when the links are built. They can be appended later.

Open source. Each member can create, modify and delete the Wiki pages at will. Wiki content is not reviewed by anyone before publication, and is updated upon being saved.

Maintainability. Wiki maintains a version database, which records its historical revision and content, thus enabling version management.

To run a Wiki-based site, it is necessary to deploy a Wiki platform. The requirements of a Wiki platform include editing, links, version management, sandboxes (test-bed), and search functions. Many Wiki platforms

(4)

have been developed and used in various ﬁelds. For example, MediaWiki (http://www.mediawiki.org/wiki/ MediaWik), which is used by Wikipedia, is a widely used tool. PBwiki (http://pbwiki.com/), which is devel-oped by PHP languages, is adopted by many libraries.

2.3. Rapid prototyping

System development is a continuous and fundamental task in many domains, so effective and efficient approaches to system development are demanded by developers. Traditional system development life cycle paradigms, such as the waterfall model, focus on verifying the system requirements during the early stages, and then go through the whole process to generate the perfect product. However, this approach is not suitable when the requirement cannot be clearly defined at the beginning. Therefore, evolutionary approaches, such as rapid prototyping, are proposed to alleviate the limitations.

There have not been a clear definition of rapid prototyping (RP) though this technology has been success-fully used in many fields, including commercial, military and academic applications (Gordon & Bieman, 1995). Generally speaking, RP is recognized as technologies which rapidly realize the conceptual model of a final product of system without incurring too much cost. The purpose of RP is to incrementally clarify the require-ment and refine the prototype. The techniques for rapid prototyping could date back to the late 1980s and were mainly used in manufacturing. Nowadays, RP is applied for many other domains, even in educational applications. For example, RP has been used to create the lecture contents of the IT SoC certificate program (Kim & Park, 2007).

Many RP methods have been proposed in the literature. Although these procedures are not exactly the same, they conform to the following workﬂow:

1. initial definition of requirements; 2. rapid implementation of a prototype; 3. user evaluation and requirement refinement; 4. implementation of refined requirements; 5. repeat step 3 and step 4 until completion.

An early model simply relates traditional design steps to prototypes, consisting of need assessment and analysis, prototype building, prototype utilization, system installation and maintenance (Tripp & Bichelmeyer, 1990). Another example, used to design computer-based courseware, is a three-stage model: analysis, develop-ment and evaluation (Yang, Moore, & Burton, 1995).

RP has also been applied to instructional design for both generating high quality product and reducing development time. Jones and Richey reported eight RP applications in instructional design (Jones & Richey, 2000), including educational software design, instructional videos, etc. The aforementioned researches illus-trate that RP is an eﬀective approach to system development in educational applications. In the case of adap-tive e-Learning, educators are usually encouraged to rapidly develop various personalized teaching materials. The challenges result from varied requirements and timely pressure, which are similar to the motivation of RP. In this work, we adopt the concept of RP to design teaching materials, facilitating the rapid authoring process. 2.4. Related work

Due to the advances in information technologies and the requirements of courseware, more and more teachers are able and willing to design their own teaching materials and make them accessible on the Web (Iorio, Feliziani, Mirri, Salomoni, & Vitali, 2006; Lanzilotti, Ardito, Costabile, & Angeli, 2006; Sierra, Fern-a´ndez-Valmayor, Guinea, & Hernanz, 2006). In addition, a growing number of large-scale projects aim to con-struct learning content repositories (Kassahun, Beulens, & Hartog, 2006; Kiu & Lee, 2006). For example, in 2002, the National Science Council of Taiwan approved a resolution on the ‘‘National Science and Technol-ogy Program for e-Learning,’’ planning to spend $120 million within a 5-year period (ELNP, 2002). These educational contents are mainly based on Sharable Content Object Reference Model (SCORM) (SCORM, 2004), which has become a popular standard for creating sharable and reusable teaching materials for

(5)

e-Learning. With the popularization of e-Learning, how to ﬁnd and reuse these existing materials becomes an important issue.

Teaching materials are one of the important elements in instruction and learning activities. A large amount of work has been devoted to the methodology of teaching-material design. Some scholars (Nkambou, Fras-son, & Gauthier, 1998) presented an authoring environment for the development of course materials. How-ever, this approach depends on educational experts to participate in the development process. In order to facilitate the reuse of SCORM learning objects and customization of course materials, a system named Teach-ing-Material Design Center (Wang & Hsu, 2006) was proposed, reusing e-material from diﬀerent providers and integrating them for a particular course. Nevertheless, this system still relies on human experts to design a satisfactory teaching material. The recent works ofCoﬀey (2007)andWang (2007)addressed the issues of courseware maintenance and enhancement. The former designed a meta-cognitive tool for courseware devel-opment and reuse, and the latter presented a course material enhancement process.

Brieﬂy speaking, aforementioned approaches to designing teaching materials are time-consuming. Also, expensive human-resource costs are involved.

3. Approach

To speed up the development process of teaching materials, our idea is to use a rapid prototyping approach which is based on automatic draft generation and Wiki-based revision. In this section, the teaching material design problem is presented ﬁrst. Then, the proposed approach and its components are described.

3.1. Problem description

Typically, grid infrastructure is built with a suite of middleware. Common middleware platforms, such as Globus Toolkits (Foster & Kesselman, 1997) and Condor (Condor, 2004), are based on a master-slave para-digm. Hence, we represent the grid by a master-slave model. Also, real-time information is included to model the dynamic grid. A grid is a star graph G =ÆNG, EGæ that consists of a finite set of node NG, and a finite set of edges EG. NGrepresents the set of sites in the grid. One node P02 NGis specified as the master node, and other nodes are slave nodes. Each edge in EGconnects the master node and a slave node.

In the SCORM standard, a Content Package (CP) is deﬁned as a package of learning materials, and a Learning Object Repository (LOR) is a database where the Content Packages are stored. A Content Package is modeled as a k-ary tree with three levels: root, chapter, and section, where the section nodes contain the content and the internal nodes represent the structural information, as shown inFig. 1. The content is repre-sented by a vector. In addition, a CP is associated with a set of Metadata. To enable content-based retrieval, the well-known Vector Space Model is applied to represent the text content. Also, the LOM metadata is included in this model of CP. Hereafter, teaching materials, SCORM-compliant documents, and Content Packages are used interchangeably. The feature vector of a CP is represented by a term-weighting vector, vCP.

vCP¼ hw1; w2; . . . ; wjV ji

(6)

where V means the set of vocabulary and |V| is its size. The term-weighting wiis evaluated using the extended Vector Space Model proposed by Trotman (2005). The idea of this weighting scheme is to emphasize the importance of some structure. For example, the same word appearing in Abstract and the last Chapter of a book has diﬀerent signiﬁcance.

A query Q is modeled as a set of keywords. The feature vector of a query Q is denoted by vQ, vQ¼ hq1; q2; . . . ; qjV ji

where V means the set of vocabulary and |V| is its size. The term-weighting qiis 1 if the ith keyword in the vocabulary, V, is a term in the query. Otherwise, qiis 0.

We will now deﬁne the notion of similarity between a query and a content package, which means the rel-evance of the content package to the query. Let Q be a query with feature vector vQ, and CP be a content package with feature vector vCP. The Similarity sim(Q, CP) is deﬁned by:

simðQ; CPÞ ¼ vQ vCP

where the operation is inner product of vectors.

The editor, who submits the query to develop a teaching material, is usually a teacher who is not necessarily an expert in courseware design. Domain taxonomy and a thesaurus are assumed to be available for the mate-rial development process. Also, the designer can reuse any existing teaching matemate-rials in LORs of the e-Learn-ing grid.

We assume that the course ontology, built by educational experts, is available, as shown inFig. 2. The sub-ject matter is mathematics for nine-year coherence curriculum at low-grade elementary-school level, according to Ministry of Education in Taiwan. The course ontology is modeled as a rooted tree O =ÆNO, EO,rootæ that consists of a ﬁnite set of node NO, a ﬁnite set of edges EO, and a root node in NO. NOrepresents the set of nodes in the tree. Each node in NOrepresents a concept in this ontology and is associated with a set of key-words, which describe the concept. For example, the set of keywords of the node ‘‘Arithmetic Operations’’ is {‘‘addition’’, ‘‘subtraction’’, ‘‘multiplication’’, ‘‘division’’}. An edge in EOconnects a node and its child node, which expresses the hierarchical relation of the two nodes. For instance, the edge connecting ‘‘Geometry’’ and ‘‘Shapes’’ means the former is a more general concept than the latter. Finally, the root node in this example is the ‘‘Mathematics’’ node.

Based on the deﬁnitions above, the Teaching-Material Designing Problem (TMDP) can be described as fol-lows. For a query given by a teaching-material editor, design a teaching-material, where the designer can inter-actively consult the editor to elicit the meaning of the query; the existing materials in m LORs can be reused; also, a course ontology is available. The objective is to minimize the total development time.

3.2. WARP

To help teachers rapidly develop course materials, we propose the WARP (Wiki-based Authoring by Rapid Prototyping) method, based on the idea of a rapid prototyping approach. The key aspects of WARP include: reuse, automation and collaborative authoring. We use domain expertise to search for useful teaching

(7)

rials in learning object repositories and reuse them. By means of knowledge acquisition tools and automated merging algorithm developed by ourselves, the process can be automated and sped up. Finally, we use a Wiki-based authoring environment to exploit collaborative intelligence for revision.

As shown inFig. 3, the WARP approach is composed of ﬁve phases.

Phase 1: requirement verification. The purpose of this phase is to clarify users’ information need specified by query terms. The system verifies the scope of the query in the domain ontology by asking questions to the user interactively.

Phase 2: query expansion.This phase aims to increase the searching performance by expanding the query. A rule-based method is utilized to represent the searching heuristics and to infer appropriate searching strategies.

Phase 3: teaching-material retrieval. Relevant teaching materials stored in the e-Learning grid are retrieved in this phase. A global index, built in a bottom-up way, is used to speed up the access process.

Phase 4: draft generation. The ﬁrst version of the draft is automatically generated by merging the teaching materials found in the previous phase.

Phase 5: Wiki-based revision. We use a Wiki-based authoring tool to facilitate collaborative revision for the draft.

In the following paragraphs, these phases are described in detail, respectively.

Phase 1 (Requirement verification). This phase aims to clarify users’ information need speciﬁed by query terms. Because the initial queries submitted by novice editors are usually vague and ambiguous, documents found by the searching engine might not answer to their expectations. Lee and Liu modeled query intention with a goal-driven approach (Lee & Liu, 2005). In spite of its powerful functionality, it is not suitable for our rapid prototyping approach in terms of runtime overhead. Therefore, we design a lightweight method to clarify users’ intention. To begin with, a single-concept query is deﬁned as follows:

Deﬁnition. Single-concept queries. A query is said single-concept with respect to an ontology if its terms appear in the set of keywords of only one node in the ontology.

The following example is presented to illustrate this deﬁnition.

Example 1 (Single-concept queries). Let a query Q be {‘‘addition’’, ‘‘multiplication’’}. As shown inFig. 4, the keyword set of the node ‘‘Arithmetic Operations’’, KAO, is {‘‘addition’’, ‘‘subtraction’’, ‘‘multiplication’’, ‘‘divi-sion’’}. Q is a subset of KAO, and its terms do not appear in other ontology nodes. Therefore, Q is a single-concept query with respect to the ontology.

This method uses the ontology of courses to identify users’ intentions. By means of ﬁnding the ontology nodes whose keywords include the query terms, the system can validate the search scope of the query. Users’ requirements are veriﬁed through dialogues between users and computers. The algorithm is presented as follows.

(8)

Algorithm 1 (Requirement verification algorithm (QVAlg)). Symbols deﬁnition:

Q: a set of keywords; the query submitted by a user

O: a treeÆN, Eæ, where N is the set of nodes and E is the set of edges; representing an ontology L: the depth of the ontology tree O

Root: the root node of the ontology tree O

S: an ontology node in N, representing the scope of the query, returned by the algorithm Input: Q, O

Output: S

Step 1: count :¼ 0

Step 2: for each node I in the ontology O

If (Q and I have some keywords in common) Then S :¼ I

count :¼ count + 1 ENDIf

Step 3: If (count6¼ 1) Then Ptr :¼ Root

For i :¼ 1 to L

Ask the user to choose a node from the children of Ptr let Ptr point to the node chosen by the user

S :¼ Ptr ENDIf Step 4: return S

The purpose of Step 2 is to check whether the query is a single-concept one with respect to the ontology. When a query involving multiple concepts is detected or query terms do not belong to the ontology keyword set, the inter-active interview process is activated, as shown in Step 3. In the past, interinter-active interviewing is an eﬀective way to acquisition what users think in knowledge engineering domain. The system uses a top-down traversal on the ontology to ask questions. Requirements are veriﬁed through dialogues between users and computers. For exam-ple, the following dialog, trigged by Step 3, can help the system clarify the subject matters of users’ intentions. Example 2 (Interactive interview).

Q: ‘‘Please select a topic:’’(1) Number (2) Geometry (3) Algebra (4) Statistics A: 1

(9)

Q: ‘‘Please select a sub-topic:’’(1) Counting (2) Addition (3) Subtraction (4) Multiplication (5) Division A: 4

According to users’ answers, the system can confirm that the user want to find documents related to ‘‘Mul-tiplication’’ in the ‘‘Number’’ topic. Therefore, the system returns a modified query containing predefined key-words in the topic.

Phase 2 (Query expansion). Searching strategies used by experts and novices are different. If we can acquire the domain expertise of searching experts, the system can assist novices to search like an expert. In this work, we focus on the strategies of query expansion. The main reason is that the query expansion technique can be well integrated with existing information retrieval technologies, which are primarily based on the Vector-Space Model (Salton & McGill, 1983).

To acquire searching heuristics of experts, we use the Repertory-Grid method (Kelly, 1955), which has been widely applied to knowledge acquisition. After the analysis of the Repertory-Grid, rules representing the knowledge of searching experts can be generated. The purpose of these rules is to choose an appropriate strat-egy according to the input values of attributes. The inference process can be illustrated inFig. 5.

In this work, four strategies of query expansion are adopted. These operations are guided by the given ontology. When the information need is focused on some node in the ontology, the four operations can be used to reformulate the query, described as follows.

Generalization: append keywords of the parent node in the ontology. Specialization: append keywords of the children nodes in the ontology. Expansion: append other keywords of the same nodes in the ontology. Shrink: remove terms of the query.

The choice of a suitable strategy depends on the values of several attributes, which could include: The number of query terms.

The number of the previous searching results. The level of the targeted students.

The following example shows a rule representing one of the expert heuristics. Example 3 (Knowledge of query expansion).

IF (Number_Query_Term < 2) THEN Expansion

This rule means that the ‘‘Expansion’’ strategy is triggered when the number of query terms is less than two. Therefore, The system selects keywords from the current node in the ontology to expand the original query. The number of selected terms depends on the system conﬁguration.

(10)

The following example illustrates the diﬀerence between a query and its expanded version. Example 4 (Query Expansion).

Q={‘‘multiplication’’}

The original query contains only one term. Assume that the current node is ‘‘Arithmetic Operations’’, as shown inFig. 2. Based on the rule inExample 3, the ‘‘Expansion’’ action is triggered. Then, the system selects another two keywords in the current ontology-node, addition and subtraction. After expansion, the query contains three terms.

Q0_{={‘‘Multiplication’’, ‘‘addition’’, ‘‘subtraction’’}}

Query expansion and search are conducted in Phase 2 and Phase 3 of the proposed WARP method, respec-tively. Therefore, the output of Phase 2, query expansion, is the input of Phase 3, search. That is, the original query is transformed into the expanded query in Phase 2, and is then processed in Phase 3 to search the repositories.

The query expansion process is independent of subsequent search technologies and engines. For example, the custom search engines from Google or Yahoo can work with the proposed WARP method. However, these custom search engines are keyword-based. They do not take the structural information and metadata of standardized learning content.

Phase 3 (Teaching-material retrieval). In this phase, the expanded query is used to search for relevant teaching materials in the grid. Learning object repositories are located at different sites, which are connected with wide area networks. Our method is to reorganize the documents in each learning object repositories into a local index. Next, all local indexes are merged into a global index. The global index is then utilized to ﬁnd the relevant documents. The primary advantage of building a global index is its suitability for centralized management and implementation on e-Learning grids, thus increasing the searching performance.

The searching algorithm is shown below.

Algorithm 2 (Teaching-material searching algorithm (TMSAlg)). Symbols deﬁnition:

VQ: the feature vector of the query Q submitted by a user k: the number of documents to be returned

VD: the feature vector of a document D or teaching material D Sim: a similarity function, cosine function

I: the global index of teaching materials

R: the k teaching materials found by the algorithm Input: VQ, k

Output: R

Step 1: For each document D indexed by I Compute Sim(VQ, VD)

Step 2: Rank all documents by the similarity values Step 3: return the ﬁrst k documents as R

Example 5 (Teaching-material searching). Let the feature vector of the query be Æ1, 2, 1æ. Assume that the repositories contain ﬁve documents, and their feature vectors are:

Document 1: Æ1, 2, 1æ. Document 2: Æ3, 2, 1æ.

(11)

Document 3: Æ2, 2, 1æ. Document 4: Æ2, 2, 3æ. Document 5: Æ3, 2, 3æ.

In this example, the similarity function is vector inner product. Then, the similarity values are 6, 8, 7, 9 and 10, respectively. Consequently, the most relevant document is Document 5, which has the highest similarity value.

Phase 4 (Draft generation). The purpose of this phase is to automatically generate the ﬁrst version of the teaching material by merging existing teaching materials found in the previous phase. The reason is based on the observation that it is easier to revise a document than to construct one from scratch.

The algorithm is based on level-wise clustering, and its main concept is union of diﬀerent learning objects from relevant teaching materials. In addition, it retains the most similar learning object as the cluster center. Also, the clustering process is guaranteed to converge. The algorithm is shown as follows.

Algorithm 3 (Teaching-material merging algorithm (TMMAlg)). Symbols deﬁnition:

T1, T2: Content Package trees; the two teaching materials to be merged L: the depth of a Content Package tree

T: a Content Package tree; the merged teaching material Input: T1and T2

Output: T

Step 1: For i :¼ L to 1

1.1: cluster nodes at the same layer

1.2: If$ a cluster with more than one node

Then retain the node with the larger similarity value Step 2: modify the links between layers

Step 3: Return the merged teaching material

Example 6 (Teaching-material merging). We assume that there are two teaching materials to be merged: A and B, as shown inFig. 6. After merging, the node with the highest similarity is retained, say, A0and B10. Phase 5 (Wiki-based revision). In the previous phase, the draft is rapidly generated by computers, but it still needs to be revised by human editors to guarantee its readability. In order to increase the efﬁciency of the con-sequent revision process, we adopt a Wiki-based approach. By means of collaborative efforts, the draft is expected to be rapidly revised. Our belief in this approach comes from the successful example of the Wikipedia project, which aims to compile an encyclopedia by web users.

We have designed a Wiki-based authoring environment to enable collaborative revision. Our strategy is to use available middleware to ease the implementation. The architecture, as shown inFig. 7, includes four main components:

(12)

Web-based interface: a user-friendly interface for users to access the system. Wiki-HTML converter: conversion between Wiki and HTML formats.

SCORM-HTML converter: conversion between SCORM and HTML formats. Grid Middleware: universal access to remote resources in the grid.

3.3. An illustrative example

An example is presented to illustrate how teachers of an elementary school use the WARP method to col-laboratively design a teaching material for the ‘‘Area’’ unit in the third-grade Mathematics course. The ontol-ogy of the ‘‘Shape’’ unit is shown inFig. 8. The keyword ‘‘Area’’ is associated with the ‘‘Shape’’ node. The overall process is summarized as follows.

Phase 1: requirement veriﬁcation.

The teachers express their requirement by specifying the keyword ‘‘area’’ and metadata ‘‘grade’’ (its value = 3). The system veriﬁes this query is a single-concept query, and then sends the original query to the next phase.

(13)

Phase 2: query expansion.

In order to increase the precision of searching, the teachers deﬁne the strategy of query expansion as ‘‘Spe-cialization.’’ After inference, the system recommends another three keywords to reﬁne the original query: ‘‘Triangle,’’ ‘‘Rectangle,’’ and ‘‘Circle.’’ The teachers adopt the ‘‘Triangle’’ as an expanded keyword. Conse-quently, the expanded query, ‘‘area and triangle,’’ is sent to the next phase for searching.

Phase 3: teaching-material retrieval.

According to the expanded query and the speciﬁed metadata, three teaching materials are found in the repositories, as shown inFig. 9.

Fig. 8. The ontology of the ‘‘Shape’’ unit.

(14)

The top two relevant teaching materials are retrieved for draft generation in the next phase. The out-lines of the two teaching materials are shown inFig. 10. The ﬁrst teaching material, with the name ‘‘Inter-esting Areas,’’ consists of ﬁve lessons. The second one, with the name ‘‘Areas and Triangles,’’ has six lessons.

Phase 4: draft generation.

The ﬁrst version of the draft is automatically generated by merging the teaching materials found in the pre-vious phase. In this phase, redundant modules are removed. For example, both teaching materials have a les-son about ‘‘square centimeters.’’ The two lesles-sons are clustered into one group, and one of them is removed from the draft. Similarly, lesson 2 of the second teaching material is removed after the clustering process. The resultant draft consists of nine lessons. The outline of the draft is shown inFig. 11.

Phase 5: Wiki-based revision.

The teachers use a Wiki-based authoring tool to facilitate collaborative revision for the draft. Through the Talk page, the revision work is coordinated. In this phase, inappropriate content is modiﬁed, and the presen-tation of content is adjusted. Finally, the teaching material is composed of six lessons, organized into two modules, as shown inFig. 12.

Fig. 11. The outline of the draft. Fig. 10. The outlines of the two teaching materials.

(15)

4. Experimental results

In this section, the implementation and evaluation design are described. Then, experimental results are pre-sented and discussed.

4.1. Implementation and design of evaluation

In order to evaluate the proposed approach, we implemented the aforementioned algorithms, and built a prototype for Wiki-based authoring. The Wiki-based authoring interface is shown inFig. 13.

The prototype is built on a grid test-bed, which is composed of four domains, as shown inFig. 14. The middleware is Globus Toolkit 4.0. To elicit the expertise of searching experts, we use the DRAMA tool (Lin, Tseng, & Tsai, 2003), which is a suite of toolkits for knowledge engineering developed by KDE Lab. of NCTU. We use this tool for rapid acquisition of searching rules.

We apply the WARP approach to an elementary school mathematics Course. Participants are 24 teachers from three elementary schools in Nantou, Taiwan. The course is mathematics for the third grade. The existing teaching materials are retrieved from repositories built by Ministry of Education, Taiwan.

The existing teaching materials are mainly retrieved from two learning content repositories built by Ministry of Education, Taiwan. One is named ‘‘A Service Station for Learning’’ (http://content.edu.tw), built in 2000. The other is named ‘‘The Six Great Learning Networks,’’ (http://learning.edu.tw), built in 2004. This site consists of six subjects: life education, health medicine, nature ecology, history and culture, humanities and arts, science and education. The content is featured by colorful and interesting presentation.

4.2. Experiment 1: evaluation of WARP

The objective of this evaluation is to answer the question: is the teaching-material development time using WARP signiﬁcantly shorter than one using a traditional approach?

(16)

Fig. 13. Interface of a Wiki-based authoring environment.

(17)

(1) Experimental design.

A two-group t-test was employed. It is a widely used method to test whether the difference between two means is significant. It can measure the difference of two groups.

(2) Tools.

The participants are provided with an internet-enabled environment. That is, they can access informa-tion and content available on the web.

(3) Sample.

Twenty-four teachers from three primary schools in Nantou, Taiwan, are selected as participants. They are randomly divided into two groups, each with twelve teachers. One is named the experimental group, and the other is named the control group.

(4) Hypothesis.

A null hypothesis was set up, which is that no signiﬁcant diﬀerence exists between the development times of the two groups.

(5) Treatment.

The experimental group was provided with the WARP environment while the control one was not. That is, teachers of the control group can only search for existing teaching materials and revise them manually. Furthermore, teachers of the experimental group formed a Wiki community, and participated in Wiki-based revision.

(6) Results.

The development times of the control group and the experimental group are illustrated in Fig. 15. ‘‘WARP-v1’’ means the development time of the first version by the WARP approach. Similarly, ‘‘WARP-v2’’ and ‘‘WARP-v3’’ represent the second and the third version respectively. The t-test gives the probability that the difference between the two means is caused by chance. The difference between sample means of the control group and ‘‘WARP-v3’’ is 42.3. Since the t-ratio is significant at 0.05 and above, the null hypothesis can be rejected. This evaluation showed that the development time of the experimental group is significantly shorter than the control group.

A total of 45 items about feedback from teachers were collected during the Wiki-based authoring process by examining postings on Talk pages, and were classiﬁed along the following four dimensions:

Postings for comments. 26 postings are related to coordination of editing activities. For example, ‘‘I would like to suggest pruning lesson 9. The content of evaluating areas seems too diﬃcult for the third-grade stu-dents’’ (Talk page for lesson 9, as shown in Fig. 16).

Replies to comments. 14 postings are responses to comments of others. For example, ‘‘I agree with that the calculation of area for a triangle is too hard for this stage’’ (Talk page for the article on lesson 9, as shown inFig. 16).

(18)

Polls. 2 voting sessions were organized by users to decide on controversial editing actions. For example, ‘‘The vote is this: Should the above paragraph be included in the lesson? The three possible answers are: Yes, No and Abstain’’ (Talk page for lesson 3).

Oﬀ-topic remarks. 3 postings are unrelated to the content. For example, ‘‘I will suggest my colleagues to try this interesting tool’’ (Talk page for Area:WARP).

4.3. Experiment 2: evaluation of query expansion

This experiment investigated whether the proposed intelligent query expansion could enhance the perfor-mance of the original query.Fig. 17shows the precision value from the 12 teachers of the experimental group who used the WARP tool to search for relevant teaching materials. The precision values ranged from 0.7 to 1.0 with the WARP and from 0.2 to 0.7 with the original query. The next experiment measured the recall value, as shown inFig. 18. Similarly, the WARP approach could improve the performance of the original query.

4.4. Discussion

The Wiki-based revision for teaching material produces both individual and collective benefits. The indi-vidual who makes a knowledge contribution can see it immediately published, thus observing the contribution outcome without delay and with pride of authorship. This immediacy between action and positive outcome may very well create a positive reinforcement effect for the author. Immediacy of results has social impacts as well. First, any published result is visible and therefore potentially beneficial to others right away. As others

Fig. 16. Screenshot of Wiki’s Talk pages.

(19)

see useful contributions being made, they can use these contributions, as well as build upon them and add their own associated knowledge.

The teaching material produced by the proposed approach has two advantages: variety and innovation. On the one hand, the draft is generated from several relevant teaching materials, which results in its variety of content. On the other hand, the draft is revised by many authors. In this process, diﬀerent ideas are added in the draft, thus resulting in its innovation.

Wikis enable instant publication of content. As soon as an author saves the new content, it becomes imme-diately visible to all readers viewing the page. No coordinator is involved in the publication process. Never-theless, there are safeguards. For instance, Wikis maintain a temporal database of earlier page versions, and roll-back to an earlier version requires only a few clicks. However, from the viewpoint of Wiki designers and administrators, the storage and management for temporal revision are challenges when the Wiki system scales up. Grid platform is a suitable solution to these problems, which can provide resource for the storage and operation of temporal revision.

We discuss the quality of teaching materials produced by WARP in two aspects: content and presentation. First, the quality of the final version heavily depends on the effort of involved authors. The proposed merge algorithm can help to automatically collect relevant learning objects. However, it depends on human authors to refine the draft, such as course sequence, content selection, etc. For example, as shown inFigs. 11 and 12, the draft has evolved from a flat course structure to a two-level hierarchy, which is more organized and under-standable for students. Second, currently available Wiki platforms are mostly text-based, and allow users to upload image files. However, multimedia learning objects can not be easily edited on current Wiki platforms. Therefore, in this work, most of the multimedia learning objects in original teaching materials are skipped

Fig. 18. Comparison of recall.

(20)

because of the limitation of Wiki platforms. Consequently, the ﬁnal version is mainly composed of texts and ﬁgures, as shown inFig. 19.

5. Conclusions and future work

This paper describes a Wiki-based rapid prototyping approach to designing teaching materials for e-Learn-ing grids. It is characterized by a time-save-Learn-ing development process, minimal human involvement, reduce-Learn-ing redundant eﬀort and high-quality teaching materials. The evaluation was carried out using a two-group t-test design. Experimental results indicate that teaching materials can be rapidly generated with the proposed approach. In the near future, we will conduct experiments to evaluate the quality of teaching materials devel-oped by the proposed approach. Also, more participants with diﬀerent background knowledge and teaching experiences will be invited to evaluate the proposed system. We expect that the WARP approach can assist novice as well as experienced teachers to develop useful course materials rapidly and easily. Furthermore, elic-itation of users’ requirements and content management for e-Learning grids will be investigated.

Acknowledgement

This research was partially supported by National Science Council of Republic of China under the number of NSC95-2520-S009-007-MY3 and NSC95-2520-S009-008-MY3.

References

ADDIE (2004). ADDIE design [on-line]. Available fromhttp://ed.isu.edu/addie/design/design.html(retrieved 07.03.07).

Coﬀey, J. W. (2007). A meta-cognitive tool for courseware development, maintenance, and reuse. Computers and Education, 48, 548–566. Condor (2004). Condor project 2006. Available fromhttp://www.cs.wisc.edu/condor/.

Dimitrakos, T., Randal, D. M., & Ritrovato, P. (2002). LeGE-WG: A European network working towards the establishment of a European infrastructure for global technology enhanced learning. Available fromhttp://grid.ucy.ac.cy/axgrids04/AxGrids/papers/ E00-1776736304.pdf.

ELNP (2002). National science and technology program for e-Learning. Available from http://elnp.ncu.edu.tw/ (retrieved 12.03.07).

Foster, I. (2002). The grid: A new infrastructure for 21st century science. Physics Today, 55(2), 42–47.

Foster, I., & Kesselman, C. (1997). Globus: A metacomputing infrastructure toolkit. International Journal of Supercomputer Applications and High Performance Computing, 11(2), 115–128.

Foster, I., Kesselman, C., & Nick, J. M. (2002). The physiology of the grid: An open grid services architecture for distributed systems integration. Available fromhttp://www.globus.org/research/papers/ogsa.pdf.

Gaeta, M., Ritrovato, P., & Salerno, S. (2003). ELeGI: The European learning grid infrastructure. Paper presented at the Proceedings of 3rd international LeGE-WG workshop: GRID infrastructure to support future technology enhanced learning.

Gordon, V. S., & Bieman, J. M. (1995). Rapid prototyping: Lessons learned. IEEE Software, 12(1), 85–95.

Iorio, A. D., Feliziani, A. A., Mirri, S., Salomoni, P., & Vitali, F. (2006). Automatically producing accessible learning objects. Journal of Educational Technology and Society, 9(4), 3–16.

Jones, T. S., & Richey, R. C. (2000). Rapid prototyping methodology in action: A development study. Educational Technology Research and Development, 48(2), 63–80.

Joseph, J., Emest, M., & Fellenstein, C. (2004). Evolution grid computing architecture and grid adoption model. IBM System Journals, 43(4), 624–645.

Kassahun, A., Beulens, A., & Hartog, R. (2006). Providing author-deﬁned state data storage to learning objects. Journal of Educational Technology and Society, 9(2), 19–32.

Kelly, G. A. (1955). The Psychology of Personal Constructs (vol. 1). New York: W.W. Norton.

Kim, E. O., & Park, J. H. (2007). Study on the rapid prototyping methodology of the lecture contents for the IT SoC certiﬁcate program. Paper presented at the IEEE international conference on microelectronic systems education (MSE’07).

Kiu, C.-C., & Lee, C.-S. (2006). Ontology mapping and merging through OntoDNA for learning object reusability. Journal of Educational Technology and Society, 9(3), 27–42.

Lanzilotti, R., Ardito, C., Costabile, M. F., & Angeli, A. D. (2006). eLSE methodology: A systematic approach to the e-learning systems evaluation. Journal of Educational Technology and Society, 9(4), 42–53.

Lee, C. -H. L., & Liu, A. (2005). Modeling the query intention with goals.

Li, Y., Yang, S., Jiang, J., & Shi, M. (2006). Building grid-enabled large-scale collaboration environment in e-Learning grid. Expert Systems with Applications, 31, 4–15.

(21)

Lin, Y. T., Tseng, S. S., & Tsai, C.-F. (2003). Design and implementation of new object-oriented rule base management system. Expert Systems with Applications, 25(3), 369–385.

Litzkow, M., Livny, M., & Mutka, M. (1988). Condor: A hunter of idle workstations. Paper presented at the eighth international conference on distributed computing systems, San Jose, CA.

Louridas, P. (2006). Using wikis in software development. Software, IEEE, 23(2), 88–91. Luqi (1989). Software evolution through rapid prototyping. Computer, 13–25.

Nkambou, R., Frasson, C., & Gauthier, G. (1998). A new approach to ITS-curriculum and course authoring: The authoring environment. Computers and Education, 31(1), 105–130.

O’Reilly, T. (2005). What Is Web 2.0. Available fromhttp://www.oreillynet.com/pub/a/oreilly/tim/news/2005/09/30/what-is-web-20.html (retrieved 07.03.07).

Salton, G., & McGill, M. J. (1983). Introduction to modern information retrieval. New York: McGraw & Hill. SCORM (2004). Sharable content object reference model (SCORM) 2006. Available fromhttp://www.adlnet.org/.

Sierra, J. L., Ferna´ndez-Valmayor, A., Guinea, M., & Hernanz, H. (2006). From research resources to learning objects: Process model and virtualization experiences. Journal of Educational Technology and Society, 9(3), 56–68.

Tripp, S., & Bichelmeyer, B. (1990). Rapid prototyping: An alternative instructional design strategy. Educational Technology Research and Development, 38(1), 31–44.

Trotman, A. (2005). Choosing document structure weights. Information Processing and Management, 41, 243–264.

Wang, H.-C. (2007). Performing a course material enhancement process with asynchronous interactive online system. Computers and Education, 48(4), 567–581.

Wang, H.-C., & Hsu, C.-W. (2006). Teaching-material design center: An ontology-based system for customizing reusable e-materials. Computers and Education, 46(4), 458–470.

Wikipedia (2004). Wikipedia. Available fromhttp://www.wikipedia.org/(retrieved 07.03.07).

Yang, C., Moore, D. M., & Burton, J. K. (1995). Managing courseware production: An instructional design model with a software engineering approach. Educational Technology Research and Development, 43(4), 60–70.