國 立 交 通 大 學
資訊科學與工程研究所
碩
士
論
文
一個輔助選擇社會科學研究方法之改良式知識表格
意圖發現法
A Modified Repertory Grid Based Intention Finding
Approach to Selecting Social Science Research Methods
研 究 生：王惠君
指導教授：曾憲雄 博士
ii
一個輔助選擇社會科學研究方法之改良式知識表格意圖發現法
A Modified Repertory Grid Based Intention Finding Approach to
Selecting Social Science Research Methods
研 究 生：王惠君 Student: Hui-Chun Wang
指導教授：曾憲雄博士 Advisor: Dr. Shian-Shyong Tseng
國 立 交 通 大 學
資 訊 科 學 與 工 程 研 究 所
碩 士 論 文
A Thesis
Submitted to Institute of Computer Science and Engineering College of Computer Science
National Chiao Tung University in partial Fulfillment of the Requirements
for the Degree of Master
In
Computer Science
June 2009
Hsinchu, Taiwan, Republic of China
iii
一個輔助選擇社會科學研究方法之改良式知識表格
意圖發現法
學生: 王惠君 指導教授: 曾憲雄 博士 交通大學資訊科學與工程研究所摘要
社會科學研究者在特定的問題上會運用適合的研究方法來描述及探究了 解社會現象。而且每一種研究方法對應到每一個特性的重要性都不相同，所 以選擇適合的研究方法策略也大不相同。如果初學者運用了一個不適合的研 究方法來探究，這會導致於他的研究結果無效。然而挑選適合的研究方法策 略是需要經驗累積而成的，但這對初學者來說挑選適合的研究方法策略是很 難去了解。因此，對於社會科學研究而言在特定問題上如何去選出適合的研 究方法是一件很重要且很難的步驟。據我們所知，目前有關於輔助挑選適合 的研究方法之研究相當少，所以我們提出了一個改良式知識表格意圖發現法 (Modified Repertory Grid Based Intention Finding Approach, MRGBIFA) 來 找 出初 學 者的 意 圖， 並 且還 提 出智 慧 型社 會 科學 研 究方 法 輔助 系統 (Intelligent Social Science Research Methods Selection Assistant, ISSRMSA)來找出初學者的迷失概念，依據他們不同的迷失概念來給予適當的 輔助。我們最後實驗結果顯示我們的系統輔助初學者來挑選適合的研究方法 是很有幫助的。未來我們將會擴展現有的 ISSRMSA 去整合接下來的研究流程， 輔助初學者去規劃他們的實驗設計，並且去驗證他們的實驗設計的準確性， 最後去導引他們去挑選適合的檢定方法。 關鍵字: 意圖發現, 改良式知識表格, 社會科學研究方法iv
A Modified Repertory Grid Based Intention
Finding Approach to Selecting Social Science
Research Methods
Student: Hui-Chun Wang Advisor: Dr. Shian-Shyong Tseng Department of Computer Science
National Chiao Tung University
Abstract
Social science researchers use an appropriate research method in order to describe, explore and understand social life in a specific problem. The selection strategy may be different because the importance of each feature in various research methods is different. If the apprentices select an inappropriate research method, the research results will be useless. It needs experience to determine which features can dominate the method selection, but it is difficult for the apprentices to understand. Therefore, how to select an appropriate research method from the specific problem is an important and difficult issue in social science research. As far as we know, there is less study about research method selection assistance; so we want to assist the apprentices to find appropriate research method according to their intentions. In this thesis, we propose the Modified Repertory Grid Based Intention Finding Approach (MRGBIFA), which is to acquire apprentices' intentions; and the Intelligent Social Science Research Methods Selection Assistant (ISSRMSA) is to discover apprentices' misconceptions and give proper assistance according to diverse misconceptions. Experimental results show that our ISSRMSA is useful for the apprentices to select research methods. In the near future, the ISSRMSA could be extended to plan apprentices' experiment design, verify the validation of their experiment design, and guide them to choose appropriate Statistical Tests.
Keyword: Intention Finding, Modified Repertory Grid, Social Science
v
致謝
首先誠摯的感謝指導教授：曾憲雄博士。曾教授在這兩年碩士生涯中， 對我的細心教導以及諄諄教誨，不管在學習研究方面抑或是做人處事，時常 與我分享他過去寶貴的經驗，也在我感到迷惘時給予當頭棒喝，真的非常感 謝曾教授給予我的耐心、用心及苦心，讓學生獲益匪淺，特別要在這對教授 獻上十二萬分的感謝。接著要特地感謝洪宗貝教授、楊鎮華教授和袁賢銘教 授，特別撥冗來參加學生的論文口試審查，也給予我非常多的建議及鼓勵， 讓本篇論文更加豐富、更加有價值，非常謝謝您們。 在研究生涯中，也非常感謝林喚宇學長和曲衍旭學長，謝謝您們這段期 間的指導及幫忙，不管在學術研究、做人處事以及生活上都給予很多鼓勵和 建議，在這邊也要深深的獻上我的感激！同時也感謝實驗室的所有學長們， 尤其是翁瑞峰學長和楊哲青學長，這兩年間深深受到您們的照顧及鼓勵，非 常感謝您們。對於兩年同窗的好友：小雅、祖淵、啟珺、士緯、世恒以及專 班的同學們：雅鈴、家銘、碧玲、運璟、冠穎，還有最貼心的怡利和學弟妹 們，也要特別謝謝你們這段期間的互相鼓勵及支持，我才能堅持努力完成這 篇碩士論文，謝謝你們。 最後要謝謝我最愛的家人以及身邊的好朋友們(曉涵、大學七小妞、高中 三人幫)，在我最無助最失落時總有你們陪著我，不管身在何方，真的非常謝 謝你們。 惠君 2009 年 7 月于新竹vi
Table of Content
Page 摘要 ... iii Abstract ... iv 致謝 ... v Table of Content ... viList of Tables... viii
List of Figures ... ix
List of Algorithms ... x
Chapter 1 Introduction ... 1
Chapter 2 Preliminaries ... 4
2.1 Social Science Research Process ... 4
2.2 Traditional Social Science Research Method Learning ... 5
2.3 Knowledge Acquisition ... 6
Chapter 3 Modified Repertory Grid based Intention Finding Approach ... 10
3.1 Knowledge Transformation ... 11
3.2 Intention Finding Process ... 12
Chapter 4 Intelligent Social Science Research Methods Selection Assistant .... 18
4.1 System Architecture ... 18
4.2 Rule-based Guiding Controller ... 20
4.3 Social Science Research Methods Misconceptions Finding ... 25
vii
Chapter 5 Experiments ... 29
5.1 System Implementation ... 29
5.2 Experiment Design and Result ... 30
Chapter 6 Conclusions and Future Works ... 32
viii
List of Tables
Page Table 1: Type of user status facts ... 23 Table 2: Trusty rate of answer status ... 26 Table 3: Results of satisfaction questionnaire survey ... 31
ix
List of Figures
Page
Figure 1: The research process ... 5
Figure 2: An example of repertory grid ... 8
Figure 3 : An example of EMCUD and AOT ... 9
Figure 4: Hierarchical repertory grid of research method ... 12
Figure 5: AOT of research method ... 12
Figure 6: Reverse spiral model for intention finding process ... 13
Figure 7: An overview of intention finding process ... 16
Figure 8: An example of repertory grid and AOT ... 17
Figure 9: An example of intention finding process ... 17
Figure 10: System Architecture of ISSRMSA ... 19
Figure 11: System process of each step in the ISSRMSA ... 21
Figure 12: Partial rule classes ... 24
Figure 13: An example of partial repertory grid ... 28
Figure 14: The user view of the MRGBIFA ... 30
x
List of Algorithms
Page Algorithm 1 MRGBIFA Dialogue Algorithm... 14
1
Chapter 1 Introduction
Social science research including social policy, human geography, political science, social anthropology and education is based on logic and empirical observations, and social science researchers should use an appropriate research method in order to describe, explore and understand social life in a specific problem. A research method consists of the data collection through observation and experimentation, and the hypotheses verification by formulating and testing [2].
The apprentices must be familiar with the specific research method before they verify the inquiry. If the apprentices select an inappropriate research method, the research results will be useless because different research methods are suitable to some limited hypotheses. However, the selection strategy may be different according to operational definitions even under the same hypotheses. Besides, research methods possess a lot of features. The research method selection strategy is complicated because the importance of each feature in various research methods is different. It needs experience to determine which features can dominate the method selection, but it is difficult for the apprentices to understand. Hence, how to select an appropriate research method from the specific problem is an important and difficult issue in social science research.
However, traditional social science research learning focuses on the process learning of research methods. The apprentices are still difficult to select the appropriate research method without enough experience. Moreover, experience
2
rule depends on how apprentices' hypotheses are defined, and it is difficult to transform these rules into standard procedure to instruct the apprentices. In this thesis, we hope to assist the apprentices to select the appropriate research method according to their intentions and let them gain the experience of research method selection from the selecting process. Hence, our idea is to use the interview approach to acquire apprentices' intentions and assist them step by step.
For assistance, we have to elicit the expertise in advance. The expertise of social science research methods selection can be reduced to a classification problem, which finds the significant features in accordance with the comprehension of research methods. Hence, we apply the repertory grid, which was devised by G. A. kelly [3], to represent the knowledge since the repertory grid is an interview technique and can be used as a method to solve the classification problem in the knowledge acquisition. However, our purpose is to acquire apprentices' intentions; we propose the Modified Repertory Grid Based
Intention Finding Approach (MRGBIFA), which is a dialogue mechanism
based on the expertise. The difference between modified repertory grid and original repertory grid is that the former whose attributes are chosen by the apprentices and the goal is to find out apprentices' intentions; the latter whose attributes are given by experts and the goal is to acquire the expertise.
Eventually, our proposed Intelligent Social Science Research Methods
Selection Assistant (ISSRMSA) is to discover apprentices' misconceptions and
give proper assistance according to diverse misconceptions. For revealing the misconception, we construct the partial grid according to three candidate research
3
methods and selected attribute to find out the partial learning status from the apprentices, and then compare the difference between the apprentices' and the expert's grid. Finally, in accordance with various misconceptions we give apprentices the proper help and guiding approach.
The organization of this thesis is as follows. In Chapter 2, we introduce some preliminaries about the techniques used in the system. The MRGBIFA we propose is introduced in Chapter 3. Chapter 4 shows the overall system architecture of the ISSRMSA and describes the system in detail. The implementation and experimentation of ISSRMSA is discussed in Chapter 5. At the end, concluding remarks are given in Chapter 6.
4
Chapter 2 Preliminaries
As far as we know, there is less study about research method selection assistance. Hence, some related background knowledge and techniques we used in the system will be introduced in this chapter.
2.1 Social Science Research Process
In the research process, the apprentices need to have the hypotheses in advance, and then select a research method according to the properties of the hypotheses. Hence, we want to find out the properties of the hypotheses and the misconceptions from the apprentices in the intention finding process, and then assist them to select an appropriate research method. For providing assistance, we have to understand the elements of research and find out the misconceptions from each element. We regard this research process, which is proposed by Earl Babbie [1], as our criteria of intention finding process. Figure 1 shows the entire research process including formulating research questions, operational definition, choice of research method, sampling, measurement, research design, data analysis, and writing the research paper [6][7].
5
Figure 1: The research process
We may divide the research process into three parts: the upper section is the dynamic process, which varies from different research issues; the middle section is the transformation from the dynamic process to the static process, which represents the issues in more detail; the lower section is the static process, which is a standard procedure of research doing. Hence, we observe that research method selection approach is decided by apprentices' ideas and operational definition of hypotheses from the research process. In this thesis, our intention finding process mainly focuses on the middle and lower section of research process.
2.2 Traditional Social Science Research Method
Learning
6
learning by doing. Lecture gives lessons about basic concepts to the apprentices, but the concept may be too abstract to apply in real experiment [1]. Case study is to consult relevant books, such as imitating related research methods to design the whole experiment. However, these imitating learning approaches will lead to that the apprentices can only use known solution to solve every research problem [1]. In learning by doing, the apprentices can be inspired to obtain a deeper knowledge of the subjects they're studying, but experience rule is difficult to transform these rules into standard procedure to instruct the apprentices for the teachers. Generally speaking, the apprentices usually lack experiences for selecting research method. Even though the apprentices determine the method by known features, they may be stumble in selecting an appropriate research method. Because the research methods have a huge number of features, and some implications of the features are difficult to determine by the apprentices. In summary, how to help the apprentices who have little selecting experience to find appropriate research method is an important issue.
2.3 Knowledge Acquisition
For giving proper assistance, we require apprentices' misconception based on the domain expertise. Firstly, we have to acquire the domain expertise. The followings are two knowledge acquisition tools to acquire the expertise.
7
2.3.1 Repertory Grid
The Repertory Grid was devised by George Kelly [3] and is based on his Personal Constructs theory of personality which posits that individuals interpret the world in terms of their own personal set of constructs to distinguish between similar and different elements in the world. It is an interviewing technique for identifying the ways that a person explains his or her experiences.
In a repertory grid, a set of elements is listed across the top of the grid to represent the objects to be classified. Various combinations of three elements are then presented to the expert, and the question “Think of an important trait which make two of the elements different from the third” is asked. The trait given by the expert is listed on the left hand side of the grid, and its opposite is listed on the right hand side. Each pair of a trait and its opposite is called a construct. After the set of constructs is ready, the expert is asked to fill the grid with ratings. A 5-scale rating mechanism is usually used in filling the grid.
Example 1: An example of repertory grid
Take Figure 2 as an example, we elicit the elements, which include Coffee A, Coffee B, Coffee C and Coffee D, from the expert. And then we elicit constructs from the expert. Each time three elements are chosen to ask experts for a construct to distinguish one element from the other two. If the traits Bitter, Cold, Light, Strong and their opposites Sweet, Hot, Dark, Weak are given by the expert. Finally, we let the expert rate each entry of the grid. The values range from 5 to 1, where 5 means that the element is very likely to have the trait of the right hand
8
side and 1 means that the element is very likely to have the trait of the left hand side. If the expert thinks that Coffee B is neither too bitter nor too sweet, it might get a 3 on the Bitter-Sweet line.
Coffee A Coffee B Coffee C Coffee D
Bitter 5 3 2 1 Sweet
Cold 4 4 5 4 Hot
Light 2 4 3 5 Dark
Strong 1 3 2 4 Weak
Figure 2: An example of repertory grid
The repertory grid can represent the expertise, but it cannot explain the embedded meanings whose importance of each feature in various research methods.
2.3.2 EMCUD (Embedded Meaning Capturing under
Uncertainty Deciding)
EMCUD, devised by Hwang and Tseng [8][9], is a new Repertory Grid-based knowledge acquisition methodology to elicit the embedded meanings of knowledge. In EMCUD, the multi-type representation is used instead of pure Boolean values. The Attribute-Ordering Table (AOT), which is used to record the importance of each attribute to each object, is employed to capture the embedded meanings of the resulting grids. The value of each AOT entry may be labeled 'X', 'D' or an integer number. 'X' means that the attribute has no relationship with the
9
object. 'D' means that the attribute dominates the object. An integer number means that the attribute does not dominate the object but is of some degree of importance relative to other attributes.
Example 2: An example of EMCUD and AOT
The construction of EMCUD is similar to original repertory grid, and the construction of AOT is that each EMCUD[Object, attribute] entry will be transformed into a question to ask experts filling out each AOT[Object, attribute] entry. An example of construct an AOT is shown as follows:
EMCUD: If A1 is not equal to any element of {9, 10, 12}, is it possible for Obj1 to
be implied? Expert: No.
The answer means that A1 dominates Obj1, and hence AOT[Obj1, A1] = 'D'. The
whole EMCUD and AOT are shown in Figure 3.
Figure 3 : An example of EMCUD and AOT
EMCUD can be used to construct a rule based expert system which infers the conclusion from the information.
10
Chapter 3 Modified Repertory Grid based
Intention Finding Approach
As mentioned above, how to select an appropriate research method is an important and difficult step, so we want to assist the apprentices to select the appropriate research method according to their intentions. We need to acquire the expertise before providing assistance. However, the expertise of research method selection includes the importance of each feature in various research methods. We use EMCUD to acquire the expertise of social science research methods selection. In order to acquire apprentices' intentions immediately, the Modified Repertory
Grid Based Intention Finding Approach (MRGBIFA) is proposed. MRGBIFA
is a dialogue mechanism that is based on the expertise of research method selection to obtain apprentices' intentions. There are three steps in the intention finding process including System List Candidate Objects, Apprentices Select Distinguishable Attribute and Apprentices Determine Candidate Objects' Attribute. In the first step, three candidate objects and the entire attributes are listed each time. In the second step, the MRGBIFA is used to ask the apprentices to select an attribute, which can judge whether these objects are appropriate for the hypotheses, to acquire apprentices' intentions. In the third step, we have to find out their misconceptions in order to provide assistance. The intention finding process is not discontinued until the apprentices have distinguished the entire objects. Finally, the appropriate research method and explanation are listed to the apprentices. The detailed discussion about the MRGBIFA is described in the
11
following sections.
3.1 Knowledge Transformation
The expertise of research method selection has two parts including: research method selection knowledge, which is the knowledge to understand how to select the appropriate research method from features, and research method feature knowledge, which is the knowledge to understand the value of features. Therefore, in order to acquire the knowledge we apply a hierarchical repertory grid which is composed of two levels: the upper level is research method selection knowledge, and the lower level is research method feature knowledge. Figure 4 represents hierarchical repertory grid of research method. In the left grid of Figure 4, objects indicate research methods, attributes are used to distinguish the research methods, and each attribute to each object produces the corresponding value, which contains different amount according to diverse attributes. Data types of values are divided into three kinds: integer, symbolic type, and computation type, which is symbolic type and can be transformed into integer. But the values of each attribute are not acquirable, so an acquisition process is started and the object part becomes the values of each attribute as shown in the right grid of Figure 4.
12 RG 調查研究 實驗研究 內容分析 觀察研究 … 樣本大小 大樣本 小樣本 中樣本 中樣本 … 研究設計 因果性 探索性 敘述性 探索性 … 對象 個人 個人 其他 個人 … 要做實驗 No Yes No No … … … … … RG 大樣本 中樣本 小樣本 樣本個數 N>100 30≦N≦100 N<30 樣本特性 具母體 代表性 配額樣本 隨機樣本 … … … … RG 因果性 探索性 敘述性 目的 辨認 因果 敘述現象 獲得觀點 關連性 X 因果性 探索性 … … … … …
Figure 4: Hierarchical repertory gridof research method
AOT of research method is shown in Figure 5. Objects and attributes are the same as the left grid of Figure 4. Each attribute to each object produces the corresponding value, which contains only one and is the importance of each feature in various research methods.
AOT 調查研究 實驗研究 內容分析 觀察研究 … 樣本大小 1 X D 1 … 研究設計 D 1 1 2 … 對象 2 1 2 D … 要做實驗 2 D 1 1 … … … … …
Figure 5: AOT of research method
3.2 Intention Finding Process
For assisting the apprentices to select the appropriate research method, their intentions need to be acquired. In order to iteratively obtain their intention, a spiral
13
model [10] is adopted to design the process, which contains 3 steps: System List Candidate Objects, Apprentices Select Distinguishable Attribute and Apprentices Determine Candidate Objects' Attribute as shown in Figure 6. A MRGBIFA (MRGBIFDAlg) is proposed to interact with the apprentices to acquire their intentions based on the spiral model. The original spiral model is a system development method (SDM) used in information technology (IT). These phases iteratively go through in order to eliminate all the problems. The progress of original spiral mode is to finish tasks incrementally to approach the final goal, but our reverse spiral model is gathering more apprentices' intentions and discarding the inappropriate research method to find out the most appropriate one. Every round of the MRGBIFDAlg deals with extracting the appropriate research method. System Lists Candidate Objects Apprentices Select Distinguishable Attribute Apprentices Determine Candidate Objects' Attribute
Figure 6: Reverse spiral model for intention finding process
In the intention finding process, we classify the apprentices' behaviors into three statuses:object misconception, attribute misconception and wrong answer. Object misconception means that the apprentice is not familiar with the research
14
method. Attribute misconception means that the apprentice does not understand the feature of research method. Wrong answer means that the apprentice's value is
not as same as the expert's. The MRGBIFA Dialogue Algorithm is shown in Algorithm 1.
Algorithm 1 MRGBIFA Dialogue Algorithm
Algorithm 1 : MRGBIFA Dialogue Algorithm (MRGBIFDAlg) Symbol Definition :
AtrributeSet: a set of all attributes. IntentionSet: a set of intention.
MethodSet: a set of research methods which have not been discriminated.
LSSet: a set of learning status {"object misconception", "attribute misconception", "wrong answer"}.
IFQ: intention finding question is used to acquire the intention TQ: test question is used to find out the misconception
Input : EMCUD, apprentices' answers Output : The appropriate research method Step1: (Initialization) IntentionSet =ψ Step2: System Lists Candidate Objects
Step2.1: Three research methods are chosen from MethodSet.
Step2.2: Candidate Objects are regarded as the subject, like the sentence
"選擇可以判斷你的假設跟 Candidate Objects 比較相似的特性" and AtrributeSet are regarded as the choices in IFQ.
Step3: Apprentices Select Distinguishable Attribute
Step3.1: The apprentices select an attribute from AtrributeSet in IFQ. Step3.2: Apprentices' answers which include their intentions are fetched. Step3.3: Evaluate apprentices' answers to get LSSet.
Step3.3: If "attribute has misconception" LSSet
Show warning message "要不要換別的特性來分辨".
Step3.4: The answers are added to IntentionSet.
Step4: Apprentices Determine Candidate Objects' Attribute Step4.1: If "object has misconception" LSSet
Decrease the values of selected attribute in TQ.
Step4.2: Candidate Objects are regarded as the subject and the values of
selected attribute is regarded as the choices in TQ.
Step4.3: Apprentices respond the value of selected attribute in TQ, like the
sentence "Candidate Object 的 selected attribute 是什麼特性值". Step4.4: Apprentices' answers which include their LS are fetched.
15
Go to Step 3.
Step5: If |MethodSet| ≧ 3
Go to Step 2. Else
Terminate the process.
Step6: System Finds The Final Research Method
Return MethodSet.
MRGBIFDAlg, an iterative procedure, can be used to find out the appropriate research method according to apprentices' intentions. Once all research methods have been distinguished, this process will be terminated. First, in the process ''System List Candidate Objects'' is used that three candidate research methods are selected, which have not been discriminated, from research method list and the entire attributes are displayed. Then, in the process
''Apprentices Select Distinguishable Attribute'' indicates that the apprentices are asked to select an attribute, which can judge their intention is similar to which object. This step is different to original repertory grid, and the whole attributes are provided in order to acquire their intentions of research method selection. After the intentions are found out, the assistance strategy is produced according to diverse misconceptions of everyone. In the process ''Apprentices Determine Candidate Objects' Attribute'' is used that apprentices' misconceptions are observed. The partial grid is generated according to three candidate research methods and selected attribute, and the apprentices are requested to fill out the grid in order to reveal their learning status. Then the assistant strategy is decided by analyzing their learning status.
Figure 7 gives an overview of intention finding process. Firstly, we have to request experts to apply repertory grid and AOT through EMCUD to present the
16
domain expertise. The MRGBIF Dialogue mechanism is used to generate some questions based on the expertise and the apprentices are asked. The MRGBIF Dialogue mechanism may carry out continuously. Finally, the appropriate research method is found out according to apprentices' answer process.
System List Candidate Objects Apprentices Select Distinguishable Attribute Apprentices Determine Candidate Objects' Attribute
System Find The Final Research
Method Apprentices
MRGBIFA Dialogue mechanism Expert EMCUD RG 調查研究 實驗研究 … 樣本大小 大樣本 小樣本 … 研究設計 因果性 探索性 … … … … … AOT 調查研究 實驗研究 … 樣本大小 1 X … 研究設計 D 1 … … … … …
Figure 7: An overview of intention finding process
Example 3: An example of intention finding process
Here we illustrate the architecture in more detail by showing an example of MRGBIFA. We assume that the repertory grid and AOT of social science research methods have been acquired from experts. As shown in Figure 8, there are five research methods and four attributes in the repertory grid and AOT.
17 RG 調查研究 實驗研究 內容分析 觀察研究 個案研究 樣本大小 大樣本 小樣本 中樣本 中樣本 小樣本 研究設計 因果性 探索性 敘述性 探索性 敘述性 對象 個人 個人 其他 個人 其他 要做實驗 No Yes No No No AOT 調查研究 實驗研究 內容分析 觀察研究 個案研究 樣本大小 1 X D 1 D 研究設計 D 1 1 2 1 對象 2 1 2 D 2 要做實驗 2 D 1 1 2
Figure 8: An example of repertory grid and AOT
The questions are produced to ask the apprentice. An example of these questions based on the expertise is shown in Figure 9. Three research methods, which are not distinguished by the apprentice, are listed and all the attributes in the first step and the apprentice is asked to select an attribute, which can distinguish the candidate objects by his intention. In the second step, the apprentice selects the "樣本大小" attribute and the "中樣本" value. The answers are viewed as his intention. In the third step, the partial grid is filled out by the apprentice, and then some misconceptions with the "內容分析" object or the "樣 本大小" attribute are discovered by the MRGBIF.
Step 1:List Candidate Object List
Step 2:Select Distinguishable Attribute
Step 3:Determine Candidate Objects’ Attribute
Q：請回答下列問題 實驗研究的樣本大小 內容分析的樣本大小 觀察研究的樣本大小 大樣本 大樣本 大樣本 中樣本 中樣本 中樣本 小樣本 小樣本 小樣本 Q：請選一個可以 判斷你的假設跟實 驗研究、內容分析 和觀察研究比較相 似的特性? 對象 要做實驗 樣本大小 研究設計 Q：請選一個可以 判斷你的假設跟實 驗研究、內容分析 和觀察研究比較相 似的特性? 對象 要做實驗 樣本大小 研究設計 Q：請選一個特 性值來分辨? 大樣本 中樣本 小樣本
18
Chapter 4 Intelligent Social Science
Research Methods Selection Assistant
As mentioned before, we want to assist the apprentices to find appropriate research method according to their learning status. Therefore, the following two issues should be solved: (1) how to determine apprentices' misconceptions from the records of a knowledge acquisition process, (2) how to assist the apprentices in selecting research method according to the misconceptions.
The idea for solving this problem is based on the MRGBIFA so that we acquire apprentices' misconceptions in what respect from their intentions. There is less study about research method selection assistance. Based upon this idea, the
Intelligent Social Science Research Methods Selection Assistant (ISSRMSA)
is proposed. Our approach is based on the experts' selection strategy of research method. The steps of the ISSRMSA are described as follows: enumerating the research method list, choosing the attributes and eliminating inappropriate research methods. We apply this selection strategy to the apprentices, and then observe their behaviors in selection process. Finally, we revise the ISSRMSA in accordance with the feedback. The details of the ISSRMSA will be described in Chapter 4.
19
Figure 10 represents the system architecture in the ISSRMSA. The components of the ISSRMSA are composed of Dialogue-Based Knowledge Acquisition Interface and Rule-based Guiding Controller. The Dialogue-Based Knowledge Acquisition Interface is to find apprentice's intentions. The detailed discussion is mentioned above. The Rule-based Guiding Controller is to detect apprentice's learning status and give proper help according to various learning status. Learner Status Facts database and Path Database are generated from the third step of MRGBIFA. Rule Base is constructed according to the possible conditions from the apprentices in advance.
Rule-based Guiding Controller Learner Status Facts Path Database Rule Base Apprentices Dialogue-Based Intention Finding Interface Meta Rule AttributeList Determination Rule Class Intention Evaluation Rule Class Misconception Finding Rule Class
Figure 10: System Architecture of ISSRMSA
Example 4: An example of ISSRMSA
Take Figure 9 as an example, when the apprentice enters our ISSRMSA, the questions are generated to ask him, and then his intentions, which are the "樣本大
20
小" attribute and the "中樣本" value, are discovered in the Dialogue-Based Knowledge Acquisition Interface. His answer process and intention are transformed to Rule-based Guiding Controller, which can compare the expert's repertory grid to find out apprentice's misconceptions, and then the assistance strategy is inferred to the apprentice from their misconceptions. We assume that he has some misconceptions with the "樣本大小" attribute, so the hint is provided to let the apprentices easily understand the attribute. The detailed discussion is described in the following sections.
4.2 Rule-based Guiding Controller
The questions which are generated by the MRGBIFA can find out not only the apprentices' intention but also their learning status. Hence, their learning status is found out in each step. Figure 11 indicates the system process of each step in the ISSRMSA.
21
MRGBIF Dialogue mechanism
Fetch Learner Status Fact Infer by AttributeList Determination Rule Class Show Candidate AttributeList Get Selected Attribute Get User-given Value Compare value with expert’s grid
Calculate Doubt Degree Calculate Certainty Factor Establish Fact
Step 1： AttributeList Determination
Step 3： Misconception Finding Step 2： Intention Evaluation
Fetch Learner Status Fact
Infer by Intention Evaluation
Rule Class
Show warning & MRG Question
Infer by Misconception Finding
Rule Class Show feedback & go
through next step
Figure 11: System process of each step in the ISSRMSA
The process ''System List Candidate Objects'' means that when three candidate research methods are selected, they are observed whether there exists any misconception in Learner Status Facts. If exists, the facts are transferred to the inference engine and the assistance strategy is showed in the KA interface. For example, the choices of the following questions will be decreased. Then, the process ''Apprentices Select Distinguishable Attribute'' indicates that when the apprentices select an attribute, the attribute is observed whether there exists any misconception in Learner Status Facts. If exists, the fact is transferred to the inference engine and the assistance strategy is showed in the KA interface. For example, the warning will be showed up to demand that the apprentices should select another attributes. The process ''Apprentices Determine Candidate Objects'
22
Attribute'' means that when the apprentices fill out the partial grid according to candidate objects, the grid are compared with expert's one. The doubt degrees of each object and attribute are calculated according to the result of expert and apprentices' repertory grid in order to find out the misconceptions. The certainty factors of the candidate objects are computed according to the importance of AOT in order to discard the inappropriate research methods. These calculating results are transformed into facts. Then, the facts are stored to Learner Status Facts and transferred to the inference engine. Finally, the assistance strategy is showed in the KA interface. For example, a hint is provided for the apprentices when they answered incorrectly and have misconception with the attribute.
4.2.1 User Status Facts
In the ISSRMSA, we induce the possible behaviors from the apprentices in advance. For example, when the apprentices face the questions, they may not understand the research methods or attributes, etc. Hence, the user status facts are divided into three parts: (1) Facts of Misconception, (2) Facts of Current Answer, (3) Fact of Research Method, and the facts are listed in Table 1.
23
Table 1: Type of user status facts
Fact
Type
Facts of Misconception
Object has misconception. Attribute has misconception.
Facts of Current Answer
Apprentices respond right to the values.
Apprentices respond wrong to the important values. Apprentices respond wrong to the unimportant values.
Fact of Research Method
The research method is not appropriate.
4.2.2 Guiding Rule Classes
Based upon the different considerations of each step in the MRGBIFA, meta rule is divided into three rule classes: Objects Rule Class, Attributes Rule Class and Values Rule Class. Objects Rule Class checks the apprentices' learning status of the candidate objects in the process ''System List Candidate Objects''. Attributes Rule Class checks the apprentices' learning status of the attribute in the process ''Apprentices Select Distinguishable Attribute''. Values Rule Class checks the apprentices' answer status of the values in the process ''Apprentices Determine Candidate Objects' Attribute''. The partial rule classes are shown in Figure 12. For example, the rules are as follows in the process ''System List Candidate Objects'': RULE: Object is appropriate ∩ ～(Object has misconception) → ''不採取任何 動作''
RULE: Object is appropriate ∩ Object has misconception → ''減少問題選項'' RULE: ～(Object is appropriate) → ''挑其它未分辨過的 Object''
24
System List Candidate Objects
適合的 object 不適合的object 不採取 任何動作 減少問 題選項 挑其他未 分辨的 Object Object 無迷失概念 有迷失概念Object
Apprentices Select Distinguishable Attribute
不採取
任何動作 警告標語Show Attribute
無迷失概念 有迷失概念Attribute
Apprentices Determine Candidate Objects' Attribute
Attribute 懷疑程度下降 &之前Object 懷疑程度上升 Attribute 懷疑程度上升 Value 都答對 Value答錯 但不重要 Value答錯 但重要 問下一層 Grid 給hint Attribute 第一次選 Attribute 非第一次選 Attribute 懷疑程度下降 &之前Object 懷疑程度上升 下一層 Value答對 下一層 Value答錯 Attribute 懷疑程度上升 Attribute 懷疑程度下降 Attribute 懷疑程度上升 &再重挑 Attribute Value 答對 Value答錯
25
4.3 Social
Science
Research
Methods
Misconceptions Finding
In order to give proper help, we acquire apprentices' learning status and find out their misconceptions. The apprentices have their own understanding degree of research method for each object and attribute. However, the apprentices still have some misconceptions in fact. Doubt Degree is used to calculate the possibility of misconception.
Formula 1: Doubt Degree
Doubt Degree = Weight(candidate object, selected attribute) * Trusty Rate + Old Doubt Degree, where
Weight(candidate object, selected attribute) means the corresponding value of selected
attribute to candidate object in AOT, Trusty Rate means the status of current answer and Old Doubt Degree means the former doubt degree.
The value of Doubt Degree is positive or negative, when it is positive that means the object or attribute has no misconception for the apprentices, else means the object or attribute may has misconception. Moreover, we define the threshold which is used to determine whether the misconception exists. If the value of Doubt Degree is negative and greater than the threshold that means the object or attribute has misconception exactly.
Definition 1: Weight(object, selected attribute)
Weight(object, selected attribute) {D, 2, 1, X}
D means that the attribute dominates the object, so we give a possibility 100%.
26
2 means that the attribute does not dominate the object but it has some degree of importance relative to other attributes, so we give a possibility 75%. 1 means that the attribute does not dominate the object but it has a little bit
degree of importance relative to other attributes, so we give a possibility 50%.
X means that the attribute has no relationship with the object, so we give a possibility 25%.
Definition 2: Trusty Rate
The status of current answer, in Table 2, consists of all problems correctly, two of the problems correctly and missed one, one of the problems correctly and missed two and all problems incorrectly.
Table 2: Trusty rate of answer status
Answer Status
Trusty Rate
all problems correctly trusty rate of Attribute is +1 trusty rates of Objects are +1
two of the problems correctly and missed one
rusty rate of Attribute is +0.5
trusty rates of correct Objects are +1 and trusty rates of incorrect Object is -1
one of the problems correctly and missed two
rusty rate of Attribute is -0.5
trusty rate of correct Object is +1 and trusty rates of incorrect Objects are -0.75
all problems incorrectly trusty rate of Attribute is -1
trusty rates of Objects are -0.25
4.4 Research
Methods
Selecting
Certainty
Factor
For discarding inappropriate research methods, Certainty Factor is used to calculate the possibility of research method.
27
Formula 2: Certainty Factor (CF)
CF candidate object = Weight(candidate object, selected attribute) * Value Similarity(candidate object, selected attribute) + Old
CF candidate object, where Weight(candidate object, selected attribute) means the corresponding
value of selected attribute to candidate object in AOT, Value Similarity(candidate
object, selected attribute) means the corresponding value of selected attribute to candidate
object in Repertory Grid and Old CF means the former certainty factor of candidate object.
The value of Certainty Factor is positive or negative, when it is positive means the candidate object is appropriate for the apprentices, else means the candidate object may be inappropriate. Moreover, we define the threshold which is used to determine whether the candidate object is inappropriate. If the value of Certainty Factor is negative and greater than the threshold that means the candidate object is inappropriate exactly.
Definition 3: Value Similarity(candidate object, selected attribute)
Value Similarity(candidate object, selected attribute) {1, -0.5, -1}
When the apprentices' answer corresponds to the expert's, we assign the value 1.
When the apprentices' answer does not correspond to the expert's but is kind of similar in conceptual level, we assign the value -0.5.
When the apprentices' answer is totally different with the expert's answer, we assign the value -1.
28 Example 5: An example of CF RG 調查研究 實驗研究 內容分析 樣本大小 大樣本 小樣本 中樣本 CF 0 1.25 2.5 AOT 調查研究 實驗研究 內容分析 樣本大小 1 X D
Figure 13: An example of partial repertory grid
Take Figure 13 as an example, we assume that apprentices' intention is the '' 小樣本'' value of the ''樣本大小'' attribute and the candidate objects are ''調查研 究'', ''實驗研究'' and ''內容分析''. These candidate objects are calculated as follows:
New CF調查研究 =0. 5 * (-1) + 0 = -0.5
New CF實驗研究 =0.25 * 1 + 1.25 = 1.5
New CF內容分析 =1 * (-0.5) + 2.5 = 2
In the ''調查研究'' object, Weight(調查研究, 樣本大小) is the value 1, we give a
possibility 50%; Value Similarity(調查研究, 樣本大小) is the ''大樣本'' value and it totally
has no similarity between the apprentices' intention and the expert's grid, we assign the value -1; Old CF調查研究 is the value 0. Hence, CF調查研究 is turned into the
value -0.5. We suppose that the ''調查研究'' object may be inappropriate because CF調查研究 is negative.
29
Chapter 5 Experiments
5.1 System Implementation
In order to implement a MRGBIFA on web-based environment, some tools are used to assist implementing.
Dialogue-Based Intention Finding Interface
Microsoft Visual Studio 2005 ASP.NET with C# is used to construct the interface. It provides very convenient tools and integrates well with database. Rule base and Inference Engine
DRAMA is used to construct rule base and inference engine in the system. It contains useful tools, like rule verification tool, knowledge acquisition assistant tool and the inference server.
At last, the following figures are some snapshots of the system. Firstly, the system asks the apprentice to select an attribute, which can distinguish the candidate objects by his intention (see the Figure 14(a)). Next, the system asks the apprentice to choose a value from the selected attribute (see the Figure 14(b)), and then let them fill out the partial grid according to candidate objects and selected attribute (see the Figure 14(c)). In the intention finding process, the system will infer the selected item and show the inference result in each step.
30
(a) (b)
(c)
Figure 14: The user view of the MRGBIFA
5.2 Experiment Design and Result
We want to evaluate the system whether the system can provide enough assistants for the apprentices. We collect the satisfaction of the apprentices for their comments about using the system to assist them. We invite 30 graduate students who have learned research method from National Chiao Tung University. Let these students use the system to assist them selecting research method. After they finished their selection, they filled a satisfaction questionnaire. 5 questions
31
are designed and Likert 5 scale is used to evaluate the degree of apprentices' satisfaction, from very disagree (1) to very agree (5). The results of satisfaction questionnaire survey are shown in Table 3.
Table 3: Results of satisfaction questionnaire survey
Question Satisfaction
Degree Q1. This system can assist selecting research method
correctly.
3.4
Q2. This system can find out all intentions. 3.6
Q3. This system provides rich information to assist selecting research method.
4.0
Q4. The assistance strategy is useful to select research method.
3.9
Q5. I will use this system again in the future. 3.7
Q1 is the lowest points between the five questions which are evaluated. This means our ISSRMSA needs to consider more factors when we calculate the certainty factor. The results show that our ISSRMSA is useful for the apprentices to select research method and gain rich information. Finally, some students hope that our ISSRMSA can provide assistance about experiment design because the assistance strategy is just used to select research methods but not for the experiment design.
32
Chapter 6 Conclusions and Future Works
In this thesis, we want to assist the apprentices to select the appropriate research method according to their intentions and let them gain the experience of research method selection from the selecting process. Hence, we propose the Modified Repertory Grid Based Intention Finding Approach (MRGBIFA), which is a dialogue mechanism, to discover apprentices' intentions; and the Intelligent Social Science Research Methods Selection Assistant (ISSRMSA) is to discover apprentices' misconceptions and give proper assistance according to diverse misconceptions. There are three steps in the MRGBIFA including System List Candidate Objects, Apprentices Select Distinguishable Attribute and Apprentices Determine Candidate Objects' Attribute. In the first and second steps, our major target is to find the intention. In the third step, our major target is to reveal the misconception. We construct the partial grid based on repertory grid approach to acquire the partial learning status from the apprentices, and then compare the difference between the apprentices' and the expert's grid. Finally, in accordance with various misconceptions we give the proper help and guiding approach. The MRGBIFA is not discontinued until the apprentices have distinguished the entire objects. Finally, the appropriate research method and explanation are listed to the apprentices. Experimental results show that our ISSRMSA is useful for the apprentices to select research method and gain rich information.
In the near future, we will help apprentices to plan their experiment design when we realize which research method is appropriate for apprentices. Moreover,
33
we can verify the validation of the experiment design according to the intention finding process to find out another misconception and immediately give proper help. In addition, we can guide apprentices to choose appropriate Statistical Tests. Eventually, we hope to achieve the goal that the whole study is more complete.
34
Reference
[1] E. R. Babbie, The Practice of Social Research. Wadsworth, 1975: Thomson
Learning Inc.
[2] Scientific method, Merriam-Webster Dictionary. [Online]
http://www.merriam-webster.com/
[3] G. A. Kelly, The psychology of personal constructs. Norton, New York,
1955.
[4] W. Lawrence Neuman, Social Research Methods: Qualitative and
Quantitative Approaches. 1991: Allyn & Bacon.
[5] G. D. Garson, Advances in Social Science and Computers. A research
annual, 1989. H61.3 A38 v.1.
[6] B. Anderson, The Psychology Experiment: An Introduction to the Scientific
Method. Wadsworth, 1966.
[7] P. Godfrey Smith, Theory and Reality: An introduction to the philosophy of
35
[8] G. J. Hwang and S. S. Tseng, "EMCUD: A knowledge acquisition method
which captures embedded meanings under uncertainty," Int’l Journal of Man
Machine Studies, Vol. 33, pp. 431-451, 1990.
[9] S. S. Tseng and G. J. Hwang, "A new table-based knowledge acquisition
method," International Computer Symposium, Taipei, Taiwan, R.O.C.,
1988.
[10] B. Boehm, "A Spiral Model of Software Development and Enhancement",
