1.1 Background
The famous search engine, Google1, receives more than 200 million queries every day.
Automatic question answering becomes one of killer applications associated with natural language techniques and information retrieval to deal with. So it is desirable for computer scientists to propose efficient QA systems to extract the answers automatically.
Question answering researches have become popular since TREC2 (Text REtrieval Conference) 1999. In TREC QA track, the QA systems proposed by the participators try to find the answer for a set of given questions from the collected documents provided by TREC. During the last decade, some QA systems have been proposed, such as START3 presented by Katz et al. (2003). START is a Web-based QA system for several general domains including geography, science, arts, entertainment, history, and culture.
Recently, some researchers (Zhang et al. 2004; Niu et al. 2004; Wu et al. 2005) consider that specific domain QA has great potential. Specific domain QA is presented by using domain ontology. For example, Wu et al. (2005) divide the QA system into the question part and the answer part. They use the ontology proposed by Yeh et al. (2004) to calculate the distance of keyword concepts in the question part
1 Google http://www.google.com.tw
2 TREC http://trec.nist.gov/
3 START http://start.csail.mit.edu/
and the casual relations in the answer part in order to retrieve the possible answer passages. Niu et al. (2004) consider the ontology as the specific expansion, such as hypernym expansion. Zhang et al. (2004) tag the categories for the nouns in the question and documents by using the ontology. The authors use okapi function to measure the similarity of categories between the question and the documents to retrieve the answer passages from the documents. In fact, how to utilize the domain knowledge is the main difference between open domain QA and specific domain QA.
We discuss this topic in next section.
1.2 Specific Domain QA and Open Domain QA
Open domain QA processing involves question processing, information retrieval and answer extraction (Niu et al. 2004; John et al. 2004). Question processing is to understand what the question is asked about. The main purpose is to identify the answer type of a question so as to spot the answer. For open domain QA, the answer type can be identified by the interrogative word only. However, the interrogative word is not sufficient to understand query intention for specific domain. Take the questions
“Who invented the toothbrush?” in open domain and “Who is at the greatest risk for heat-related illness?” in specific domain as the examples. We consider the answer type for two questions as person name according to the interrogative word. But the answer type is not person name for specific domain question. The details of examples are showed in Table 1.
Table 1. Examples of open domain and specific domain
Question Answer
Open Domain Who invented the toothbrush? William Addis Specific Domain Who is at the greatest risk for
heat-related illness?
Infants and children up to four years of age, people 65 years of age and older …
The information retrieval module is to retrieve the relevant documents for the inputted question. In open domain QA, most of the questions are factoid questions, such as person, place, time, place or object. These questions are data-driven because their answers are always single. However, the domain knowledge is required for specific domain QA to understand the question and to consider whether the retrieved documents are relevant or not. So the specific domain questions are recognized as the knowledge-driven.
The answer extraction is to spot the answers from the relevant documents according to the information provided by the component of question processing. The strategy to locate the answers is calculating the similarity between the given question and the documents or passages. For example, the syntactic structure and named-entity are considered to spot the possible answers and the answers are ranked by the similarity score. In open domain QA, there is an explicit answer for each question, such as date, person name, or place name. But in specific domain QA, most of the specific domain questions are to concern the explanation.
1.3 Motivation
In this thesis, we concern the need to propose an efficient method for answering medical questions generated from people. The medical FAQs from the Web are the main data set for us to develop the medical QA because the questions of FAQs are generated by people and the answers of FAQs are provided by domain experts. They are good materials to propose specific domain QA.
For the medical QA, we use UMLS4 (Unified Medical Language System) as knowledge base and PubMed5 as the document source to deal with medical questions.
First, the medical FAQs and medical literatures are collected from the Web. For the medical literatures, we extract the syntactic pattern as the form of NP-Verb-NP patterns. After concept identification for the noun phrases by using UMLS, the NP-Verb-NP patterns are transformed into Concept-Verb-Concept patterns. For the medical FAQs, the questions are used to train the question classifier. We also use the ontology to expand the query presented in (Hersh et al. 2000). When the question is inputted, the question is analyzed and the syntactic pattern with concept is identified by UMLS. The relevant texts which the answer may contain in are retrieved and ranked by scoring the weight of concept patterns and the weight of keywords.
There are three indicators for evaluating our method. The first indicator which we use to evaluate the performance of the method is the mean reciprocal rank (MRR).
If the k-th abstract returned by the search engine contains the answer, the value of reciprocal rank is 1/k. The second indicator is human effort (HE). It is defined as the
4 UMLS http://www.nlm.nih.gov/research/umls/
5 PubMed http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed
user finds the answer in the least rank of passages returned by the system. The third indicator is recall at top five passages returned. We take 203 questions from FAQs to evaluate the method. The experimental results show that there are 0.63 in MRR, 2.55 in human effort and 80% recall at top five passages for our proposed method.
The rest of the thesis is organized as follows. The related work is surveyed in Chapter 2. Medical question answering is described in Chapter 3. The evaluation and analysis are showed in Chapter 4. The conclusion and future work are given in Chapter 5.