Chapter 1 Introduction
3.3 Reduce the Cross-Language Translation Ambiguity
Chinese Query
Vocabulary Translation
Monolingual English Retrieval
process Dictionary
Candidate words
Word Sense Disambiguation
WordNet Ontology chain
English documents correct Image retrieval
process
Image database Un-annotated
Related images user
Figure 3-4: The cross medical image system architecture
Figure 3-4 is the overview of our cross medical image system architecture.
User can use Chinese terms to retrieve related medical document. The associated medical images of document can be used as image query example. The image process retrieves the similar image by image features.
The Dictionaries and WordNet approach are designed for general domain resource and the creation time is very expansive. The relationship definitions are still not enough to gain good results. The ambiguous translation is the main problem that gets bad result. Davis (1997) had discussed that use category will reduce the ambiguity problem. In this proposed we want to solve the ambiguity problem by
using proposed ontology chain method.
In the corpus approach need large amount parallel bilingual document for reference. In this proposed we try to find similar concept bilingual document by similar medical images. When user uses Chinese language to retrieve English language document case, we can get relevance medical images. In the multilingual database, we can create a relationship between English document and French document by similar medical image. Those document have similar medical images, it must have similar concept in some aspect.
3.3.1 Cross-Language Retrieval System Query by Keyword
Figure 3-5: Proposed system flowchart
Figure 3-5 is the proposed cross-language system flowchart for keyword based retrieval. It contains three modules that are: Query Translation, Resolving Translation Ambiguity, and Monolingual Information Retrieval. Describe detail as following:
Query Translation: Translate the Chinese query keyword into possible English
words by dictionary and its synonym, Hypernym and hyponym expansion by WordNet.
Resolving Translation Ambiguity: After query Translation process, the produced possible translation English words may cause ambiguous. We proposed an ontology chain approach to conquer the ambiguity problem. The ontology was created by expert. We use the category to construct an ontology map. We use ontology chain to refine the candidate translated keywords, puck up the meaningful candidate keyword as the query terms for monolingual retrieval.
Monolingual Retrieval Process: Find the document that related to the query terms.
Ontology is a formal explicit specification. It constructs a common conceptualization for users. Ontology contains principal concept and relationship among concepts, especially usefully in specific domain. Figure 3-6 is an example of ontology. In our previous works, we use the St Andrews data set from ImageClef2004 to evaluate our proposed methods. The St Andrews dataset consists of 28,133 photographs from St Andrews University Library photographic collection which holds one of the largest and most important collections of historic photography in Scotland.
Figure 3-6: An example of ontology
The St Andrews data set from CLEF2004, each document belong to more than one category. It contains 946 categories. We arrange the categories to construct an ontology map that define the relationship between each category. Creating the ontology by category is easier than construct ontology by terms in time complexity.
When the translations of query terms contain multiple meaning will cause ambiguity result. We use the ontological chain to evaluate the most appropriate meaning.
The related similarity between ontology node Li and query express Q is defined as Eq. (22).
N t Q
L Sim
N
j ij i
∑
== 1
2
) ,
( (22)
Where tij is the frequencies of translated English terms, N is the number of Chinese query terms.
We let arbitrary two node of ontology have a distance. For example as show in Figure 3-6 the “herring” and “fish processing” have a path 3. Based on the similarity of query terms and the distance we can define a relationship between two nodes.
Rel(Li, Lj) = Figure 3-7 is a relationship map between nodes. We set a threshold 1.5 to eliminate less important relationships. The connected nodes will construct many semantic concepts. We can eliminate the translated English term that do no fall in the connected nodes. The sum of relationships of each island connected nodes can be used as important degree for ranking candidate English terms.
drama
Figure 3-7: Semantic relationship map
3.4.1 Experiment Result for Medical Image Collections
The entire ImageCLEFmed library consists of multiple collections. Each collection is organized as cases that represent a group of related images and
annotations. Each case consists of a group of images and an optional annotation.
Each image is part of a case and has optional associated annotations consisting of metadata (e.g., HEAL tagging) and/or a textual annotation.
In practical medical images retrieval systems, it is natural for doctors to retrieve the related images by the descriptions of requirement. After textual query, the system retrieves related images by annotations for users to browse. The user can further
rank the images in visual. Combining the textual and visually features will help the user to find the desire image more precise. In ImageCLEF 2005, medical image retrieval task contains 25 queries for evaluation, the queries mixed visual images and semantic textual to retrieve desire images. The visual queries use image examples to find similar images, each topic contain at least one image example. The semantic textual queries allow user query by a sentence, which high-level semantic concept are hard to be derived from images directly. The goal of this task is to examine how the visual feature can improve the query result.
All submissions of participants in this task were classified into automatic runs and manual runs. The automatic runs mean that the system at the query process without human manual intervened. In the automatic category, the methods can be classified into three sub-categories: Text only, Visual only and Mixed retrieval (visual and textual) according to the feature used. The category “Text only” means that systems use textual feature only to retrieve relevant images. “Visual only”
category means that systems only use visual image feature without combine textual annotation to retrieve similar images. Mixed retrieval means the systems combine the visual and textual feature to retrieve images.
In this task, we have submitted ten runs for the mixed retrieval of automatic runs and six runs for the visual only of automatic runs. In the content-based approach, we combine four proposed image features by weighted adjusting to retrieve related images. The weight of features we set at the system initial and do not have any further user intervention while query is processing. Table 3-1 lists the query result of visual only runs and the setting weight of four image features. Table 3-2 lists the result of mixed retrieval runs and the setting weight of image features and textual features. The difference of each runs is the weighted setting of features.
The query topics contain color and gray images. We first examine the queries
image is color or gray image by color/gray feature. According to the image is color or gray set different weight for image features. In the Table 3-1, “C” denotes that query image is color image and “G” denotes that query image is gray image. We submit six runs for visual only category. The run, “nctu_visual_auto_a8”, has the better result in our experiment. The weight of each feature are set equal, it means that four image features have the same importance. The result also shows that visual only approach has a bottleneck because the query topics contain semantic queries.
Submission runs The weight of Image features Result
Image features Coherence Gray HIS Color HIS Facade MAP Rank of runs detected color or gray C G C G C G C G
visual_auto_a1 0.3 0.2 0.3 0.5 1 0.2 1 1 0.0628 14
visual_auto_a2 0.3 0.2 0.5 0.3 0.3 0.5 1 1 0.0649 10
visual_auto_a3 0.5 1 0.5 1 1 0.5 1 1 0.0661 8
visual_auto_a5 0.1 0.2 0.1 0.5 1 0.5 0.5 1 0.0631 13
visual_auto_a7 0.3 0.2 0.3 0.5 1 0.2 1 0.5 0.0644 11
visual_auto_a8 1 1 1 1 1 1 1 1 0.0672 7
Table 3-1: The query result of visual only runs and the weight of visual image features
Submission runs The weight of Image features
Image features Coherenc e
Gray HIS Color HIS Facade visual textual
Detected color or gray
C G C G C G C G C G C G
MAP Rank
visual+Text_auto_1 0.3 0.2 0.3 0.5 1 0.2 1 1 1 1 0.8 0.1 0.2276 10 visual+Text_auto_2 0.3 0.2 0.5 0.3 0.3 0.5 1 1 1 1 0.8 0.1 0.2127 14 visual+Text_auto_3 0.5 1 0.5 1 1 0.5 1 1 1 1 0.8 0.1 0.2286 9 visual+Text_auto_4 0.3 0.2 0.3 0.5 1 0.2 1 1 1 1 1 0.2 0.2389 3 visual+Text_auto_5 0.1 0.2 0.1 0.5 1 0.5 0.5 1 1 1 0.8 0.1 0.2246 12 visual+Text_auto_6 0.3 0.2 0.3 0.5 1 0.2 1 1 1 1 1 1 0.2318 7 visual+Text_auto_7 0.3 0.2 0.3 0.5 1 0.2 1 0.5 1 1 0.8 0.1 0.2265 11 visual+Text_auto_8 1 1 1 1 1 1 1 1 1 1 1 1 0.2324 6 visual+Text_auto_9 1 1 1 1 1 1 1 1 1 1 0.1 0.1 0.0906 22 visual+Text_auto_10 1 1 1 1 1 1 1 1 0.1 0.1 1 1 0.1941 15
Table 3-2: The result of mixed retrieval runs and the weight of visual image features and textual features
The setting weights of mixed runs and results are listed in the Table 3-2. The result of run8, run9 and run10 illustrate that combine the visual and textual feature will get better results than single features. Run8 assume that the significant of visual and textual feature are equal. Run9 emphasizes the weight of visual features and Run10 emphasizes the weight of textual features. The result shows that text-based approach is better than content-based approach, but the content-based approach can improve the textual result.
3.4.2 Experiment Result for St Andrews collections
We test our method using the CLEF English-Chinese ImageCLEF [11] test collection. The collection includes about 30,000 historic photographs with British English semi-structured captions. ImageCLEF is the cross-language image retrieval track that is run as part of the Cross Language Evaluation Forum (CLEF) campaign.
The campaign is run in a similar manner as the TREC and NTCIR information retrieval evaluations. This is a bilingual query translation task from a source language to English. The goal of this task lies in finding as many relevant images as possible from the St. Andrews image collection.
Each image has an accompanying textual description consisting of 8 distinct fields. These fields can be used individually or collectively to facilitate image retrieval. The 28,133 captions consist of 44,085 terms and 1,348,474 word occurrences; the maximum caption length is 316 words, but on average 48 words in length. All captions are written in British English, although the language also contains colloquial expressions. Approximately 81% of captions contain text in all fields, the rest generally without the description field. In most cases the image description is a grammatical sentence of around 15 words.
descriptive categories, e.g. “airports”, “airships”, “flowers”, “beach scenes” and
“breweries”. On average, each image is assigned to 4 categories. We use the categories to manually construct the ontology. Related categories will be gathered into a class from bottom to up by experts to construct a hierarchic structure, which is the ontology used in this paper. Totally the ontology has 946 categories.
The language used to express the associated texts or textual queries should not affect retrieval, i.e. an image with a caption written in English should be searchable in languages other than English. In the experiment we will show that ontology approach will improve the precision by 13%, which is better than the result without dis-ambiguity process.
The cross language evaluation forum includes data set, 25 query topics and query answer to evaluate related systems. We evaluate three models; they are Mono-lingual IR (Mono-lingual Information Retrieval), CLIR (Cross Language Information Retrieval), and Ontology-based CLIR. The Mono-lingual IR uses the original English queries with expansion by WordNet to retrieve related documents; it is the baseline in evaluation.
The CLIR model uses Chinese queries to retrieve related documents; it first translates the Chinese terms to all possible English terms by a dictionary. After translation, the CLIR model also expands synonym by WordNet and uses the possible English terms as query without dis-ambiguous analysis. The CLIR model does not consider the ambiguity problem that is used to compare with Ontology-based CLIR model. The Ontology-based CLIR analysis the candidate terms based on Ontology chain trying to reduce the ambiguity translation problem.
The performance is normally evaluated based on precision and recall as defined in the following equations:
retrieved , of
number total
retrieved answer
correct precision= numberof
answer . actual
retrieved answer
correct of
of number total
number recall=
We used the UMASS and Lemur versions of the standard trec_eval tool to compute the precision, recall and mean average precision scores. This provides the "standard"
information retrieval evaluation measures, e.g. precision at a given rank cut-off, average precision across 11 recall points, and single-valued summaries for each measure.
Figure 3-8 is the precision/recall of experiment, the result show that ontology-based CLIR approach is better than normal CLIR and close to the Mono-lingual IR.
0 0.2 0.4 0.6 0.8 1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Recall
Precision
Mono-lingual IR Ontology-based CLIR
Figure 3-8: The precision vs. recall
Table 3-3 is the experiment result. The mean average precision (MAP) of normal CLIR is 49.18% and ontology based CLIR is 55.18. The Mono-lingual IR directly uses native language for query without translation ambiguity for comparison. It shows that Ontology-based CLIR improve the performance from 49% to 55 %, and reach to the 92% precision of mono lingual IR. The mono-lingual IR using the original English query avoids the translator ambiguity, so that it has the best performance in the experiment result.
The normal CLIR translated Chinese to English may have miss translation or ambiguity translated will cause find more irrelevant documents lowering the precision. And the ontology based CLIR furthermore filter out irrelevant translated terms. The experiment result shows that the Ontology chain is useful to recognize the acceptation of query. Translated terms must have the same concept to consist a query. Based on the specific domain, compare to the query context can reducing the ambiguity. The proposed Ontology based CLIR effective to locate the correct translation increasing a 10% rate comparing to monolingual IR.
Mono IR CLIR Ontology
MAP 60.63% 49.18% 55.18%
Table 3-3: The mean average precision
A novel ontology-based approach for reducing the ambiguity of bilingual translation has been presented in this paper. Using a dictionary to translate a Chinese term into English may cause the problem of translation ambiguity. We successfully find the best translation by analyzing the relevance of all possible translated terms. A metric based on the ontology for co-occurrence concept evaluation has been proposed. The experimental results show that the proposed approach can effectively remove the irrelevant results and improve the precision.
The Ontology is very useful in the synonym and polysemy analysis. A word in different domain may have different lexical meaning. The ontology is domain dependent. In this paper we define the ontology by manual for St. Andrews corpus;
we now try to cluster the words into concept capable of automatic constructing the ontology.
In the WWW application, each page can be viewed as an ontology node. In the digital library the similar pages will be placed together. We can reference the link to construct related concept. Our proposed mechanism will easy apply to WWW environment.
Chapter 4
Medical Image Retrieval with Relevance Feedback
In this chapter we will describe the relevance feedback mechanism help to improve the retrieval result for image query. In this dissertation we design a new model for user to do relevance feedback in image retrieval. The proposed feedback method not only allow user to choice positive examples and negative examples but also can observe the features of user’s interest. We expect that our new relevance feedback mechanism will get more information from user and fast tune the system to suit user’s interesting.
Content-based image retrieval (CBIR) is a process to find images similar in visual content to a given query from an image database. It is usually using low level features, such as color, texture or shape features, extracted from the images themselves to compare similarity. There is much research addressing the performance of content-based image retrieval method are still limited especially in the two aspects of retrieval accuracy and response time.
The limited retrieval accuracy is because of the gay between semantic concepts and low-level image features, which is a problem in content-based image retrieval.
An image can be explained in many aspects by different people. For example, different types of features have different significance; user sometime want to find similar color images and sometime want to find similar shape images that is very subjective. An important issue is how to derive the weighting of different features.
For the flexibility, the system in the initial weighting of features always equal, but there is no universal formula for all queries. The relevance feedback technique can
be used to reduce the gap [13] [36] [38] [57].
Relevance feedback, developed from information retrieval [54], is a supervised learning technique used to improve the effectiveness of information retrieval systems. User marks the image of query result as positive or negative examples to improve the system’s performance. For a given query, the system first retrieves a list of ranked images according to predefined similarity metrics. Then, the user select a set of positive and negative example form the result images, and the system reformulate the query and retrieves a new list of images. The main problem is how to incorporate positive and negative examples to refine the query and how to adjust the similarity measure according to the feedback.