The Experiment and Result

We use the ImageCLEF 2004 evaluation to evaluate the performance of our system. The dataset for the medical retrieval task is called CasImage and consists of 8,725 anonymised medical images, e.g. scans, and X–rays from the University Hospitals of Geneva. The majority of images are associated with case notes, a written description of a previous diagnosis for an illness the image identifies. Case notes are written in XML and consist of several fields including: a diagnosis, free-text description, clinical presentation, keywords and title. The task is multilingual because case notes are mixed language written in either English or French (approx. 80%).

An example case notes field for description and corresponding images is shown in Figure 2-4. Not all case notes have entries for each field and the text itself reflects real clinical data in that it contains mixed–case text, spelling errors, erroneous French accents and un–grammatical sentences as well as some entirely empty case notes. In the dataset there are 2,078 cases to be exploited during retrieval (e.g. for query expansion). Around 1,500 of the 8,725 images in the collection are not attached to case notes and 207 case notes are empty. The case notes may be used to refine images which are visually similar to ensure they match modality and anatomic region.

The process of evaluation and the format of results employ the trec_eval tool.

There are 26 queries. The corresponding answer images of every query were judged as either relevant or partially relevant by at least 2 assessors.

Figure 2-4: An example of medical case note and associated images

In this task, we have two experiments. The first run, VIS, uses the visual features of the query image to query the database. The second run, VWF, is the result where the user manually selects the relevant images as positive examples. In the results summary, the mean average precision of the first run VIS of our system is 0.3788. The mean average precision of run2 (VWF) is 0.4474 (refer to appendix).

Figure 2-5 shows the precision and recall graphs.

The results show that the image features we propose can represent the medical image content well. The medical image’s background is very similar. Relevance feedback can extract the dominant features; thus it can improve the performance strongly. This result is better than that of the GIFT system (MAP=0.3791), used as the baseline for medical imageCLEF 2004 [12]. It is also even better than the best work in visual method (MAP=0.4214) [12]. For convenience, we have tabulated them in table 2-1 where ‘1’,’2’,’3’ indicate the top 3 result of the medical imageCLEF 2004 published in [12], respectively.

In this paper we consider that the contrast of a grayscale image dominates human perception. We use a relative normalization method to reduce the impact of illumination. Figure 2-6 is the result of an example query returned by our system. It can be observed from Figure 2-6: that F_10/9719.jpg and F_17/16870.jpg are darker than the query image (def_queries/1.jpg), but our system still can find them out.

Figure 2-7 and Figure 2-8 are some of the query results of our system.

Rank 1 2 3 Ours

MAP 0.4214 0.4189 0.3824 0.4474 Table 2-1 Results reported at imageCLEF2004

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Recall

Precision

VIS VWF

Figure 2-5: Precision vs. Recall graphs without and with feedback. VIS is the result without relevance feedback. VWF is the result with manual feedback

Figure 2-6: Result of an example query ‘Pelvic’

Figure 2-7: Result of example query ‘lung’

Figure 2-8: Result of example query ‘hand’

The first run has accuracy above 50% in the first 20 images. The really similar images may have similar features in some aspect. The misjudged images are always less consistent. So we try to refine the initial result by the automatic feed back mechanism. We cluster the first 20 images into six classes. If the class contains diverse images, the center of the class will become farther, and consequently more different, from the query image. Thus we can improve the result by our feedback method.

In this section we propose several image features to represent medical images.

Although the color histogram of content-based image retrieval methods has good performance in general-purpose color images, unlike general-purpose color images the X-ray images only contain gray level pixels. Thus, we concentrate on the contrast representation of images.

The image representations we propose have obtained good results in this task. Our

representation is immune to defective illumination. A total of 322 features are used.

It is very efficient in computation.

An image represents thousands of words. An image can be viewed from various aspects; furthermore, different people may have different interpretations of the same image. This means that many parameters need to be tuned. In the future, we will try to learn the user behavior and tune those parameters by learning methods.

Chapter 3

Combine Text and Visual Feature for Medical Image Retrieval

Images by their very nature are language independent, but often they are accompanied by texts semantically related to the image (e.g. textual captions or metadata). Images can then be retrieved using primitive features based on pixels with form the contents of an image (e.g. using a visual exemplar), abstracted features expressed through text or a combination of both. The language used to express the associated texts or textual queries should not affect retrieval, i.e. an image with a caption written in English should be searchable in languages other than English.

There is an increasing amount of full text material in various languages available through the Internet and other information suppliers. The world of globalization is increasing, many countries have been unified. The European project to unify European countries is a very important example in order to eliminate broader for the cooperation, global and large market, real international and free business. The high-developed technologies in network infrastructure and Internet set the platform of the cooperation and globalization. Indeed, the business should be global and worldwide oriented. Thus, the issues of the multilinguality arise in order to overcome the remaining technical barriers that still separate countries and cultures.

There are 6,700 languages spoken in 228 countries and English is used as the native language only 6 % of the World population, but English is the dominant language of the collections, resources and services in the Internet. Actually, the

English language is widely used on the Internet. About 60 % of the world online population is represented by English and 30% by European languages [44].

Approximately 147 M people are connected to the Internet (US and Canada about 87 M, Europe 33.25 M, Asia/Pacific 22 M, and Africa 800,000). However, the size of Web sites and Internet users from other countries (non-English countries) is increasing progressively, so that the multilingual problem becomes more and more important.

Recently, a lot of digital libraries will be set up containing large collections of information in a large number of languages. However, it is impractical to submit a query in each language in order to retrieve these multilingual documents. Therefore, a multilingual retrieval environment is essential for benefiting from worldwide information resources. Most research and development activities are focused on only one language. But, this is not the objective of the digital libraries and Internet philosophy. Both technologies aim at establishing a global digital library containing all information resources from different areas, different countries and in different languages.

The access to those materials should be possible for the worldwide community and never be restricted because of non-understanding languages. Thus, the EU funded some projects addressing the multilingual issues. For instance, in the ESPRIT project EMIR (European Multilingual Information Retrieval) a commercial information retrieval system SPIRIT has been developed which supports French, English, German, Dutch and Russian. UNESCO launched some projects in order to democratize and globalize the access to the world cultural patrimony, such as Memory of the World. Recently, UNESCO has started the MEDLIB⁴ project which aims at creating a virtual library for the Mediterranean region.

Cross-language Information retrieval allows user query document in another language. In 1996, ACM SIGIR Workshop for Multilingual Information Retrieval names it as Cross-Language Information Retrieval. Many international conferences focus on Cross-Language information retrieval issues, like as ACL Annual Meeting，ACM SIGIR99 (SIGIR00), etc. The organizations TREC, CLEF, and NTCIR are three major Cross-Language Evaluations to evaluate Cross-language technologies.