Organization of This Dissertation

The remainder of this thesis is organized as follows. In Chapter 2, we review previous related visual medical image researches and proposed an efficient representation correspond to user’s viewpoint and similar matching metric. In Chapter 3, we combine text and visual feature to improve the accuracy of query

result and propose an ontology method to conquer the ambiguity problem of cross language retrieval. In Chapter 4, relevance feedback methods were surveyed and a novel feedback mechanism is proposed. Last, the conclusion and further works is described in Chapter 5.

Chapter 2

Content Based Medical Image Retrieval

The number of digitally produced medical images is rising strongly. In the radiology department of the University Hospital of Geneva alone, the number of images produced per day in 2002 was 12,000, and it is still rising. The management and the access to these large image repositories are increasing the need for tools that effectively filter and efficiently search through large amounts of visual data. Most access to these systems is based on the patient identification or study characteristics (modality, study description) [35] as it is also defined in the DICOM standard [52].

Imaging systems and image archives have often been described as an important economic and clinical factor in the hospital environment [22]. Several methods from computer vision and image processing have already been proposed for the use in medicine [51]. Medical images have often been used for retrieval systems, and the medical domain is often cited as one of the principal application domains for content-based access technologies [6] [32] [48] [64] in terms of potential impact.

Two exceptions seem to be the Assert system on the classification of high resolution CTs of the lung [1] [63] and the Image Retrieval in Medical Applications (IRMA) system for the classification of images into anatomical areas, modalities and view points [33]. Assert system allow the physician to circles one or more Pathology Bearing Regions (PBR) in the query image. The system then retrieves the n most visually similar images from the database using an index comprised of a combination of localized features of the PBRs and of the global image. The database consists of High Resolution Computed Tomography of the lung. CBIR is particularly needed for this domain because the current state of the art in diagnosis,

for those cases not immediately recognizable, is to consult a published atlas of lung pathologies. Assert system saves the radiologist from the laborious task of paging through the atlas looking for an image that matches the pathology of their current patient.

Image Retrieval in Medical Applications (IRMA) is a cooperative project of the Department of Diagnostic Radiology, the Department of Medical Informatics, Division of Medical Image Processing and the Chair of Computer Science VI at the Aachen University of Technology. Aim of the project is the development and implementation of high-level methods for content-based image retrieval with prototypical application to medico-diagnostic tasks on a radiological image archive.

They want to perform semantic and formalized queries on the medical image database which includes intra- and inter-individual variance and diseases. Example tasks are the staging of a patient's therapy or the retrieval of images with similar diagnostic findings in large electronic archives. Formal content-based queries also take into account the technical conditions of the examination and the image acquisition modalities.

The system ought to classify and register radiological images in a general way without restriction to a certain diagnostic problem or question. Methods of pattern recognition and structural analysis are used to describe the image content in a feature based, formal and generalized way. The formalized and normalized description of the images is then used as a mean to compare images in the archive which allows a fast and reliable retrieval.

In addition to the queries on an existing electronic archive, the automatic classification and indexing allows a simple insertion of conventional radiographs into the system. Automated classification of radiographs based on global features with respect to imaging modality, direction, body region examined and biological

system under investigation.

Identification of image features that are relevant for medical diagnosis these features are derived from a-priori classified and registered images. The resulting system must retrieve images similar to a query image with respect to a selected set of features. These features can, for example, be based on the visual similarity of certain image structures. Figure 2-1 is a concept view of content based image retrieval system.

Indexing System

Text and Image Data

GUI Image

Representation &

Landmark

Calculate Feature

Feature Vector

Build RDB Table

Build FV Tree

Feature Vector Organization Collect Text Data

Figure 2-1: A concept view of content based image retrieval system

Content-based retrieval has also been proposed several times from the medical community for the inclusion into various applications [7] [69], often without any implementation. Still for a real medical application of content-based retrieval methods and the integration of these tools into medical practice a very close cooperation between the two fields is necessary for a longer period of time and not simply an exchange of data or a list of the necessary functionality.

Most existing content-based image retrieval (CBIR) systems rely on computing a global signature for each image based on color, texture and shape information [65].

However, medical image usually consists of gray level variations in highly localized regions in the image. Thus, the content-based image retrieval method in color based can’t apply to medical images directly. In the gray level images, the contrast of gray level is more important than illuminate in human’s viewpoint. In this dissertation, we consider the human’s conceptual and we design a wavelet-based image representation method. Wavelets have proven their efficiency in image retrieval, for their capability in capturing texture and shape information [1] [42] [49]. Making use of wavelet sub-band decomposition, relevant information about the structure of data can be computed, which can serve as a low dimensional feature vector. A wavelet-based medical image retrieval system, which is mainly based on textural information, has shown satisfactory results in retrieving HRCT lung images [31].

One of the most significant problems in content-based image retrieval results from the lack of a common test-bed for researchers. Although many published articles report on content-based retrieval results using color photographs, there has been little effort in establishing a benchmark set of images and queries. It is very important that image databases are made available free of charge for the comparison and verification of algorithms. Only such reference databases allow comparing systems and to have a reference for the evaluation that is done based on the same images. ImageCLEF¹ [12] offers numerous medical images for evaluation and has many benefits in advancing the technology and utilization of content-based image retrieval systems.

Compared to text retrieval little is known about how to search for images, although it has been an extremely active domain in the fields of computer vision and information retrieval [41][56][64][69]. Benchmarks such as ImageCLEF [12] allow evaluating algorithms compared to other systems and deliver insights into the

1 http://clef.isti.cnr.it/

techniques that perform well. They offered a default CBIR system that is GIFT². This software tool is open source and can be used by other participants of ImageCLEF.

The feature sets that are used by GIFT are:

−Local color features at different scales by partitioning the images successively into four equally sized regions (four times) and taking the mode color of each region as a descriptor;

−Global color features in the form of a color histogram, compared by a simple histogram intersection;

−Local texture features by partitioning the image and applying Gabor filters in various scales and directions, quantized into 10 strengths;

−Global texture features represented as a simple histogram of responses of the local Gabor filters in various directions and scales.

In this dissertation we want to study a medical image retrieval system that contains variety of type images for clinical student to learning or patient to understand his condition. The image contains variety of type image, thus we consider global and local image features expect to describe variety of type images.

We based on the wavelet coefficient to extract dominate gray value and texture information. For efficient retrieving we design several new representations of features. The following is our proposed feature representation methods.