Thomas M. Deserno, RWTH Aachen University, Germany
L. Rodney Long, National Library of Medicine, NIH, USA
Hayit Greenspan, Tel Aviv University, Israel
In most biomedical disciplines, digital image data is rapidly expanding in quantity and heterogeneity, and there is an increasing trend towards the formation of archives adequate to support diagnostics and preventive medicine. Exploration, exploitation, and consolidation of the immense image collections require tools to access structurally different data for research, diagnostics and teaching. Currently, image data is linked to textual descriptions, and data access is provided only via these textual additives. There are virtually no tools available to access medical images directly by their content or to cope with their structural differences. Therefore, visualbased (i.e. content-based) indexing and retrieval based on information contained in the pixel data of biomedical images is expected to have a great impact on biomedical image databases. However, existing systems for content-based image retrieval (CBIR) are not applicable to the biomedical imagery special needs, and novel methodologies are urgently needed.
This special issue grew from the workshop, Content-Based Image Retrieval: Major Challenges for Medical Applications at SPIE's International Symposium on Medical Imaging 2008 (Content-Based, 2008), which was convened to assess status of CBIR within the biomedical clinical and research worlds, and to collect opinion from leading CBIR researchers about the most productive way forward. The workshop was structured around the concept of "gaps" (Deserno, Antani, & Long, 2008) between desired capabilities and use for medical CBIR, and what has actually been realized.
Panel participants in the workshop were asked to comment about a particular category of CBIR gap-content, features, performance, or usability-and also to provide informed opinion about critical gating factors affecting the use of CBIR in medicine (see (Deserno, et al., 2008) or the article by Long et al. in this issue for a discussion of these categories). Most of the articles in this issue are authored by panel participants and reflect this theme of "bridging gaps" in existing biomedical CBIR implementations.
Audience participants contributed a number of valuable comments, including one extended critique (see Acknowledgment) which we believe was particularly useful; we take the liberty to paraphrase it as "for CBIR to be valuable in medicine, know your data-images and related pathology and patient data-thoroughly; and incorporate the totality of this information into your system". This same point is made independently and with elaboration in the article of Müller & Kalpathy-Cramer (in this issue).
In total, six articles that have passed the international review process were collected within this special issue. Brief introductions to each of the articles are given below.
Long et al. have provided an overview of the gap approach, illustrate its use in comparing four implemented CBIR systems, and argue for more attention in the CBIR research community to the areas of performance and usability, based on the relative inattention to these areas in the past, as measured by a survey of technical literature over the past seven years.
Horsthemke et al. contribute to understanding medical CBIR systems' semantic content and evaluation of performance with a focused investigation of mapping low-level mathematical shape-related features, including Fourier descriptors and a novel Radial Normal Index feature that they introduce, to the semantic features of spiculation, lobulation, and sphericity that are used by radiologists to describe shape of nodules in lung images. In their evaluation of the ability of the low-level features to predict the semantic interpretations of radiologists, the authors emphasize the significance of high variability of medical expert interpretation in establishing a reference "truth" standard for judging the success for algorithmic approaches in medicine.
Kim et al. describe two prototype applications for content-based image retrieval in a highdimensional feature spaces. The first application is in 4D dynamic PET, and the second is dualmodal PET/CT images. These multi-dimensional imaging modalities are requiring new innovative techniques for content-based retrieval to fully utilize the massive amount of information in a meaningful and practical manner. The authors summarize research in bridging the feature gap for retrieval of multi-dimensional biomedical images.
Traina et al. focus on the performance gap in CBIR, including considerations that affect the user not only in perceived time of system response, but in effectiveness of the response in providing relevant information to satisfy the user's information goal. For example, they discuss techniques to reduce dimensionality of feature vectors not only with the goal of reducing search time, but also to improve returned results as measured by precision vs. recall evaluations. In addition, they discuss and provide results to illustrate performance gains possible by database indexing of high-dimensional feature vectors, by analysis and optimization of query components, and by iterative relevance feedback from the user.
Greenspan addresses technical approaches to feature representation of image content and also the significant issue of representing not only global, but also local, region-of-interest content in medical images. She provides a detailed discussion of medical image retrieval with two approaches: (1) image landmark retrieval, where images are first segmented and key regions labeled; then the images are retrieved by labels; and (2) image-to-image retrieval, where an image is used as part or all of the query. She illustrates recent image content representation by the Gaussian Mixture Modeling-Kullback- Leibler (GMM-KL) method, as well as image "codebook" representation by image patches.
Müller & Kalpathy-Cramer specifically address the content gap in CBIR, contrasting the significance of purely visual image queries, which has been the main emphasis of biomedical CBIR researchers, versus queries that incorporate as much as possible of the complete patient context. They argue for the latter as a critical factor in advancing the use of CBIR in medicine. They contend that more than knowledge of imaging modality and viewing angle are required; rather, numerous case-specific contextual factors may be necessary to make images truly useful, and CBIR systems should address the need to incorporate these factors. Important factors that they provide for illustration in particular cases include patient age, medical goal of the image acquisition, and patient radiation treatment history.
In this issue we have provided expert responses for addressing specific problematic concerns in the field of medical CBIR, rather than a comprehensive survey. For such a survey, we refer the reader to the 2004 work of Müller, Michoux, Bandon, and Geissbuhler (2004). To those who have helped make this special issue possible by the contribution of their technical articles, and to all those researchers throughout the world who work to make and share progress in this highly challenging, sometimes frustrating, and continually exciting enterprise of biomedical Content-based Image Retrieval, we express our deep gratitude.
We thank Dr. Shih-Chung Ben Lo of the ISIS Center, Georgetown University, Washington, DC, for his thorough and helpful comments from the audience during the workshop.
Thomas M. Deserno (born as Lehmann) received the diploma (masters degree) in electrical engineering, the PhD (summa cum laude) in computer science, and the habilitation in medical informatics from the RWTH Aachen University, Aachen, Germany, in 1992, 1998, and 2004, respectively. Since 2007, he is full professor for medical informatics. His research interests include medical image processing applied to quantitative measurements for computer-assisted diagnoses and content-based image retrieval from large medical databases. He currently serves on the international editorial board of Dentomaxillofacial Radiology as well as Methods of Information in Medicine, and he is co-editor of Europe of the International Journal of Healthcare Information Systems and Informatics.
L. Rodney Long is an electronics engineer for the Communications Engineering Branch at the National Library of Medicine, where he has worked since 1990. Prior to his current job, he worked for 14 years in industry as a software developer and as a systems engineer. His research interests are in telecommunications, image processing, and scientific/biomedical databases. He has an MA in applied mathematics from the University of Maryland.
Hayit Greenspan received the BSc and MSc degrees from the electrical engineering department of the Technion, Israel, in 1986 and 1989, respectively, and the PhD degree from the Electrical Engineering Dept. at CALTECH—California Institute of Technology, in 1994. Following the PhD she was a postdoc at the Computer Science Division at U.C. Berkeley. In 1997 she joined the Biomedical Engineering Dept. of the Faculty of Engineering, Tel-Aviv University, Israel, where she is currently a faculty member. Her research interests include medical image processing and analysis, content-based image and video search and retrieval, statistical image modeling and segmentation.