Scalable medical image understanding by fusing cross-modal object recognition with formal domain semantics

Recent advances in medical imaging technology have dramatically increased the amount of clinical image data. In contrast, techniques for efficiently exploiting the rich semantic information in medical images have evolved much slower. Despite the research outcomes in image understanding, current image databases are still indexed by manually assigned subjective keywords instead of the semantics of the images. Indeed, most current content-based image search applications index image features that do not generalize well and use inflexible queries. This slow progress is due to the lack of scalable and generic information representation systems which can abstract over the high dimensional nature of medical images as well as semantically model the results of object recognition techniques. We propose a system combining medical imaging information with ontological formalized semantic knowledge that provides a basis for building universal knowledge repositories and gives clinicians fully cross-lingual and cross-modal access to biomedical information.