Medical image understanding through the integration of cross-modal object recognition with formal domain knowledge

Rapid advances in medical imaging scanner technology have increased dramatically in the last decade the amount of medical image data generated every day. By contrast, the software technology that would allow the efficient exploitation of the highly informational content of medical images has evolved much slower. Despite the research outcomes in image understanding and semantic modeling, current image databases are still indexed by keywords assigned by humans and not by the image content. The reason for this slow progress is the lack of scalable and generic information representations capable of overcoming the high-dimensional nature of image data. Indeed, most of the current content-based image search applications are focused on the indexing of certain image features that do not generalize well and use inflexible queries. We propose a system combining medical imaging information with semantic background knowledge from formalized ontologies, that provides a basis for building universal knowledge repositories, giving clinicians a fully cross-lingual and cross-modal access to biomedical information of all forms.