Multi-Modal Understanding and Generation for Medical Images and Text via Vision-Language Pre-Training.

Recently a number of studies demonstrated impressive performance on diverse vision-language multi-modal tasks such as image captioning and visual question answering by extending the BERT architecture with multi-modal pre-training objectives. In this work we explore a broad set of multi-modal representation learning tasks in the medical domain, specifically using radiology images and the unstructured report. We propose Medical Vision Language Learner (MedViLL), which adopts a BERT-based architecture combined with a novel multi-modal attention masking scheme to maximize generalization performance for both vision-language understanding tasks (diagnosis classification, medical image-report retrieval, medical visual question answering) and vision-language generation task (radiology report generation).

Computer-aided recognition of myopic tilted optic disc using deep learning algorithms in fundus photography.

Background: It is necessary to consider myopic optic disc tilt as it seriously impacts normal ocular parameters. However, ophthalmologic measurements are within inter-observer variability and time-consuming to get. This study aimed to develop and evaluate deep learning models that automatically recognize a myopic tilted optic disc in fundus photography.