M-VAAL: Multimodal Variational Adversarial Active Learning for Downstream Medical Image Analysis Tasks.

Acquiring properly annotated data is expensive in the medical field as it requires experts, time-consuming protocols, and rigorous validation. Active learning attempts to minimize the need for large annotated samples by actively sampling the most informative examples for annotation. These examples contribute significantly to improving the performance of supervised machine learning models, and thus, active learning can play an essential role in selecting the most appropriate information in deep learning-based diagnosis, clinical assessments, and treatment planning.

Improving Medical Image Classification in Noisy Labels Using only Self-supervised Pretraining.

Noisy labels hurt deep learning-based supervised image classification performance as the models may overfit the noise and learn corrupted feature extractors. For natural image classification training with noisy labeled data, model initialization with contrastive self-supervised pretrained weights has shown to reduce feature corruption and improve classification performance. However, no works have explored: i) how other self-supervised approaches, such as pretext task-based pretraining, impact the learning with noisy label, and ii) any self-supervised pretraining methods alone for medical images in noisy label settings.

Investigating Transformer Encoding Techniques to Improve Data-Driven Volume-to-Surface Liver Registration for Image-Guided Navigation.

Due to limited direct organ visualization, minimally invasive interventions rely extensively on medical imaging and image guidance to ensure accurate surgical instrument navigation and target tissue manipulation. In the context of laparoscopic liver interventions, intra-operative video imaging only provides a limited field-of-view of the liver surface, with no information of any internal liver lesions identified during diagnosis using pre-procedural imaging. Hence, to enhance intra-procedural visualization and navigation, the registration of pre-procedural, diagnostic images and anatomical models featuring target tissues to be accessed or manipulated during surgery entails a sufficient accurate registration of the pre-procedural data into the intra-operative setting.

Comparative Analysis of Non-Rigid Registration Techniques for Liver Surface Registration.

Non-rigid surface-based soft tissue registration is crucial for surgical navigation systems, but its adoption still faces several challenges due to the large number of degrees of freedom and the continuously varying and complex surface structures present in the intra-operative data. By employing non-rigid registration, surgeons can integrate the pre-operative images into the intra-operative guidance environment, providing real-time visualization of the patient's complex pre- and intra-operative anatomy in a common coordinate system to improve navigation accuracy. However, many of the existing registration methods, including those for liver applications, are inaccessible to the broader community.

Inpainting surgical occlusion from laparoscopic video sequences for robot-assisted interventions.

Purpose: Medical technology for minimally invasive surgery has undergone a paradigm shift with the introduction of robot-assisted surgery. However, it is very difficult to track the position of the surgical tools in a surgical scene, so it is crucial to accurately detect and identify surgical tools. This task can be aided by deep learning-based semantic segmentation of surgical video frames.

Disparity refinement framework for learning-based stereo matching methods in cross-domain setting for laparoscopic images.

Purpose: Stereo matching methods that enable depth estimation are crucial for visualization enhancement applications in computer-assisted surgery. Learning-based stereo matching methods have shown great promise in making accurate predictions on laparoscopic images. However, they require a large amount of training data, and their performance may be degraded due to domain shifts.

Deformable Image Registration Using Vision Transformers for Cardiac Motion Estimation from Cine Cardiac MRI Images.

Accurate cardiac motion estimation is a crucial step in assessing the kinematic and contractile properties of the cardiac chambers, thereby directly quantifying the regional cardiac function, which plays an important role in understanding myocardial diseases and planning their treatment. Since the cine cardiac magnetic resonance imaging (MRI) provides dynamic, high-resolution 3D images of the heart that depict cardiac motion throughout the cardiac cycle, cardiac motion can be estimated by finding the optical flow representation between the consecutive 3D volumes from a 4D cine cardiac MRI dataset, thereby formulating it as an image registration problem. Therefore, we propose a hybrid convolutional neural network (CNN) and Vision Transformer (ViT) architecture for deformable image registration of 3D cine cardiac MRI images for consistent cardiac motion estimation.

Deep learning architecture for 3D image super-resolution of late gadolinium enhanced cardiac MRI.

Purpose: High-resolution late gadolinium enhanced (LGE) cardiac magnetic resonance imaging (MRI) volumes are difficult to acquire due to the limitations of the maximal breath-hold time achievable by the patient. This results in anisotropic 3D volumes of the heart with high in-plane resolution, but low-through-plane resolution. Thus, we propose a 3D convolutional neural network (CNN) approach to improve the through-plane resolution of the cardiac LGE-MRI volumes.

Learning Deep Representations of Cardiac Structures for 4D Cine MRI Image Segmentation through Semi-Supervised Learning.

Learning good data representations for medical imaging tasks ensures the preservation of relevant information and the removal of irrelevant information from the data to improve the interpretability of the learned features. In this paper, we propose a semi-supervised model-namely, combine-all in semi-supervised learning (CSL)-to demonstrate the power of a simple combination of a disentanglement block, variational autoencoder (VAE), generative adversarial network (GAN), and a conditioning layer-based reconstructor for performing two important tasks in medical imaging: segmentation and reconstruction. Our work is motivated by the recent progress in image segmentation using semi-supervised learning (SSL), which has shown good results with limited labeled data and large amounts of unlabeled data.

Toward a Quasi-dynamic Pulsed Field Electroporation Numerical Model for Cardiac Ablation: Predicting Tissue Conductance Changes and Ablation Lesion Patterns.

Pulsed field ablation (PFA) has the potential to evolve into an efficient alternative to traditional RF ablation for atrial fibrillation treatment. However, achieving irreversible tissue electroporation is critical to suppressing arrhythmic pathways, raising the need for accurate lesion characterization. To understand the physics behind the tissue response PFA, we propose a quasi-dynamic model that quantifies tissue conductance at end-electroporation and identifies regions that have undergone fully irreversible electroporation (IRE).

A Disparity Refinement Framework for Learning-based Stereo Matching Methods in Cross-domain Setting for Laparoscopic Images.

Stereo matching methods that enable depth estimation are crucial for visualization enhancement applications in computer-assisted surgery (CAS). Learning-based stereo matching methods are promising to predict accurate results for applications involving video images. However, they require a large amount of training data, and their performance may be degraded due to domain shifts.

Developing and Evaluating the Fidelity of Patient Specific Kidney Emulating Phantoms for Image-guided Intervention Applications.

Patient specific organ and tissue mimicking phantoms are used routinely to develop and assess new image-guided intervention tools and techniques in laboratory settings, enabling scientists to maintain acceptable anatomical relevance, while avoiding animal studies when the developed technology is still in its infancy. Gelatin phantoms, specifically, offer a cost-effective and readily available alternative to the traditional manufacturing of anatomical phantoms, and also provide the necessary versatility to mimic various stiffness properties specific to various organs or tissues. In this study, we describe the protocol to develop patient specific anthropomorphic gelatin kidney phantoms and we also assess the faithfulness of the developed phantoms against the patient specific CT images and corresponding virtual anatomical models used to generate the phantoms.

A Deep Learning Framework to Estimate Elastic Modulus from Ultrasound Measured Displacement Fields.

Ultrasound (US) elastography is a technique that enables non-invasive quantification of material properties, such as stiffness, from ultrasound images of deforming tissue. The displacement field is measured from the US images using image matching algorithms, and then a parameter, often the elastic modulus, is inferred or subsequently measured to identify potential tissue pathologies, such as cancerous tissues. Several traditional inverse problem approaches, loosely grouped as either direct or iterative, have been explored to estimate the elastic modulus.

Investigating the impact of class-dependent label noise in medical image classification.

Label noise is inevitable in medical image databases developed for deep learning due to the inter-observer variability caused by the different levels of expertise of the experts annotating the images, and, in some cases, the automated methods that generate labels from medical reports. It is known that incorrect annotations or label noise can degrade the actual performance of supervised deep learning models and can bias the model's evaluation. Existing literature show that noise in one class has minimal impact on the model's performance for another class in natural image classification problems where different target classes have a relatively distinct shape and share minimal visual cues for knowledge transfer among the classes.

Learning feature descriptors for pre- and intra-operative point cloud matching for laparoscopic liver registration.

Purpose: In laparoscopic liver surgery, preoperative information can be overlaid onto the intra-operative scene by registering a 3D preoperative model to the intra-operative partial surface reconstructed from the laparoscopic video. To assist with this task, we explore the use of learning-based feature descriptors, which, to our best knowledge, have not been explored for use in laparoscopic liver registration. Furthermore, a dataset to train and evaluate the use of learning-based descriptors does not exist.

A Feature-based Affine Registration Method for Capturing Background Lung Tissue Deformation for Ground Glass Nodule Tracking.

Lung nodule tracking assessment relies on cross-sectional measurements of the largest lesion profile depicted in initial and follow-up computed tomography (CT) images. However, apparent changes in nodule size assessed via simple image-based measurements may also be compromised by the effect of the background lung tissue deformation on the GGN between the initial and follow-up images, leading to erroneous conclusions about nodule changes due to disease. To compensate for the lung deformation and enable consistent nodule tracking, here we propose a feature-based affine registration method and study its performance vis-a-vis several other registration methods.

SPIE Medical Imaging 50th anniversary: historical review of the Image-Guided Procedures, Robotic Interventions, and Modeling conference.

Purpose: Among the conferences comprising the Medical Imaging Symposium is the MI104 conference currently titled Image-Guided Procedures, Robotic Interventions, and Modeling, although its name has evolved through at least nine iterations over the last 30 years. Here, we discuss the important role that this forum has presented for researchers in the field during this time.

Approach: The origins of the conference are traced from its roots in Image Capture and Display in the late 1980s, and some of the major themes for which the conference and its proceedings have provided a valuable forum are highlighted.

A 3D Convolutional Neural Network with Gradient Guidance for Image Super-Resolution of Late Gadolinium Enhanced Cardiac MRI.

In this paper, we describe a 3D convolutional neural network (CNN) framework to compute and generate super-resolution late gadolinium enhanced (LGE) cardiac magnetic resonance imaging (MRI) images. The proposed CNN framework consists of two branches: a super-resolution branch with a 3D dense deep back-projection network (DBPN) as the backbone to learn the mapping of low-resolution LGE cardiac volumes to high-resolution LGE cardiac volumes, and a gradient branch that learns the mapping of the gradient map of low resolution LGE cardiac volumes to the gradient map of their high-resolution counterparts. The gradient branch of the CNN provides additional cardiac structure information to the super-resolution branch to generate structurally more accurate super-resolution LGE MRI images.

Endoscope Localization and Dense Surgical Scene Reconstruction for Stereo Endoscopy by Unsupervised Optical Flow and Kanade-Lucas-Tomasi Tracking.

In image-guided surgery, endoscope tracking and surgical scene reconstruction are critical, yet equally challenging tasks. We present a hybrid visual odometry and reconstruction framework for stereo endoscopy that leverages unsupervised learning-based and traditional optical flow methods to enable concurrent endoscope tracking and dense scene reconstruction. More specifically, to reconstruct texture-less tissue surfaces, we use an unsupervised learning-based optical flow method to estimate dense depth maps from stereo images.

Investigating uncertainty in augmented reality enhanced renal navigation using in vitro patient-specific tissue-mimicking phantoms.

To improve the outcome of minimally invasive renal interventions, traditional video-guided needle navigation can be enhanced by tracking the needle, guiding the needle using video imaging, and augmenting the surgical scene with pre-procedural images or models of the anatomy. In our previous work we studied, both through simulations and in vitro experiments, the uncertainty associated with the model-to-phantom registration, as well as the camera-tracker calibration and video-guided navigation. In this work, we characterize the overall navigation uncertainty using tissue emulating patient-specific kidney phantoms featuring both virtual and physical internal targets.

Joint Segmentation and Uncertainty Estimation of Ventricular Structures from Cardiac MRI using a Bayesian CondenseUNet.

While convolutional neural networks (CNNs) have shown potential in segmenting cardiac structures from magnetic resonance (MR) images, their clinical applications still fall short of providing reliable cardiac segmentation. As a result, it is critical to quantify segmentation uncertainty in order to identify which segmentations might be troublesome. Moreover, quantifying uncertainty is critical in real-world scenarios, where input distributions are frequently moved from the training distribution due to sample bias and non-stationarity.

A Weakly Supervised Learning Approach for Surgical Instrument Segmentation from Laparoscopic Video Sequences.

Fully supervised learning approaches for surgical instrument segmentation from video images usually require a time-consuming process of generating accurate ground truth segmentation masks. We propose an alternative way of labeling surgical instruments for binary segmentation that first commences with rough, scribble-like annotations of the surgical instruments using a disc-shaped brush. We then present a framework that starts with a graph-model-based method for generating initial segmentation labels based on the user-annotated paint-brush scribbles and then proceeds with a deep learning model that learns from the noisy, initial segmentation labels.

A Deep Learning Framework for Image Super-Resolution for Late Gadolinium Enhanced Cardiac MRI.

Cardiac magnetic resonance imaging (MRI) provides 3D images with high-resolution in-plane information, however, they are known to have low through-plane resolution due to the trade-off between resolution, image acquisition time and signal-to-noise ratio. This results in anisotropic 3D images which could lead to difficulty in diagnosis, especially in late gadolinium enhanced (LGE) cardiac MRI, which is the reference imaging modality for locating the extent of myocardial fibrosis in various cardiovascular diseases like myocardial infarction and atrial fibrillation. To address this issue, we propose a self-supervised deep learning-based approach to enhance the through-plane resolution of the LGE MRI images.