Artificial intelligence in cardiovascular imaging and intervention.

Recent progress in artificial intelligence (AI) includes generative models, multimodal foundation models, and federated learning, which enable a wide spectrum of novel exciting applications and scenarios for cardiac image analysis and cardiovascular interventions. The disruptive nature of these novel technologies enables concurrent text and image analysis by so-called vision-language transformer models. They not only allow for automatic derivation of image reports, synthesis of novel images conditioned on certain textual properties, and visual questioning and answering in an oral or written dialogue style, but also for the retrieval of medical images from a large database based on a description of the pathology or specifics of the dataset of interest.

Surgical phase and instrument recognition: how to identify appropriate dataset splits.

Purpose: Machine learning approaches can only be reliably evaluated if training, validation, and test data splits are representative and not affected by the absence of classes. Surgical workflow and instrument recognition are two tasks that are complicated in this manner, because of heavy data imbalances resulting from different length of phases and their potential erratic occurrences. Furthermore, sub-properties like instrument (co-)occurrence are usually not particularly considered when defining the split.

Surgical training of minimally invasive mitral valve repair on a patient-specific simulator improves surgical skills.

Objectives: Minimally invasive mitral valve repair (MVR) is considered one of the most challenging operations in cardiac surgery and requires much practice and experience. Simulation-based surgical training might be a method to support the learning process and help to flatten the steep learning curve of novices. The purpose of this study was to show the possible effects on learning of surgical training using a high-fidelity simulator with patient-specific mitral valve replicas.

Comparative evaluation of three commercially available markerless depth sensors for close-range use in surgical simulation.

Purpose: Minimally invasive surgeries have restricted surgical ports, demanding a high skill level from the surgeon. Surgical simulation potentially reduces this steep learning curve and additionally provides quantitative feedback. Markerless depth sensors show great promise for quantification, but most such sensors are not designed for accurate reconstruction of complex anatomical forms in close-range.

Surgical Rehearsal for Mitral Valve Repair: Personalizing Surgical Simulation by 3D Printing.

Purpose: The goal of this study was to show possible effects of performing the actual procedure of mitral valve repair (MVR) on personalized silicone models 1 day before operation.

Description: Based on preoperative 3-dimensional echocardiography recordings, flexible 3-dimensional replicas of the depicted pathologic mitral valves could be produced and used for a simulation of reconstructive techniques analogous to the upcoming MVR procedure. We integrated this step of personalized surgical planning into the clinical routine of 6 MVR cases with 3 different surgeons.

Point detection through multi-instance deep heatmap regression for sutures in endoscopy.

Purpose: Mitral valve repair is a complex minimally invasive surgery of the heart valve. In this context, suture detection from endoscopic images is a highly relevant task that provides quantitative information to analyse suturing patterns, assess prosthetic configurations and produce augmented reality visualisations. Facial or anatomical landmark detection tasks typically contain a fixed number of landmarks, and use regression or fixed heatmap-based approaches to localize the landmarks.

Mutually Improved Endoscopic Image Synthesis and Landmark Detection in Unpaired Image-to-Image Translation.

The CycleGAN framework allows for unsupervised image-to-image translation of unpaired data. In a scenario of surgical training on a physical surgical simulator, this method can be used to transform endoscopic images of phantoms into images which more closely resemble the intra-operative appearance of the same surgical target structure. This can be viewed as a novel augmented reality approach, which we coined Hyperrealism in previous work.