Shape completion in the dark: completing vertebrae morphology from 3D ultrasound.

Purpose: Ultrasound (US) imaging, while advantageous for its radiation-free nature, is challenging to interpret due to only partially visible organs and a lack of complete 3D information. While performing US-based diagnosis or investigation, medical professionals therefore create a mental map of the 3D anatomy. In this work, we aim to replicate this process and enhance the visual representation of anatomical structures.

From quantitative metrics to clinical success: assessing the utility of deep learning for tumor segmentation in breast surgery.

Purpose: Preventing positive margins is essential for ensuring favorable patient outcomes following breast-conserving surgery (BCS). Deep learning has the potential to enable this by automatically contouring the tumor and guiding resection in real time. However, evaluation of such models with respect to pathology outcomes is necessary for their successful translation into clinical practice.

Real-time integration between Microsoft HoloLens 2 and 3D Slicer with demonstration in pedicle screw placement planning.

Purpose: Up to date, there has been a lack of software infrastructure to connect 3D Slicer to any augmented reality (AR) device. This work describes a novel connection approach using Microsoft HoloLens 2 and OpenIGTLink, with a demonstration in pedicle screw placement planning.

Methods: We developed an AR application in Unity that is wirelessly rendered onto Microsoft HoloLens 2 using Holographic Remoting.

The use of 3D digital anatomy model improves the communication with patients presenting with prostate disease: The first experience in Senegal.

Objectives: We hypothesized that the use of an interactive 3D digital anatomy model can improve the quality of communication with patients about prostate disease.

Methods: A 3D digital anatomy model of the prostate was created from an MRI scan, according to McNeal's zonal anatomy classification. During urological consultation, the physician presented the digital model on a computer and used it to explain the disease and available management options.

Use of Machine Learning to Assess Cataract Surgery Skill Level With Tool Detection.

Purpose: To develop a method for objective analysis of the reproducible steps in routine cataract surgery.

Design: Prospective study; machine learning.

Participants: Deidentified faculty and trainee surgical videos.

Sensor-Based Automated Detection of Electrosurgical Cautery States.

In computer-assisted surgery, it is typically required to detect when the tool comes into contact with the patient. In activated electrosurgery, this is known as the . By continuously tracking the electrosurgical tools' location using a navigation system, energy events can help determine locations of sensor-classified tissues.

Bridging 3D Slicer and ROS2 for Image-Guided Robotic Interventions.

Developing image-guided robotic systems requires access to flexible, open-source software. For image guidance, the open-source medical imaging platform 3D Slicer is one of the most adopted tools that can be used for research and prototyping. Similarly, for robotics, the open-source middleware suite robot operating system (ROS) is the standard development framework.

Real-time automatic tumor segmentation for ultrasound-guided breast-conserving surgery navigation.

Purpose: Ultrasound-based navigation is a promising method in breast-conserving surgery, but tumor contouring often requires a radiologist at the time of surgery. Our goal is to develop a real-time automatic neural network-based tumor contouring process for intraoperative guidance. Segmentation accuracy is evaluated by both pixel-based metrics and expert visual rating.

Training in soft tissue resection using real-time visual computer navigation feedback from the Surgery Tutor: A randomized controlled trial.

Background: In competency-based medical education, surgery trainees are often required to learn procedural skills in a simulated setting before proceeding to the clinical environment. The Surgery Tutor computer navigation platform allows for real-time proctor-less assessment of open soft tissue resection skills; however, the use of this platform as an aid in acquisition of procedural skills is yet to be explored.

Methods: In this prospective randomized controlled trial, 20 final year medical students were randomized to receive either training with real-time computer navigation feedback (Intervention, n = 10) or simulation training without navigation feedback (Control, n = 10) during resection of simulated non-palpable soft tissue tumors.

System for Central Venous Catheterization Training Using Computer Vision-Based Workflow Feedback.

Objective: To develop a system for training central venous catheterization that does not require an expert observer. We propose a training system that uses video-based workflow recognition and electromagnetic tracking to provide trainees with real-time instruction and feedback.

Methods: The system provides trainees with prompts about upcoming tasks and visual cues about workflow errors.

Design of an Ultrasound-Navigated Prostate Cancer Biopsy System for Nationwide Implementation in Senegal.

This paper presents the design of NaviPBx, an ultrasound-navigated prostate cancer biopsy system. NaviPBx is designed to support an affordable and sustainable national healthcare program in Senegal. It uses spatiotemporal navigation and multiparametric transrectal ultrasound to guide biopsies.

Three-dimensional ultrasound for knee osteophyte depiction: a comparative study to computed tomography.

Purpose: Osteophytes are common radiographic markers of osteoarthritis. However, they are not accurately depicted using conventional imaging, thus hampering surgical interventions that rely on pre-operative images. Studies have shown that ultrasound (US) is promising at detecting osteophytes and monitoring the progression of osteoarthritis.

Electromagnetic (EM) catheter path tracking in ultrasound-guided brachytherapy of the breast.

Purpose: To evaluate a novel navigation system for breast brachytherapy, based on ultrasound (US)-guided catheter needle implantations followed by electromagnetic (EM) tracking of catheter paths.

Methods: Breast phantoms were produced, containing US-visible tumors. Ultrasound was used to localize the tumor pose and volume within the phantom, followed by planning an optimal catheter pattern through the tumor using navigation software.

Navigated tissue characterization during skin cancer surgery.

Purpose: Basal cell carcinoma (BCC) is the most commonly diagnosed skin cancer and is treated by surgical resection. Incomplete tumor removal requires surgical revision, leading to significant healthcare costs and impaired cosmesis. We investigated the clinical feasibility of a surgical navigation system for BCC surgery, based on molecular tissue characterization using rapid evaporative ionization mass spectrometry (REIMS).

Automatic Spine Ultrasound Segmentation for Scoliosis Visualization and Measurement.

Objective: Integrate tracked ultrasound and AI methods to provide a safer and more accessible alternative to X-ray for scoliosis measurement. We propose automatic ultrasound segmentation for 3-dimensional spine visualization and scoliosis measurement to address difficulties in using ultrasound for spine imaging.

Methods: We trained a convolutional neural network for spine segmentation on ultrasound scans using data from eight healthy adult volunteers.

Real-time electromagnetic navigation for breast-conserving surgery using NaviKnife technology: A matched case-control study.

Breast-conserving surgery (BCS) is a mainstay in breast cancer treatment. For nonpalpable breast cancers, current strategies have limited accuracy, contributing to high positive margin rates. We developed NaviKnife, a surgical navigation system based on real-time electromagnetic (EM) tracking.

Machine learning methods for automated technical skills assessment with instructional feedback in ultrasound-guided interventions.

Objective: Currently, there is a worldwide shift toward competency-based medical education. This necessitates the use of automated skills assessment methods during self-guided interventions training. Making assessment methods that are transparent and configurable will allow assessment to be interpreted into instructional feedback.

Surgery Tutor for Computational Assessment of Technical Proficiency in Soft-Tissue Tumor Resection in a Simulated Setting.

Background: In competency-based medical education, progression between milestones requires reliable and valid methods of assessment. Surgery Tutor is an open-source motion tracking platform developed to objectively assess technical proficiency during open soft-tissue tumor resections in a simulated setting. The objective of our study was to provide evidence in support of construct validity of the scores obtained by Surgery Tutor.

Self-guided training for deep brain stimulation planning using objective assessment.

Objective: Deep brain stimulation (DBS) is an increasingly common treatment for neurodegenerative diseases. Neurosurgeons must have thorough procedural, anatomical, and functional knowledge to plan electrode trajectories and thus ensure treatment efficacy and patient safety. Developing this knowledge requires extensive training.

Objective assessment of colonoscope manipulation skills in colonoscopy training.

Objective: Manipulation of the colonoscope is a technical challenge for novice clinicians which is best learned in a simulated environment. It involves the coordination of scope tip steering with scope insertion, using a rotated image as reference. The purpose of this work is to develop and validate a system which objectively assesses colonoscopy technical skills proficiency in an arbitrary training environment, allowing novices to assess their technical proficiency prior to real patient encounters.

Utility of 3D Reconstruction of 2D Liver Computed Tomography/Magnetic Resonance Images as a Surgical Planning Tool for Residents in Liver Resection Surgery.

Objective: A fundamental aspect of surgical planning in liver resections is the identification of key vessel tributaries to preserve healthy liver tissue while fully resecting the tumor(s). Current surgical planning relies primarily on the surgeon's ability to mentally reconstruct 2D computed tomography/magnetic resonance (CT/MR) images into 3D and plan resection margins. This creates significant cognitive load, especially for trainees, as it relies on image interpretation, anatomical and surgical knowledge, experience, and spatial sense.

1.5 T augmented reality navigated interventional MRI: paravertebral sympathetic plexus injections.

Purpose: The high contrast resolution and absent ionizing radiation of interventional magnetic resonance imaging (MRI) can be advantageous for paravertebral sympathetic nerve plexus injections. We assessed the feasibility and technical performance of MRI-guided paravertebral sympathetic injections utilizing augmented reality navigation and 1.5 T MRI scanner.

Model-based registration of preprocedure MR and intraprocedure US of the lumbar spine.

Purpose: Epidural and spinal needle insertions, as well as facet joint denervation and injections are widely performed procedures on the lumbar spine for delivering anesthesia and analgesia. Ultrasound (US)-based approaches have gained popularity for accurate needle placement, as they use a non-ionizing, inexpensive and accessible modality for guiding these procedures. However, due to the inherent difficulties in interpreting spinal US, they yet to become the clinical standard-of-care.

Development and Evaluation of a Simulation-based Curriculum for Ultrasound-guided Central Venous Catheterization.

Objective: To develop a simulation-based curriculum for residents to learn ultrasound-guided (USG) central venous catheter (CVC) insertion, and to study the volume and type of practice that leads to technical proficiency.

Methods: Ten post-graduate year two residents from the Departments of Emergency Medicine and Anesthesiology completed four training sessions of two hours each, at two week intervals, where they engaged in a structured program of deliberate practice of the fundamental skills of USG CVC insertion on a simulator. Progress during training was monitored using regular hand motion analysis (HMA) and performance benchmarks were determined by HMA of local experts.

Increasing the impact of medical image computing using community-based open-access hackathons: The NA-MIC and 3D Slicer experience.

The National Alliance for Medical Image Computing (NA-MIC) was launched in 2004 with the goal of investigating and developing an open source software infrastructure for the extraction of information and knowledge from medical images using computational methods. Several leading research and engineering groups participated in this effort that was funded by the US National Institutes of Health through a variety of infrastructure grants. This effort transformed 3D Slicer from an internal, Boston-based, academic research software application into a professionally maintained, robust, open source platform with an international leadership and developer and user communities.