A key challenge in understanding subcellular organization is quantifying interpretable measurements of intracellular structures with complex multi-piece morphologies in an objective, robust and generalizable manner. Here we introduce a morphology-appropriate representation learning framework that uses 3D rotation invariant autoencoders and point clouds. This framework is used to learn representations of complex multi-piece morphologies that are independent of orientation, compact, and easy to interpret. We apply our framework to intracellular structures with punctate morphologies (e.g. DNA replication foci) and polymorphic morphologies (e.g. nucleoli). We systematically compare our framework to image-based autoencoders across several intracellular structure datasets, including a synthetic dataset with pre-defined rules of organization. We explore the trade-offs in the performance of different models by performing multi-metric benchmarking across efficiency, generative capability, and representation expressivity metrics. We find that our framework, which embraces the underlying morphology of multi-piece structures, facilitates the unsupervised discovery of sub-clusters for each structure. We show how our approach can also be applied to phenotypic profiling using a dataset of nucleolar images following drug perturbations. We implement and provide all representation learning models using CytoDL, a python package for flexible and configurable deep learning experiments.
Deep learning models are used to minimize the number of polyps that goes unnoticed by the experts and to accurately segment the detected polyps during interventions. Although state-of-the-art models are proposed, it remains a challenge to define representations that are able to generalize well and that mediate between capturing low-level features and higher-level semantic details without being redundant. Another challenge with these models is that they are computation and memory intensive, which can pose a problem with real-time applications.
The estimation of the pose of surgical instruments is important in Robot-assisted Minimally Invasive Surgery (RMIS) to assist surgical navigation and enable autonomous robotic task execution. The performance of current instrument pose estimation methods deteriorates significantly in the presence of partial tool visibility, occlusions, and changes in the surgical scene. In this work, a vision-based framework is proposed for markerless estimation of the 6DoF pose of surgical instruments.
A thorough understanding of surgical anatomy is essential for preparing and training medical students to become competent and skilled surgeons. While Virtual Reality (VR) has shown to be a suitable interaction paradigm for surgical training, traditional anatomical VR models often rely on simple labels and arrows pointing to relevant landmarks. Yet, studies have indicated that such visual settings could benefit from knowledge maps as such representations explicitly illustrate the conceptual connections between anatomical landmarks.
Non-invasive brain-computer interfaces (BCI) hold great promise in the field of neurorehabilitation. They are easy to use and do not require surgery, particularly in the area of motor imagery electroencephalography (EEG). However, motor imagery EEG signals often have a low signal-to-noise ratio and limited spatial and temporal resolution.
Thirty-four per cent of deaths among Americans aged 1-46 are due to injury, and many of these deaths could be prevented if all hospitals performed as well as the highest-performing hospitals. The Institute of Medicine and the National Academies of Science, Engineering and Medicine have called for learning health systems, with emphasis on clinical practice guidelines (CPGs) as a means of limiting preventable deaths. Reduction in mortality has been demonstrated when evidence-based trauma CPGs are adhered to; however, guidelines are variably updated, redundant, absent, inaccessible, or perceived as irrelevant.
