Large Language Models and Large Multimodal Models in Medical Imaging: A Primer for Physicians.

Large language models (LLMs) are poised to have a disruptive impact on health care. Numerous studies have demonstrated promising applications of LLMs in medical imaging, and this number will grow as LLMs further evolve into large multimodal models (LMMs) capable of processing both text and images. Given the substantial roles that LLMs and LMMs will have in health care, it is important for physicians to understand the underlying principles of these technologies so they can use them more effectively and responsibly and help guide their development.

Portable head CT motion artifact correction via diffusion-based generative model.

Portable head CT images often suffer motion artifacts due to the prolonged scanning time and critically ill patients who are unable to hold still. Image-domain motion correction is attractive for this application as it does not require CT projection data. This paper describes and evaluates a generative model based on conditional diffusion to correct motion artifacts in portable head CT scans.

Motion-compensated 4DCT reconstruction from single-beat cardiac CT scans using convolutional networks.

We proposed a deep learning-based method for single-heartbeat 4D cardiac CT reconstruction, where a single cardiac cycle was split into multiple phases for reconstruction. First, we pre-reconstruct each phase using the projection data from itself and the neighboring phases. The pre-reconstructions are fed into a supervised registration network to generate the deformation fields between different phases.

Zero shot health trajectory prediction using transformer.

Integrating modern machine learning and clinical decision-making has great promise for mitigating healthcare's increasing cost and complexity. We introduce the Enhanced Transformer for Health Outcome Simulation (ETHOS), a novel application of the transformer deep-learning architecture for analyzing high-dimensional, heterogeneous, and episodic health data. ETHOS is trained using Patient Health Timelines (PHTs)-detailed, tokenized records of health events-to predict future health trajectories, leveraging a zero-shot learning approach.

MA-SAM: Modality-agnostic SAM adaptation for 3D medical image segmentation.

The Segment Anything Model (SAM), a foundation model for general image segmentation, has demonstrated impressive zero-shot performance across numerous natural image segmentation tasks. However, SAM's performance significantly declines when applied to medical images, primarily due to the substantial disparity between natural and medical image domains. To effectively adapt SAM to medical images, it is important to incorporate critical third-dimensional information, i.

Motion correction and super-resolution for multi-slice cardiac magnetic resonance imaging via an end-to-end deep learning approach.

Accurate reconstruction of a high-resolution 3D volume of the heart is critical for comprehensive cardiac assessments. However, cardiac magnetic resonance (CMR) data is usually acquired as a stack of 2D short-axis (SAX) slices, which suffers from the inter-slice misalignment due to cardiac motion and data sparsity from large gaps between SAX slices. Therefore, we aim to propose an end-to-end deep learning (DL) model to address these two challenges simultaneously, employing specific model components for each challenge.

Texture-preserving low dose CT image denoising using Pearson divergence.

The mean squared error (MSE), also known asloss, has been widely used as a loss function to optimize image denoising models due to its strong performance as a mean estimator of the Gaussian noise model. Recently, various low-dose computed tomography (LDCT) image denoising methods using deep learning combined with the MSE loss have been developed; however, this approach has been observed to suffer from the regression-to-the-mean problem, leading to over-smoothed edges and degradation of texture in the image.To overcome this issue, we propose a stochastic function in the loss function to improve the texture of the denoised CT images, rather than relying on complicated networks or feature space losses.

Estimate and compensate head motion in non-contrast head CT scans using partial angle reconstruction and deep learning.

Background: Patient head motion is a common source of image artifacts in computed tomography (CT) of the head, leading to degraded image quality and potentially incorrect diagnoses. The partial angle reconstruction (PAR) means dividing the CT projection into several consecutive angular segments and reconstructing each segment individually. Although motion estimation and compensation using PAR has been developed and investigated in cardiac CT scans, its potential for reducing motion artifacts in head CT scans remains unexplored.

Surviving ChatGPT in healthcare.

At the dawn of of Artificial General Intelligence (AGI), the emergence of large language models such as ChatGPT show promise in revolutionizing healthcare by improving patient care, expanding medical access, and optimizing clinical processes. However, their integration into healthcare systems requires careful consideration of potential risks, such as inaccurate medical advice, patient privacy violations, the creation of falsified documents or images, overreliance on AGI in medical education, and the perpetuation of biases. It is crucial to implement proper oversight and regulation to address these risks, ensuring the safe and effective incorporation of AGI technologies into healthcare systems.

Artificial general intelligence for radiation oncology.

The emergence of artificial general intelligence (AGI) is transforming radiation oncology. As prominent vanguards of AGI, large language models (LLMs) such as GPT-4 and PaLM 2 can process extensive texts and large vision models (LVMs) such as the Segment Anything Model (SAM) can process extensive imaging data to enhance the efficiency and precision of radiation therapy. This paper explores full-spectrum applications of AGI across radiation oncology including initial consultation, simulation, treatment planning, treatment delivery, treatment verification, and patient follow-up.

Assessment of valve regurgitation severity via contrastive learning and multi-view video integration.

. This paper presents a novel approach for addressing the intricate task of diagnosing aortic valve regurgitation (AR), a valvular disease characterized by blood leakage due to incompetence of the valve closure. Conventional diagnostic techniques require detailed evaluations of multi-modal clinical data, frequently resulting in labor-intensive and time-consuming procedures that are vulnerable to varying subjective assessment of regurgitation severity.

Anatomically Guided PET Image Reconstruction Using Conditional Weakly-Supervised Multi-Task Learning Integrating Self-Attention.

To address the lack of high-quality training labels in positron emission tomography (PET) imaging, weakly-supervised reconstruction methods that generate network-based mappings between prior images and noisy targets have been developed. However, the learned model has an intrinsic variance proportional to the average variance of the target image. To suppress noise and improve the accuracy and generalizability of the learned model, we propose a conditional weakly-supervised multi-task learning (MTL) strategy, in which an auxiliary task is introduced serving as an anatomical regularizer for the PET reconstruction main task.

Differentiating ChatGPT-Generated and Human-Written Medical Texts: Quantitative Study.

Background: Large language models, such as ChatGPT, are capable of generating grammatically perfect and human-like text content, and a large number of ChatGPT-generated texts have appeared on the internet. However, medical texts, such as clinical notes and diagnoses, require rigorous validation, and erroneous medical content generated by ChatGPT could potentially lead to disinformation that poses significant harm to health care and the general public.

Objective: This study is among the first on responsible artificial intelligence-generated content in medicine.

Spach Transformer: Spatial and Channel-Wise Transformer Based on Local and Global Self-Attentions for PET Image Denoising.

Position emission tomography (PET) is widely used in clinics and research due to its quantitative merits and high sensitivity, but suffers from low signal-to-noise ratio (SNR). Recently convolutional neural networks (CNNs) have been widely used to improve PET image quality. Though successful and efficient in local feature extraction, CNN cannot capture long-range dependencies well due to its limited receptive field.

PET image denoising based on denoising diffusion probabilistic model.

Purpose: Due to various physical degradation factors and limited counts received, PET image quality needs further improvements. The denoising diffusion probabilistic model (DDPM) was a distribution learning-based model, which tried to transform a normal distribution into a specific data distribution based on iterative refinements. In this work, we proposed and evaluated different DDPM-based methods for PET image denoising.

SwinCross: Cross-modal Swin transformer for head-and-neck tumor segmentation in PET/CT images.

Background: Radiotherapy (RT) combined with cetuximab is the standard treatment for patients with inoperable head and neck cancers. Segmentation of head and neck (H&N) tumors is a prerequisite for radiotherapy planning but a time-consuming process. In recent years, deep convolutional neural networks (DCNN) have become the de facto standard for automated image segmentation.

Point-of-care AI-assisted stepwise ultrasound pneumothorax diagnosis.

. Ultrasound is extensively utilized as a convenient and cost-effective method in emergency situations. Unfortunately, the limited availability of skilled clinicians in emergency hinders the wider adoption of point-of-care ultrasound.

Unsupervised PET logan parametric image estimation using conditional deep image prior.

Recently, deep learning-based denoising methods have been gradually used for PET images denoising and have shown great achievements. Among these methods, one interesting framework is conditional deep image prior (CDIP) which is an unsupervised method that does not need prior training or a large number of training pairs. In this work, we combined CDIP with Logan parametric image estimation to generate high-quality parametric images.

End-to-end deep learning for interior tomography with low-dose x-ray CT.

There are several x-ray computed tomography (CT) scanning strategies used to reduce radiation dose, such as (1) sparse-view CT, (2) low-dose CT and (3) region-of-interest (ROI) CT (called interior tomography). To further reduce the dose, sparse-view and/or low-dose CT settings can be applied together with interior tomography. Interior tomography has various advantages in terms of reducing the number of detectors and decreasing the x-ray radiation dose.

Connectivity-based Cortical Parcellation via Contrastive Learning on Spatial-Graph Convolution.

. Objective of this work is the development and evaluation of a cortical parcellation framework based on tractography-derived brain structural connectivity. .