Publications by authors named "Joel Saltz"

Objective: With the increasing energy surrounding the development of artificial intelligence and machine learning (AI/ML) models, the use of the same external validation dataset by various developers allows for a direct comparison of model performance. Through our High Throughput Truthing project, we are creating a validation dataset for AI/ML models trained in the assessment of stromal tumor-infiltrating lymphocytes (sTILs) in triple negative breast cancer (TNBC).

Materials And Methods: We obtained clinical metadata for hematoxylin and eosin-stained glass slides and corresponding scanned whole slide images (WSIs) of TNBC core biopsies from two US academic medical centers.

View Article and Find Full Text PDF

Introducing interpretability and reasoning into Multiple Instance Learning (MIL) methods for Whole Slide Image (WSI) analysis is challenging, given the complexity of gigapixel slides. Traditionally, MIL interpretability is limited to identifying salient regions deemed pertinent for downstream tasks, offering little insight to the end-user (pathologist) regarding the rationale behind these selections. To address this, we propose Self-Interpretable MIL (SI-MIL), a method intrinsically designed for interpretability from the very outset.

View Article and Find Full Text PDF
Article Synopsis
  • Diffusion models can improve image generation in specialized fields like histopathology and satellite imagery by utilizing self-supervised learning (SSL) embeddings as stand-ins for human labels, which are hard to obtain.
  • This new method allows for high-quality images to be created from these embeddings, and it can even generate larger images by combining smaller patches while maintaining their spatial consistency.
  • The approach enhances classifier performance on both small patch-level and larger scale classification tasks and shows strong adaptability, successfully working with unseen datasets and different input sources, including text descriptions for image synthesis.
View Article and Find Full Text PDF

In the medical diagnostics domain, pathology and histology are pivotal for the precise identification of diseases. Digital histopathology, enhanced by automation, facilitates the efficient analysis of massive amount of biopsy images produced on a daily basis, streamlining the evaluation process. This study focuses in Stain Color Normalization (SCN) within a Whole-Slide Image (WSI) cohort, aiming to reduce batch biases.

View Article and Find Full Text PDF

Estimating uncertainty of a neural network is crucial in providing transparency and trustworthiness. In this paper, we focus on uncertainty estimation for digital pathology prediction models. To explore the large amount of unlabeled data in digital pathology, we propose to adopt novel learning method that can fully exploit unlabeled data.

View Article and Find Full Text PDF

Introduction: Deep learning models hold great promise for digital pathology, but their opaque decision-making processes undermine trust and hinder clinical adoption. Explainable AI methods are essential to enhance model transparency and reliability.

Methods: We developed HIPPO, an explainable AI framework that systematically modifies tissue regions in whole slide images to generate image counterfactuals, enabling quantitative hypothesis testing, bias detection, and model evaluation beyond traditional performance metrics.

View Article and Find Full Text PDF
Article Synopsis
  • A study investigated the prevalence of vestibular disorders in patients with COVID-19 compared to those without the virus using data from the National COVID Cohort Collaborative database.
  • Results showed that individuals with COVID-19 were significantly more likely to experience vestibular disorders, with the highest risk associated with the omicron 23A variant (OR of 8.80).
  • The findings underscore the need for further research on the long-term effects of vestibular disorders in COVID-19 patients and implications for patient counseling.
View Article and Find Full Text PDF

Multiplex Immunohistochemistry (mIHC) is a cost-effective and accessible method for in situ labeling of multiple protein biomarkers in a tissue sample. By assigning a different stain to each biomarker, it allows the visualization of different types of cells within the tumor vicinity for downstream analysis. However, to detect different types of stains in a given mIHC image is a challenging problem, especially when the number of stains is high.

View Article and Find Full Text PDF

To achieve high-quality results, diffusion models must be trained on large datasets. This can be notably prohibitive for models in specialized domains, such as computational pathology. Conditioning on labeled data is known to help in data-efficient model training.

View Article and Find Full Text PDF

In digital pathology, the spatial context of cells is important for cell classification, cancer diagnosis and prognosis. To model such complex cell context, however, is challenging. Cells form different mixtures, lineages, clusters and holes.

View Article and Find Full Text PDF
Article Synopsis
  • K17 expression influences the immune environment in pancreatic ductal adenocarcinoma (PDAC), affecting tumor growth and patient survival.
  • A study using advanced imaging techniques found that higher K17 levels lead to reduced presence of important immune cells, such as CD8+ T cells and macrophages, in and around tumors.
  • These findings suggest that targeting K17 could enhance the immune response against PDAC, potentially improving immunotherapy strategies for this aggressive cancer.
View Article and Find Full Text PDF

Background: The immune microenvironment impacts tumor growth, invasion, metastasis, and patient survival and may provide opportunities for therapeutic intervention in pancreatic ductal adenocarcinoma (PDAC). Although never studied as a potential modulator of the immune response in most cancers, Keratin 17 (K17), a biomarker of the most aggressive (basal) molecular subtype of PDAC, is intimately involved in the histogenesis of the immune response in psoriasis, basal cell carcinoma, and cervical squamous cell carcinoma. Thus, we hypothesized that K17 expression could also impact the immune cell response in PDAC, and that uncovering this relationship could provide insight to guide the development of immunotherapeutic opportunities to extend patient survival.

View Article and Find Full Text PDF

A growing body of research supports stromal tumour-infiltrating lymphocyte (TIL) density in breast cancer to be a robust prognostic and predicive biomarker. The gold standard for stromal TIL density quantitation in breast cancer is pathologist visual assessment using haematoxylin and eosin-stained slides. Artificial intelligence/machine-learning algorithms are in development to automate the stromal TIL scoring process, and must be validated against a reference standard such as pathologist visual assessment.

View Article and Find Full Text PDF
Article Synopsis
  • Large-scale collaboration in oncology research is essential for advancing cancer biology, precision oncology, and population sciences, which requires innovative data management and analytic tools.
  • The informatics community plays a crucial role in automating the organization of diverse clinical data types, including molecular tests and diagnostic imaging, to address the complexities of cancer progression.
  • The paper presents a new Clinical & Research Data Warehouse (CRDW) that supports multimodal data, such as genomics and radiology images, integrating machine-learning tools for deeper insights into tumor characteristics beyond traditional methods.
View Article and Find Full Text PDF

This work puts forth and demonstrates the utility of a reporting framework for collecting and evaluating annotations of medical images used for training and testing artificial intelligence (AI) models in assisting detection and diagnosis. AI has unique reporting requirements, as shown by the AI extensions to the Consolidated Standards of Reporting Trials (CONSORT) and Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) checklists and the proposed AI extensions to the Standards for Reporting Diagnostic Accuracy (STARD) and Transparent Reporting of a Multivariable Prediction model for Individual Prognosis or Diagnosis (TRIPOD) checklists. AI for detection and/or diagnostic image analysis requires complete, reproducible, and transparent reporting of the annotations and metadata used in training and testing data sets.

View Article and Find Full Text PDF

Introduction: Research driven by real-world clinical data is increasingly vital to enabling learning health systems, but integrating such data from across disparate health systems is challenging. As part of the NCATS National COVID Cohort Collaborative (N3C), the N3C Data Enclave was established as a centralized repository of deidentified and harmonized COVID-19 patient data from institutions across the US. However, making this data most useful for research requires linking it with information such as mortality data, images, and viral variants.

View Article and Find Full Text PDF

Recent advances in the field of immuno-oncology have brought transformative changes in the management of cancer patients. The immune profile of tumours has been found to have key value in predicting disease prognosis and treatment response in various cancers. Multiplex immunohistochemistry and immunofluorescence have emerged as potent tools for the simultaneous detection of multiple protein biomarkers in a single tissue section, thereby expanding opportunities for molecular and immune profiling while preserving tissue samples.

View Article and Find Full Text PDF

Digital pathology has seen a proliferation of deep learning models in recent years, but many models are not readily reusable. To address this challenge, we developed WSInfer: an open-source software ecosystem designed to streamline the sharing and reuse of deep learning models for digital pathology. The increased access to trained models can augment research on the diagnostic, prognostic, and predictive capabilities of digital pathology.

View Article and Find Full Text PDF

We propose DiRL, a Diversity-inducing Representation Learning technique for histopathology imaging. Self-supervised learning (SSL) techniques, such as contrastive and non-contrastive approaches, have been shown to learn rich and effective representations of digitized tissue samples with limited pathologist supervision. Our analysis of vanilla SSL-pretrained models' attention distribution reveals an insightful observation: sparsity in attention, i.

View Article and Find Full Text PDF

Quantifying tumor-infiltrating lymphocytes (TILs) in breast cancer tumors is a challenging task for pathologists. With the advent of whole slide imaging that digitizes glass slides, it is possible to apply computational models to quantify TILs for pathologists. Development of computational models requires significant time, expertise, consensus, and investment.

View Article and Find Full Text PDF

Clinical trials have been the center of progress in modern medicine. In oncology, we are fortunate to have a structure in place through the National Clinical Trials Network (NCTN). The NCTN provides the infrastructure and a forum for scientific discussion to develop clinical concepts for trial design.

View Article and Find Full Text PDF

Inflammatory bowel disease (IBD) is characterized by chronic, dysregulated inflammation in the gastrointestinal tract. The heterogeneity of IBD is reflected through two major subtypes, Crohn's Disease (CD) and Ulcerative Colitis (UC). CD and UC differ across symptomatic presentation, histology, immune responses, and treatment.

View Article and Find Full Text PDF

The clinical significance of the tumor-immune interaction in breast cancer is now established, and tumor-infiltrating lymphocytes (TILs) have emerged as predictive and prognostic biomarkers for patients with triple-negative (estrogen receptor, progesterone receptor, and HER2-negative) breast cancer and HER2-positive breast cancer. How computational assessments of TILs might complement manual TIL assessment in trial and daily practices is currently debated. Recent efforts to use machine learning (ML) to automatically evaluate TILs have shown promising results.

View Article and Find Full Text PDF

Modern histologic imaging platforms coupled with machine learning methods have provided new opportunities to map the spatial distribution of immune cells in the tumor microenvironment. However, there exists no standardized method for describing or analyzing spatial immune cell data, and most reported spatial analyses are rudimentary. In this review, we provide an overview of two approaches for reporting and analyzing spatial data (raster versus vector-based).

View Article and Find Full Text PDF

An aneurysm is a pathological widening of a blood vessel. Aneurysms of the aorta are often asymptomatic until they rupture, killing approximately 10,000 Americans per year. Fortunately, rupture can be prevented through early detection and surgical repair.

View Article and Find Full Text PDF