Machine learning in microscopy - insights, opportunities and challenges.

Machine learning (ML) is transforming the field of image processing and analysis, from automation of laborious tasks to open-ended exploration of visual patterns. This has striking implications for image-driven life science research, particularly microscopy. In this Review, we focus on the opportunities and challenges associated with applying ML-based pipelines for microscopy datasets from a user point of view.

Cluster analysis for localisation-based data sets: dos and don'ts when quantifying protein aggregates.

Many proteins display a non-random distribution on the cell surface. From dimers to nanoscale clusters to large, micron-scale aggregations, these distributions regulate protein-protein interactions and signalling. Although these distributions show organisation on length-scales below the resolution limit of conventional optical microscopy, single molecule localisation microscopy (SMLM) can map molecule locations with nanometre precision.

A framework for evaluating the performance of SMLM cluster analysis algorithms.

Single-molecule localization microscopy (SMLM) generates data in the form of coordinates of localized fluorophores. Cluster analysis is an attractive route for extracting biologically meaningful information from such data and has been widely applied. Despite a range of cluster analysis algorithms, there exists no consensus framework for the evaluation of their performance.

Event-driven acquisition for content-enriched microscopy.

A common goal of fluorescence microscopy is to collect data on specific biological events. Yet, the event-specific content that can be collected from a sample is limited, especially for rare or stochastic processes. This is due in part to photobleaching and phototoxicity, which constrain imaging speed and duration.

Fluorescent d-Amino Acids for Super-resolution Microscopy of the Bacterial Cell Wall.

Fluorescent d-amino acids (FDAAs) have previously been developed to enable highlighting of locations of bacterial cell wall growth. Most bacterial cells lie at the edge of the diffraction limit of visible light; thus, resolving the precise details of peptidoglycan (PG) biosynthesis requires super-resolution microscopy after probe incorporation. Single molecule localization microscopy (SMLM) has stringent requirements on the fluorophore photophysical properties and therefore has remained challenging in this context.

Correction of multiple-blinking artifacts in photoactivated localization microscopy.

Photoactivated localization microscopy (PALM) produces an array of localization coordinates by means of photoactivatable fluorescent proteins. However, observations are subject to fluorophore multiple blinking and each protein is included in the dataset an unknown number of times at different positions, due to localization error. This causes artificial clustering to be observed in the data.

Single-molecule localization microscopy.

Single-molecule localization microscopy (SMLM) describes a family of powerful imaging techniques that dramatically improve spatial resolution over standard, diffraction-limited microscopy techniques and can image biological structures at the molecular scale. In SMLM, individual fluorescent molecules are computationally localized from diffraction-limited image sequences and the localizations are used to generate a super-resolution image or a time course of super-resolution images, or to define molecular trajectories. In this Primer, we introduce the basic principles of SMLM techniques before describing the main experimental considerations when performing SMLM, including fluorescent labelling, sample preparation, hardware requirements and image acquisition in fixed and live cells.

Multi-color Molecular Visualization of Signaling Proteins Reveals How C-Terminal Src Kinase Nanoclusters Regulate T Cell Receptor Activation.

Elucidating the mechanisms that controlled T cell activation requires visualization of the spatial organization of multiple proteins on the submicron scale. Here, we use stoichiometrically accurate, multiplexed, single-molecule super-resolution microscopy (DNA-PAINT) to image the nanoscale spatial architecture of the primary inhibitor of the T cell signaling pathway, Csk, and two binding partners implicated in its membrane association, PAG and TRAF3. Combined with a newly developed co-clustering analysis framework, we find that Csk forms nanoscale clusters proximal to the plasma membrane that are lost post-stimulation and are re-recruited at later time points.

Machine learning for cluster analysis of localization microscopy data.

Quantifying the extent to which points are clustered in single-molecule localization microscopy data is vital to understanding the spatial relationships between molecules in the underlying sample. Many existing computational approaches are limited in their ability to process large-scale data sets, to deal effectively with sample heterogeneity, or require subjective user-defined analysis parameters. Here, we develop a supervised machine-learning approach to cluster analysis which is fast and accurate.

An agent-based model of molecular aggregation at the cell membrane.

Molecular clustering at the plasma membrane has long been identified as a key process and is associated with regulating signalling pathways across cell types. Recent advances in microscopy, in particular the rise of super-resolution, have allowed the experimental observation of nanoscale molecular clusters in the plasma membrane. However, modelling approaches capable of recapitulating these observations are in their infancy, partly because of the extremely complex array of biophysical factors which influence molecular distributions and dynamics in the plasma membrane.

Differential nanoscale organisation of LFA-1 modulates T-cell migration.

Effector T-cells rely on integrins to drive adhesion and migration to facilitate their immune function. The heterodimeric transmembrane integrin LFA-1 (αLβ2 integrin) regulates adhesion and migration of effector T-cells through linkage of the extracellular matrix with the intracellular actin treadmill machinery. Here, we quantified the velocity and direction of F-actin flow in migrating T-cells alongside single-molecule localisation of transmembrane and intracellular LFA-1.

Strategies for increasing the throughput of super-resolution microscopies.

Fluorescence microscopy methods have been developed to circumvent the diffraction limit by exploiting nonlinearities in the interactions between light and fluorophores. Initially, these methods were up to orders of magnitude slower than standard microscopies. In recent years, a wide array of technological advances have increased the throughput of super-resolution microscopies, through parallelization, smart scanning or data processing, and sample expansion.

Quantitative fibre analysis of single-molecule localization microscopy data.

Single molecule localization microscopy (SMLM) methods produce data in the form of a spatial point pattern (SPP) of all localized emitters. Whilst numerous tools exist to quantify molecular clustering in SPP data, the analysis of fibrous structures has remained understudied. Taking the SMLM localization coordinates as input, we present an algorithm capable of tracing fibrous structures in data generated by SMLM.

3D Bayesian cluster analysis of super-resolution data reveals LAT recruitment to the T cell synapse.

Single-molecule localisation microscopy (SMLM) allows the localisation of fluorophores with a precision of 10-30 nm, revealing the cell's nanoscale architecture at the molecular level. Recently, SMLM has been extended to 3D, providing a unique insight into cellular machinery. Although cluster analysis techniques have been developed for 2D SMLM data sets, few have been applied to 3D.

Quantification of fibrous spatial point patterns from single-molecule localization microscopy (SMLM) data.

Motivation: Unlike conventional microscopy which produces pixelated images, SMLM produces data in the form of a list of localization coordinates-a spatial point pattern (SPP). Often, such SPPs are analyzed using cluster analysis algorithms to quantify molecular clustering within, for example, the plasma membrane. While SMLM cluster analysis is now well developed, techniques for analyzing fibrous structures remain poorly explored.

A Bayesian cluster analysis method for single-molecule localization microscopy data.

Cell function is regulated by the spatiotemporal organization of the signaling machinery, and a key facet of this is molecular clustering. Here, we present a protocol for the analysis of clustering in data generated by 2D single-molecule localization microscopy (SMLM)-for example, photoactivated localization microscopy (PALM) or stochastic optical reconstruction microscopy (STORM). Three features of such data can cause standard cluster analysis approaches to be ineffective: (i) the data take the form of a list of points rather than a pixel array; (ii) there is a non-negligible unclustered background density of points that must be accounted for; and (iii) each localization has an associated uncertainty in regard to its position.

Superresolution imaging of the cytoplasmic phosphatase PTPN22 links integrin-mediated T cell adhesion with autoimmunity.

Integrins are heterodimeric transmembrane proteins that play a fundamental role in the migration of leukocytes to sites of infection or injury. We found that protein tyrosine phosphatase nonreceptor type 22 (PTPN22) inhibits signaling by the integrin lymphocyte function-associated antigen-1 (LFA-1) in effector T cells. PTPN22 colocalized with its substrates at the leading edge of cells migrating on surfaces coated with the LFA-1 ligand intercellular adhesion molecule-1 (ICAM-1).

Bayesian cluster identification in single-molecule localization microscopy data.

Single-molecule localization-based super-resolution microscopy techniques such as photoactivated localization microscopy (PALM) and stochastic optical reconstruction microscopy (STORM) produce pointillist data sets of molecular coordinates. Although many algorithms exist for the identification and localization of molecules from raw image data, methods for analyzing the resulting point patterns for properties such as clustering have remained relatively under-studied. Here we present a model-based Bayesian approach to evaluate molecular cluster assignment proposals, generated in this study by analysis based on Ripley's K function.

The nanoscale organization of signaling domains at the plasma membrane.

In this chapter, we present an overview of the role of the nanoscale organization of signaling domains in regulating key cellular processes. In particular, we illustrate the importance of protein and lipid nanodomains as triggers and mediators of cell signaling. As particular examples, we summarize the state of the art of understanding the role of nanodomains in the mounting of an immune response, cellular adhesion, intercellular communication, and cell proliferation.

Topographic prominence as a method for cluster identification in single-molecule localisation data.

Single-molecule localisation based super-resolution fluorescence imaging produces maps of the coordinates of fluorescent molecules in a region of interest. Cluster analysis algorithms provide information concerning the clustering characteristics of these molecules, often through the generation of cluster heat maps based on local molecular density. The goal of this study was to generate a new cluster analysis method based on a topographic approach.