On learning what to learn: Heterogeneous observations of dynamics and establishing possibly causal relations among them.

Before we attempt to (approximately) learn a function between two sets of observables of a physical process, we must first decide what the and of the desired function are going to be. Here we demonstrate two distinct, data-driven ways of first deciding "the right quantities" to relate through such a function, and then proceeding to learn it. This is accomplished by first processing simultaneous heterogeneous data streams (ensembles of time series) from observations of a physical system: records of multiple of the system.

Data-driven discovery of chemotactic migration of bacteria via coordinate-invariant machine learning.

Background: E. coli chemotactic motion in the presence of a chemonutrient field can be studied using wet laboratory experiments or macroscale-level partial differential equations (PDEs) (among others). Bridging experimental measurements and chemotactic Partial Differential Equations requires knowledge of the evolution of all underlying fields, initial and boundary conditions, and often necessitates strong assumptions.

Machine learning for the identification of phase transitions in interacting agent-based systems: A Desai-Zwanzig example.

Deriving closed-form analytical expressions for reduced-order models, and judiciously choosing the closures leading to them, has long been the strategy of choice for studying phase- and noise-induced transitions for agent-based models (ABMs). In this paper, we propose a data-driven framework that pinpoints phase transitions for an ABM-the Desai-Zwanzig model-in its mean-field limit, using a smaller number of variables than traditional closed-form models. To this end, we use the manifold learning algorithm Diffusion Maps to identify a parsimonious set of data-driven latent variables, and we show that they are in one-to-one correspondence with the expected theoretical order parameter of the ABM.

Transformations establishing equivalence across neural networks: When have two networks learned the same task?

Transformations are a key tool in the qualitative study of dynamical systems: transformations to a normal form, for example, underpin the study of instabilities and bifurcations. In this work, we test, and when possible establish, an equivalence between two different artificial neural networks by attempting to construct a data-driven transformation between them, using diffusion maps with a Mahalanobis-like metric. If the construction succeeds, the two networks can be thought of as belonging to the same equivalence class.

Tipping points of evolving epidemiological networks: Machine learning-assisted, data-driven effective modeling.

We study the tipping point collective dynamics of an adaptive susceptible-infected-susceptible (SIS) epidemiological network in a data-driven, machine learning-assisted manner. We identify a parameter-dependent effective stochastic differential equation (eSDE) in terms of physically meaningful coarse mean-field variables through a deep-learning ResNet architecture inspired by numerical stochastic integrators. We construct an approximate effective bifurcation diagram based on the identified drift term of the eSDE and contrast it with the mean-field SIS model bifurcation diagram.

Task-oriented machine learning surrogates for tipping points of agent-based models.

We present a machine learning framework bridging manifold learning, neural networks, Gaussian processes, and Equation-Free multiscale approach, for the construction of different types of effective reduced order models from detailed agent-based simulators and the systematic multiscale numerical analysis of their emergent dynamics. The specific tasks of interest here include the detection of tipping points, and the uncertainty quantification of rare events near them. Our illustrative examples are an event-driven, stochastic financial market model describing the mimetic behavior of traders, and a compartmental stochastic epidemic model on an Erdös-Rényi network.

Learning black- and gray-box chemotactic PDEs/closures from agent based Monte Carlo simulation data.

We propose a machine learning framework for the data-driven discovery of macroscopic chemotactic Partial Differential Equations (PDEs)-and the closures that lead to them- from high-fidelity, individual-based stochastic simulations of Escherichia coli bacterial motility. The fine scale, chemomechanical, hybrid (continuum-Monte Carlo) simulation model embodies the underlying biophysics, and its parameters are informed from experimental observations of individual cells. Using a parsimonious set of collective observables, we learn effective, coarse-grained "Keller-Segel class" chemotactic PDEs using machine learning regressors: (a) (shallow) feedforward neural networks and (b) Gaussian Processes.

Gentlest Ascent Dynamics on Manifolds Defined by Adaptively Sampled Point-Clouds.

Finding saddle points of dynamical systems is an important problem in practical applications, such as the study of rare events of molecular systems. Gentlest ascent dynamics (GAD) (10.1088/0951-7715/24/6/008) is one of a number of algorithms in existence that attempt to find saddle points.

Machine learning-assisted crystal engineering of a zeolite.

It is shown that Machine Learning (ML) algorithms can usefully capture the effect of crystallization composition and conditions (inputs) on key microstructural characteristics (outputs) of faujasite type zeolites (structure types FAU, EMT, and their intergrowths), which are widely used zeolite catalysts and adsorbents. The utility of ML (in particular, Geometric Harmonics) toward learning input-output relationships of interest is demonstrated, and a comparison with Neural Networks and Gaussian Process Regression, as alternative approaches, is provided. Through ML, synthesis conditions were identified to enhance the Si/Al ratio of high purity FAU zeolite to the hitherto highest level (i.

Depolymerization of plastics by means of electrified spatiotemporal heating.

Depolymerization is a promising strategy for recycling waste plastic into constituent monomers for subsequent repolymerization. However, many commodity plastics cannot be selectively depolymerized using conventional thermochemical approaches, as it is difficult to control the reaction progress and pathway. Although catalysts can improve the selectivity, they are susceptible to performance degradation.

Black and gray box learning of amplitude equations: Application to phase field systems.

We present a data-driven approach to learning surrogate models for amplitude equations and illustrate its application to interfacial dynamics of phase field systems. In particular, we demonstrate learning effective partial differential equations describing the evolution of phase field interfaces from full phase field data. We illustrate this on a model phase field system, where analytical approximate equations for the dynamics of the phase field interface (a higher-order eikonal equation and its approximation, the Kardar-Parisi-Zhang equation) are known.

Learning effective stochastic differential equations from microscopic simulations: Linking stochastic numerics to deep learning.

We identify effective stochastic differential equations (SDEs) for coarse observables of fine-grained particle- or agent-based simulations; these SDEs then provide useful coarse surrogate models of the fine scale dynamics. We approximate the drift and diffusivity functions in these effective SDEs through neural networks, which can be thought of as effective stochastic ResNets. The loss function is inspired by, and embodies, the structure of established stochastic numerical integrators (here, Euler-Maruyama and Milstein); our approximations can thus benefit from backward error analysis of these underlying numerical schemes.

Limits of entrainment of circadian neuronal networks.

Circadian rhythmicity lies at the center of various important physiological and behavioral processes in mammals, such as sleep, metabolism, homeostasis, mood changes, and more. Misalignment of intrinsic neuronal oscillations with the external day-night cycle can disrupt such processes and lead to numerous disorders. In this work, we computationally determine the limits of circadian synchronization to external light signals of different frequency, duty cycle, and simulated amplitude.

On the parameter combinations that matter and on those that do not: data-driven studies of parameter (non)identifiability.

We present a data-driven approach to characterizing nonidentifiability of a model's parameters and illustrate it through dynamic as well as steady kinetic models. By employing Diffusion Maps and their extensions, we discover the minimal combinations of parameters required to characterize the output behavior of a chemical system: a set of for the model. Furthermore, we introduce and use a Conformal Autoencoder Neural Network technique, as well as a kernel-based Jointly Smooth Function technique, to disentangle the parameter combinations that do not affect the output behavior from the ones that do.

Machine Learning Memory Kernels as Closure for Non-Markovian Stochastic Processes.

Finding the dynamical law of observable quantities lies at the core of physics. Within the particular field of statistical mechanics, the generalized Langevin equation (GLE) comprises a general model for the evolution of observables covering a great deal of physical systems with many degrees of freedom and an inherently stochastic nature. Although formally exact, GLE brings its own great challenges.

Long-term p21 and p53 dynamics regulate the frequency of mitosis events and cell cycle arrest following radiation damage.

Radiation exposure of healthy cells can halt cell cycle temporarily or permanently. In this work, we analyze the time evolution of p21 and p53 from two single cell datasets of retinal pigment epithelial cells exposed to several levels of radiation, and in particular, the effect of radiation on cell cycle arrest. Employing various quantification methods from signal processing, we show how p21 levels, and to a lesser extent p53 levels, dictate whether the cells are arrested in their cell cycle and how frequently these mitosis events are likely to occur.

Learning emergent partial differential equations in a learned emergent space.

We propose an approach to learn effective evolution equations for large systems of interacting agents. This is demonstrated on two examples, a well-studied system of coupled normal form oscillators and a biologically motivated example of coupled Hodgkin-Huxley-like neurons. For such types of systems there is no obvious space coordinate in which to learn effective evolution laws in the form of partial differential equations.

Programmable heating and quenching for efficient thermochemical synthesis.

Conventional thermochemical syntheses by continuous heating under near-equilibrium conditions face critical challenges in improving the synthesis rate, selectivity, catalyst stability and energy efficiency, owing to the lack of temporal control over the reaction temperature and time, and thus the reaction pathways. As an alternative, we present a non-equilibrium, continuous synthesis technique that uses pulsed heating and quenching (for example, 0.02 s on, 1.

High-entropy nanoparticles: Synthesis-structure-property relationships and data-driven discovery.

High-entropy nanoparticles have become a rapidly growing area of research in recent years. Because of their multielemental compositions and unique high-entropy mixing states (i.e.

Learning the temporal evolution of multivariate densities via normalizing flows.

In this work, we propose a method to learn multivariate probability distributions using sample path data from stochastic differential equations. Specifically, we consider temporally evolving probability distributions (e.g.

Learning Poisson Systems and Trajectories of Autonomous Systems via Poisson Neural Networks.

We propose the Poisson neural networks (PNNs) to learn Poisson systems and trajectories of autonomous systems from data. Based on the Darboux-Lie theorem, the phase flow of a Poisson system can be written as the composition of: 1) a coordinate transformation; 2) an extended symplectic map; and 3) the inverse of the transformation. In this work, we extend this result to the unknotted trajectories of autonomous systems.

It doesn't always pay to be fit: success landscapes.

Landscapes play an important role in many areas of biology, in which biological lives are deeply entangled. Here we discuss a form of landscape in evolutionary biology which takes into account (1) initial growth rates, (2) mutation rates, (3) resource consumption by organisms, and (4) cyclic changes in the resources with time. The long-term equilibrium number of surviving organisms as a function of these four parameters forms what we call a success landscape, a landscape we would claim is qualitatively different from fitness landscapes which commonly do not include mutations or resource consumption/changes in mapping genomes to the final number of survivors.

Initializing LSTM internal states via manifold learning.

We present an approach, based on learning an intrinsic data manifold, for the initialization of the internal state values of long short-term memory (LSTM) recurrent neural networks, ensuring consistency with the initial observed input data. Exploiting the generalized synchronization concept, we argue that the converged, "mature" internal states constitute a function on this learned manifold. The dimension of this manifold then dictates the length of observed input time series data required for consistent initialization.

Global and local reduced models for interacting, heterogeneous agents.

Large collections of coupled, heterogeneous agents can manifest complex dynamical behavior presenting difficulties for simulation and analysis. However, if the collective dynamics lie on a low-dimensional manifold, then the original agent-based model may be approximated with a simplified surrogate model on and near the low-dimensional space where the dynamics live. Analytically identifying such simplified models can be challenging or impossible, but here we present a data-driven coarse-graining methodology for discovering such reduced models.