Stabilizing machine learning prediction of dynamics: Novel noise-inspired regularization tested with reservoir computing.

Recent work has shown that machine learning (ML) models can skillfully forecast the dynamics of unknown chaotic systems. Short-term predictions of the state evolution and long-term predictions of the statistical patterns of the dynamics ("climate") can be produced by employing a feedback loop, whereby the model is trained to predict forward only one time step, then the model output is used as input for multiple time steps. In the absence of mitigating techniques, however, this feedback can result in artificially rapid error growth ("instability").

Using data assimilation to train a hybrid forecast system that combines machine-learning and knowledge-based components.

We consider the problem of data-assisted forecasting of chaotic dynamical systems when the available data are in the form of noisy partial measurements of the past and present state of the dynamical system. Recently, there have been several promising data-driven approaches to forecasting of chaotic dynamical systems using machine learning. Particularly promising among these are hybrid approaches that combine machine learning with a knowledge-based model, where a machine-learning technique is used to correct the imperfections in the knowledge-based model.

Separation of chaotic signals by reservoir computing.

We demonstrate the utility of machine learning in the separation of superimposed chaotic signals using a technique called reservoir computing. We assume no knowledge of the dynamical equations that produce the signals and require only training data consisting of finite-time samples of the component signals. We test our method on signals that are formed as linear combinations of signals from two Lorenz systems with different parameters.

Dynamics of analog logic-gate networks for machine learning.

We describe the continuous-time dynamics of networks implemented on Field Programable Gate Arrays (FPGAs). The networks can perform Boolean operations when the FPGA is in the clocked (digital) mode; however, we run the programed FPGA in the unclocked (analog) mode. Our motivation is to use these FPGA networks as ultrafast machine-learning processors, using the technique of reservoir computing.

Attractor reconstruction by machine learning.

A machine-learning approach called "reservoir computing" has been used successfully for short-term prediction and attractor reconstruction of chaotic dynamical systems from time series data. We present a theoretical framework that describes conditions under which reservoir computing can create an empirical model capable of skillful short-term forecasts and accurate long-term ergodic behavior. We illustrate this theory through numerical experiments.

Hybrid forecasting of chaotic processes: Using machine learning in conjunction with a knowledge-based model.

A model-based approach to forecasting chaotic dynamical systems utilizes knowledge of the mechanistic processes governing the dynamics to build an approximate mathematical model of the system. In contrast, machine learning techniques have demonstrated promising results for forecasting chaotic systems purely from past time series measurements of system state variables (training data), without prior knowledge of the system dynamics. The motivation for this paper is the potential of machine learning for filling in the gaps in our underlying mechanistic knowledge that cause widely-used knowledge-based models to be inaccurate.

Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data.

We use recent advances in the machine learning area known as "reservoir computing" to formulate a method for model-free estimation from data of the Lyapunov exponents of a chaotic process. The technique uses a limited time series of measurements as input to a high-dimensional dynamical system called a "reservoir." After the reservoir's response to the data is recorded, linear regression is used to learn a large set of parameters, called the "output weights.

Defining chaos.

In this paper, we propose, discuss, and illustrate a computationally feasible definition of chaos which can be applied very generally to situations that are commonly encountered, including attractors, repellers, and non-periodically forced systems. This definition is based on an entropy-like quantity, which we call "expansion entropy," and we define chaos as occurring when this quantity is positive. We relate and compare expansion entropy to the well-known concept of topological entropy to which it is equivalent under appropriate conditions.

Comment on "Long time evolution of phase oscillator systems" [Chaos 19, 023117 (2009)].

In a recent paper by Ott and Antonsen [Chaos 19, 023117 (2009)], it was shown for the case of Lorentzian distributions of oscillator frequencies that the dynamics of a very general class of large systems of coupled phase oscillators time-asymptotes to a particular simplified form given by Ott and Antonsen [Chaos 18, 037113 (2008)]. This comment extends this previous result to a broad class of oscillator distribution functions.

Duplication count distributions in DNA sequences.

We study quantitative features of complex repetitive DNA in several genomes by studying sequences that are sufficiently long that they are unlikely to have repeated by chance. For each genome we study, we determine the number of identical copies, the "duplication count," of each sequence of length 40, that is of each "40-mer." We say a 40-mer is "repeated" if its duplication count is at least 2.

Weighted percolation on directed networks.

We present and numerically test an analysis of the percolation transition for general node removal strategies valid for locally treelike directed networks. On the basis of heuristic arguments we predict that, if the probability of removing node i is p(i), the network disintegrates if p(i) is such that the largest eigenvalue of the matrix with entries A(ij)(1-p(i)) is less than 1, where A is the adjacency matrix of the network. The knowledge or applicability of a Markov network model is not required by our theory, thus making it applicable to situations not covered by previous works.

Improving Phrap-based assembly of the rat using "reliable" overlaps.

The assembly methods used for whole-genome shotgun (WGS) data have a major impact on the quality of resulting draft genomes. We present a novel algorithm to generate a set of "reliable" overlaps based on identifying repeat k-mers. To demonstrate the benefits of using reliable overlaps, we have created a version of the Phrap assembly program that uses only overlaps from a specific list.

Approximating the largest eigenvalue of network adjacency matrices.

The largest eigenvalue of the adjacency matrix of a network plays an important role in several network processes (e.g., synchronization of oscillators, percolation on directed networks, and linear stability of equilibria of network coupled systems).

Characterizing the dynamical importance of network nodes and links.

The largest eigenvalue of the adjacency matrix of networks is a key quantity determining several important dynamical processes on complex networks. Based on this fact, we present a quantitative, objective characterization of the dynamical importance of network nodes and links in terms of their effect on the largest eigenvalue. We show how our characterization of the dynamical importance of nodes can be affected by degree-degree correlations and network community structure.

Emergence of coherence in complex networks of heterogeneous dynamical systems.

We present a general theory for the onset of coherence in collections of heterogeneous maps interacting via a complex connection network. Our method allows the dynamics of the individual uncoupled systems to be either chaotic or periodic, and applies generally to networks for which the number of connections per node is large. We find that the critical coupling strength at which a transition to synchrony takes place depends separately on the dynamics of the individual uncoupled systems and on the largest eigenvalue of the adjacency matrix of the coupling network.

Scale dependence of branching in arterial and bronchial trees.

Although models of branching in arterial and bronchial trees often predict a dependence of bifurcation parameters on the scale of the bifurcating vessels, direct verification of this dependence by comparison with data is uncommon. We compare measurements of bifurcation parameters of airways and arterial trees of different mammals as a function of scale to general features predicted by theoretical models based on minimization of pumping power and network volume. We find that the size dependence is more complex than existing theories based solely on energy and volume minimization explain, and suggest additional factors that may govern the branching at different scales.

Synchronization in large directed networks of coupled phase oscillators.

We study the emergence of collective synchronization in large directed networks of heterogeneous oscillators by generalizing the classical Kuramoto model of globally coupled phase oscillators to more realistic networks. We extend recent theoretical approximations describing the transition to synchronization in large undirected networks of coupled phase oscillators to the case of directed networks. We also consider the case of networks with mixed positive-negative coupling strengths.

Formation of multifractal population patterns from reproductive growth and local resettlement.

We consider the general character of the spatial distribution of a population that grows through reproduction and subsequent local resettlement of new population members. We present several simple one- and two-dimensional point placement models to illustrate possible generic behavior of these distributions. We show, numerically and analytically, that these models all lead to multifractal spatial distributions of population.

Convex error growth patterns in a global weather model.

We investigate the error growth, that is, the growth in the distance E between two typical solutions of a weather model. Typically E grows until it reaches a saturation value E(s). We find two distinct broad log-linear regimes, one for E below 2% of E(s) and the other for E above.

Onset of synchronization in large networks of coupled oscillators.

We study the transition from incoherence to coherence in large networks of coupled phase oscillators. We present various approximations that describe the behavior of an appropriately defined order parameter past the transition and generalize recent results for the critical coupling strength. We find that, under appropriate conditions, the coupling strength at which the transition occurs is determined by the largest eigenvalue of the adjacency matrix.

A preprocessor for shotgun assembly of large genomes.

The whole-genome shotgun (WGS) assembly technique has been remarkably successful in efforts to determine the sequence of bases that make up a genome. WGS assembly begins with a large collection of short fragments that have been selected at random from a genome. The sequence of bases at each end of the fragment is determined, albeit imprecisely, resulting in a sequence of letters called a "read.

Localized error bursts in estimating the state of spatiotemporal chaos.

We consider the problem of estimating the current state of an evolving spatiotemporally chaotic system from noisy observations of the system state and a model of the system dynamics. Using a simple scheme for state estimation, we show the possible occurrence of temporally and spatially intermittent large bursts in the estimation error. We discuss the similarity of these bursts with those occurring at the bubbling transition in the synchronization of low dimensional chaotic dynamical systems.

Desynchronization waves and localized instabilities in oscillator arrays.

We consider a ring of identical or near-identical coupled periodic oscillators in which the connections have randomly heterogeneous strength. We use the master stability function method to determine the possible patterns at the desynchronization transition that occurs as the coupling strengths are increased. We demonstrate Anderson localization of the modes of instability and show that such localized instability generates waves of desynchronization that spread to the whole array.

Reducing storage requirements for biological sequence comparison.

Motivation: Comparison of nucleic acid and protein sequences is a fundamental tool of modern bioinformatics. A dominant method of such string matching is the 'seed-and-extend' approach, in which occurrences of short subsequences called 'seeds' are used to search for potentially longer matches in a large database of sequences. Each such potential match is then checked to see if it extends beyond the seed.

Spatial patterns of desynchronization bursts in networks.

We adapt a previous model and analysis method (the master stability function), extensively used for studying the stability of the synchronous state of networks of identical chaotic oscillators, to the case of oscillators that are similar but not exactly identical. We find that bubbling induced desynchronization bursts occur for some parameter values. These bursts have spatial patterns, which can be predicted from the network connectivity matrix and the unstable periodic orbits embedded in the attractor.