Placing Approach-Avoidance Conflict Within the Framework of Multi-objective Reinforcement Learning.

Many psychiatric disorders are marked by impaired decision-making during an approach-avoidance conflict. Current experiments elicit approach-avoidance conflicts in bandit tasks by pairing an individual's actions with consequences that are simultaneously desirable (reward) and undesirable (harm). We frame approach-avoidance conflict tasks as a multi-objective multi-armed bandit.

A Competition of Critics in Human Decision-Making.

Recent experiments and theories of human decision-making suggest positive and negative errors are processed and encoded differently by serotonin and dopamine, with serotonin possibly serving to oppose dopamine and protect against risky decisions. We introduce a temporal difference (TD) model of human decision-making to account for these features. Our model involves two critics, an optimistic learning system and a pessimistic learning system, whose predictions are integrated in time to control how potential decisions compete to be selected.

Rescorla-Wagner Models with Sparse Dynamic Attention.

The Rescorla-Wagner (R-W) model describes human associative learning by proposing that an agent updates associations between stimuli, such as events in their environment or predictive cues, proportionally to a prediction error. While this model has proven informative in experiments, it has been posited that humans selectively attend to certain cues to overcome a problem with the R-W model scaling to large cue dimensions. We formally characterize this scaling problem and provide a solution that involves limiting attention in a R-W model to a sparse set of cues.

Critically slow learning in flashcard learning models.

Algorithmic education theory examines, among other things, the trade-off between reviewing old material and studying new material: time spent learning the new comes at the expense of reviewing and solidifying one's understanding of the old. This trade-off is captured in the "Slow Flashcard System" (SFS)-a system that has been studied not only for its applications in educational software but also for its critical properties; it is a simple discrete deterministic system capable of remarkable complexity, with standing conjectures regarding its longterm behavior. Here, we introduce a probabilistic model of SFS and further derive a continuous time, continuous space partial differential equation model.

Estimating Intermittent Individual Spawning Behavior via Disaggregating Group Data.

In order to understand fish biology and reproduction, it is important to know the fecundity patterns of individual fish, as frequently established by recording the output of mixed-sex groups of fish in a laboratory setting. However, for understanding individual reproductive health and modeling purposes it is important to estimate individual fecundity from group fecundity. We created a multistage method that disaggregates group-level data into estimates for individual-level clutch size and spawning interval distributions.

Altruistic aging: The evolutionary dynamics balancing longevity and evolvability.

Altruism is typically associated with traits or behaviors that benefit the population as a whole, but are costly to the individual. We propose that, when the environment is rapidly changing, senescence (age-related deterioration) can be altruistic. According to numerical simulations of an agent-based model, while long-lived individuals can outcompete their short lived peers, populations composed of long-lived individuals are more likely to go extinct during periods of rapid environmental change.

Frequency adjustment and synchrony in networks of delayed pulse-coupled oscillators.

We introduce a system of pulse-coupled oscillators that can change both their phases and frequencies and prove that when there is a separation of time scales between phase and frequency adjustment the system converges to exact synchrony on strongly connected graphs with time delays. The analysis involves decomposing the network into a forest of tree-like structures that capture causality. These results provide a robust method of sensor net synchronization as well as demonstrate a new avenue of possible pulse-coupled oscillator research.

Probabilistic convergence guarantees for type-II pulse-coupled oscillators.

We show that a large class of pulse-coupled oscillators converge with high probability from random initial conditions on a large class of graphs with time delays. Our analysis combines previous local convergence results, probabilistic network analysis, and a classification scheme for type-II phase response curves to produce rigorous lower bounds for convergence probabilities based on network density. These results suggest methods for the analysis of pulse-coupled oscillators, and provide insights into the balance of excitation and inhibition in the operation of biological type-II phase response curves and also the design of decentralized and minimal clock synchronization schemes in sensor nets.

Robust convergence in pulse-coupled oscillators with delays.

We show that for pulse-coupled oscillators a class of phase response curves with both excitation and inhibition exhibit robust convergence to synchrony on arbitrary aperiodic connected graphs with delays. We describe the basins of convergence and give explicit bounds on the convergence times. These results provide new and more robust methods for synchronization of sensor nets and also have biological implications.