Future views on neuroscience and AI.

The relationship between neuroscience and artificial intelligence (AI) has evolved rapidly over the past decade. These two areas of study influence and stimulate each other. We invited experts to share their perspectives on this exciting intersection, focusing on current achievements, unsolved questions, and future directions.

Pre-existing visual responses in a projection-defined dopamine population explain individual learning trajectories.

A key challenge of learning a new task is that the environment is high dimensional-there are many different sensory features and possible actions, with typically only a small reward-relevant subset. Although animals can learn to perform complex tasks that involve arbitrary associations between stimuli, actions, and rewards, a consistent and striking result across varied experimental paradigms is that in initially acquiring such tasks, large differences between individuals are apparent in the learning process. What neural mechanisms contribute to initial task acquisition, and why do some individuals learn a new task much more quickly than others? To address these questions, we recorded longitudinally from dopaminergic (DA) axon terminals in mice learning a visual decision-making task.

A feature-specific prediction error model explains dopaminergic heterogeneity.

The hypothesis that midbrain dopamine (DA) neurons broadcast a reward prediction error (RPE) is among the great successes of computational neuroscience. However, recent results contradict a core aspect of this theory: specifically that the neurons convey a scalar, homogeneous signal. While the predominant family of extensions to the RPE model replicates the classic model in multiple parallel circuits, we argue that these models are ill suited to explain reports of heterogeneity in task variable encoding across DA neurons.

Prioritizing replay when future goals are unknown.

Although hippocampal place cells replay nonlocal trajectories, the computational function of these events remains controversial. One hypothesis, formalized in a prominent reinforcement learning account, holds that replay plans routes to current goals. However, recent puzzling data appear to contradict this perspective by showing that replayed destinations lag current goals.

Independent inhibitory control mechanisms for aggressive motivation and action.

Social behaviors often consist of a motivational phase followed by action. Here we show that neurons in the ventromedial hypothalamus ventrolateral area (VMHvl) of mice encode the temporal sequence of aggressive motivation to action. The VMHvl receives local inhibitory input (VMHvl shell) and long-range input from the medial preoptic area (MPO) with functional coupling to neurons with specific temporal profiles.

Neural circuit models for evidence accumulation through choice-selective sequences.

Decision making is traditionally thought to be mediated by populations of neurons whose firing rates persistently accumulate evidence across time. However, recent decision-making experiments in rodents have observed neurons across the brain that fire sequentially as a function of spatial position or time, rather than persistently, with the subset of neurons in the sequence depending on the animal's choice. We develop two new candidate circuit models, in which evidence is encoded either in the relative firing rates of two competing chains of neurons or in the network location of a stereotyped pattern ("bump") of neural activity.

A neural mechanism for learning from delayed postingestive feedback.

Animals learn the value of foods based on their postingestive effects and thereby develop aversions to foods that are toxic and preferences to those that are nutritious. However, it remains unclear how the brain is able to assign credit to flavors experienced during a meal with postingestive feedback signals that can arise after a substantial delay. Here, we reveal an unexpected role for postingestive reactivation of neural flavor representations in this temporal credit assignment process.

Overlapping representations of food and social stimuli in mouse VTA dopamine neurons.

Dopamine neurons of the ventral tegmental area (VTA) respond to food and social stimuli and contribute to both forms of motivation. However, it is unclear whether the same or different VTA neurons encode these different stimuli. To address this question, we performed two-photon calcium imaging in mice presented with food and conspecifics and found statistically significant overlap in the populations responsive to both stimuli.

Overlapping representations of food and social stimuli in VTA dopamine neurons.

Dopamine neurons of the ventral tegmental area (VTA ) respond to food and social stimuli and contribute to both forms of motivation. However, it is unclear if the same or different VTA neurons encode these different stimuli. To address this question, we performed 2-photon calcium imaging in mice presented with food and conspecifics, and found statistically significant overlap in the populations responsive to both stimuli.

A neural substrate of sex-dependent modulation of motivation.

While there is emerging evidence of sex differences in decision-making behavior, the neural substrates that underlie such differences remain largely unknown. Here we demonstrate that in mice performing a value-based decision-making task, while choices are similar between the sexes, motivation to engage in the task is modulated by action value more strongly in females than in males. Inhibition of activity in anterior cingulate cortex (ACC) neurons that project to the dorsomedial striatum (DMS) preferentially disrupts this relationship between value and motivation in females, without affecting choice in either sex.

Corrigendum: An accumulation-of-evidence task using visual pulses for mice navigating in virtual reality.

[This corrects the article DOI: 10.3389/fnbeh.2018.

Perineuronal Nets in the Dorsomedial Striatum Contribute to Behavioral Dysfunction in Mouse Models of Excessive Repetitive Behavior.

Background: Excessive repetitive behavior is a debilitating symptom of several neuropsychiatric disorders. Parvalbumin-positive inhibitory interneurons in the dorsal striatum have been linked to repetitive behavior, and a sizable portion of these cells are surrounded by perineuronal nets (PNNs), specialized extracellular matrix structures. Although PNNs have been associated with plasticity and neuropsychiatric disease, no previous studies have investigated their involvement in excessive repetitive behavior.

Behavioural and dopaminergic signatures of resilience.

Chronic stress can have lasting adverse consequences in some individuals, yet others are resilient to the same stressor. Susceptible and resilient individuals exhibit differences in the intrinsic properties of mesolimbic dopamine (DA) neurons after the stressful experience is over. However, the causal links between DA, behaviour during stress and individual differences in resilience are unknown.

Choice-selective sequences dominate in cortical relative to thalamic inputs to NAc to support reinforcement learning.

How are actions linked with subsequent outcomes to guide choices? The nucleus accumbens, which is implicated in this process, receives glutamatergic inputs from the prelimbic cortex and midline regions of the thalamus. However, little is known about whether and how representations differ across these input pathways. By comparing these inputs during a reinforcement learning task in mice, we discovered that prelimbic cortical inputs preferentially represent actions and choices, whereas midline thalamic inputs preferentially represent cues.

Opponent control of behavior by dorsomedial striatal pathways depends on task demands and internal state.

A classic view of the striatum holds that activity in direct and indirect pathways oppositely modulates motor output. Whether this involves direct control of movement, or reflects a cognitive process underlying movement, remains unresolved. Here we find that strong, opponent control of behavior by the two pathways of the dorsomedial striatum depends on the cognitive requirements of a task.

Inferring spikes from calcium imaging in dopamine neurons.

Calcium imaging has led to discoveries about neural correlates of behavior in subcortical neurons, including dopamine (DA) neurons. However, spike inference methods have not been tested in most populations of subcortical neurons. To address this gap, we simultaneously performed calcium imaging and electrophysiology in DA neurons in brain slices and applied a recently developed spike inference algorithm to the GCaMP fluorescence.

Standardized and reproducible measurement of decision-making in mice.

Progress in science requires standardized assays whose results can be readily shared, compared, and reproduced across laboratories. Reproducibility, however, has been a concern in neuroscience, particularly for measurements of mouse behavior. Here, we show that a standardized task to probe decision-making in mice produces reproducible results across multiple laboratories.

A Comparison of Dopaminergic and Cholinergic Populations Reveals Unique Contributions of VTA Dopamine Neurons to Short-Term Memory.

We systematically compare the contributions of two dopaminergic and two cholinergic ascending populations to a spatial short-term memory task in rats. In ventral tegmental area dopamine (VTA-DA) and nucleus basalis cholinergic (NB-ChAT) populations, trial-by-trial fluctuations in activity during the delay period relate to performance with an inverted-U, despite the fact that both populations have low activity during that time. Transient manipulations reveal that only VTA-DA neurons, and not the other three populations we examine, contribute causally and selectively to short-term memory.

Distinct signals in medial and lateral VTA dopamine neurons modulate fear extinction at different times.

Dopamine (DA) neurons are to encode reward prediction error (RPE), in addition to other signals, such as salience. While RPE is known to support learning, the role of salience in learning remains less clear. To address this, we recorded and manipulated VTA DA neurons in mice during fear extinction.

Striatal circuits for reward learning and decision-making.

The striatum is essential for learning which actions lead to reward and for implementing those actions. Decades of experimental and theoretical work have led to several influential theories and hypotheses about how the striatal circuit mediates these functions. However, owing to technical limitations, testing these hypotheses rigorously has been difficult.

Specialized coding of sensory, motor and cognitive variables in VTA dopamine neurons.

There is increased appreciation that dopamine neurons in the midbrain respond not only to reward and reward-predicting cues, but also to other variables such as the distance to reward, movements and behavioural choices. An important question is how the responses to these diverse variables are organized across the population of dopamine neurons. Whether individual dopamine neurons multiplex several variables, or whether there are subsets of neurons that are specialized in encoding specific behavioural variables remains unclear.