A common stay-on-goal mechanism in anterior cingulate cortex for information and effort choices.

Humans and non-humans alike often make choices to gain information, even when the information cannot be used to change the outcome. Prior research has shown the anterior cingulate cortex (ACC) is important for evaluating options involving reward-predictive information. Here we studied the role of ACC in information choices using optical inhibition to evaluate the contribution of this region during specific epochs of decision making.

Dopamine Release in the Nucleus Accumbens Core Encodes the General Excitatory Components of Learning.

Dopamine release in the nucleus accumbens core (NAcC) is generally considered to be a proxy for phasic firing of the ventral tegmental area dopamine (VTA) neurons. Thus, dopamine release in NAcC is hypothesized to reflect a unitary role in reward prediction error signaling. However, recent studies reveal more diverse roles of dopamine neurons, which support an emerging idea that dopamine regulates learning differently in distinct circuits.

Dopamine projections to the basolateral amygdala drive the encoding of identity-specific reward memories.

To make adaptive decisions, we build an internal model of the associative relationships in an environment and use it to make predictions and inferences about specific available outcomes. Detailed, identity-specific cue-reward memories are a core feature of such cognitive maps. Here we used fiber photometry, cell-type and pathway-specific optogenetic manipulation, Pavlovian cue-reward conditioning and decision-making tests in male and female rats, to reveal that ventral tegmental area dopamine (VTA) projections to the basolateral amygdala (BLA) drive the encoding of identity-specific cue-reward memories.

Context learning: Dividing up the camp.

A new study reveals that ventral hippocampus integrates information about where we can get rewards into a decision to seek out those rewards. This helps maximise the number of rewards received, while reducing the effort we expend to procure them.

The cognitive (lateral) hypothalamus.

Despite the physiological complexity of the hypothalamus, its role is typically restricted to initiation or cessation of innate behaviors. For example, theories of lateral hypothalamus argue that it is a switch to turn feeding 'on' and 'off' as dictated by higher-order structures that render when feeding is appropriate. However, recent data demonstrate that the lateral hypothalamus is critical for learning about food-related cues.

BehaviorDEPOT is a simple, flexible tool for automated behavioral detection based on markerless pose tracking.

Quantitative descriptions of animal behavior are essential to study the neural substrates of cognitive and emotional processes. Analyses of naturalistic behaviors are often performed by hand or with expensive, inflexible commercial software. Recently, machine learning methods for markerless pose estimation enabled automated tracking of freely moving animals, including in labs with limited coding expertise.

Dopamine errors drive excitatory and inhibitory components of backward conditioning in an outcome-specific manner.

For over two decades, phasic activity in midbrain dopamine neurons was considered synonymous with the prediction error in temporal-difference reinforcement learning. Central to this proposal is the notion that reward-predictive stimuli become endowed with the scalar value of predicted rewards. When these cues are subsequently encountered, their predictive value is compared to the value of the actual reward received, allowing for the calculation of prediction errors.

Higher-Order Conditioning and Dopamine: Charting a Path Forward.

Higher-order conditioning involves learning causal links between multiple events, which then allows one to make novel inferences. For example, observing a correlation between two events (e.g.

The prediction-error hypothesis of schizophrenia: new data point to circuit-specific changes in dopamine activity.

Schizophrenia is a severe psychiatric disorder affecting 21 million people worldwide. People with schizophrenia suffer from symptoms including psychosis and delusions, apathy, anhedonia, and cognitive deficits. Strikingly, schizophrenia is characterised by a learning paradox involving difficulties learning from rewarding events, whilst simultaneously 'overlearning' about irrelevant or neutral information.

The effect of stress and reward on encoding future fear memories.

Prior experience changes the way we learn about our environment. Stress predisposes individuals to developing psychological disorders, just as positive experiences protect from this eventuality (Kirkpatrick & Heller, 2014; Koenigs & Grafman, 2009; Pechtel & Pizzagalli, 2011). Yet current models of how the brain processes information often do not consider a role for prior experience.

Past experience shapes the neural circuits recruited for future learning.

Experimental research controls for past experience, yet prior experience influences how we learn. Here, we tested whether we could recruit a neural population that usually encodes rewards to encode aversive events. Specifically, we found that GABAergic neurons in the lateral hypothalamus (LH) were not involved in learning about fear in naïve rats.

Prior Cocaine Use Alters the Normal Evolution of Information Coding in Striatal Ensembles during Value-Guided Decision-Making.

Substance use disorders (SUDs) are characterized by maladaptive behavior. The ability to properly adjust behavior according to changes in environmental contingencies necessitates the interlacing of existing memories with updated information. This can be achieved by assigning learning in different contexts to compartmentalized "states.

Responding to preconditioned cues is devaluation sensitive and requires orbitofrontal cortex during cue-cue learning.

The orbitofrontal cortex (OFC) is necessary for inferring value in tests of model-based reasoning, including in sensory preconditioning. This involvement could be accounted for by representation of value or by representation of broader associative structure. We recently reported neural correlates of such broader associative structure in OFC during the initial phase of sensory preconditioning (Sadacca et al.

Causal evidence supporting the proposal that dopamine transients function as temporal difference prediction errors.

Reward-evoked dopamine transients are well established as prediction errors. However, the central tenet of temporal difference accounts-that similar transients evoked by reward-predictive cues also function as errors-remains untested. In the present communication we addressed this by showing that optogenetically shunting dopamine activity at the start of a reward-predicting cue prevents second-order conditioning without affecting blocking.

Dopamine transients do not act as model-free prediction errors during associative learning.

Dopamine neurons are proposed to signal the reward prediction error in model-free reinforcement learning algorithms. This term represents the unpredicted or 'excess' value of the rewarding event, value that is then added to the intrinsic value of any antecedent cues, contexts or events. To support this proposal, proponents cite evidence that artificially-induced dopamine transients cause lasting changes in behavior.

Modulation of attention and action in the medial prefrontal cortex of rats.

Theories of functioning in the medial prefrontal cortex are distinct across appetitively and aversively motivated procedures. In the appetitive domain, it is argued that the medial prefrontal cortex is important for producing adaptive behavior when circumstances change. This view advocates a role for this region in using higher-order information to bias performance appropriate to that circumstance.

An Integrated Model of Action Selection: Distinct Modes of Cortical Control of Striatal Decision Making.

Making decisions in environments with few choice options is easy. We select the action that results in the most valued outcome. Making decisions in more complex environments, where the same action can produce different outcomes in different conditions, is much harder.

Author Correction: Dopamine transients are sufficient and necessary for acquisition of model-based associations.

In the version of this article initially published, the laser activation at the start of cue X in experiment 1 was described in the first paragraph of the Results and in the third paragraph of the Experiment 1 section of the Methods as lasting 2 s; in fact, it lasted only 1 s. The error has been corrected in the HTML and PDF versions of the article.

What a relief! A role for dopamine in positive (but not negative) valence.

We have long known that dopamine encodes the predictive relationship between cues and rewards. But what about relief learning? In this issue of Neuropsychopharmacology, Mayer et al. show that the same circuits encoding rewarding events also encode relief from aversive events.

Evaluation of the hypothesis that phasic dopamine constitutes a cached-value signal.

The phasic dopamine error signal is currently argued to be synonymous with the prediction error in Sutton and Barto (1987, 1998) model-free reinforcement learning algorithm (Schultz et al., 1997). This theory argues that phasic dopamine reflects a cached-value signal that endows reward-predictive cues with the scalar value inherent in reward.

Model-based predictions for dopamine.

Phasic dopamine responses are thought to encode a prediction-error signal consistent with model-free reinforcement learning theories. However, a number of recent findings highlight the influence of model-based computations on dopamine responses, and suggest that dopamine prediction errors reflect more dimensions of an expected outcome than scalar reward value. Here, we review a selection of these recent results and discuss the implications and complications of model-based predictions for computational theories of dopamine and learning.