A framework for understanding and advancing intertemporal choice research using rodent models.

Intertemporal choices are common and consequential to private and public life. Thus, there is considerable interest in understanding the neural basis of intertemporal decision making. In this minireview, we briefly describe conceptual and psychological perspectives on intertemporal choice and then provide a comprehensive evaluation of the neural structures and signals that comprise the underlying cortico-limbic-striatal circuit.

The Lateral Habenula Circuitry: Reward Processing and Cognitive Control.

There has been a growing interest in understanding the role of the lateral habenula (LHb) in reward processing, affect regulation, and goal-directed behaviors. The LHb gets major inputs from the habenula-projecting globus pallidus and the mPFC, sending its efferents to the dopaminergic VTA and SNc, serotonergic dorsal raphe nuclei, and the GABAergic rostromedial tegmental nucleus. Recent studies have made advances in our understanding of the LHb circuit organization, yet the precise mechanisms of its involvement in complex behaviors are largely unknown.

Learning to Thrive: Building Diverse Scientists' Access to Community and Resources through the BRAINS Program.

Broadening the Representation of Academic Investigators in NeuroScience is a National Institutes of Health-funded, national program that addresses challenges to the persistence of diverse early-career neuroscientists. In doing so, BRAINS aims to advance diversity in neuroscience by increasing career advancement and retention of post-PhD, early-career neuroscientists from underrepresented groups (URGs). The comprehensive professional development program is structured to catalyze conversations specific to URGs in neuroscience and explicitly addresses factors known to impact persistence such as a weak sense of belonging to the scientific community, isolation and solo status, inequitable access to resources that impact career success, and marginalization from informal networks and mentoring relationships.

Analysis of morphine-induced changes in the activity of periaqueductal gray neurons in the intact rat.

Microinjection of morphine into the periaqueductal gray (PAG) produces antinociception. In vitro slice recordings indicate that all PAG neurons are sensitive to morphine either by direct inhibition or indirect disinhibition. We tested the hypothesis that all PAG neurons respond to opioids in vivo by examining the extracellular activity of PAG neurons recorded in lightly anesthetized and awake rats.

Ongoing behavioral state information signaled in the lateral habenula guides choice flexibility in freely moving rats.

The lateral habenula (LHb) plays a role in a wide variety of behaviors ranging from maternal care, to sleep, to various forms of cognition. One prominent theory with ample supporting evidence is that the LHb serves to relay basal ganglia and limbic signals about negative outcomes to midbrain monoaminergic systems. This makes it likely that the LHb is critically involved in behavioral flexibility as all of these systems have been shown to contribute when flexible behavior is required.

Integrative hippocampal and decision-making neurocircuitry during goal-relevant predictions and encoding.

It has become clear that the hippocampus plays a critical role in the identification of new contexts and for the detection of changes in familiar contexts. The hippocampus accomplishes these goals through a continual process of comparing predicted features of a context or situation to those actually experienced. A mismatch between expected and experienced context expectations is thought to lead to the generation of a context prediction error (Mizumori, 2013) that functionally alerts connected brain areas to alter subsequent decision making and response selection.

A role for the lateral dorsal tegmentum in memory and decision neural circuitry.

A role for the hippocampus in memory is clear, although the mechanism for its contribution remains a matter of debate. Converging evidence suggests that hippocampus evaluates the extent to which context-defining features of events occur as expected. The consequence of mismatches, or prediction error, signals from hippocampus is discussed in terms of its impact on neural circuitry that evaluates the significance of prediction errors: Ventral tegmental area (VTA) dopamine cells burst fire to rewards or cues that predict rewards (Schultz, Dayan, & Montague, 1997).

Cost-benefit decision circuitry: proposed modulatory role for acetylcholine.

In order to select which action should be taken, an animal must weigh the costs and benefits of possible outcomes associate with each action. Such decisions, called cost-benefit decisions, likely involve several cognitive processes (including memory) and a vast neural circuitry. Rodent models have allowed research to begin to probe the neural basis of three forms of cost-benefit decision making: effort-, delay-, and risk-based decision making.

Context prediction analysis and episodic memory.

Events that happen at a particular place and time come to define our episodic memories. Extensive experimental and clinical research illustrate that the hippocampus is central to the processing of episodic memories, and this is in large part due to its analysis of context information according to spatial and temporal references. In this way, hippocampus defines ones expectations for a given context as well as detects errors in predicted contextual features.

Homeostatic regulation of memory systems and adaptive decisions.

While it is clear that many brain areas process mnemonic information, understanding how their interactions result in continuously adaptive behaviors has been a challenge. A homeostatic-regulated prediction model of memory is presented that considers the existence of a single memory system that is based on a multilevel coordinated and integrated network (from cells to neural systems) that determines the extent to which events and outcomes occur as predicted. The "multiple memory systems of the brain" have in common output that signals errors in the prediction of events and/or their outcomes, although these signals differ in terms of what the error signal represents (e.

Effects of prefrontal cortical inactivation on neural activity in the ventral tegmental area.

Dopamine (DA) cells have been suggested to signal discrepancies between expected and actual rewards in reinforcement learning. DA cells in the ventral tegmental area (VTA) receive direct projections from the medial prefrontal cortex (mPFC), a structure known to be one of the brain areas that represents expected future rewards. To investigate whether the mPFC contributes to generating reward prediction error signals of DA cells, we recorded VTA cells from rats foraging for different amounts of reward in a spatial working memory task.

Age-associated changes in the hippocampal-ventral striatum-ventral tegmental loop that impact learning, prediction, and context discrimination.

Studies of the neural mechanisms of navigation and context discrimination have generated a powerful heuristic for understanding how neural codes, circuits, and computations contribute to accurate behavior as animals traverse and learn about spatially extended environments. It is assumed that memories are updated as a result of spatial experience. The mechanism, however, for such a process is not clear.

Neural systems analysis of decision making during goal-directed navigation.

The ability to make adaptive decisions during goal-directed navigation is a fundamental and highly evolved behavior that requires continual coordination of perceptions, learning and memory processes, and the planning of behaviors. Here, a neurobiological account for such coordination is provided by integrating current literatures on spatial context analysis and decision-making. This integration includes discussions of our current understanding of the role of the hippocampal system in experience-dependent navigation, how hippocampal information comes to impact midbrain and striatal decision making systems, and finally the role of the striatum in the implementation of behaviors based on recent decisions.

Complimentary roles of the hippocampus and retrosplenial cortex in behavioral context discrimination.

Complex cognitive functions, such as learning and memory, arise from the interaction of multiple brain regions that comprise functional circuits and different components of these circuits make unique contributions to learning. The hippocampus and the retrosplenial cortex (RSC) are anatomically interconnected and both regions are involved in learning and memory. Previous studies indicate that the hippocampus exhibits unique firing patterns for different contexts and that RSC neurons selectively respond to cues that predict reinforcement or the need for a behavioral response, suggesting a hippocampal role in encoding contexts and an RSC role in encoding behaviorally significant cues.

Activation of dopamine neurons is critical for aversive conditioning and prevention of generalized anxiety.

Generalized anxiety is thought to result, in part, from impairments in contingency awareness during conditioning to cues that predict aversive or fearful outcomes. Dopamine neurons of the ventral midbrain exhibit heterogeneous responses to aversive stimuli that are thought to provide a critical modulatory signal to facilitate orientation to environmental changes and assignment of motivational value to unexpected events. Here we describe a mouse model in which activation of dopamine neurons in response to an aversive stimulus is attenuated by conditional genetic inactivation of functional NMDA receptors on dopamine neurons.

Ventral tegmental area and substantia nigra neural correlates of spatial learning.

The ventral tegmental area (VTA) and substantia nigra pars compacta (SNc) may provide modulatory signals that, respectively, influence hippocampal (HPC)- and striatal-dependent memory. Electrophysiological studies investigating neural correlates of learning and memory of dopamine (DA) neurons during classical conditioning tasks have found DA neural activity in VTA and SNc to be tightly coupled with reinforcement expectations. Also, VTA integrity and DA in HPC have been found to regulate the encoding of HPC-dependent memories.

Independent neural coding of reward and movement by pedunculopontine tegmental nucleus neurons in freely navigating rats.

Phasic firing of dopamine (DA) neurons in the ventral tegmental area (VTA) and substantia nigra (SN) is likely to be crucial for reward processing that guides learning. One of the key structures implicated in the regulation of this DA burst firing is the pedunculopontine tegmental nucleus (PPTg), which projects to both the VTA and SN. Different literatures suggest that the PPTg serves as a sensory-gating area for DA cells or it regulates voluntary movement.

Hippocampal episode fields develop with learning.

Several recent studies have shown that hippocampal neurons fire during the delay period in between trials and that these firing patterns differ when different behaviors are required, suggesting that the neuronal responses may be involved in maintaining the memories needed for the upcoming trial. In particular, one study found that hippocampal neurons reliably fired at particular times, referred to as "episode fields" (EFs), during the delay period of a spatial alternation task (Pastalkova et al. (2008) Science 321:1322-1327).

Conjunctive encoding of movement and reward by ventral tegmental area neurons in the freely navigating rodent.

As one of the two main sources of brain dopamine, the ventral tegmental area (VTA) is important for several complex functions, including motivation, reward prediction, and contextual learning. Although many studies have identified the potential neural substrate of VTA dopaminergic activity in reward prediction functions during Pavlovian and operant conditioning tasks, less is understood about the role of VTA neuronal activity in motivated behaviors and more naturalistic forms of context-dependent learning. Therefore, VTA neural activity was recorded as rats performed a spatial memory task under varying contextual conditions.

Ventral tegmental area disruption selectively affects CA1/CA2 but not CA3 place fields during a differential reward working memory task.

Hippocampus (HPC) receives dopaminergic (DA) projections from the ventral tegmental area (VTA) and substantia nigra. These inputs appear to provide a modulatory signal that influences HPC dependent behaviors and place fields. We examined how efferent projections from VTA to HPC influence spatial working memory and place fields when the reward context changes.

Neuronal representation of conditioned taste in the basolateral amygdala of rats.

Animals develop robust learning and long lasting taste aversion memory once they experience a new taste that is followed by visceral discomfort. A large body of literature has supported the hypothesis that basolateral amygdala (BLA) plays a critical role in the acquisition and extinction of such conditioned taste aversions (CTA). Despite the evidence that BLA is crucially engaged during CTA training, it is unclear how BLA neural activity represents the conditioned tastes.

Context dependent effects of ventral tegmental area inactivation on spatial working memory.

Rats were tested on a hippocampus dependent win-shift working memory task in familiar or novel environments after receiving bilateral ventral tegmental area infusions of baclofen. Baclofen infusion disrupted working memory performance in both familiar and novel environments. In addition, baclofen infusion selectively disrupted short-term working memory in the novel environment.

Disruption of NMDAR-dependent burst firing by dopamine neurons provides selective assessment of phasic dopamine-dependent behavior.

Midbrain dopamine (DA) neurons fire in 2 characteristic modes, tonic and phasic, which are thought to modulate distinct aspects of behavior. However, the inability to selectively disrupt these patterns of activity has hampered the precise definition of the function of these modes of signaling. Here, we addressed the role of phasic DA in learning and other DA-dependent behaviors by attenuating DA neuron burst firing and subsequent DA release, without altering tonic neural activity.