Neural substrates of parallel devaluation-sensitive and devaluation-insensitive Pavlovian learning in humans.

We aim to differentiate the brain regions involved in the learning and encoding of Pavlovian associations sensitive to changes in outcome value from those that are not sensitive to such changes by combining a learning task with outcome devaluation, eye-tracking, and functional magnetic resonance imaging in humans. Contrary to theoretical expectation, voxels correlating with reward prediction errors in the ventral striatum and subgenual cingulate appear to be sensitive to devaluation. Moreover, regions encoding state prediction errors appear to be devaluation insensitive.

Neural substrates of parallel devaluation-sensitive and devaluation-insensitive Pavlovian learning in humans.

Pavlovian learning depends on multiple and parallel associations leading to distinct classes of conditioned responses that vary in their flexibility following changes in the value of an associated outcome. Here, we aimed to differentiate brain areas involved in learning and encoding associations that are sensitive to changes in the value of an outcome from those that are not sensitive to such changes. To address this question, we combined a Pavlovian learning task with outcome devaluation, eye-tracking and functional magnetic resonance imaging.

Mesolimbic Neurobehavioral Mechanisms of Reward Motivation in Anorexia Nervosa: A Multimodal Imaging Study.

Diminished motivation to pursue and obtain primary and secondary rewards has been demonstrated in anorexia nervosa (AN). However, the neurobehavioral mechanisms underlying the behavioral activation component of aberrant reward motivation remains incompletely understood. This work aims to explore this underexplored facet of reward motivation in AN.

White matter tracts characteristics in habitual decision-making circuit underlie ritual behaviors in anorexia nervosa.

Anorexia nervosa (AN) is a difficult to treat, pernicious psychiatric disorder that has been linked to decision-making abnormalities. We examined the structural characteristics of habitual and goal-directed decision-making circuits and their connecting white matter tracts in 32 AN and 43 healthy controls across two independent data sets of adults and adolescents as an explanatory sub-study. Total bilateral premotor/supplementary motor area-putamen tracts in the habit circuit had a significantly higher volume in adults with AN, relative to controls.

Restructuring of amygdala subregion apportion across adolescence.

Total amygdala volumes develop in association with sex and puberty, and postmortem studies find neuronal numbers increase in a nuclei specific fashion across development. Thus, amygdala subregions and composition may evolve with age. Our goal was to examine if amygdala subregion absolute volumes and/or relative proportion varies as a function of age, sex, or puberty in a large sample of typically developing adolescents (N = 408, 43 % female, 10-17 years).

Behavioural evidence for parallel outcome-sensitive and outcome-insensitive Pavlovian learning systems in humans.

There is a dichotomy in instrumental conditioning between goal-directed actions and habits that are distinguishable on the basis of their relative sensitivity to changes in outcome value. It is less clear whether a similar distinction applies in Pavlovian conditioning, where responses have been found to be predominantly outcome sensitive. To test for both devaluation insensitive and devaluation sensitive Pavlovian conditioning in humans, we conducted four experiments combining Pavlovian conditioning and outcome devaluation procedures while measuring multiple conditioned responses.

Evidence for model-based encoding of Pavlovian contingencies in the human brain.

Prominent accounts of Pavlovian conditioning successfully approximate the frequency and intensity of conditioned responses under the assumption that learning is exclusively model-free; that animals do not develop a cognitive map of events. However, these model-free approximations fall short of comprehensively capturing learning and behavior in Pavlovian conditioning. We therefore performed multivoxel pattern analysis of high-resolution functional MRI data in human participants to test for the encoding of stimulus-stimulus associations that could support model-based computations during Pavlovian conditioning.

A high-resolution probabilistic in vivo atlas of human subcortical brain nuclei.

Recent advances in magnetic resonance imaging methods, including data acquisition, pre-processing and analysis, have benefited research on the contributions of subcortical brain nuclei to human cognition and behavior. At the same time, these developments have led to an increasing need for a high-resolution probabilistic in vivo anatomical atlas of subcortical nuclei. In order to address this need, we constructed high spatial resolution, three-dimensional templates, using high-accuracy diffeomorphic registration of T- and T- weighted structural images from 168 typical adults between 22 and 35 years old.

Neural computations underlying inverse reinforcement learning in the human brain.

In inverse reinforcement learning an observer infers the reward distribution available for actions in the environment solely through observing the actions implemented by another agent. To address whether this computational process is implemented in the human brain, participants underwent fMRI while learning about slot machines yielding hidden preferred and non-preferred food outcomes with varying probabilities, through observing the repeated slot choices of agents with similar and dissimilar food preferences. Using formal model comparison, we found that participants implemented inverse RL as opposed to a simple imitation strategy, in which the actions of the other agent are copied instead of inferring the underlying reward structure of the decision problem.

Distinct prediction errors in mesostriatal circuits of the human brain mediate learning about the values of both states and actions: evidence from high-resolution fMRI.

Prediction-error signals consistent with formal models of "reinforcement learning" (RL) have repeatedly been found within dopaminergic nuclei of the midbrain and dopaminoceptive areas of the striatum. However, the precise form of the RL algorithms implemented in the human brain is not yet well determined. Here, we created a novel paradigm optimized to dissociate the subtypes of reward-prediction errors that function as the key computational signatures of two distinct classes of RL models-namely, "actor/critic" models and action-value-learning models (e.

Learning, Reward, and Decision Making.

In this review, we summarize findings supporting the existence of multiple behavioral strategies for controlling reward-related behavior, including a dichotomy between the goal-directed or model-based system and the habitual or model-free system in the domain of instrumental conditioning and a similar dichotomy in the realm of Pavlovian conditioning. We evaluate evidence from neuroscience supporting the existence of at least partly distinct neuronal substrates contributing to the key computations necessary for the function of these different control systems. We consider the nature of the interactions between these systems and show how these interactions can lead to either adaptive or maladaptive behavioral outcomes.

In vivo delineation of subdivisions of the human amygdaloid complex in a high-resolution group template.

The nuclei of the human amygdala remain difficult to distinguish in individual subject structural magnetic resonance images. However, interpretation of the amygdala's role in whole brain networks requires accurate localization of functional activity to a particular nucleus or subgroup of nuclei. To address this, high spatial resolution, three-dimensional templates, using joint high accuracy diffeomorphic registration of T1- and T2-weighted structural images from 168 typical adults between 22 and 35 years old released by the Human Connectome Project were constructed.

Regional specialization within the human striatum for diverse psychological functions.

Decades of animal and human neuroimaging research have identified distinct, but overlapping, striatal zones, which are interconnected with separable corticostriatal circuits, and are crucial for the organization of functional systems. Despite continuous efforts to subdivide the human striatum based on anatomical and resting-state functional connectivity, characterizing the different psychological processes related to each zone remains a work in progress. Using an unbiased, data-driven approach, we analyzed large-scale coactivation data from 5,809 human imaging studies.

Distinct Contributions of Ventromedial and Dorsolateral Subregions of the Human Substantia Nigra to Appetitive and Aversive Learning.

Unlabelled: The role of neurons in the substantia nigra (SN) and ventral tegmental area (VTA) of the midbrain in contributing to the elicitation of reward prediction errors during appetitive learning has been well established. Less is known about the differential contribution of these midbrain regions to appetitive versus aversive learning, especially in humans. Here we scanned human participants with high-resolution fMRI focused on the SN and VTA while they participated in a sequential Pavlovian conditioning paradigm involving an appetitive outcome (a pleasant juice), as well as an aversive outcome (an unpleasant bitter and salty flavor).

Inhibiting PKMζ reveals dorsal lateral and dorsal medial striatum store the different memories needed to support adaptive behavior.

Evidence suggests that two regions of the striatum contribute differential support to instrumental response selection. The dorsomedial striatum (DMS) is thought to support expectancy-mediated actions, and the dorsolateral striatum (DLS) is thought to support habits. Currently it is unclear whether these regions store task-relevant information or just coordinate the learning and retention of these solutions by other brain regions.

Expectancy, ambiguity, and behavioral flexibility: separable and complementary roles of the orbital frontal cortex and amygdala in processing reward expectancies.

Appetitive goal-directed behavior can be associated with a cue-triggered expectancy that it will lead to a particular reward, a process thought to depend on the OFC and basolateral amygdala complex. We developed a biologically informed neural network model of this system to investigate the separable and complementary roles of these areas as the main components of a flexible expectancy system. These areas of interest are part of a neural network with additional subcortical areas, including the central nucleus of amygdala, ventral (limbic) and dorsomedial (associative) striatum.

Computational models of cognitive control.

Cognitive control refers to the ability to perform task-relevant processing in the face of other distractions or other forms of interference, in the absence of strong environmental support. It depends on the integrity of the prefrontal cortex and associated biological structures (e.g.

Emotional salience changes the focus of spatial attention.

The interaction of emotional stimulus processing and attentional mechanisms has become a topic of intensive investigation. The present study combined aversive conditioning with a spatial attention paradigm that by simultaneously recording event-related potentials (ERPs) allowed for accessing the distribution of auditory spatial attention. In an initial conditioning phase, seven spatial locations (conditioned stimuli; CS-) were associated with an emotionally neutral and one additional location (CS+) with an emotionally aversive sound (US).

Attentional control of associative learning--a possible role of the central cholinergic system.

How does attention interact with learning? Kruschke [Kruschke, J.K. (2001).