Distributional reinforcement learning in prefrontal cortex.

The prefrontal cortex is crucial for learning and decision-making. Classic reinforcement learning (RL) theories center on learning the expectation of potential rewarding outcomes and explain a wealth of neural data in the prefrontal cortex. Distributional RL, on the other hand, learns the full distribution of rewarding outcomes and better explains dopamine responses.

A cognitive map for value-guided choice in ventromedial prefrontal cortex.

The prefrontal cortex is crucial for economic decision-making and representing the value of options. However, how such representations facilitate flexible decisions remains unknown. We reframe economic decision-making in prefrontal cortex in line with representations of structure within the medial temporal lobe because such cognitive map representations are known to facilitate flexible behaviour.

A Diversity of Intrinsic Timescales Underlie Neural Computations.

Neural processing occurs across a range of temporal scales. To facilitate this, the brain uses fast-changing representations reflecting momentary sensory input alongside more temporally extended representations, which integrate across both short and long temporal windows. The temporal flexibility of these representations allows animals to behave adaptively.

A circuit mechanism for decision-making biases and NMDA receptor hypofunction.

Decision-making biases can be features of normal behaviour, or deficits underlying neuropsychiatric symptoms. We used behavioural psychophysics, spiking-circuit modelling and pharmacological manipulations to explore decision-making biases during evidence integration. Monkeys showed a pro-variance bias (PVB): a preference to choose options with more variable evidence.

Combined model-free and model-sensitive reinforcement learning in non-human primates.

Contemporary reinforcement learning (RL) theory suggests that potential choices can be evaluated by strategies that may or may not be sensitive to the computational structure of tasks. A paradigmatic model-free (MF) strategy simply repeats actions that have been rewarded in the past; by contrast, model-sensitive (MS) strategies exploit richer information associated with knowledge of task dynamics. MF and MS strategies should typically be combined, because they have complementary statistical and computational strengths; however, this tradeoff between MF/MS RL has mostly only been demonstrated in humans, often with only modest numbers of trials.

Visual fixation patterns during economic choice reflect covert valuation processes that emerge with learning.

Visual fixations play a vital role in decision making. Recent studies have demonstrated that the longer subjects fixate an option, the more likely they are to choose it. However, the role of evaluating stimuli covertly (i.

Triple dissociation of attention and decision computations across prefrontal cortex.

Naturalistic decision-making typically involves sequential deployment of attention to choice alternatives to gather information before a decision is made. Attention filters how information enters decision circuits, thus implying that attentional control may shape how decision computations unfold. We recorded neuronal activity from three subregions of the prefrontal cortex (PFC) while monkeys performed an attention-guided decision-making task.

Reconciling persistent and dynamic hypotheses of working memory coding in prefrontal cortex.

Competing accounts propose that working memory (WM) is subserved either by persistent activity in single neurons or by dynamic (time-varying) activity across a neural population. Here, we compare these hypotheses across four regions of prefrontal cortex (PFC) in an oculomotor-delayed-response task, where an intervening cue indicated the reward available for a correct saccade. WM representations were strongest in ventrolateral PFC neurons with higher intrinsic temporal stability (time-constant).

Correction: Approach-Induced Biases in Human Information Sampling.

[This corrects the article DOI: 10.1371/journal.pbio.

Approach-Induced Biases in Human Information Sampling.

Information sampling is often biased towards seeking evidence that confirms one's prior beliefs. Despite such biases being a pervasive feature of human behavior, their underlying causes remain unclear. Many accounts of these biases appeal to limitations of human hypothesis testing and cognition, de facto evoking notions of bounded rationality, but neglect more basic aspects of behavioral control.

Autocorrelation structure at rest predicts value correlates of single neurons during reward-guided choice.

Correlates of value are routinely observed in the prefrontal cortex (PFC) during reward-guided decision making. In previous work (Hunt et al., 2015), we argued that PFC correlates of chosen value are a consequence of varying rates of a dynamical evidence accumulation process.

Neural Signatures of Value Comparison in Human Cingulate Cortex during Decisions Requiring an Effort-Reward Trade-off.

Unlabelled: Integrating costs and benefits is crucial for optimal decision-making. Although much is known about decisions that involve outcome-related costs (e.g.

Prioritising the relevant information for learning and decision making within orbital and ventromedial prefrontal cortex.

Our environment and internal states are frequently complex, ambiguous and dynamic, meaning we need to have selection mechanisms to ensure we are basing our decisions on currently relevant information. Here, we review evidence that orbitofrontal (OFC) and ventromedial prefrontal cortex (VMPFC) play conserved, critical but distinct roles in this process. While OFC may use specific sensory associations to enhance task-relevant information, particularly in the context of learning, VMPFC plays a role in ensuring irrelevant information does not impinge on the decision in hand.

Capturing the temporal evolution of choice across prefrontal cortex.

Activity in prefrontal cortex (PFC) has been richly described using economic models of choice. Yet such descriptions fail to capture the dynamics of decision formation. Describing dynamic neural processes has proven challenging due to the problem of indexing the internal state of PFC and its trial-by-trial variation.

Behavioral modeling of human choices reveals dissociable effects of physical effort and temporal delay on reward devaluation.

There has been considerable interest from the fields of biology, economics, psychology, and ecology about how decision costs decrease the value of rewarding outcomes. For example, formal descriptions of how reward value changes with increasing temporal delays allow for quantifying individual decision preferences, as in animal species populating different habitats, or normal and clinical human populations. Strikingly, it remains largely unclear how humans evaluate rewards when these are tied to energetic costs, despite the surge of interest in the neural basis of effort-guided decision-making and the prevalence of disorders showing a diminished willingness to exert effort (e.

Single-neuron mechanisms underlying cost-benefit analysis in frontal cortex.

Effective decision-making requires consideration of costs and benefits. Previous studies have implicated orbitofrontal cortex (OFC), dorsolateral prefrontal cortex (DLPFC), and anterior cingulate cortex (ACC) in cost-benefit decision-making. Yet controversy remains about whether different decision costs are encoded by different brain areas, and whether single neurons integrate costs and benefits to derive a subjective value estimate for each choice alternative.

Contrasting reward signals in the orbitofrontal cortex and anterior cingulate cortex.

Damage to the orbitofrontal cortex (OFC) and anterior cingulate cortex (ACC) impairs decision making, but the underlying value computations that cause such impairments remain unclear. Both the OFC and ACC encode a wide variety of signals correlated with decision making. The current challenge is to determine how these two different areas support decision-making processes.

Double dissociation of value computations in orbitofrontal and anterior cingulate neurons.

Damage to prefrontal cortex (PFC) impairs decision-making, but the underlying value computations that might cause such impairments remain unclear. Here we report that value computations are doubly dissociable among PFC neurons. Although many PFC neurons encoded chosen value, they used opponent encoding schemes such that averaging the neuronal population extinguished value coding.

Decision making and reward in frontal cortex: complementary evidence from neurophysiological and neuropsychological studies.

Patients with damage to the prefrontal cortex (PFC)--especially the ventral and medial parts of PFC--often show a marked inability to make choices that meet their needs and goals. These decision-making impairments often reflect both a deficit in learning concerning the consequences of a choice, as well as deficits in the ability to adapt future choices based on experienced value of the current choice. Thus, areas of PFC must support some value computations that are necessary for optimal choice.

Oscillatory phase coupling coordinates anatomically dispersed functional cell assemblies.

Hebb proposed that neuronal cell assemblies are critical for effective perception, cognition, and action. However, evidence for brain mechanisms that coordinate multiple coactive assemblies remains lacking. Neuronal oscillations have been suggested as one possible mechanism for cell assembly coordination.

Heterogeneous reward signals in prefrontal cortex.

Neurons encode upcoming rewards throughout frontal cortex. Recent papers have helped to determine that these signals play different roles in different frontal regions. Neurons in orbitofrontal cortex (PFo) appear to be responsible for calculating the specific value of an expected reward, information that can help efficiently guide decision-making.

Encoding of reward and space during a working memory task in the orbitofrontal cortex and anterior cingulate sulcus.

Several lines of research indicate that emotional and motivational information may be useful in guiding the allocation of attentional resources. Two areas of the frontal lobe that are particularly implicated in the encoding of motivational information are the orbital prefrontal cortex (PFo) and the dorsomedial region of prefrontal cortex, specifically the anterior cingulate sulcus (PFcs). However, it remains unclear whether these areas use this information to influence spatial attention.

Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variables.

Damage to the frontal lobe can cause severe decision-making impairments. A mechanism that may underlie this is that neurons in the frontal cortex encode many variables that contribute to the valuation of a choice, such as its costs, benefits and probability of success. However, optimal decision-making requires that one considers these variables, not only when faced with the choice, but also when evaluating the outcome of the choice, in order to adapt future behaviour appropriately.