Neural dynamics in the orbitofrontal cortex reveal cognitive strategies.

Behavior is sloppy: a multitude of cognitive strategies can produce similar behavioral read-outs. An underutilized approach is to combine multifaceted behavioral analyses with neural recordings to resolve cognitive strategies. Here we show that rats performing a decision-making task exhibit distinct strategies over training, and these cognitive strategies are decipherable from orbitofrontal cortex (OFC) neural dynamics.

Dopamine transients encode reward prediction errors independent of learning rates.

Biological accounts of reinforcement learning posit that dopamine encodes reward prediction errors (RPEs), which are multiplied by a learning rate to update state or action values. These values are thought to be represented by corticostriatal synaptic weights, which are updated by dopamine-dependent plasticity. This suggests that dopamine release reflects the product of the learning rate and RPE.

Acetylcholine demixes heterogeneous dopamine signals for learning and moving.

Midbrain dopamine neurons promote reinforcement learning and movement vigor. A major outstanding question is how dopamine-recipient neurons in the striatum parse these heterogeneous signals. Here we characterized dopamine and acetylcholine release in the dorsomedial striatum (DMS) of rats performing a decision-making task.

Dopamine transients encode reward prediction errors independent of learning rates.

Biological accounts of reinforcement learning posit that dopamine encodes reward prediction errors (RPEs), which are multiplied by a learning rate to update state or action values. These values are thought to be represented in synaptic weights in the striatum, and updated by dopamine-dependent plasticity, suggesting that dopamine release might reflect the product of the learning rate and RPE. Here, we leveraged the fact that animals learn faster in volatile environments to characterize dopamine encoding of learning rates in the nucleus accumbens core (NAcc).

Compositional pretraining improves computational efficiency and matches animal behavior on complex tasks.

1Recurrent neural networks (RNN) are ubiquitously used in neuroscience to capture both neural dynamics and behaviors of living systems. However, when it comes to complex cognitive tasks, training RNNs with traditional methods can prove difficult and fall short of capturing crucial aspects of animal behavior. Here we propose a principled approach for identifying and incorporating compositional tasks as part of RNN training.

Distinct value computations support rapid sequential decisions.

The value of the environment determines animals' motivational states and sets expectations for error-based learning. How are values computed? Reinforcement learning systems can store or cache values of states or actions that are learned from experience, or they can compute values using a model of the environment to simulate possible futures. These value computations have distinct trade-offs, and a central question is how neural systems decide which computations to use or whether/how to combine them.

Distinct value computations support rapid sequential decisions.

The value of the environment determines animals' motivational states and sets expectations for error-based learning. How are values computed? Reinforcement learning systems can store or "cache" values of states or actions that are learned from experience, or they can compute values using a model of the environment to simulate possible futures3. These value computations have distinct trade-offs, and a central question is how neural systems decide which computations to use or whether/how to combine them.

Transformation of acoustic information to sensory decision variables in the parietal cortex.

The process by which sensory evidence contributes to perceptual choices requires an understanding of its transformation into decision variables. Here, we address this issue by evaluating the neural representation of acoustic information in the auditory cortex-recipient parietal cortex, while gerbils either performed a two-alternative forced-choice auditory discrimination task or while they passively listened to identical acoustic stimuli. During task engagement, stimulus identity decoding performance from simultaneously recorded parietal neurons significantly correlated with psychometric sensitivity.

Subpopulations of neurons in lOFC encode previous and current rewards at time of choice.

Studies of neural dynamics in lateral orbitofrontal cortex (lOFC) have shown that subsets of neurons that encode distinct aspects of behavior, such as value, may project to common downstream targets. However, it is unclear whether reward history, which may subserve lOFC's well-documented role in learning, is represented by functional subpopulations in lOFC. Previously, we analyzed neural recordings from rats performing a value-based decision-making task, and we documented trial-by-trial learning that required lOFC (Constantinople et al.

Tracking self-performance in the prefrontal cortex: It's layered.

In this issue of Cell, Spellman and colleagues record and manipulate the activity of neurons in the medial prefrontal cortex of mice performing a task in which they must pay attention to different stimuli. They show that this brain region is important for monitoring the animals' performance, and neurons that appear to contribute to behavior reside in deep cortical layers.

Parietal Cortex Is Required for the Integration of Acoustic Evidence.

Sensory-driven decisions are formed by accumulating information over time. Although parietal cortex activity is thought to represent accumulated evidence for sensory-based decisions, recent perturbation studies in rodents and non-human primates have challenged the hypothesis that these representations actually influence behavior. Here, we asked whether the parietal cortex integrates acoustic features from auditory cortical inputs during a perceptual decision-making task.

Lateral orbitofrontal cortex promotes trial-by-trial learning of risky, but not spatial, biases.

Individual choices are not made in isolation but are embedded in a series of past experiences, decisions, and outcomes. The effects of past experiences on choices, often called sequential biases, are ubiquitous in perceptual and value-based decision-making, but their neural substrates are unclear. We trained rats to choose between cued guaranteed and probabilistic rewards in a task in which outcomes on each trial were independent.

An Analysis of Decision under Risk in Rats.

In 1979, Daniel Kahneman and Amos Tversky published a ground-breaking paper titled "Prospect Theory: An Analysis of Decision under Risk," which presented a behavioral economic theory that accounted for the ways in which humans deviate from economists' normative workhorse model, Expected Utility Theory [1, 2]. For example, people exhibit probability distortion (they overweight low probabilities), loss aversion (losses loom larger than gains), and reference dependence (outcomes are evaluated as gains or losses relative to an internal reference point). We found that rats exhibited many of these same biases, using a task in which rats chose between guaranteed and probabilistic rewards.

Fronto-parietal Cortical Circuits Encode Accumulated Evidence with a Diversity of Timescales.

Decision-making in dynamic environments often involves accumulation of evidence, in which new information is used to update beliefs and select future actions. Using in vivo cellular resolution imaging in voluntarily head-restrained rats, we examined the responses of neurons in frontal and parietal cortices during a pulse-based accumulation of evidence task. Neurons exhibited activity that predicted the animal's upcoming choice, previous choice, and graded responses that reflected the strength of the accumulated evidence.

Sources of noise during accumulation of evidence in unrestrained and voluntarily head-restrained rats.

Decision-making behavior is often characterized by substantial variability, but its source remains unclear. We developed a visual accumulation of evidence task designed to quantify sources of noise and to be performed during voluntary head restraint, enabling cellular resolution imaging in future studies. Rats accumulated discrete numbers of flashes presented to the left and right visual hemifields and indicated the side that had the greater number of flashes.

Effects and mechanisms of wakefulness on local cortical networks.

Mammalian brains generate internal activity independent of environmental stimuli. Internally generated states may bring about distinct cortical processing modes. To investigate how brain state impacts cortical circuitry, we recorded intracellularly from the same neurons, under anesthesia and subsequent wakefulness, in rat barrel cortex.

Quantitative analysis of neurons with Kv3 potassium channel subunits, Kv3.1b and Kv3.2, in macaque primary visual cortex.

Voltage-gated potassium channels that are composed of Kv3 subunits exhibit distinct electrophysiological properties: activation at more depolarized potentials than other voltage-gated K+ channels and fast kinetics. These channels have been shown to contribute to the high-frequency firing of fast-spiking (FS) GABAergic interneurons in the rat and mouse brain. In the rodent neocortex there are distinct patterns of expression for the Kv3.