Does phasic dopamine release cause policy updates?

Phasic dopamine activity is believed to both encode reward-prediction errors (RPEs) and to cause the adaptations that these errors engender. If so, a rat working for optogenetic stimulation of dopamine neurons will repeatedly update its policy and/or action values, thus iteratively increasing its work rate. Here, we challenge this view by demonstrating stable, non-maximal work rates in the face of repeated optogenetic stimulation of midbrain dopamine neurons.

The priming effect of rewarding brain stimulation in rats depends on both the cost and strength of reward but survives blockade of D2-like dopamine receptors.

Receipt of an intense reward boosts motivation to work for more of that reward. This phenomenon is called the priming effect of rewards. Using a novel measurement method, we show that the priming effect of rewarding electrical brain stimulation depends on the cost, as well as on the strength, of the anticipated reward.

The Convergence Model of Brain Reward Circuitry: Implications for Relief of Treatment-Resistant Depression by Deep-Brain Stimulation of the Medial Forebrain Bundle.

Deep-brain stimulation of the medial forebrain bundle (MFB) can provide effective, enduring relief of treatment-resistant depression. Panksepp provided an explanatory framework: the MFB constitutes the core of the neural circuitry subserving the anticipation and pursuit of rewards: the "SEEKING" system. On that view, the SEEKING system is hypoactive in depressed individuals; background electrical stimulation of the MFB alleviates symptoms by normalizing activity.

The trade-off between pulse duration and power in optical excitation of midbrain dopamine neurons approximates Bloch's law.

Optogenetic experiments reveal functional roles of specific neurons. However, functional inferences have been limited by widespread adoption of a restricted set of stimulation parameters. Broader exploration of the parameter space can deepen insight into the mapping between selective neural activity and behavior.

Dopamine neurons do not constitute an obligatory stage in the final common path for the evaluation and pursuit of brain stimulation reward.

The neurobiological study of reward was launched by the discovery of intracranial self-stimulation (ICSS). Subsequent investigation of this phenomenon provided the initial link between reward-seeking behavior and dopaminergic neurotransmission. We re-evaluated this relationship by psychophysical, pharmacological, optogenetic, and computational means.

The priming effect of food persists following blockade of dopamine receptors.

The priming effect of rewards is a boost in the vigor of reward seeking resulting from the previous receipt of a reward. Extensive work has been carried out on the priming effect of electrical brain stimulation, but much less research exists on the priming effect of natural rewards, such as food. While both reinforcement and motivation are linked with dopamine transmission in the brain, the priming effect of rewards does not appear to be dopamine-dependent.

Learning to use past evidence in a sophisticated world model.

Humans and other animals are able to discover underlying statistical structure in their environments and exploit it to achieve efficient and effective performance. However, such structure is often difficult to learn and use because it is obscure, involving long-range temporal dependencies. Here, we analysed behavioural data from an extended experiment with rats, showing that the subjects learned the underlying statistical structure, albeit suffering at times from immediate inferential imperfections as to their current state within it.

17β-estradiol locally increases phasic dopamine release in the dorsal striatum.

Studies using in vivo microdialysis have shown that 17β-estradiol (E2) increases dopamine (DA) transmission in the dorsal striatum. Both systemic administration of E2 and local infusion into the dorsal striatum rapidly enhance amphetamine-induced DA release. However, it is not known to what degree these effects reflect tonic and/or phasic DA release.

Valuation of opportunity costs by rats working for rewarding electrical brain stimulation.

Pursuit of one goal typically precludes simultaneous pursuit of another. Thus, each exclusive activity entails an "opportunity cost:" the forgone benefits from the next-best activity eschewed. The present experiment estimates, in laboratory rats, the function that maps objective opportunity costs into subjective ones.

Ventral Midbrain NMDA Receptor Blockade: From Enhanced Reward and Dopamine Inactivation.

Glutamate stimulates ventral midbrain (VM) N-Methyl-D-Aspartate receptors (NMDAR) to initiate dopamine (DA) burst firing activity, a mode of discharge associated with enhanced DA release and reward. Blockade of VM NMDAR, however, enhances brain stimulation reward (BSR), the results can be explained by a reduction in the inhibitory drive on DA neurons that is also under the control of glutamate. In this study, we used fast-scan cyclic voltammetry (FSCV) in anesthetized animals to determine whether this enhancement is associated with a change in phasic DA release in the nucleus accumbens.

The Effects of Electrical and Optical Stimulation of Midbrain Dopaminergic Neurons on Rat 50-kHz Ultrasonic Vocalizations.

Rationale: Adult rats emit ultrasonic vocalizations (USVs) at around 50-kHz; these commonly occur in contexts that putatively engender positive affect. While several reports indicate that dopaminergic (DAergic) transmission plays a role in the emission of 50-kHz calls, the pharmacological evidence is mixed. Different modes of dopamine (DA) release (i.

Robust optical fiber patch-cords for in vivo optogenetic experiments in rats.

In vivo optogenetic experiments commonly employ long lengths of optical fiber to connect the light source (commonly a laser) to the optical fiber implants in the brain. Commercially available patch cords are expensive and break easily. Researchers have developed methods to build these cables in house for in vivo experiments with rodents [1-4].

dcc Haploinsufficiency results in blunted sensitivity to cocaine enhancement of reward seeking.

Mesocortical dopamine connectivity continues to mature during adolescence. This protracted development confers increased vulnerability for environmental and genetic factors to disrupt mesocortical wiring and subsequently influence responses to drugs of abuse in adulthood. The netrin-1 receptor, DCC, orchestrates medial prefrontal cortex dopamine input during adolescence and dictates the functional organization of local circuitry.

Some work and some play: microscopic and macroscopic approaches to labor and leisure.

Given the option, humans and other animals elect to distribute their time between work and leisure, rather than choosing all of one and none of the other. Traditional accounts of partial allocation have characterised behavior on a macroscopic timescale, reporting and studying the mean times spent in work or leisure. However, averaging over the more microscopic processes that govern choices is known to pose tricky theoretical problems, and also eschews any possibility of direct contact with the neural computations involved.

The effect of probability discounting on reward seeking: a three-dimensional perspective.

Rats will work for electrical stimulation of the medial forebrain bundle. The rewarding effect arises from the volleys of action potentials fired by the stimulation and subsequent spatio-temporal integration of their post-synpatic impact. The proportion of time allocated to self-stimulation depends on the intensity of the rewarding effect as well as on other key determinants of decision-making, such as subjective opportunity costs and reward probability.

Optimal indolence: a normative microscopic approach to work and leisure.

Dividing limited time between work and leisure when both have their attractions is a common everyday decision. We provide a normative control-theoretic treatment of this decision that bridges economic and psychological accounts. We show how our framework applies to free-operant behavioural experiments in which subjects are required to work (depressing a lever) for sufficient total time (called the price) to receive a reward.

Validation and extension of the reward-mountain model.

The reward-mountain model relates the vigor of reward seeking to the strength and cost of reward. Application of this model provides information about the stage of processing at which manipulations such as drug administration, lesions, deprivation states, and optogenetic interventions act to alter reward seeking. The model has been updated by incorporation of new information about frequency following in the directly stimulated neurons responsible for brain stimulation reward and about the function that maps objective opportunity costs into subjective ones.

Role of dopamine tone in the pursuit of brain stimulation reward.

Dopaminergic neurons contribute to intracranial self-stimulation (ICSS) and other reward-seeking behaviors, but it is not yet known where dopaminergic neurons intervene in the neural circuitry underlying reward pursuit or which psychological processes are involved. In rats working for electrical stimulation of the medial forebrain bundle, we assessed the effect of GBR-12909 (1-[2-[bis(4-fluorophenyl)-methoxy]ethyl]-4-[3- phenylpropyl]piperazine), a specific blocker of the dopamine transporter. Operant performance was measured as a function of the strength and cost of electrical stimulation.

Scarce means with alternative uses: robbins' definition of economics and its extension to the behavioral and neurobiological study of animal decision making.

Almost 80 years ago, Lionel Robbins proposed a highly influential definition of the subject matter of economics: the allocation of scarce means that have alternative ends. Robbins confined his definition to human behavior, and he strove to separate economics from the natural sciences in general and from psychology in particular. Nonetheless, I extend his definition to the behavior of non-human animals, rooting my account in psychological processes and their neural underpinnings.

Cannabinoid receptor blockade reduces the opportunity cost at which rats maintain operant performance for rewarding brain stimulation.

There is ample evidence that blockade of CB(1) receptors reduces reward seeking. However, the reported effects of CB(1) blockade on performance for rewarding electrical brain stimulation stand out as an exception. By applying a novel method for conceptualizing and measuring reward seeking, we show that AM-251, a CB(1) receptor antagonist, does indeed decrease performance for rewarding electrical stimulation of the medial forebrain bundle in rats.

At what stage of neural processing does cocaine act to boost pursuit of rewards?

Dopamine-containing neurons have been implicated in reward and decision making. One element of the supporting evidence is that cocaine, like other drugs that increase dopaminergic neurotransmission, powerfully potentiates reward seeking. We analyze this phenomenon from a novel perspective, introducing a new conceptual framework and new methodology for determining the stage(s) of neural processing at which drugs, lesions and physiological manipulations act to influence reward-seeking behavior.

Rattus Psychologicus: construction of preferences by self-stimulating rats.

Behavioral economists have proposed that human preferences are constructed during their elicitation and are thus influenced by the elicitation procedure. For example, different preferences are expressed when options are encountered one at a time or concurrently. This phenomenon has been attributed to differences in the "evaluability" of a particular attribute when comparison to an option with a different value of this attribute is or is not available.

Dynamic changes in dopamine tone during self-stimulation of the ventral tegmental area in rats.

In a prior study, phasic release of dopamine (DA) in the nucleus accumbens (NAc) was only transiently and rarely detected by means of fast-scan cyclic voltammetry (FCSV) in rats already trained to work for electrical stimulation of the ventral tegmental area (VTA) on a continuous reinforcement schedule. However, in rats receiving rewarding electrical stimulation via lateral hypothalamic (LH) electrodes, elevated DA tone in the NAc terminal field was detected via microdialysis for up to 2h, even when short (1.5s) inter-train intervals were employed.

The reinforcement mountain: allocation of behavior as a function of the rate and intensity of rewarding brain stimulation.

The single-operant matching law has been used to describe the relationship between time allocated to pursuit of brain stimulation reward (BSR) and the obtained rate of reinforcement. We generalize this relationship to a third dimension by including the strength of the stimulation (the number of pulses per train) as an independent dimension, and we dub the resulting 3-dimensional structure "the reinforcement mountain." The validity of generalizing the single-operant matching law in this way was assessed by determining the changes in the position of the mountain produced by increasing the stimulation current or the train duration.