Does phasic dopamine release cause policy updates?

Phasic dopamine activity is believed to both encode reward-prediction errors (RPEs) and to cause the adaptations that these errors engender. If so, a rat working for optogenetic stimulation of dopamine neurons will repeatedly update its policy and/or action values, thus iteratively increasing its work rate. Here, we challenge this view by demonstrating stable, non-maximal work rates in the face of repeated optogenetic stimulation of midbrain dopamine neurons.

Dopamine neurons do not constitute an obligatory stage in the final common path for the evaluation and pursuit of brain stimulation reward.

The neurobiological study of reward was launched by the discovery of intracranial self-stimulation (ICSS). Subsequent investigation of this phenomenon provided the initial link between reward-seeking behavior and dopaminergic neurotransmission. We re-evaluated this relationship by psychophysical, pharmacological, optogenetic, and computational means.

Learning to use past evidence in a sophisticated world model.

Humans and other animals are able to discover underlying statistical structure in their environments and exploit it to achieve efficient and effective performance. However, such structure is often difficult to learn and use because it is obscure, involving long-range temporal dependencies. Here, we analysed behavioural data from an extended experiment with rats, showing that the subjects learned the underlying statistical structure, albeit suffering at times from immediate inferential imperfections as to their current state within it.

Valuation of opportunity costs by rats working for rewarding electrical brain stimulation.

Pursuit of one goal typically precludes simultaneous pursuit of another. Thus, each exclusive activity entails an "opportunity cost:" the forgone benefits from the next-best activity eschewed. The present experiment estimates, in laboratory rats, the function that maps objective opportunity costs into subjective ones.

The Effects of Electrical and Optical Stimulation of Midbrain Dopaminergic Neurons on Rat 50-kHz Ultrasonic Vocalizations.

Rationale: Adult rats emit ultrasonic vocalizations (USVs) at around 50-kHz; these commonly occur in contexts that putatively engender positive affect. While several reports indicate that dopaminergic (DAergic) transmission plays a role in the emission of 50-kHz calls, the pharmacological evidence is mixed. Different modes of dopamine (DA) release (i.

The effect of probability discounting on reward seeking: a three-dimensional perspective.

Rats will work for electrical stimulation of the medial forebrain bundle. The rewarding effect arises from the volleys of action potentials fired by the stimulation and subsequent spatio-temporal integration of their post-synpatic impact. The proportion of time allocated to self-stimulation depends on the intensity of the rewarding effect as well as on other key determinants of decision-making, such as subjective opportunity costs and reward probability.

Optimal indolence: a normative microscopic approach to work and leisure.

Dividing limited time between work and leisure when both have their attractions is a common everyday decision. We provide a normative control-theoretic treatment of this decision that bridges economic and psychological accounts. We show how our framework applies to free-operant behavioural experiments in which subjects are required to work (depressing a lever) for sufficient total time (called the price) to receive a reward.

Validation and extension of the reward-mountain model.

The reward-mountain model relates the vigor of reward seeking to the strength and cost of reward. Application of this model provides information about the stage of processing at which manipulations such as drug administration, lesions, deprivation states, and optogenetic interventions act to alter reward seeking. The model has been updated by incorporation of new information about frequency following in the directly stimulated neurons responsible for brain stimulation reward and about the function that maps objective opportunity costs into subjective ones.

Role of dopamine tone in the pursuit of brain stimulation reward.

Dopaminergic neurons contribute to intracranial self-stimulation (ICSS) and other reward-seeking behaviors, but it is not yet known where dopaminergic neurons intervene in the neural circuitry underlying reward pursuit or which psychological processes are involved. In rats working for electrical stimulation of the medial forebrain bundle, we assessed the effect of GBR-12909 (1-[2-[bis(4-fluorophenyl)-methoxy]ethyl]-4-[3- phenylpropyl]piperazine), a specific blocker of the dopamine transporter. Operant performance was measured as a function of the strength and cost of electrical stimulation.

Cannabinoid receptor blockade reduces the opportunity cost at which rats maintain operant performance for rewarding brain stimulation.

There is ample evidence that blockade of CB(1) receptors reduces reward seeking. However, the reported effects of CB(1) blockade on performance for rewarding electrical brain stimulation stand out as an exception. By applying a novel method for conceptualizing and measuring reward seeking, we show that AM-251, a CB(1) receptor antagonist, does indeed decrease performance for rewarding electrical stimulation of the medial forebrain bundle in rats.

At what stage of neural processing does cocaine act to boost pursuit of rewards?

Dopamine-containing neurons have been implicated in reward and decision making. One element of the supporting evidence is that cocaine, like other drugs that increase dopaminergic neurotransmission, powerfully potentiates reward seeking. We analyze this phenomenon from a novel perspective, introducing a new conceptual framework and new methodology for determining the stage(s) of neural processing at which drugs, lesions and physiological manipulations act to influence reward-seeking behavior.

The effects of reinforcer magnitude on timing in rats.

The relation between reinforcer magnitude and timing behavior was studied using a peak procedure. Four rats received multiple consecutive sessions with both low and high levels of brain stimulation reward (BSR). Rats paused longer and had later start times during sessions when their responses were reinforced with low-magnitude BSR.

Prolonged rewarding stimulation of the rat medial forebrain bundle: neurochemical and behavioral consequences.

Extracellular dopamine levels were measured in the rat nucleus accumbens by means of in vivo microdialysis. Delivery of rewarding medial forebrain bundle stimulation at a low rate (5 trains/min) produced a sustained elevation of dopamine levels, regardless of whether train onset was predictable. When the rate of train delivery was increased to 40 trains/min, dopamine levels rose rapidly during the first 40 min but then declined toward the baseline range.

Growth of brain stimulation reward as a function of duration and stimulation strength.

The strength of a train of rewarding brain stimulation required to support a criterion level of operant performance declines hyperbolically as the duration is increased. This finding has been attributed to a process of leaky integration. However, the rate at which integration approaches asymptote has been shown to depend on stimulation strength, a finding that differs from the behavior of a simple leaky integrator.

Operant tempo varies with reinforcement rate: implications for measurement of reward efficacy.

Herrnstein's melioration theory has been used to account for the hyperbolic form of the single operant matching law and to scale the effectiveness of reinforcing brain stimulation. Underlying this scaling method is the assumption that the mean rate of responding during operant bouts (the response 'tempo') is fixed and does not vary with the rate of reinforcement. The validity of this account was assessed by testing the constant-tempo assumption via a survivor analysis of the distributions of inter-response times at different variable-intervals (VIs) in rats responding for rewarding electrical stimulation of the lateral hypothalamus.