People exert more effort to avoid losses than to obtain gains.

Loss aversion is a fundamental tenet of behavioral economics and has led to many real-world applications. These applications, and some laboratory studies, show that people perform better under loss-avoidance than under gain incentives. This increased performance under loss-avoidance incentives has ubiquitously been explained by the notion that loss aversion causes people to exert more effort to avoid losses than to obtain gains.

Mathematical Characterization of Changes in Fear During Exposure Therapy.

Background: During exposure therapy, patients report increases in fear that generally decrease within and across exposure sessions. Our main aim was to characterize these changes in fear ratings mathematically; a secondary aim was to test whether the resulting model would help to predict treatment outcome.

Methods: We applied tools of computational psychiatry to a previously published dataset in which 30 women with spider phobia were randomly assigned to virtual-reality exposures in a single context or in multiple contexts (n = 15 each).

Dopaminergic Disturbances in Tourette Syndrome: An Integrative Account.

Tourette syndrome (TS) is thought to involve dopaminergic disturbances, but the nature of those disturbances remains controversial. Existing hypotheses suggest that TS involves 1) supersensitive dopamine receptors, 2) overactive dopamine transporters that cause low tonic but high phasic dopamine, 3) presynaptic dysfunction in dopamine neurons, or 4) dopaminergic hyperinnervation. We review evidence that contradicts the first two hypotheses; we also note that the last two hypotheses have traditionally been considered too narrowly, explaining only small subsets of findings.

Premonitory urges and tics in Tourette syndrome: computational mechanisms and neural correlates.

Tourette syndrome is characterized by open motor behaviors - tics - but another crucial aspect of the disorder is the presence of premonitory urges: uncomfortable sensations that typically precede tics and are temporarily alleviated by tics. We review the evidence implicating the somatosensory cortices and the insula in premonitory urges and the motor cortico-basal ganglia-thalamo-cortical loop in tics. We consider how these regions interact during tic execution, suggesting that the insula plays an important role as a nexus linking the sensory and emotional character of premonitory urges with their translation into tics.

The Roles of Phasic and Tonic Dopamine in Tic Learning and Expression.

Tourette syndrome (TS) prominently involves dopaminergic disturbances, but the precise nature of those disturbances has remained elusive. A substantial body of empirical work and recent computational models have characterized the specific roles of phasic and tonic dopamine (DA) in action learning and selection, respectively. Using insights from this work and models, we suggest that TS involves increases in both phasic and tonic DA, which produce increased propensities for tic learning and expression, respectively.

A Roadmap for the Development of Applied Computational Psychiatry.

Background: Computational psychiatry is a burgeoning field that utilizes mathematical approaches to investigate psychiatric disorders, derive quantitative predictions, and integrate data across multiple levels of description. Computational psychiatry has already led to many new insights into the neurobehavioral mechanisms that underlie several psychiatric disorders, but its usefulness from a clinical standpoint is only now starting to be considered.

Methods: Examples of computational psychiatry are highlighted, and a phase-based pipeline for the development of clinical computational-psychiatry applications is proposed, similar to the phase-based pipeline used in drug development.

An Integrative Perspective on the Role of Dopamine in Schizophrenia.

We propose that schizophrenia involves a combination of decreased phasic dopamine responses for relevant stimuli and increased spontaneous phasic dopamine release. Using insights from computational reinforcement-learning models and basic-science studies of the dopamine system, we show that each of these two disturbances contributes to a specific symptom domain and explains a large set of experimental findings associated with that domain. Reduced phasic responses for relevant stimuli help to explain negative symptoms and provide a unified explanation for the following experimental findings in schizophrenia, most of which have been shown to correlate with negative symptoms: reduced learning from rewards; blunted activation of the ventral striatum, midbrain, and other limbic regions for rewards and positive prediction errors; blunted activation of the ventral striatum during reward anticipation; blunted autonomic responding for relevant stimuli; blunted neural activation for aversive outcomes and aversive prediction errors; reduced willingness to expend effort for rewards; and psychomotor slowing.

Computational psychiatry as a bridge from neuroscience to clinical applications.

Translating advances in neuroscience into benefits for patients with mental illness presents enormous challenges because it involves both the most complex organ, the brain, and its interaction with a similarly complex environment. Dealing with such complexities demands powerful techniques. Computational psychiatry combines multiple levels and types of computation with multiple types of data in an effort to improve understanding, prediction and treatment of mental illness.

Changes in corticostriatal connectivity during reinforcement learning in humans.

Many computational models assume that reinforcement learning relies on changes in synaptic efficacy between cortical regions representing stimuli and striatal regions involved in response selection, but this assumption has thus far lacked empirical support in humans. We recorded hemodynamic signals with fMRI while participants navigated a virtual maze to find hidden rewards. We fitted a reinforcement-learning algorithm to participants' choice behavior and evaluated the neural activity and the changes in functional connectivity related to trial-by-trial learning variables.

Reduced functional connectivity within the limbic cortico-striato-thalamo-cortical loop in unmedicated adults with obsessive-compulsive disorder.

Cortico-striato-thalamo-cortical (CSTC) loops project from the cortex to the striatum, then from the striatum to the thalamus via the globus pallidus, and finally from the thalamus back to the cortex again. These loops have been implicated in Obsessive-Compulsive Disorder (OCD) with particular focus on the limbic CSTC loop, which encompasses the orbitofrontal and anterior cingulate cortices, as well as the ventral striatum. Resting state functional-connectivity MRI (rs-fcMRI) studies, which examine temporal correlations in neural activity across brain regions at rest, have examined CSTC loop connectivity in patients with OCD and suggest hyperconnectivity within these loops in medicated adults with OCD.

Conscious and unconscious processes in cognitive control: a theoretical perspective and a novel empirical approach.

Controlled processing is often referred to as "voluntary" or "willful" and therefore assumed to depend entirely on conscious processes. Recent studies using subliminal-priming paradigms, however, have started to question this assumption. Specifically, these studies have shown that subliminally presented stimuli can induce adjustments in control.

Evidence that conditioned avoidance responses are reinforced by positive prediction errors signaled by tonic striatal dopamine.

We conducted an experiment in which hedonia, salience and prediction error hypotheses predicted different patterns of dopamine (DA) release in the striatum during learning of conditioned avoidance responses (CARs). The data strongly favor the latter hypothesis. It predicts that during learning of the 2-way active avoidance CAR task, positive prediction errors generated when rats do not receive an anticipated footshock (which is better than expected) cause DA release that reinforces the instrumental avoidance action.

A neurocomputational approach to obsessive-compulsive disorder.

A recent article shows that a change in a single parameter in a neural-network model of brain dynamics leads to repetitive behaviors that resist termination and towards which the network tends. These findings may have implications for obsessive-compulsive disorder and are consistent with evidence of glutamatergic hyperactivity in this disorder.

Adaptation to conflict via context-driven anticipatory signals in the dorsomedial prefrontal cortex.

Behavioral interference elicited by competing response tendencies adapts to contextual changes. Recent nonhuman primate research suggests a key mnemonic role of distinct prefrontal cells in supporting such context-driven behavioral adjustments by maintaining conflict information across trials, but corresponding prefrontal functions have yet to be probed in humans. Using event-related functional magnetic resonance imaging, we investigated the human neural substrates of contextual adaptations to conflict.

The neural circuits that generate tics in Tourette's syndrome.

Objective: The purpose of this study was to examine neural activity and connectivity within cortico-striato-thalamo-cortical circuits and to reveal circuit-based neural mechanisms that govern tic generation in Tourette's syndrome.

Method: Functional magnetic resonance imaging data were acquired from 13 individuals with Tourette's syndrome and 21 healthy comparison subjects during spontaneous or simulated tics. Independent component analysis with hierarchical partner matching was used to isolate neural activity within functionally distinct regions of cortico-striato-thalamo-cortical circuits.

Abnormal amygdalar activation and connectivity in adolescents with attention-deficit/hyperactivity disorder.

Objective: Emotional reactivity is one of the most disabling symptoms associated with attention-deficit/hyperactivity disorder (ADHD). We aimed to identify neural substrates associated with emotional reactivity and to assess the effects of stimulants on those substrates.

Method: We used functional magnetic resonance imaging (fMRI) to assess neural activity in adolescents with (n = 15) and without (n = 15) ADHD while they performed a task involving the subliminal presentation of fearful faces.

The attenuation of dysfunctional emotional processing with stimulant medication: an fMRI study of adolescents with ADHD.

Functional neuroimaging studies of attention-deficit/hyperactivity disorder (ADHD) have focused on the neural correlates of cognitive control. However, for many youths with ADHD, emotional lability is an important clinical feature of the disorder. We aimed to identify the neural substrates associated with emotional lability that were distinct from impairments in cognitive control and to assess the effects that stimulants have on those substrates.

From reinforcement learning models to psychiatric and neurological disorders.

Over the last decade and a half, reinforcement learning models have fostered an increasingly sophisticated understanding of the functions of dopamine and cortico-basal ganglia-thalamo-cortical (CBGTC) circuits. More recently, these models, and the insights that they afford, have started to be used to understand important aspects of several psychiatric and neurological disorders that involve disturbances of the dopaminergic system and CBGTC circuits. We review this approach and its existing and potential applications to Parkinson's disease, Tourette's syndrome, attention-deficit/hyperactivity disorder, addiction, schizophrenia and preclinical animal models used to screen new antipsychotic drugs.

Basal ganglia surface morphology and the effects of stimulant medications in youth with attention deficit hyperactivity disorder.

Objective: Disturbances in the basal ganglia portions of cortico-striato-thalamo-cortical circuits likely contribute to the symptoms of attention deficit hyperactivity disorder (ADHD). The authors examined the morphologic features of the basal ganglia nuclei (caudate, putamen, and globus pallidus) in children with ADHD.

Method: A total of 104 individuals (combined-type ADHD patients: N=47; healthy comparison subjects: N=57), aged 7 to 18 years, were examined in a cross-sectional case-control study using anatomical magnetic resonance imaging.

Two-factor theory, the actor-critic model, and conditioned avoidance.

Two-factor theory (Mowrer, 1947, 1951, 1956) remains one of the most influential theories of avoidance, but it is at odds with empirical findings that demonstrate sustained avoidance responding in situations in which the theory predicts that the response should extinguish. This article shows that the well-known actor-critic model seamlessly addresses the problems with two-factor theory, while simultaneously being consistent with the core ideas that underlie that theory. More specifically, the article shows that (1) the actor-critic model bears striking similarities to two-factor theory and explains all of the empirical phenomena that two-factor theory explains, in much the same way, and (2) there are subtle but important differences between the actor-critic model and two-factor theory, which result in the actor-critic model predicting the persistence of avoidance responses that is found empirically.

Reinforcement learning, conditioning, and the brain: Successes and challenges.

The field of reinforcement learning has greatly influenced the neuroscientific study of conditioning. This article provides an introduction to reinforcement learning followed by an examination of the successes and challenges using reinforcement learning to understand the neural bases of conditioning. Successes reviewed include (1) the mapping of positive and negative prediction errors to the firing of dopamine neurons and neurons in the lateral habenula, respectively; (2) the mapping of model-based and model-free reinforcement learning to associative and sensorimotor cortico-basal ganglia-thalamo-cortical circuits, respectively; and (3) the mapping of actor and critic to the dorsal and ventral striatum, respectively.

Fear conditioning and social groups: statistics, not genetics.

Humans display more conditioned fear when the conditioned stimulus in a fear conditioning paradigm is a picture of an individual from another race than when it is a picture of an individual from their own race (Olsson, Ebert, Banaji, & Phelps, 2005). These results have been interpreted in terms of a genetic "preparedness" to learn to fear individuals from different social groups (Ohman, 2005; Olsson et al., 2005).

Functional disturbances within frontostriatal circuits across multiple childhood psychopathologies.

Objective: Neuroimaging studies of healthy individuals inform us about the normative maturation of the frontostriatal circuits that subserve self-regulatory control processes. Findings from these studies can be used as a reference frame against which to compare the aberrant development of these processes in individuals across a wide range of childhood psychopathologies.

Method: The authors reviewed extensive neuroimaging evidence for the presence of abnormalities in frontostriatal circuits in children and adults with Tourette's syndrome and obsessive-compulsive disorder (OCD) as well as a more limited number of imaging studies of adolescents and adults with anorexia nervosa or bulimia nervosa that, together, implicate dysregulation of frontostriatal control systems in the pathogenesis of these eating disorders.