Revisiting the Role of Uncertainty-Driven Exploration in a (Perceived) Non-Stationary World.

Humans are often faced with an exploration-versus-exploitation trade-off. A commonly used paradigm, multi-armed bandit, has shown humans to exhibit an "uncertainty bonus", which combines with estimated reward to drive exploration. However, previous studies often modeled belief updating using either a Bayesian model that assumed the reward contingency to remain stationary, or a reinforcement learning model.

Devaluation of Unchosen Options: A Bayesian Account of the Provenance and Maintenance of Overly Optimistic Expectations.

Humans frequently overestimate the likelihood of desirable events while underestimating the likelihood of undesirable ones: a phenomenon known as . Previously, it was suggested that unrealistic optimism arises from asymmetric belief updating, with a relatively reduced coding of undesirable information. Prior studies have shown that a reinforcement learning (RL) model with asymmetric learning rates (greater for a positive prediction error than a negative prediction error) could account for unrealistic optimism in a bandit task, in particular the tendency of human subjects to persistently choosing a single option when there are multiple equally good options.

Leveraging Computer Vision Face Representation to Understand Human Face Representation.

Face processing plays a critical role in human social life, from differentiating friends from enemies to choosing a life mate. In this work, we leverage various computer vision techniques, combined with human assessments of similarity between pairs of faces, to investigate human face representation. We find that combining a shape- and texture-feature based model (Active Appearance Model) with a particular form of metric learning, not only achieves the best performance in predicting human similarity judgments on held-out data (both compared to other algorithms and to humans), but also performs better or comparable to alternative approaches in modeling human social trait judgment (e.

Distinct motivations to seek out information in healthy individuals and problem gamblers.

As massive amounts of information are becoming available to people, understanding the mechanisms underlying information-seeking is more pertinent today than ever. In this study, we investigate the underlying motivations to seek out information in healthy and addicted individuals. We developed a novel decision-making task and a novel computational model which allows dissociating the relative contribution of two motivating factors to seek out information: a desire for novelty and a general desire for knowledge.

From likely to likable: The role of statistical typicality in human social assessment of faces.

Humans readily form social impressions, such as attractiveness and trustworthiness, from a stranger's facial features. Understanding the provenance of these impressions has clear scientific importance and societal implications. Motivated by the efficient coding hypothesis of brain representation, as well as Claude Shannon's theoretical result that maximally efficient representational systems assign shorter codes to statistically more typical data (quantified as log likelihood), we suggest that social "liking" of faces increases with statistical typicality.

Bayesian computational markers of relapse in methamphetamine dependence.

Methamphetamine use disorder is associated with a high likelihood of relapse. Identifying robust predictors of relapse that have explanatory power is critical to develop secondary prevention based on a mechanistic understanding of relapse. Computational approaches have the potential to identify such predictive markers of psychiatric illness, with the advantage of providing a finer mechanistic explanation of the cognitive processes underlying psychiatric vulnerability.

Demystifying excessively volatile human learning: A Bayesian persistent prior and a neural approximation.

Understanding how humans and animals learn about statistical regularities in stable and volatile environments, and utilize these regularities to make predictions and decisions, is an important problem in neuroscience and psychology. Using a Bayesian modeling framework, specifically the Dynamic Belief Model (DBM), it has previously been shown that humans tend to make the assumption that environmental statistics undergo abrupt, unsignaled changes, even when environmental statistics are actually stable. Because exact Bayesian inference in this setting, an example of switching state space models, is computationally intensive, a number of approximately Bayesian and heuristic algorithms have been proposed to account for learning/prediction in the brain.

Motor Preparation Disrupts Proactive Control in the Stop Signal Task.

In a study of the stop signal task (SST) we employed Bayesian modeling to compute the estimated likelihood of stop signal or P(Stop) trial by trial and identified regional processes of conflict anticipation and response slowing. A higher P(Stop) is associated with prolonged go trial reaction time (goRT)-a form of sequential effect-and reflects proactive control of motor response. However, some individuals do not demonstrate a sequential effect despite similar go and stop success (SS) rates.

Publisher Correction: Learning the value of information and reward over time when solving exploration-exploitation problems.

A correction to this article has been published and is linked from the HTML and PDF versions of this paper. The error has been fixed in the paper.

Learning the value of information and reward over time when solving exploration-exploitation problems.

To flexibly adapt to the demands of their environment, animals are constantly exposed to the conflict resulting from having to choose between predictably rewarding familiar options (exploitation) and risky novel options, the value of which essentially consists of obtaining new information about the space of possible rewards (exploration). Despite extensive research, the mechanisms that subtend the manner in which animals solve this exploitation-exploration dilemma are still poorly understood. Here, we investigate human decision-making in a gambling task in which the informational value of each trial and the reward potential were separately manipulated.

Anhedonia and anxiety underlying depressive symptomatology have distinct effects on reward-based decision-making.

Depressive pathology, which includes both heightened negative affect (e.g., anxiety) and reduced positive affect (e.

Reduced Neural Recruitment for Bayesian Adjustment of Inhibitory Control in Methamphetamine Dependence.

Delineating the processes that contribute to the progression and maintenance of substance dependence is critical to understanding and preventing addiction. Several previous studies have shown inhibitory control deficits in individuals with stimulant use disorder. We used a Bayesian computational approach to examine potential neural deficiencies in the dynamic predictive processing underlying inhibitory function among recently abstinent methamphetamine-dependent individuals (MDIs), a population at high risk of relapse.

Inseparability of Go and Stop in Inhibitory Control: Go Stimulus Discriminability Affects Stopping Behavior.

Inhibitory control, the ability to stop or modify preplanned actions under changing task conditions, is an important component of cognitive functions. Two lines of models of inhibitory control have previously been proposed for human response in the classical stop-signal task, in which subjects must inhibit a default go response upon presentation of an infrequent stop signal: (1) the race model, which posits two independent go and stop processes that race to determine the behavioral outcome, go or stop; and (2) an optimal decision-making model, which posits that observers decides whether and when to go based on continually (Bayesian) updated information about both the go and stop stimuli. In this work, we probe the relationship between go and stop processing by explicitly manipulating the discrimination difficulty of the go stimulus.

Altered Statistical Learning and Decision-Making in Methamphetamine Dependence: Evidence from a Two-Armed Bandit Task.

Understanding how humans weigh long-term and short-term goals is important for both basic cognitive science and clinical neuroscience, as substance users need to balance the appeal of an immediate high vs. the long-term goal of sobriety. We use a computational model to identify learning and decision-making abnormalities in methamphetamine-dependent individuals (MDI, n = 16) vs.

Bayesian neural adjustment of inhibitory control predicts emergence of problem stimulant use.

Bayesian ideal observer models quantify individuals' context- and experience-dependent beliefs and expectations about their environment, which provides a powerful approach (i) to link basic behavioural mechanisms to neural processing; and (ii) to generate clinical predictors for patient populations. Here, we focus on (ii) and determine whether individual differences in the neural representation of the need to stop in an inhibitory task can predict the development of problem use (i.e.

Statistical learning and adaptive decision-making underlie human response time variability in inhibitory control.

Response time (RT) is an oft-reported behavioral measure in psychological and neurocognitive experiments, but the high level of observed trial-to-trial variability in this measure has often limited its usefulness. Here, we combine computational modeling and psychophysics to examine the hypothesis that fluctuations in this noisy measure reflect dynamic computations in human statistical learning and corresponding cognitive adjustments. We present data from the stop-signal task (SST), in which subjects respond to a go stimulus on each trial, unless instructed not to by a subsequent, infrequently presented stop signal.

Impaired Bayesian learning for cognitive control in cocaine dependence.

Background: Cocaine dependence is associated with cognitive control deficits. Here, we apply a Bayesian model of stop-signal task (SST) performance to further characterize these deficits in a theory-driven framework.

Methods: A "sequential effect" is commonly observed in SST: encounters with a stop trial tend to prolong reaction time (RT) on subsequent go trials.

Cost-sensitive Bayesian control policy in human active sensing.

An important but poorly understood aspect of sensory processing is the role of active sensing, the use of self-motion such as eye or head movements to focus sensing resources on the most rewarding or informative aspects of the sensory environment. Here, we present behavioral data from a visual search experiment, as well as a Bayesian model of within-trial dynamics of sensory processing and eye movements. Within this Bayes-optimal inference and control framework, which we call C-DAC (Context-Dependent Active Controller), various types of behavioral costs, such as temporal delay, response error, and sensor repositioning cost, are explicitly minimized.

Altered neural processing of the need to stop in young adults at risk for stimulant dependence.

Identification of neurocognitive predictors of substance dependence is an important step in developing approaches to prevent addiction. Given evidence of inhibitory control deficits in substance abusers (Monterosso et al., 2005; Fu et al.

Bayesian prediction and evaluation in the anterior cingulate cortex.

The dorsal anterior cingulate cortex (dACC) has been implicated in a variety of cognitive control functions, among them the monitoring of conflict, error, and volatility, error anticipation, reward learning, and reward prediction errors. In this work, we used a Bayesian ideal observer model, which predicts trial-by-trial probabilistic expectation of stop trials and response errors in the stop-signal task, to differentiate these proposed functions quantitatively. We found that dACC hemodynamic response, as measured by functional magnetic resonance imaging, encodes both the absolute prediction error between stimulus expectation and outcome, and the signed prediction error related to response outcome.

Emotion and decision-making: affect-driven belief systems in anxiety and depression.

Emotion processing and decision-making are integral aspects of daily life. However, our understanding of the interaction between these constructs is limited. In this review, we summarize theoretical approaches that link emotion and decision-making, and focus on research with anxious or depressed individuals to show how emotions can interfere with decision-making.

Rational decision-making in inhibitory control.

An important aspect of cognitive flexibility is inhibitory control, the ability to dynamically modify or cancel planned actions in response to changes in the sensory environment or task demands. We formulate a probabilistic, rational decision-making framework for inhibitory control in the stop signal paradigm. Our model posits that subjects maintain a Bayes-optimal, continually updated representation of sensory inputs, and repeatedly assess the relative value of stopping and going on a fine temporal scale, in order to make an optimal decision on when and whether to go on each trial.

Dynamics of attentional selection under conflict: toward a rational Bayesian account.

The brain exhibits remarkable facility in exerting attentional control in most circumstances, but it also suffers apparent limitations in others. The authors' goal is to construct a rational account for why attentional control appears suboptimal under conditions of conflict and what this implies about the underlying computational principles. The formal framework used is based on Bayesian probability theory, which provides a convenient language for delineating the rationale and dynamics of attentional selection.

Sequential effects: Superstition or rational behavior?

In a variety of behavioral tasks, subjects exhibit an automatic and apparently suboptimal : they respond more rapidly and accurately to a stimulus if it reinforces a local pattern in stimulus history, such as a string of repetitions or alternations, compared to when it violates such a pattern. This is often the case even if the local trends arise by chance in the context of a randomized design, such that stimulus history has no real predictive power. In this work, we use a normative Bayesian framework to examine the hypothesis that such idiosyncrasies may reflect the inadvertent engagement of mechanisms critical for adapting to a changing environment.