Belief inference for hierarchical hidden states in spatial navigation.

Uncertainty abounds in the real world, and in environments with multiple layers of unobservable hidden states, decision-making requires resolving uncertainties based on mutual inference. Focusing on a spatial navigation problem, we develop a Tiger maze task that involved simultaneously inferring the local hidden state and the global hidden state from probabilistically uncertain observation. We adopt a Bayesian computational approach by proposing a hierarchical inference model.

Confidence modulates the decodability of scene prediction during partially-observable maze exploration in humans.

Prediction ability often involves some degree of uncertainty-a key determinant of confidence. Here, we sought to assess whether predictions are decodable in partially-observable environments where one's state is uncertain, and whether this information is sensitive to confidence produced by such uncertainty. We used functional magnetic resonance imaging-based, partially-observable maze navigation tasks in which subjects predicted upcoming scenes and reported their confidence regarding these predictions.

Model-based and model-free pain avoidance learning.

While there is good evidence that reward learning is underpinned by two distinct decision control systems - a cognitive 'model-based' and a habitbased 'model-free' system, a comparable distinction for punishment avoidance has been much less clear. We implemented a pain avoidance task that placed differential emphasis on putative model-based and model-free processing, mirroring a paradigm and modelling approach recently developed for reward-based decision-making. Subjects performed a two-step decision-making task with probabilistic pain outcomes of different quantities.

Classification and characterisation of brain network changes in chronic back pain: A multicenter study.

Chronic pain is a common, often disabling condition thought to involve a combination of peripheral and central neurobiological factors. However, the extent and nature of changes in the brain is poorly understood. We investigated brain network architecture using resting-state fMRI data in chronic back pain patients in the UK and Japan (41 patients, 56 controls), as well as open data from USA.

The control of tonic pain by active relief learning.

Tonic pain after injury characterises a behavioural state that prioritises recovery. Although generally suppressing cognition and attention, tonic pain needs to allow effective relief learning to reduce the cause of the pain. Here, we describe a central learning circuit that supports learning of relief and concurrently suppresses the level of ongoing pain.

Thermosensory Perceptual Learning Is Associated with Structural Brain Changes in Parietal-Opercular (SII) Cortex.

The location of a sensory cortex for temperature perception remains a topic of substantial debate. Both the parietal-opercular (SII) and posterior insula have been consistently implicated in thermosensory processing, but neither region has yet been identified as the locus of fine temperature discrimination. Using a perceptual learning paradigm in male and female humans, we show improvement in discrimination accuracy for subdegree changes in both warmth and cool detection over 5 d of repetitive training.

Decisions about decisions.

A major puzzle of decision making is how the brain decides which decision system to use at any one time. In this issue of Neuron, Lee et al. (2014) provide a theoretical, behavioral, and neurobiological account of a prefrontal reliability-based arbitration system.

Uncertainty increases pain: evidence for a novel mechanism of pain modulation involving the periaqueductal gray.

Predictions about sensory input exert a dominant effect on what we perceive, and this is particularly true for the experience of pain. However, it remains unclear what component of prediction, from an information-theoretic perspective, controls this effect. We used a vicarious pain observation paradigm to study how the underlying statistics of predictive information modulate experience.

Prices need no preferences: social trends determine decisions in experimental markets for pain relief.

Objective: A standard view in health economics is that, although there is no market that determines the "prices" for health states, people can nonetheless associate health states with monetary values (or other scales, such as quality adjusted life year [QALYs] and disability adjusted life year [DALYs]). Such valuations can be used to shape health policy, and a major research challenge is to elicit such values from people; creating experimental "markets" for health states is a theoretically attractive way to address this. We explore the possibility that this framework may be fundamentally flawed-because there may not be any stable values to be revealed.

Neural mechanisms of belief inference during cooperative games.

Humans have the arguably unique ability to understand the mental representations of others. For success in both competitive and cooperative interactions, however, this ability must be extended to include representations of others' belief about our intentions, their model about our belief about their intentions, and so on. We developed a "stag hunt" game in which human subjects interacted with a computerized agent using different degrees of sophistication (recursive inferences) and applied an ecologically valid computational model of dynamic belief inference.

Cooperation and heterogeneity of the autistic mind.

Individuals with autism spectrum conditions (ASCs) have a core difficulty in recursively inferring the intentions of others. The precise cognitive dysfunctions that determine the heterogeneity at the heart of this spectrum, however, remains unclear. Furthermore, it remains possible that impairment in social interaction is not a fundamental deficit but a reflection of deficits in distinct cognitive processes.

Hierarchical rule switching in prefrontal cortex.

Most real-world decision-making problems involve consideration of numerous possible actions, and it is often impossible to evaluate all of them before settling on preferred strategy. In such situations, humans might explore actions more efficiently by searching only the most likely subspace of the whole action space. To study how the brain solves such action selection problems, we designed a Multi Feature Sorting Task in which the task rules defining an optimal action have a hierarchical structure and studied concurrent brain activity using it.

Altruistic learning.

The origin of altruism remains one of the most enduring puzzles of human behaviour. Indeed, true altruism is often thought either not to exist, or to arise merely as a miscalculation of otherwise selfish behaviour. In this paper, we argue that altruism emerges directly from the way in which distinct human decision-making systems learn about rewards.

Game theory of mind.

This paper introduces a model of 'theory of mind', namely, how we represent the intentions and goals of others to optimise our mutual interactions. We draw on ideas from optimum control and game theory to provide a 'game theory of mind'. First, we consider the representations of goals in terms of value functions that are prescribed by utility or rewards.

Resolution of uncertainty in prefrontal cortex.

Making optimal decisions in the face of uncertain or incomplete information arises as a common problem in everyday behavior, but the neural processes underlying this ability remain poorly understood. A typical case is navigation, in which a subject has to search for a known goal from an unknown location. Navigating under uncertain conditions requires making decisions on the basis of the current belief about location and updating that belief based on incoming information.

Control of exploitation-exploration meta-parameter in reinforcement learning.

In reinforcement learning (RL), the duality between exploitation and exploration has long been an important issue. This paper presents a new method that controls the balance between exploitation and exploration. Our learning scheme is based on model-based RL, in which the Bayes inference with forgetting effect estimates the state-transition probability of the environment.