Individual differences in information demand have a low dimensional structure predicted by some curiosity traits.

To understand human learning and progress, it is crucial to understand curiosity. But how consistent is curiosity's conception and assessment across scientific research disciplines? We present the results of a large collaborative project assessing the correspondence between curiosity measures in personality psychology and cognitive science. A total of 820 participants completed 15 personality trait measures and 9 cognitive tasks that tested multiple aspects of information demand.

A neural code supporting prospective probabilistic reasoning for instrumental information demand in humans.

When making adaptive decisions, we actively demand information, but relatively little is known about the mechanisms of active information gathering. An open question is how the brain prospectively estimates the information gains that are expected to accrue from various sources by integrating simpler quantities of prior certainty and the reliability (diagnosticity) of a source. We examine this question using fMRI in a task in which people placed bids to obtain information in conditions that varied independently in the rewards, decision uncertainty, and information diagnosticity.

The reinforcement metalearner as a biologically plausible meta-learning framework.

We argue that the type of meta-learning proposed by Binz et al. generates models with low interpretability and falsifiability that have limited usefulness for neuroscience research. An alternative approach to meta-learning based on hyperparameter optimization obviates these concerns and can generate empirically testable hypotheses of biological computations.

The parieto-occipital cortex is a candidate neural substrate for the human ability to approximate Bayesian inference.

Adaptive decision-making often requires one to infer unobservable states based on incomplete information. Bayesian logic prescribes that individuals should do so by estimating the posterior probability by integrating the prior probability with new information, but the neural basis of this integration is incompletely understood. We record fMRI during a task in which participants infer the posterior probability of a hidden state while we independently modulate the prior probability and likelihood of evidence regarding the state; the task incentivizes participants to make accurate inferences and dissociates expected value from posterior probability.

A Distinct Neural Code Supports Prospection of Future Probabilities During Instrumental Information-Seeking.

To make adaptive decisions, we must actively demand information, but relatively little is known about the mechanisms of active information gathering. An open question is how the brain estimates expected information gains (EIG) when comparing the current decision uncertainty with the uncertainty that is expected after gathering information. We examined this question using fMRI in a task in which people placed bids to obtain information in conditions that varied independently by prior decision uncertainty, information diagnosticity, and the penalty for an erroneous choice.

Beta traveling waves in monkey frontal and parietal areas encode recent reward history.

Brain function depends on neural communication, but the mechanisms of this communication are not well understood. Recent studies suggest that one form of neural communication is through traveling waves (TWs)-patterns of neural oscillations that propagate within and between brain areas. We show that TWs are robust in microarray recordings in frontal and parietal cortex and encode recent reward history.

Inefficient prioritization of task-relevant attributes during instrumental information demand.

In natural settings, people evaluate complex multi-attribute situations and decide which attribute to request information about. Little is known about how people make this selection and specifically, how they identify individual observations that best predict the value of a multi-attribute situation. Here show that, in a simple task of information demand, participants inefficiently query attributes that have high individual value but are relatively uninformative about a total payoff.

A Reinforcement Meta-Learning framework of executive function and information demand.

Gathering information is crucial for maximizing fitness, but requires diverting resources from searching directly for primary rewards to actively exploring the environment. Optimal decision-making thus maximizes information while reducing effort costs, but little is known about the neuro-computational implementation of this tradeoff. We present a Reinforcement Meta-Learning (RML) computational model that solves the trade-off between the value and costs of gathering information.

Uncertainty modulates visual maps during noninstrumental information demand.

Animals are intrinsically motivated to obtain information independently of instrumental incentives. This motivation depends on two factors: a desire to resolve uncertainty by gathering accurate information and a desire to obtain positively-valenced observations, which predict favorable rather than unfavorable outcomes. To understand the neural mechanisms, we recorded parietal cortical activity implicated in prioritizing stimuli for spatial attention and gaze, in a task in which monkeys were free (but not trained) to obtain information about probabilistic non-contingent rewards.

Superstitious learning of abstract order from random reinforcement.

Humans and other animals often infer spurious associations among unrelated events. However, such superstitious learning is usually accounted for by conditioned associations, raising the question of whether an animal could develop more complex cognitive structures independent of reinforcement. Here, we tasked monkeys with discovering the serial order of two pictorial sets: a "learnable" set in which the stimuli were implicitly ordered and monkeys were rewarded for choosing the higher-rank stimulus and an "unlearnable" set in which stimuli were unordered and feedback was random regardless of the choice.

Humans monitor learning progress in curiosity-driven exploration.

Curiosity-driven learning is foundational to human cognition. By enabling humans to autonomously decide when and what to learn, curiosity has been argued to be crucial for self-organizing temporally extended learning curricula. However, the mechanisms driving people to set intrinsic goals, when they are free to explore multiple learning activities, are still poorly understood.

Reward uncertainty asymmetrically affects information transmission within the monkey fronto-parietal network.

A central hypothesis in research on executive function is that controlled information processing is costly and is allocated according to the behavioral benefits it brings. However, while computational theories predict that the benefits of new information depend on prior uncertainty, the cellular effects of uncertainty on the executive network are incompletely understood. Using simultaneous recordings in monkeys, we describe several mechanisms by which the fronto-parietal network reacts to uncertainty.

Parietal neurons encode information sampling based on decision uncertainty.

During natural behavior, animals actively gather information that is relevant for learning or actions; however, the mechanisms of active sampling are rarely investigated. We tested parietal neurons involved in oculomotor control in a task in which monkeys made saccades to gather visual information relevant to a subsequent action. We show that the neurons encode, before the saccade, the information gain (reduction in decision uncertainty) that the saccade was expected to bring for the following action.

Diverse motives for human curiosity.

Curiosity-our desire to know-is a fundamental drive in human behaviour, but its mechanisms are poorly understood. A classical question concerns the curiosity motives. What drives individuals to become curious about some but not other sources of information? Here we show that curiosity about probabilistic events depends on multiple aspects of the distribution of these events.

Towards a neuroscience of active sampling and curiosity.

In natural behaviour, animals actively interrogate their environments using endogenously generated 'question-and-answer' strategies. However, in laboratory settings participants typically engage with externally imposed stimuli and tasks, and the mechanisms of active sampling remain poorly understood. We review a nascent neuroscientific literature that examines active-sampling policies and their relation to attention and curiosity.

Understanding active sampling strategies: Empirical approaches and implications for attention and decision research.

In natural behavior we actively gather information using attention and active sensing behaviors (such as shifts of gaze) to sample relevant cues. However, while attention and decision making are naturally coordinated, in the laboratory they have been dissociated. Attention is studied independently of the actions it serves.

Parietal neurons encode expected gains in instrumental information.

In natural behavior, animals have access to multiple sources of information, but only a few of these sources are relevant for learning and actions. Beyond choosing an appropriate action, making good decisions entails the ability to choose the relevant information, but fundamental questions remain about the brain's information sampling policies. Recent studies described the neural correlates of seeking information about a reward, but it remains unknown whether, and how, neurons encode choices of instrumental information, in contexts in which the information guides subsequent actions.

Spatial and non-spatial aspects of visual attention: Interactive cognitive mechanisms and neural underpinnings.

In humans and non-human primates, the parietal lobe plays a key role in spatial attention - the ability to extract information from regions of space. This role is thought to be mediated by "priority" maps that highlight attention-worthy locations, and provide top-down feedback for motor orienting and attentional allocation. Traditionally, priority signals within the parietal cortex have been characterized as being purely spatial, i.

Intrinsically motivated oculomotor exploration guided by uncertainty reduction and conditioned reinforcement in non-human primates.

Intelligent animals have a high degree of curiosity--the intrinsic desire to know--but the mechanisms of curiosity are poorly understood. A key open question pertains to the internal valuation systems that drive curiosity. What are the cognitive and emotional factors that motivate animals to seek information when this is not reinforced by instrumental rewards? Using a novel oculomotor paradigm, combined with reinforcement learning (RL) simulations, we show that monkeys are intrinsically motivated to search for and look at reward-predictive cues, and that their intrinsic motivation is shaped by a desire to reduce uncertainty, a desire to obtain conditioned reinforcement from positive cues, and individual variations in decision strategy and the cognitive costs of acquiring information.

Eye movements reveal epistemic curiosity in human observers.

Saccadic (rapid) eye movements are primary means by which humans and non-human primates sample visual information. However, while saccadic decisions are intensively investigated in instrumental contexts where saccades guide subsequent actions, it is largely unknown how they may be influenced by curiosity - the intrinsic desire to learn. While saccades are sensitive to visual novelty and visual surprise, no study has examined their relation to epistemic curiosity - interest in symbolic, semantic information.

A Multi-Area Stochastic Model for a Covert Visual Search Task.

Decisions typically comprise several elements. For example, attention must be directed towards specific objects, their identities recognized, and a choice made among alternatives. Pairs of competing accumulators and drift-diffusion processes provide good models of evidence integration in two-alternative perceptual choices, but more complex tasks requiring the coordination of attention and decision making involve multistage processing and multiple brain areas.

Active learning and decision making: an introduction to the collection.

The importance of exploratory behaviors by which agents actively sample information has been long appreciated in a wide range of disciplines ranging from machine and robot learning to neuroscience and psychology.  Given the complexity of these behaviors, progress in understanding them will require a confluence of ideas from these multiple fields.  This collection of articles in F1000Research aims to provide a home for a broad range of studies addressing this topic, including full length research articles, brief communications, single figure studies, and review/opinion articles, and studies using computational, behavioral or neural approaches.

Attention, reward, and information seeking.

Decision making is thought to be guided by the values of alternative options and involve the accumulation of evidence to an internal bound. However, in natural behavior, evidence accumulation is an active process whereby subjects decide when and which sensory stimulus to sample. These sampling decisions are naturally served by attention and rapid eye movements (saccades), but little is known about how saccades are controlled to guide future actions.