Little is known about how humans solve the exploitation/exploration trade-off. In particular, the evidence for uncertainty-driven exploration is mixed. The current study proposes a novel hypothesis of exploration that helps reconcile prior findings that may seem contradictory at first. According to this hypothesis, uncertainty-driven exploration involves a dilemma between two motives: (i) to speed up learning about the unknown, which may beget novel reward opportunities; (ii) to avoid the unknown because it is potentially dangerous. We provide evidence for our hypothesis using both behavioral and simulated data, and briefly point to recent evidence that the brain differentiates between these two motives.
Download full-text PDF |
Source |
|---|---|
| http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3472893 | PMC |
| http://dx.doi.org/10.3389/fnins.2012.00150 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
Individual differences in information demand have a low dimensional structure predicted by some curiosity traits.
Proc Natl Acad Sci U S A
November 2024
Neuroscience Department, Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027.
To understand human learning and progress, it is crucial to understand curiosity. But how consistent is curiosity's conception and assessment across scientific research disciplines? We present the results of a large collaborative project assessing the correspondence between curiosity measures in personality psychology and cognitive science. A total of 820 participants completed 15 personality trait measures and 9 cognitive tasks that tested multiple aspects of information demand.
View Article and Find Full Text PDFDecomposing high uncertainty in greenhouse gas mitigation pathways in emerging regions: An approach to evaluate risks toward carbon neutrality.
Sci Total Environ
November 2024
Key Lab of Urban Environment and Health, Xiamen Key Lab of Urban Metabolism, Research Center of Urban Carbon Neutrality, Institute of Urban Environment, Chinese Academy of Sciences, Xiamen 361021, China. Electronic address:
Article Synopsis
- This study focuses on the importance of identifying and quantifying factors contributing to uncertainty in greenhouse gas (GHG) mitigation strategies in developing regions.
- It introduces a new method that combines Quasi-Monte Carlo simulation with Sobol variance decomposition, demonstrating significant uncertainty in GHG emissions projections for Anhui Province, China, with a 6.2% emission uncertainty by 2030 and 68.6% by 2060.
- The research highlights that economic factors, particularly per capita GDP, and energy aspects, like renewable energy and UHV electricity import, play crucial roles in this uncertainty, urging policymakers to consider uncertainty analysis in their GHG reduction planning.
Exploring and Exploiting Multi-Modality Uncertainty for Tumor Segmentation on PET/CT.
IEEE J Biomed Health Inform
September 2024
Despite the success of deep learning methods in multi-modality segmentation tasks, they typically produce a deterministic output, neglecting the underlying uncertainty. The absence of uncertainty could lead to over-confident predictions with catastrophic consequences, particularly in safety-critical clinical applications. Recently, uncertainty estimation has attracted increasing attention, offering a measure of confidence associated with machine decisions.
View Article and Find Full Text PDFDopamine encoding of novelty facilitates efficient uncertainty-driven exploration.
PLoS Comput Biol
April 2024
MRC Brain Network Dynamics Unit, University of Oxford, Oxford, United Kingdom.
When facing an unfamiliar environment, animals need to explore to gain new knowledge about which actions provide reward, but also put the newly acquired knowledge to use as quickly as possible. Optimal reinforcement learning strategies should therefore assess the uncertainties of these action-reward associations and utilise them to inform decision making. We propose a novel model whereby direct and indirect striatal pathways act together to estimate both the mean and variance of reward distributions, and mesolimbic dopaminergic neurons provide transient novelty signals, facilitating effective uncertainty-driven exploration.
View Article and Find Full Text PDFMultiple and subject-specific roles of uncertainty in reward-guided decision-making.
bioRxiv
September 2024
INSERM-CEA Cognitive Neuroimaging Unit (UNICOG), NeuroSpin Center, CEA Paris-Saclay, Gif-sur-Yvette, France Université de Paris, Paris, France.
Decision-making in noisy, changing, and partially observable environments entails a basic tradeoff between immediate reward and longer-term information gain, known as the exploration-exploitation dilemma. Computationally, an effective way to balance this tradeoff is by leveraging uncertainty to guide exploration. Yet, in humans, empirical findings are mixed, from suggesting uncertainty-seeking to indifference and avoidance.
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!