Rethinking dopamine-guided action sequence learning.

As opposed to those requiring a single action for reward acquisition, tasks necessitating action sequences demand that animals learn action elements and their sequential order and sustain the behaviour until the sequence is completed. With repeated learning, animals not only exhibit precise execution of these sequences but also demonstrate enhanced smoothness and efficiency. Previous research has demonstrated that midbrain dopamine and its major projection target, the striatum, play crucial roles in these processes.

Neurocomputational model of compulsivity: deviating from an uncertain goal-directed system.

Despite a theory that an imbalance in goal-directed versus habitual systems serve as building blocks of compulsions, research has yet to delineate how this occurs during arbitration between the two systems in obsessive-compulsive disorder. Inspired by a brain model in which the inferior frontal cortex selectively gates the putamen to guide goal-directed or habitual actions, this study aimed to examine whether disruptions in the arbitration process via the fronto-striatal circuit would underlie imbalanced decision-making and compulsions in patients. Thirty patients with obsessive-compulsive disorder [mean (standard deviation) age = 26.

Controlling human causal inference through in silico task design.

Learning causal relationships is crucial for survival. The human brain's functional flexibility allows for effective causal inference, underlying various learning processes. While past studies focused on environmental factors influencing causal inference, a fundamental question remains: can these factors be manipulated for strategic causal inference control? This paper presents a task control framework for orchestrating causal learning task design.

The emergence of division of labour through decentralized social sanctioning.

Human ecological success relies on our characteristic ability to flexibly self-organize into cooperative social groups, the most successful of which employ substantial specialization and division of labour. Unlike most other animals, humans learn by trial and error during their lives what role to take on. However, when some critical roles are more attractive than others, and individuals are self-interested, then there is a social dilemma: each individual would prefer others take on the critical but unremunerative roles so they may remain free to take one that pays better.

Pneumatic tactile stimulus delivery system for studying brain responses evoked by active finger touch with fMRI.

Background: Primates use their hands to actively touch objects and collect information. To study tactile information processing, it is important for participants to experience tactile stimuli through active touch while monitoring brain activities.

New Method: Here, we developed a pneumatic tactile stimulus delivery system (pTDS) that delivers various tactile stimuli on a programmed schedule and allows voluntary finger touches during MRI scanning.

External globus pallidus input to the dorsal striatum regulates habitual seeking behavior in male mice.

The external globus pallidus (GPe) coordinates action-selection through GABAergic projections throughout the basal ganglia. GPe arkypallidal (arky) neurons project exclusively to the dorsal striatum, which regulates goal-directed and habitual seeking. However, the role of GPe arky neurons in reward-seeking remains unknown.

Astrocyte activities in the external globus pallidus regulate action-selection strategies in reward-seeking behaviors.

An imbalance in goal-directed and habitual behavioral control is a hallmark of decision-making-related disorders, including addiction. Although external globus pallidus (GPe) is critical for action selection, which harbors enriched astrocytes, the role of GPe astrocytes involved in action-selection strategies remained unknown. Using in vivo calcium signaling with fiber photometry, we found substantially attenuated GPe astrocytic activity during habitual learning compared to goal-directed learning.

Impairment of arbitration between model-based and model-free reinforcement learning in obsessive-compulsive disorder.

Introduction: Obsessive-compulsive disorder (OCD) is characterized by an imbalance between goal-directed and habitual learning systems in behavioral control, but it is unclear whether these impairments are due to a single system abnormality of the goal-directed system or due to an impairment in a separate arbitration mechanism that selects which system controls behavior at each point in time.

Methods: A total of 30 OCD patients and 120 healthy controls performed a 2-choice, 3-stage Markov decision-making paradigm. Reinforcement learning models were used to estimate goal-directed learning (as model-based reinforcement learning) and habitual learning (as model-free reinforcement learning).

Hippocampal integration and separation processes with different temporal and spatial dynamics during learning for associative memory.

The hippocampus is known to be critically involved in associative memory formation. However, the role of the hippocampus during the learning of associative memory is still controversial; while the hippocampus is considered to play a critical role in the integration of related stimuli, numerous studies also suggest a role of the hippocampus in the separation of different memory traces for rapid learning. Here, we employed an associative learning paradigm consisting of repeated learning cycles.

Importance of prefrontal meta control in human-like reinforcement learning.

Recent investigation on reinforcement learning (RL) has demonstrated considerable flexibility in dealing with various problems. However, such models often experience difficulty learning seemingly easy tasks for humans. To reconcile the discrepancy, our paper is focused on the computational benefits of the brain's RL.

Meta-control of social learning strategies.

Social learning, copying other's behavior without actual experience, offers a cost-effective means of knowledge acquisition. However, it raises the fundamental question of which individuals have reliable information: successful individuals versus the majority. The former and the latter are known respectively as success-based and conformist social learning strategies.

Concordance between the international standards for neurological classification of spinal cord injury motor examination and needle electromyography findings in muscles with a motor power grade of zero or trace.

To evaluate the accuracy of the International Standards for Neurological Classification of Spinal Cord Injury (ISNCSCI) motor examination in individuals with spinal cord injury (SCI) with motor grade 0 or 1 and analyze its degree of concordance with needle electromyography (EMG) findings for each key muscle. Retrospective study. University hospital in Goyang, Korea.

Prefrontal solution to the bias-variance tradeoff during reinforcement learning.

Evidence that the brain combines different value learning strategies to minimize prediction error is accumulating. However, the tradeoff between bias and variance error, which imposes different constraints on each learning strategy's performance, poses a challenge for value learning. While this tradeoff specifies the requirements for optimal learning, little has been known about how the brain deals with this issue.

Neurocomputational mechanism of controllability inference under a multi-agent setting.

Controllability perception significantly influences motivated behavior and emotion and requires an estimation of one's influence on an environment. Previous studies have shown that an agent can infer controllability by observing contingency between one's own action and outcome if there are no other outcome-relevant agents in an environment. However, if there are multiple agents who can influence the outcome, estimation of one's genuine controllability requires exclusion of other agents' possible influence.

Sick Sinus Syndrome Combined with Wallenberg Syndrome: a Case Report.

Cardiac arrhythmia is a rare manifestation of the Wallenberg syndrome; lesions are located in the brainstem, especially the lower medulla, which regulates sympathetic and parasympathetic activity. A 55-year-old man was admitted to the university hospital with symptoms including ataxia, left ptosis, decreased sensation of pain and temperature on the right side, left facial numbness, and dizziness. Brain magnetic resonance imaging revealed an infarction in the left dorsolateral medulla.

Effects of subclinical depression on prefrontal-striatal model-based and model-free learning.

Depression is characterized by deficits in the reinforcement learning (RL) process. Although many computational and neural studies have extended our knowledge of the impact of depression on RL, most focus on habitual control (model-free RL), yielding a relatively poor understanding of goal-directed control (model-based RL) and arbitration control to find a balance between the two. We investigated the effects of subclinical depression on model-based and model-free learning in the prefrontal-striatal circuitry.

Epidemiology of Spinal Cord Injury: Changes to Its Cause Amid Aging Population, a Single Center Study.

Objective: To investigate the epidemiologic and demographic characteristics of patients with spinal cord injury (SCI) who were admitted to a department of rehabilitation of a university hospital.

Methods: This was a descriptive cross-sectional study. Medical records including sex, age at injury, type of disability, traumatic or non-traumatic etiology and presence of ossification of posterior longitudinal ligament (OPLL) of patients with SCI who were admitted to the department of rehabilitation between 2012 and 2018 were reviewed.

The efficacy of computerized cognitive rehabilitation in improving attention and executive functions in acquired brain injury patients, in acute and postacute phase.

Background: Cognitive deficits, particularly executive dysfunction is common following acquired brain injury (ABI) and has detrimental effect on functional status and autonomy in daily life. Among various cognitive training methods, computerized cognitive rehabilitation (CCR) has been investigated as an alternative method to therapist-driven cognitive rehabilitation (TCR). However, previous studies have shown conflicting results on the superiority or inferiority of CCR and TCR.

Why and how the brain weights contributions from a mixture of experts.

It has long been suggested that human behavior reflects the contributions of multiple systems that cooperate or compete for behavioral control. Here we propose that the brain acts as a "Mixture of Experts" in which different expert systems propose strategies for action. It will be argued that the brain determines which experts should control behavior at any one moment in time by keeping track of the reliability of the predictions within each system, and by allocating control over behavior in a manner that depends on the relative reliabilities across experts.

Behavioral evidence for memory replay of video episodes in the macaque.

Humans recall the past by replaying fragments of events temporally. Here, we demonstrate a similar effect in macaques. We trained six rhesus monkeys with a temporal-order judgement (TOJ) task and collected 5000 TOJ trials.

Dynamic resource allocation during reinforcement learning accounts for ramping and phasic dopamine activity.

For an animal to learn about its environment with limited motor and cognitive resources, it should focus its resources on potentially important stimuli. However, too narrow focus is disadvantageous for adaptation to environmental changes. Midbrain dopamine neurons are excited by potentially important stimuli, such as reward-predicting or novel stimuli, and allocate resources to these stimuli by modulating how an animal approaches, exploits, explores, and attends.

Task complexity interacts with state-space uncertainty in the arbitration between model-based and model-free learning.

It has previously been shown that the relative reliability of model-based and model-free reinforcement-learning (RL) systems plays a role in the allocation of behavioral control between them. However, the role of task complexity in the arbitration between these two strategies remains largely unknown. Here, using a combination of novel task design, computational modelling, and model-based fMRI analysis, we examined the role of task complexity alongside state-space uncertainty in the arbitration process.

Neurostimulation Reveals Context-Dependent Arbitration Between Model-Based and Model-Free Reinforcement Learning.

While it is established that humans use model-based (MB) and model-free (MF) reinforcement learning in a complementary fashion, much less is known about how the brain determines which of these systems should control behavior at any given moment. Here we provide causal evidence for a neural mechanism that acts as a context-dependent arbitrator between both systems. We applied excitatory and inhibitory transcranial direct current stimulation over a region of the left ventrolateral prefrontal cortex previously found to encode the reliability of both learning systems.

Toward high-performance, memory-efficient, and fast reinforcement learning-Lessons from decision neuroscience.

Recent insights from decision neuroscience raise hope for the development of intelligent brain-inspired solutions to robot learning in real dynamic environments full of noise and unpredictability.