The differential effect of optogenetic serotonergic manipulation on sustained motor actions and waiting for future rewards in mice.

Serotonin is an essential neuromodulator that affects behavioral and cognitive functions. Previous studies have shown that activation of serotonergic neurons in the dorsal raphe nucleus (DRN) promotes patience to wait for future rewards. However, it is still unclear whether serotonergic neurons also regulate persistence to act for future rewards.

A whole-brain analysis of functional connectivity and immediate early gene expression reveals functional network shifts after operant learning.

Previous studies of operant learning have addressed neuronal activities and network changes in specific brain areas, such as the striatum, sensorimotor cortex, prefrontal/orbitofrontal cortices, and hippocampus. However, how changes in the whole-brain network are caused by cellular-level changes remains unclear. We, therefore, combined resting-state functional magnetic resonance imaging (rsfMRI) and whole-brain immunohistochemical analysis of early growth response 1 (EGR1), a marker of neural plasticity, to elucidate the temporal and spatial changes in functional networks and underlying cellular processes during operant learning.

Intrinsic Rewards for Exploration Without Harm From Observational Noise: A Simulation Study Based on the Free Energy Principle.

In reinforcement learning (RL), artificial agents are trained to maximize numerical rewards by performing tasks. Exploration is essential in RL because agents must discover information before exploiting it. Two rewards encouraging efficient exploration are the entropy of action policy and curiosity for information gain.

Synergizing habits and goals with variational Bayes.

Behaving efficiently and flexibly is crucial for biological and artificial embodied agents. Behavior is generally classified into two types: habitual (fast but inflexible), and goal-directed (flexible but slow). While these two types of behaviors are typically considered to be managed by two distinct systems in the brain, recent studies have revealed a more sophisticated interplay between them.

Optogenetic activation of dorsal raphe serotonin neurons induces brain-wide activation.

Serotonin is a neuromodulator that affects multiple behavioral and cognitive functions. Nonetheless, how serotonin causes such a variety of effects via brain-wide projections and various receptors remains unclear. Here we measured brain-wide responses to optogenetic stimulation of serotonin neurons in the dorsal raphe nucleus (DRN) of the male mouse brain using functional MRI with an 11.

Respiratory modulation of the heart rate: A potential biomarker of cardiorespiratory function in human.

In recent decade, wearable digital devices have shown potentials for the discovery of novel biomarkers of humans' physiology and behavior. Heart rate (HR) and respiration rate (RR) are most crucial bio-signals in humans' digital phenotyping research. HR is a continuous and non-invasive proxy to autonomic nervous system and ample evidence pinpoints the critical role of respiratory modulation of cardiac function.

Embodied bidirectional simulation of a spiking cortico-basal ganglia-cerebellar-thalamic brain model and a mouse musculoskeletal body model distributed across computers including the supercomputer Fugaku.

Embodied simulation with a digital brain model and a realistic musculoskeletal body model provides a means to understand animal behavior and behavioral change. Such simulation can be too large and complex to conduct on a single computer, and so distributed simulation across multiple computers over the Internet is necessary. In this study, we report our joint effort on developing a spiking brain model and a mouse body model, connecting over the Internet, and conducting bidirectional simulation while synchronizing them.

Neuronal Population Activity in Macaque Visual Cortices Dynamically Changes through Repeated Fixations in Active Free Viewing.

During free viewing, we move our eyes and fixate on objects to recognize the visual scene of our surroundings. To investigate the neural representation of objects in this process, we studied individual and population neuronal activity in three different visual regions of the brains of macaque monkeys (): the primary and secondary visual cortices (V1, V2) and the inferotemporal cortex (IT). We designed a task where the animal freely selected objects in a stimulus image to fixate on while we examined the relationship between spiking activity, the order of fixations, and the fixated objects.

Enhancing reinforcement learning models by including direct and indirect pathways improves performance on striatal dependent tasks.

A major advance in understanding learning behavior stems from experiments showing that reward learning requires dopamine inputs to striatal neurons and arises from synaptic plasticity of cortico-striatal synapses. Numerous reinforcement learning models mimic this dopamine-dependent synaptic plasticity by using the reward prediction error, which resembles dopamine neuron firing, to learn the best action in response to a set of cues. Though these models can explain many facets of behavior, reproducing some types of goal-directed behavior, such as renewal and reversal, require additional model components.

The Brain/MINDS Marmoset Connectivity Resource: An open-access platform for cellular-level tracing and tractography in the primate brain.

The primate brain has unique anatomical characteristics, which translate into advanced cognitive, sensory, and motor abilities. Thus, it is important that we gain insight on its structure to provide a solid basis for models that will clarify function. Here, we report on the implementation and features of the Brain/MINDS Marmoset Connectivity Resource (BMCR), a new open-access platform that provides access to high-resolution anterograde neuronal tracer data in the marmoset brain, integrated to retrograde tracer and tractography data.

Neuronal Representation of a Working Memory-Based Decision Strategy in the Motor and Prefrontal Cortico-Basal Ganglia Loops.

While animal and human decision strategies are typically explained by model-free and model-based reinforcement learning (RL), their choice sequences often follow simple procedures based on working memory (WM) of past actions and rewards. Here, we address how working memory-based choice strategies, such as win-stay-lose-switch (WSLS), are represented in the prefrontal and motor cortico-basal ganglia loops by simultaneous recording of neuronal activities in the dorsomedial striatum (DMS), the dorsolateral striatum (DLS), the medial prefrontal cortex (mPFC), and the primary motor cortex (M1). In order to compare neuronal representations when rats employ working memory-based strategies, we developed a new task paradigm, a continuous/intermittent choice task, consisting of choice and no-choice trials.

Multi-modal brain magnetic resonance imaging database covering marmosets with a wide age range.

Magnetic resonance imaging (MRI) is a non-invasive neuroimaging technique that is useful for identifying normal developmental and aging processes and for data sharing. Marmosets have a relatively shorter life expectancy than other primates, including humans, because they grow and age faster. Therefore, the common marmoset model is effective in aging research.

Body and visual instabilities functionally modulate implicit reaching corrections.

Hierarchical brain-information-processing schemes have frequently assumed that the flexible but slow voluntary action modulates a direct sensorimotor process that can quickly generate a reaction in dynamical interaction. Here we show that the quick visuomotor process for manual movement is modulated by postural and visual instability contexts that are related but remote and prior states to manual movements. A preceding unstable postural context significantly enhanced the reflexive manual response induced by a large-field visual motion during hand reaching while the response was evidently weakened by imposing a preceding random-visual-motion context.

A Spiking Neural Network Builder for Systematic Data-to-Model Workflow.

In building biological neural network models, it is crucial to efficiently convert diverse anatomical and physiological data into parameters of neurons and synapses and to systematically estimate unknown parameters in reference to experimental observations. Web-based tools for systematic model building can improve the transparency and reproducibility of computational models and can facilitate collaborative model building, validation, and evolution. Here, we present a framework to support collaborative data-driven development of spiking neural network (SNN) models based on the Entity-Relationship (ER) data description commonly used in large-scale business software development.

Social impact and governance of AI and neurotechnologies.

Advances in artificial intelligence (AI) and brain science are going to have a huge impact on society. While technologies based on those advances can provide enormous social benefits, adoption of new technologies poses various risks. This article first reviews the co-evolution of AI and brain science and the benefits of brain-inspired AI in sustainability, healthcare, and scientific discoveries.

Deploying and Optimizing Embodied Simulations of Large-Scale Spiking Neural Networks on HPC Infrastructure.

Simulating the brain-body-environment trinity in closed loop is an attractive proposal to investigate how perception, motor activity and interactions with the environment shape brain activity, and vice versa. The relevance of this embodied approach, however, hinges entirely on the modeled complexity of the various simulated phenomena. In this article, we introduce a software framework that is capable of simulating large-scale, biologically realistic networks of spiking neurons embodied in a biomechanically accurate musculoskeletal system that interacts with a physically realistic virtual environment.

A whole brain probabilistic generative model: Toward realizing cognitive architectures for developmental robots.

Building a human-like integrative artificial cognitive system, that is, an artificial general intelligence (AGI), is the holy grail of the artificial intelligence (AI) field. Furthermore, a computational model that enables an artificial system to achieve cognitive development will be an excellent reference for brain and cognitive science. This paper describes an approach to develop a cognitive architecture by integrating elemental cognitive modules to enable the training of the modules as a whole.

Forward and inverse reinforcement learning sharing network weights and hyperparameters.

This paper proposes model-free imitation learning named Entropy-Regularized Imitation Learning (ERIL) that minimizes the reverse Kullback-Leibler (KL) divergence. ERIL combines forward and inverse reinforcement learning (RL) under the framework of an entropy-regularized Markov decision process. An inverse RL step computes the log-ratio between two distributions by evaluating two binary discriminators.

Optimization and validation of diffusion MRI-based fiber tracking with neural tracer data as a reference.

Diffusion-weighted magnetic resonance imaging (dMRI) allows non-invasive investigation of whole-brain connectivity, which can reveal the brain's global network architecture and also abnormalities involved in neurological and mental disorders. However, the reliability of connection inferences from dMRI-based fiber tracking is still debated, due to low sensitivity, dominance of false positives, and inaccurate and incomplete reconstruction of long-range connections. Furthermore, parameters of tracking algorithms are typically tuned in a heuristic way, which leaves room for manipulation of an intended result.

Serotonergic projections to the orbitofrontal and medial prefrontal cortices differentially modulate waiting for future rewards.

Optogenetic activation of serotonergic neurons in the dorsal raphe nucleus (DRN) enhances patience when waiting for future rewards, and this effect is maximized by both high probability and high timing uncertainty of reward. Here, we explored which serotonin projection areas contribute to these effects using optogenetic axon terminal stimulation. We found that serotonin stimulation in the orbitofrontal cortex (OFC) is nearly as effective as that in the DRN for promoting waiting, while in the nucleus accumbens, it does not promote waiting.

Diffusion functional MRI reveals global brain network functional abnormalities driven by targeted local activity in a neuropsychiatric disease mouse model.

Diffusion functional magnetic resonance imaging (DfMRI) has been proposed as an alternative functional imaging method to detect brain activity without confounding hemodynamic effects. Here, taking advantage of this DfMRI feature, we investigated abnormalities of dynamic brain function in a neuropsychiatric disease mouse model (glial glutamate transporter-knockdown mice with obsessive-compulsive disorder [OCD]-related behavior). Our DfMRI approaches consisted of three analyses: resting state brain activity, functional connectivity, and propagation of neural information.

A biologically constrained spiking neural network model of the primate basal ganglia with overlapping pathways exhibits action selection.

Action selection has been hypothesized to be a key function of the basal ganglia, yet the nuclei involved, their interactions and the importance of the direct/indirect pathway segregation in such process remain debated. Here, we design a spiking computational model of the monkey basal ganglia derived from a previously published population model, initially parameterized to reproduce electrophysiological activity at rest and to embody as much quantitative anatomical data as possible. As a particular feature, both models exhibit the strong overlap between the direct and indirect pathways that has been documented in non-human primates.

Self-organization of action hierarchy and compositionality by reinforcement learning with recurrent neural networks.

Recurrent neural networks (RNNs) for reinforcement learning (RL) have shown distinct advantages, e.g., solving memory-dependent tasks and meta-learning.