On Predictive Planning and Counterfactual Learning in Active Inference.

Given the rapid advancement of artificial intelligence, understanding the foundations of intelligent behaviour is increasingly important. Active inference, regarded as a general theory of behaviour, offers a principled approach to probing the basis of sophistication in planning and decision-making. This paper examines two decision-making schemes in active inference based on "planning" and "learning from experience".

Federated inference and belief sharing.

This paper concerns the distributed intelligence or federated inference that emerges under belief-sharing among agents who share a common world-and world model. Imagine, for example, several animals keeping a lookout for predators. Their collective surveillance rests upon being able to communicate their beliefs-about what they see-among themselves.

Experimental validation of the free-energy principle with in vitro neural networks.

Empirical applications of the free-energy principle are not straightforward because they entail a commitment to a particular process theory, especially at the cellular and synaptic levels. Using a recently established reverse engineering technique, we confirm the quantitative predictions of the free-energy principle using in vitro networks of rat cortical neurons that perform causal inference. Upon receiving electrical stimuli-generated by mixing two hidden sources-neurons self-organised to selectively encode the two sources.

Canonical neural networks perform active inference.

This work considers a class of canonical neural networks comprising rate coding models, wherein neural activity and plasticity minimise a common cost function-and plasticity is modulated with a certain delay. We show that such neural networks implicitly perform active inference and learning to minimise the risk associated with future outcomes. Mathematical analyses demonstrate that this biological optimisation can be cast as maximisation of model evidence, or equivalently minimisation of variational free energy, under the well-known form of a partially observed Markov decision process model.

Active inference leads to Bayesian neurophysiology.

The neuronal substrates that implement the free-energy principle and ensuing active inference at the neuron and synapse level have not been fully elucidated. This Review considers possible neuronal substrates underlying the principle. First, the foundations of the free-energy principle are introduced, and then its ability to empirically explain various brain functions and psychological and biological phenomena in terms of Bayesian inference is described.

Zebrafish capable of generating future state prediction error show improved active avoidance behavior in virtual reality.

Animals make decisions under the principle of reward value maximization and surprise minimization. It is still unclear how these principles are represented in the brain and are reflected in behavior. We addressed this question using a closed-loop virtual reality system to train adult zebrafish for active avoidance.

On the Achievability of Blind Source Separation for High-Dimensional Nonlinear Source Mixtures.

For many years, a combination of principal component analysis (PCA) and independent component analysis (ICA) has been used for blind source separation (BSS). However, it remains unclear why these linear methods work well with real-world data that involve nonlinear source mixtures. This work theoretically validates that a cascade of linear PCA and ICA can solve a nonlinear BSS problem accurately-when the sensory inputs are generated from hidden sources via nonlinear mappings with sufficient dimensionality.

Inferring Neuronal Couplings From Spiking Data Using a Systematic Procedure With a Statistical Criterion.

Recent remarkable advances in experimental techniques have provided a background for inferring neuronal couplings from point process data that include a great number of neurons. Here, we propose a systematic procedure for pre- and postprocessing generic point process data in an objective manner to handle data in the framework of a binary simple statistical model, the Ising or generalized McCulloch-Pitts model. The procedure has two steps: (1) determining time bin size for transforming the point process data into discrete-time binary data and (2) screening relevant couplings from the estimated couplings.

Reverse-Engineering Neural Networks to Characterize Their Cost Functions.

This letter considers a class of biologically plausible cost functions for neural networks, where the same cost function is minimized by both neural activity and plasticity. We show that such cost functions can be cast as a variational bound on model evidence under an implicit generative model. Using generative models based on partially observed Markov decision processes (POMDP), we show that neural activity and plasticity perform Bayesian inference and learning, respectively, by maximizing model evidence.

Bayesian Filtering with Multiple Internal Models: Toward a Theory of Social Intelligence.

To exhibit social intelligence, animals have to recognize whom they are communicating with. One way to make this inference is to select among internal generative models of each conspecific who may be encountered. However, these models also have to be learned via some form of Bayesian belief updating.

Multi-context blind source separation by error-gated Hebbian rule.

Animals need to adjust their inferences according to the context they are in. This is required for the multi-context blind source separation (BSS) task, where an agent needs to infer hidden sources from their context-dependent mixtures. The agent is expected to invert this mixing process for all contexts.

The emergence of synchrony in networks of mutually inferring neurons.

This paper considers the emergence of a generalised synchrony in ensembles of coupled self-organising systems, such as neurons. We start from the premise that any self-organising system complies with the free energy principle, in virtue of placing an upper bound on its entropy. Crucially, the free energy principle allows one to interpret biological systems as inferring the state of their environment or external milieu.

In vitro neural networks minimise variational free energy.

In this work, we address the neuronal encoding problem from a Bayesian perspective. Specifically, we ask whether neuronal responses in an in vitro neuronal network are consistent with ideal Bayesian observer responses under the free energy principle. In brief, we stimulated an in vitro cortical cell culture with stimulus trains that had a known statistical structure.

Task-Related Synaptic Changes Localized to Small Neuronal Population in Recurrent Neural Network Cortical Models.

Humans have flexible control over cognitive functions depending on the context. Several studies suggest that the prefrontal cortex (PFC) controls this cognitive flexibility, but the detailed underlying mechanisms remain unclear. Recent developments in machine learning techniques allow simple PFC models written as a recurrent neural network to perform various behavioral tasks like humans and animals.

A Measure of Information Available for Inference.

The mutual information between the state of a neural network and the state of the external world represents the amount of information stored in the neural network that is associated with the external world. In contrast, the surprise of the sensory input indicates the unpredictability of the current input. In other words, this is a measure of inference ability, and an upper bound of the surprise is known as the variational free energy.

Error-Gated Hebbian Rule: A Local Learning Rule for Principal and Independent Component Analysis.

We developed a biologically plausible unsupervised learning algorithm, error-gated Hebbian rule (EGHR)-β, that performs principal component analysis (PCA) and independent component analysis (ICA) in a single-layer feedforward neural network. If parameter β = 1, it can extract the subspace that major principal components span similarly to Oja's subspace rule for PCA. If β = 0, it can separate independent sources similarly to Bell-Sejnowski's ICA rule but without requiring the same number of input and output neurons.

Spike-contrast: A novel time scale independent and multivariate measure of spike train synchrony.

Background: Synchrony within neuronal networks is thought to be a fundamental feature of neuronal networks. In order to quantify synchrony between spike trains, various synchrony measures were developed. Most of them are time scale dependent and thus require the setting of an appropriate time scale.

Learning with three factors: modulating Hebbian plasticity with errors.

Synaptic plasticity is a central theme in neuroscience. A framework of three-factor learning rules provides a powerful abstraction, helping to navigate through the abundance of models of synaptic plasticity. It is well-known that the dopamine modulation of learning is related to reward, but theoretical models predict other functional roles of the modulatory third factor; it may encode errors for supervised learning, summary statistics of the population activity for unsupervised learning or attentional feedback.

Temporal relation between neural activity and neurite pruning on a numerical model and a microchannel device with micro electrode array.

Synapse elimination and neurite pruning are essential processes for the formation of neuronal circuits. These regressive events depend on neural activity and occur in the early postnatal days known as the critical period, but what makes this temporal specificity is not well understood. One possibility is that the neural activities during the developmentally regulated shift of action of GABA inhibitory transmission lead to the critical period.

Neurogenesis enhances response specificity to spatial pattern stimulation in hippocampal cultures.

Objective: Adult neurogenesis in the hippocampus facilitates cognitive functions such as pattern separation in mammals. However, it remains unclear how newborn neurons mediate changes in neural networks to enhance pattern separation ability. Here, we developed an in vitro model of adult neurogenesis using rat hippocampal cultures in order to investigate whether newborn neurons can be directly incorporated into neural networks related to pattern separation to produce functional improvements.

Linking Neuromodulated Spike-Timing Dependent Plasticity with the Free-Energy Principle.

The free-energy principle is a candidate unified theory for learning and memory in the brain that predicts that neurons, synapses, and neuromodulators work in a manner that minimizes free energy. However, electrophysiological data elucidating the neural and synaptic bases for this theory are lacking. Here, we propose a novel theory bridging the information-theoretical principle with the biological phenomenon of spike-timing dependent plasticity (STDP) regulated by neuromodulators, which we term mSTDP.

A Local Learning Rule for Independent Component Analysis.

Humans can separately recognize independent sources when they sense their superposition. This decomposition is mathematically formulated as independent component analysis (ICA). While a few biologically plausible learning rules, so-called local learning rules, have been proposed to achieve ICA, their performance varies depending on the parameters characterizing the mixed signals.

Cultured Cortical Neurons Can Perform Blind Source Separation According to the Free-Energy Principle.

Blind source separation is the computation underlying the cocktail party effect--a partygoer can distinguish a particular talker's voice from the ambient noise. Early studies indicated that the brain might use blind source separation as a signal processing strategy for sensory perception and numerous mathematical models have been proposed; however, it remains unclear how the neural networks extract particular sources from a complex mixture of inputs. We discovered that neurons in cultures of dissociated rat cortical cells could learn to represent particular sources while filtering out other signals.

Signal transfer within a cultured asymmetric cortical neuron circuit.

Objective: Simplified neuronal circuits are required for investigating information representation in nervous systems and for validating theoretical neural network models. Here, we developed patterned neuronal circuits using micro fabricated devices, comprising a micro-well array bonded to a microelectrode-array substrate.

Approach: The micro-well array consisted of micrometre-scale wells connected by tunnels, all contained within a silicone slab called a micro-chamber.

Accurate connection strength estimation based on variational bayes for detecting synaptic plasticity.

Connection strength estimation is widely used in detecting the topology of neuronal networks and assessing their synaptic plasticity. A recently proposed model-based method using the leaky integrate-and-fire model neuron estimates membrane potential from spike trains by calculating the maximum a posteriori (MAP) path. We further enhance the MAP path method using variational Bayes and dynamic causal modeling.