Variational autoencoder: An unsupervised model for encoding and decoding fMRI activity in visual cortex.

Goal-driven and feedforward-only convolutional neural networks (CNN) have been shown to be able to predict and decode cortical responses to natural images or videos. Here, we explored an alternative deep neural network, variational auto-encoder (VAE), as a computational model of the visual cortex. We trained a VAE with a five-layer encoder and a five-layer decoder to learn visual representations from a diverse set of unlabeled images.

Task-evoked functional connectivity does not explain functional connectivity differences between rest and task conditions.

During complex tasks, patterns of functional connectivity differ from those in the resting state. However, what accounts for such differences remains unclear. Brain activity during a task reflects an unknown mixture of spontaneous and task-evoked activities.

Transferring and generalizing deep-learning-based neural encoding models across subjects.

Recent studies have shown the value of using deep learning models for mapping and characterizing how the brain represents and organizes information for natural vision. However, modeling the relationship between deep learning models and the brain (or encoding models), requires measuring cortical responses to large and diverse sets of natural visual stimuli from single subjects. This requirement limits prior studies to few subjects, making it difficult to generalize findings across subjects or for a population.

Deep Residual Network Predicts Cortical Representation and Organization of Visual Features for Rapid Categorization.

The brain represents visual objects with topographic cortical patterns. To address how distributed visual representations enable object categorization, we established predictive encoding models based on a deep residual network, and trained them to predict cortical responses to natural movies. Using this predictive model, we mapped human cortical representations to 64,000 visual objects from 80 categories with high throughput and accuracy.

Deep recurrent neural network reveals a hierarchy of process memory during dynamic natural vision.

The human visual cortex extracts both spatial and temporal visual features to support perception and guide behavior. Deep convolutional neural networks (CNNs) provide a computational framework to model cortical representation and organization for spatial visual processing, but unable to explain how the brain processes temporal information. To overcome this limitation, we extended a CNN by adding recurrent connections to different layers of the CNN to allow spatial representations to be remembered and accumulated over time.

Musical Imagery Involves Wernicke's Area in Bilateral and Anti-Correlated Network Interactions in Musicians.

Musical imagery is the human experience of imagining music without actually hearing it. The neural basis of this mental ability is unclear, especially for musicians capable of engaging in accurate and vivid musical imagery. Here, we created a visualization of an 8-minute symphony as a silent movie and used it as real-time cue for musicians to continuously imagine the music for repeated and synchronized sessions during functional magnetic resonance imaging (fMRI).

Neural Encoding and Decoding with Deep Learning for Dynamic Natural Vision.

Convolutional neural network (CNN) driven by image recognition has been shown to be able to explain cortical responses to static pictures at ventral-stream areas. Here, we further showed that such CNN could reliably predict and decode functional magnetic resonance imaging data from humans watching natural movies, despite its lack of any mechanism to account for temporal dynamics or feedback processing. Using separate data, encoding and decoding models were developed and evaluated for describing the bi-directional relationships between the CNN and the brain.

Spontaneous activity in the visual cortex is organized by visual streams.

Large-scale functional networks have been extensively studied using resting state functional magnetic resonance imaging (fMRI). However, the pattern, organization, and function of fine-scale network activity remain largely unknown. Here, we characterized the spontaneously emerging visual cortical activity by applying independent component (IC) analysis to resting state fMRI signals exclusively within the visual cortex.

Mapping white-matter functional organization at rest and during naturalistic visual perception.

Despite the wide applications of functional magnetic resonance imaging (fMRI) to mapping brain activation and connectivity in cortical gray matter, it has rarely been utilized to study white-matter functions. In this study, we investigated the spatiotemporal characteristics of fMRI data within the white matter acquired from humans both in the resting state and while watching a naturalistic movie. By using independent component analysis and hierarchical clustering, resting-state fMRI data in the white matter were de-noised and decomposed into spatially independent components, which were further assembled into hierarchically organized axonal fiber bundles.

Influences of High-Level Features, Gaze, and Scene Transitions on the Reliability of BOLD Responses to Natural Movie Stimuli.

Complex, sustained, dynamic, and naturalistic visual stimulation can evoke distributed brain activities that are highly reproducible within and across individuals. However, the precise origins of such reproducible responses remain incompletely understood. Here, we employed concurrent functional magnetic resonance imaging (fMRI) and eye tracking to investigate the experimental and behavioral factors that influence fMRI activity and its intra- and inter-subject reproducibility during repeated movie stimuli.

Broadband Electrophysiological Dynamics Contribute to Global Resting-State fMRI Signal.

Unlabelled: Spontaneous activity observed with resting-state fMRI is used widely to uncover the brain's intrinsic functional networks in health and disease. Although many networks appear modular and specific, global and nonspecific fMRI fluctuations also exist and both pose a challenge and present an opportunity for characterizing and understanding brain networks. Here, we used a multimodal approach to investigate the neural correlates to the global fMRI signal in the resting state.

Separating Fractal and Oscillatory Components in the Power Spectrum of Neurophysiological Signal.

Neurophysiological field-potential signals consist of both arrhythmic and rhythmic patterns indicative of the fractal and oscillatory dynamics arising from likely distinct mechanisms. Here, we present a new method, namely the irregular-resampling auto-spectral analysis (IRASA), to separate fractal and oscillatory components in the power spectrum of neurophysiological signal according to their distinct temporal and spectral characteristics. In this method, we irregularly resampled the neural signal by a set of non-integer factors, and statistically summarized the auto-power spectra of the resampled signals to separate the fractal component from the oscillatory component in the frequency domain.