Computational Modeling: Human Dynamic Model.

Improvements in quantitative measurements of human physical activity are proving extraordinarily useful for studying the underlying musculoskeletal system. Dynamic models of human movement support clinical efforts to analyze, rehabilitate injuries. They are also used in biomechanics to understand and diagnose motor pathologies, find new motor strategies that decrease the risk of injury, and predict potential problems from a particular procedure.

The Hierarchical Evolution in Human Vision Modeling.

Computational models of primate vision took a significant advance with David Marr's tripartite separation of the vision enterprise into the problem formulation, algorithm, and neural implementation; however, many subsequent parallel developments in robotics and modeling greatly refined the algorithm descriptions into very distinct levels that complement each other. This review traces the time course of these developments and shows how the current perspective evolved to have its alternative internal hierarchical organization.

Atari-HEAD: Atari Human Eye-Tracking and Demonstration Dataset.

Large-scale public datasets have been shown to benefit research in multiple areas of modern artificial intelligence. For decision-making research that requires human data, high-quality datasets serve as important benchmarks to facilitate the development of new methods by providing a common reproducible standard. Many human decision-making tasks require visual attention to obtain high levels of performance.

Parallel Neural Multiprocessing with Gamma Frequency Latencies.

The Poisson variability in cortical neural responses has been typically modeled using spike averaging techniques, such as trial averaging and rate coding, since such methods can produce reliable correlates of behavior. However, mechanisms that rely on counting spikes could be slow and inefficient and thus might not be useful in the brain for computations at timescales in the 10 millisecond range. This issue has motivated a search for alternative spike codes that take advantage of spike timing and has resulted in many studies that use synchronized neural networks for communication.

Humans Use Similar Posture Sequences in a Whole-Body Tracing Task.

Humans have elegant bodies that allow gymnastics, piano playing, and tool use, but understanding how they do this in detail is difficult because their musculoskeletal systems are extremely complicated. Previous studies have shown that common movements such as reaching for a coffee cup, cycling a bicycle, or playing the piano have common patterns across subjects. This paper shows that an arbitrary set of whole-body movements used to trace large closed curves have common patterns both in the trajectory of the body's limbs and in variations within those trajectories.

Modeling sensory-motor decisions in natural behavior.

Although a standard reinforcement learning model can capture many aspects of reward-seeking behaviors, it may not be practical for modeling human natural behaviors because of the richness of dynamic environments and limitations in cognitive resources. We propose a modular reinforcement learning model that addresses these factors. Based on this model, a modular inverse reinforcement learning algorithm is developed to estimate both the rewards and discount factors from human behavioral data, which allows predictions of human navigation behaviors in virtual reality with high accuracy across different subjects and with different tasks.

Modular inverse reinforcement learning for visuomotor behavior.

In a large variety of situations one would like to have an expressive and accurate model of observed animal or human behavior. While general purpose mathematical models may capture successfully properties of observed behavior, it is desirable to root models in biological facts. Because of ample empirical evidence for reward-based learning in visuomotor tasks, we use a computational model based on the assumption that the observed agent is balancing the costs and benefits of its behavior to meet its goals.

A hierarchical modular architecture for embodied cognition.

Cognition can appear complex owing to the fact that the brain is capable of an enormous repertoire of behaviors. However, this complexity can be greatly reduced when constraints of time and space are taken into account. The brain is constrained by the body to limit its goal-directed behaviors to just a few independent tasks over the scale of 1-2 min, and can pursue only a very small number of independent agendas.

Dynamic coding of signed quantities in cortical feedback circuits.

In the early sensory and motor areas of the cortex, individual neurons transmit information about specific sensory features via a peaked response. This concept has been crystallized as "labeled lines," to denote that axons communicate the specific properties of their sensory or motor parent cell. Such cells also can be characterized as being polarized, that is, as representing a signed quantity that is either positive or negative.

Dual roles for spike signaling in cortical neural populations.

A prominent feature of signaling in cortical neurons is that of randomness in the action potential. The output of a typical pyramidal cell can be well fit with a Poisson model, and variations in the Poisson rate repeatedly have been shown to be correlated with stimuli. However while the rate provides a very useful characterization of neural spike data, it may not be the most fundamental description of the signaling code.

Eye guidance in natural vision: reinterpreting salience.

Models of gaze allocation in complex scenes are derived mainly from studies of static picture viewing. The dominant framework to emerge has been image salience, where properties of the stimulus play a crucial role in guiding the eyes. However, salience-based schemes are poor at accounting for many aspects of picture viewing and can fail dramatically in the context of natural task performance.

Modelling the role of task in the control of gaze.

Gaze changes and the resultant fixations that orchestrate the sequential acquisition of information from the visual environment are the central feature of primate vision. How are we to understand their function? For the most part, theories of fixation targets have been image based: The hypothesis being that the eye is drawn to places in the scene that contain discontinuities in image features such as motion, colour, and texture. But are these features the cause of the fixations, or merely the result of fixations that have been planned to serve some visual function? This paper examines the issue and reviews evidence from various image-based and task-based sources.

Credit assignment in multiple goal embodied visuomotor behavior.

The intrinsic complexity of the brain can lead one to set aside issues related to its relationships with the body, but the field of embodied cognition emphasizes that understanding brain function at the system level requires one to address the role of the brain-body interface. It has only recently been appreciated that this interface performs huge amounts of computation that does not have to be repeated by the brain, and thus affords the brain great simplifications in its representations. In effect the brain's abstract states can refer to coded representations of the world created by the body.

Predictive feedback can account for biphasic responses in the lateral geniculate nucleus.

Biphasic neural response properties, where the optimal stimulus for driving a neural response changes from one stimulus pattern to the opposite stimulus pattern over short periods of time, have been described in several visual areas, including lateral geniculate nucleus (LGN), primary visual cortex (V1), and middle temporal area (MT). We describe a hierarchical model of predictive coding and simulations that capture these temporal variations in neuronal response properties. We focus on the LGN-V1 circuit and find that after training on natural images the model exhibits the brain's LGN-V1 connectivity structure, in which the structure of V1 receptive fields is linked to the spatial alignment and properties of center-surround cells in the LGN.

Image statistics at the point of gaze during human navigation.

Theories of efficient sensory processing have considered the regularities of image properties due to the structure of the environment in order to explain properties of neuronal representations of the visual world. The regularities imposed on the input to the visual system due to the regularities of the active selection process mediated by the voluntary movements of the eyes have been considered to a much lesser degree. This is surprising, given that the active nature of vision is well established.

Task and context determine where you look.

The deployment of human gaze has been almost exclusively studied independent of any specific ongoing task and limited to two-dimensional picture viewing. This contrasts with its use in everyday life, which mostly consists of purposeful tasks where gaze is crucially involved. To better understand deployment of gaze under such circumstances, we devised a series of experiments, in which subjects navigated along a walkway in a virtual environment and executed combinations of approach and avoidance tasks.

Learning receptive fields using predictive feedback.

Previously, it was suggested that feedback connections from higher- to lower-level areas carry predictions of lower-level neural activities, whereas feedforward connections carry the residual error between the predictions and the actual lower-level activities [Rao, R.P.N.

The role of embodied intention in early lexical acquisition.

We examine the influence of inferring interlocutors' referential intentions from their body movements at the early stage of lexical acquisition. By testing human participants and comparing their performances in different learning conditions, we find that those embodied intentions facilitate both word discovery and word-meaning association. In light of empirical findings, the main part of this article presents a computational model that can identify the sound patterns of individual words from continuous speech, using nonlinguistic contextual information, and employ body movements as deictic references to discover word-meaning associations.

Effect of parietal lobe lesions on saccade targeting and spatial memory in a naturalistic visual search task.

The eye movements of two patients with parietal lobe lesions and four normal observers were measured while they performed a visual search task with naturalistic objects. Patients were slower to perform the task than the normal observers, and the patients had more fixations per trial, longer latencies for the first saccade during the visual search, and less accurate first and second saccades to the target locations during the visual search. The increases in response times for the patients compared to the normal observers were best predicted by increases in the number of fixations.

What you see is what you need.

We studied the role of attention and task demands for implicit change detection. Subjects engaged in an object sorting task performed in a virtual reality environment, where we changed the properties of an object while the subject was manipulating it. The task assures that subjects are looking at the changed object immediately before and after the change.

Eye movements in iconic visual search.

Visual cognition depends critically on the moment-to-moment orientation of gaze. To change the gaze to a new location in space, that location must be computed and used by the oculomotor system. One of the most common sources of information for this computation is the visual appearance of an object.

Fast temporal dynamics of visual cue integration.

The integration of information from different sensors, cues, or modalities lies at the very heart of perception. We are studying adaptive phenomena in visual cue integration. To this end, we have designed a visual tracking task, where subjects track a target object among distractors and try to identify the target after an occlusion.