The visual experience dataset: Over 200 recorded hours of integrated eye movement, odometry, and egocentric video.

We introduce the Visual Experience Dataset (VEDB), a compilation of more than 240 hours of egocentric video combined with gaze- and head-tracking data that offer an unprecedented view of the visual world as experienced by human observers. The dataset consists of 717 sessions, recorded by 56 observers ranging from 7 to 46 years of age. This article outlines the data collection, processing, and labeling protocols undertaken to ensure a representative sample and discusses the potential sources of error or bias within the dataset.

Impaired stationarity perception is associated with increased virtual reality sickness.

Stationarity perception refers to the ability to accurately perceive the surrounding visual environment as world-fixed during self-motion. Perception of stationarity depends on mechanisms that evaluate the congruence between retinal/oculomotor signals and head movement signals. In a series of psychophysical experiments, we systematically varied the congruence between retinal/oculomotor and head movement signals to find the range of visual gains that is compatible with perception of a stationary environment.

Motor Signals Mediate Stationarity Perception.

Head movement relative to the stationary environment gives rise to congruent vestibular and visual optic-flow signals. The resulting perception of a stationary visual environment, referred to herein as stationarity perception, depends on mechanisms that compare visual and vestibular signals to evaluate their congruence. Here we investigate the functioning of these mechanisms and their dependence on fixation behavior as well as on the active versus passive nature of the head movement.

Natural statistics of human head orientation constrain models of vestibular processing.

Head orientation relative to gravity determines how gravity-dependent environmental structure is sampled by the visual system, as well as how gravity itself is sampled by the vestibular system. Therefore, both visual and vestibular sensory processing should be shaped by the statistics of head orientation relative to gravity. Here we report the statistics of human head orientation during unconstrained natural activities in humans for the first time, and we explore implications for models of vestibular processing.

Natural statistics of human head orientation constrain models of vestibular processing.

Head orientation relative to gravity determines how gravity-dependent environmental structure is sampled by the visual system, as well as how gravity itself is sampled by the vestibular system. Therefore, both visual and vestibular sensory processing should be shaped by the statistics of head orientation relative to gravity. Here we report the statistics of human head orientation during unconstrained natural activities in humans for the first time, and we explore implications for models of vestibular processing.

Evaluation of the Intel RealSense T265 for tracking natural human head motion.

Accurate and robust tracking of natural human head motion in natural environments is important for a number of applications including virtual and augmented reality, clinical diagnostics, as well as basic scientific research. IMU provide a versatile solution for recording inertial data including linear acceleration and angular velocity, but reconstructing head position is difficult or impossible. This problem can be solved by incorporating visual data using a technique known as visual-inertial simultaneous localization and mapping (VI-SLAM).

Positional head-eye tracking outside the lab: an open-source solution.

Simultaneous head and eye tracking has traditionally been confined to a laboratory setting and real-world motion tracking limited to measuring linear acceleration and angular velocity. Recently available mobile devices such as the Pupil Core eye tracker and the Intel RealSense T265 motion tracker promise to deliver accurate measurements outside the lab. Here, the researchers propose a hard- and software framework that combines both devices into a robust, usable, low-cost head and eye tracking system.

Towards dynamic modeling of visual-vestibular conflict detection.

Visual-vestibular mismatch is a common occurrence, with causes ranging from vehicular travel, to vestibular dysfunction, to virtual reality displays. Behavioral and physiological consequences of this mismatch include adaptation of reflexive eye movements, oscillopsia, vertigo, and nausea. Despite this significance, we still do not have a good understanding of how the nervous system evaluates visual-vestibular conflict.

Ecological Momentary Assessment of Head Motion: Toward Normative Data of Head Stabilization.

Head stabilization is fundamental for balance during locomotion but can be impaired in elderly or diseased populations. Previous studies have identified several parameters of head stability with possible diagnostic value in a laboratory setting. Recently, the ecological validity of measures obtained in such controlled contexts has been called into question.

Vestibulo-Ocular Responses and Dynamic Visual Acuity During Horizontal Rotation and Translation.

Dynamic visual acuity (DVA) provides an overall functional measure of visual stabilization performance that depends on the vestibulo-ocular reflex (VOR), but also on other processes, including catch-up saccades and likely visual motion processing. Capturing the efficiency of gaze stabilization against head movement as a whole, it is potentially valuable in the clinical context where assessment of overall patient performance provides an important indication of factors impacting patient participation and quality of life. DVA during head rotation (rDVA) has been assessed previously, but to our knowledge, DVA during horizontal translation (tDVA) has not been measured.

Insufficient compensation for self-motion during perception of object speed: The vestibular Aubert-Fleischl phenomenon.

To estimate object speed with respect to the self, retinal signals must be summed with extraretinal signals that encode the speed of eye and head movement. Prior work has shown that differences in perceptual estimates of object speed based on retinal and oculomotor signals lead to biased percepts such as the Aubert-Fleischl phenomenon (AF), in which moving targets appear slower when pursued. During whole-body movement, additional extraretinal signals, such as those from the vestibular system, may be used to transform object speed estimates from a head-centered to a world-centered reference frame.

Gravity Perception: The Role of the Cerebellum.

The cerebellum is known to support motor behaviors, including postural stability, but new research supports the view that cerebellar function is also critical for perception of spatial orientation, particularly because of its role in vestibular processing.

Psychophysical evidence for auditory motion parallax.

Distance is important: From an ecological perspective, knowledge about the distance to either prey or predator is vital. However, the distance of an unknown sound source is particularly difficult to assess, especially in anechoic environments. In vision, changes in perspective resulting from observer motion produce a reliable, consistent, and unambiguous impression of depth known as motion parallax.

Optic flow detection is not influenced by visual-vestibular congruency.

Optic flow patterns generated by self-motion relative to the stationary environment result in congruent visual-vestibular self-motion signals. Incongruent signals can arise due to object motion, vestibular dysfunction, or artificial stimulation, which are less common. Hence, we are predominantly exposed to congruent rather than incongruent visual-vestibular stimulation.

Visual-Vestibular Conflict Detection Depends on Fixation.

Visual and vestibular signals are the primary sources of sensory information for self-motion. Conflict among these signals can be seriously debilitating, resulting in vertigo [1], inappropriate postural responses [2], and motion, simulator, or cyber sickness [3-8]. Despite this significance, the mechanisms mediating conflict detection are poorly understood.

Quantification of Head Movement Predictability and Implications for Suppression of Vestibular Input during Locomotion.

Achieved motor movement can be estimated using both sensory and motor signals. The value of motor signals for estimating movement should depend critically on the stereotypy or predictability of the resulting actions. As predictability increases, motor signals become more reliable indicators of achieved movement, so weight attributed to sensory signals should decrease accordingly.

Eye Movements in Darkness Modulate Self-Motion Perception.

During self-motion, humans typically move the eyes to maintain fixation on the stationary environment around them. These eye movements could in principle be used to estimate self-motion, but their impact on perception is unknown. We had participants judge self-motion during different eye-movement conditions in the absence of full-field optic flow.

Dependence of auditory spatial updating on vestibular, proprioceptive, and efference copy signals.

Humans localize sounds by comparing inputs across the two ears, resulting in a head-centered representation of sound-source position. When the head moves, information about head movement must be combined with the head-centered estimate to correctly update the world-centered sound-source position. Spatial updating has been extensively studied in the visual system, but less is known about how head movement signals interact with binaural information during auditory spatial updating.

The effect of supine body position on human heading perception.

The use of virtual environments in functional imaging experiments is a promising method to investigate and understand the neural basis of human navigation and self-motion perception. However, the supine position in the fMRI scanner is unnatural for everyday motion. In particular, the head-horizontal self-motion plane is parallel rather than perpendicular to gravity.

Comparison of linear motion perception thresholds in vestibular migraine and Menière's disease.

Linear motion perceptual thresholds (PTs) were compared between patients with Menière's disease (MD) and vestibular migraine (VM). Twenty patients with VM, 27 patients with MD and 34 healthy controls (HC) were examined. PTs for linear motion along the inter-aural (IA), naso-occipital axes (NO), and head-vertical (HV) axis were measured using a multi-axis motion platform.

Optic flow induces nonvisual self-motion aftereffects.

There is strong evidence of shared neurophysiological substrates for visual and vestibular processing that likely support our capacity for estimating our own movement through the environment. We examined behavioral consequences of these shared substrates in the form of crossmodal aftereffects. In particular, we examined whether sustained exposure to a visual self-motion stimulus (i.

Vestibular perception and the vestibulo-ocular reflex in young and older adults.

Objectives: Quantification of the perceptual thresholds to vestibular stimuli may offer valuable complementary information to that provided by measures of the vestibulo-ocular reflex (VOR). Perceptual thresholds could be particularly important in evaluating some subjects, such as the elderly, who might have a greater potential of central as well as peripheral vestibular dysfunction. The authors hypothesized that perceptual detection and discrimination thresholds would worsen with aging, and that there would be a poor relation between thresholds and traditional measures of the angular VOR represented by gain and phase on rotational chair testing.

The importance of stimulus noise analysis for self-motion studies.

Motion simulators are widely employed in basic and applied research to study the neural mechanisms of perception and action during inertial stimulation. In these studies, uncontrolled simulator-introduced noise inevitably leads to a disparity between the reproduced motion and the trajectories meticulously designed by the experimenter, possibly resulting in undesired motion cues to the investigated system. Understanding actual simulator responses to different motion commands is therefore a crucial yet often underestimated step towards the interpretation of experimental results.

Human sensitivity to vertical self-motion.

Perceiving vertical self-motion is crucial for maintaining balance as well as for controlling an aircraft. Whereas heave absolute thresholds have been exhaustively studied, little work has been done in investigating how vertical sensitivity depends on motion intensity (i.e.

Clinical testing of otolith function: perceptual thresholds and myogenic potentials.

Cervical and ocular vestibular-evoked myogenic potential (cVEMP/oVEMP) tests are widely used clinical tests of otolith function. However, VEMP testing may not be the ideal measure of otolith function given the significant inter-individual variability in responses and given that the stimuli used to elicit VEMPs are not physiological. We therefore evaluated linear motion perceptual threshold testing compared with cVEMP and oVEMP testing as measures of saccular and utricular function, respectively.