The aftereffects of ventriloquism: patterns of spatial generalization.

We examined how visual recalibration of apparent sound location obtained at a particular location generalizes to untrained locations. Participants pointed toward the origin of tone bursts scattered along the azimuth, before and after repeated exposure to bursts in one particular location, synchronized with point flashes of light a constant distance to their left/right. Adapter tones were presented straight ahead in Experiment 1, and in the left or right periphery in Experiment 2.

Visual recalibration and selective adaptation in auditory-visual speech perception: Contrasting build-up courses.

Exposure to incongruent auditory and visual speech produces both visual recalibration and selective adaptation of auditory speech identification. In an earlier study, exposure to an ambiguous auditory utterance (intermediate between /aba/ and /ada/) dubbed onto the video of a face articulating either /aba/ or /ada/, recalibrated the perceived identity of auditory targets in the direction of the visual component, while exposure to congruent non-ambiguous /aba/ or /ada/ pairs created selective adaptation, i.e.

The aftereffects of ventriloquism: generalization across sound-frequencies.

Exposure to synchronous but spatially discordant auditory and visual inputs produces, beyond immediate cross-modal biases, adaptive recalibrations of the respective localization processes that manifest themselves in aftereffects. Such recalibrations probably play an important role in maintaining the coherence of spatial representations across the various spatial senses. The present study is part of a research program focused on the way recalibrations generalize to stimulus values different from those used for adaptation.

Recalibration of temporal order perception by exposure to audio-visual asynchrony.

The perception of simultaneity between auditory and visual information is of crucial importance for maintaining a coordinated representation of a multisensory event. Here we show that the perceptual system is able to adaptively recalibrate itself to audio-visual temporal asynchronies. Participants were exposed to a train of sounds and light flashes with a constant time lag ranging from -200 (sound first) to +200 ms (light first).

Visual recalibration of auditory speech identification: a McGurk aftereffect.

The kinds of aftereffects, indicative of cross-modal recalibration, that are observed after exposure to spatially incongruent inputs from different sensory modalities have not been demonstrated so far for identity incongruence. We show that exposure to incongruent audiovisual speech (producing the well-known McGurk effect) can recalibrate auditory speech identification. In Experiment 1, exposure to an ambiguous sound intermediate between /aba/ and /ada/ dubbed onto a video of a face articulating either /aba/ or /ada/ increased the proportion of /aba/ or /ada/ responses, respectively, during subsequent sound identification trials.

Multisensory integration, perception and ecological validity.

Studies of multimodal integration have relied to a large extent on conflict situations, in which two sensory modalities receive incongruent data concerning one aspect of the source. Exposure to such situations produces immediate crossmodal biases as well as longer lasting aftereffects, revealing recalibrations of data-to-percept matches. In the natural environment, such phenomena might be adaptive, by reducing the perturbing effects of factors like noise or growth-induced changes in receptor organs, and by enriching the percept.

Temporal ventriloquism: crossmodal interaction on the time dimension. 2. Evidence from sensorimotor synchronization.

In two experiments, we measured audio-visual crossmodal attraction on the time dimension, using a sensorimotor synchronization task. Synchronization performance made it possible to split up the total crossmodal attraction (demonstrated in earlier studies through inter-modal temporal order judgments) into its modality-specific components, the auditory bias of the visual event's perceived time of occurrence and the visual bias of the auditory event's perceived time of occurrence. Participants were asked to produce tapping movements in synchrony with a sequence of isochronously repeated pacing signals.

Temporal ventriloquism: crossmodal interaction on the time dimension. 1. Evidence from auditory-visual temporal order judgment.

In the well-known visual bias of auditory location (alias the ventriloquist effect), auditory and visual events presented in separate locations appear closer together, provided the presentations are synchronized. Here, we consider the possibility of the converse phenomenon: crossmodal attraction on the time dimension conditional on spatial proximity. Participants judged the order of occurrence of sound bursts and light flashes, respectively, separated in time by varying stimulus onset asynchronies (SOAs) and delivered either in the same or in different locations.

The aftereffects of ventriloquism: are they sound-frequency specific?

Exposing different sense modalities (like sight, hearing or touch) to repeated simultaneous but spatially discordant stimulations generally causes recalibration of localization processes in one or both of the involved modalities, which is manifested through aftereffects. These provide opportunities for determining the extent of the changes induced by the exposure. Taking the so-called ventriloquism situation, in which synchronized sounds and light flashes are delivered in different locations, we examine if auditory recalibration produced by exposing tones of one frequency to attraction by discordant light flashes generalizes to different frequencies.