Previous research has shown that listeners follow speaker gaze to mentioned objects in a shared environment to ground referring expressions, both for human and robot speakers. What is less clear is whether the benefit of speaker gaze is due to the inference of referential intentions (Staudte and Crocker, 2011) or simply the (reflexive) shifts in visual attention. That is, is gaze special in how it affects simultaneous utterance comprehension? In four eye-tracking studies we directly contrast speech-aligned speaker gaze of a virtual agent with a non-gaze visual cue (arrow). Our findings show that both cues similarly direct listeners' attention and that listeners can benefit in utterance comprehension from both cues. Only when they are similarly precise, however, does this equality extend to incongruent cueing sequences: that is, even when the cue sequence does not match the concurrent sequence of spoken referents can listeners benefit from gaze as well as arrows. The results suggest that listeners are able to learn a counter-predictive mapping of both cues to the sequence of referents. Thus, gaze and arrows can in principle be applied with equal flexibility and efficiency during language comprehension.
Download full-text PDF |
Source |
|---|---|
| http://dx.doi.org/10.1016/j.cognition.2014.06.003 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
The ECOLANG Multimodal Corpus of adult-child and adult-adult Language.
Sci Data
January 2025
Experimental Psychology, University College London, London, United Kingdom.
Communication comprises a wealth of multimodal signals (e.g., gestures, eye gaze, intonation) in addition to speech and there is a growing interest in the study of multimodal language by psychologists, linguists, neuroscientists and computer scientists.
View Article and Find Full Text PDFFacial signals shape predictions about the nature of upcoming conversational responses.
Sci Rep
January 2025
Donders Institute for Brain, Cognition and Behaviour, Radboud University, Wundtlaan 1, 6525 XD Nijmegen, Nijmegen, The Netherlands.
Increasing evidence suggests that interlocutors use visual communicative signals to form predictions about unfolding utterances, but there is little data on the predictive potential of facial signals in conversation. In an online experiment with virtual agents, we examine whether facial signals produced by an addressee may allow speakers to anticipate the response to a question before it is given. Participants (n = 80) viewed videos of short conversation fragments between two virtual humans.
View Article and Find Full Text PDFVirtual reality-based Harms tangent screen test for strabismus measurement.
Graefes Arch Clin Exp Ophthalmol
January 2025
Frankfurt Institute for Advanced Studies (FIAS), Frankfurt am Main, Germany.
Purpose: Our study presents a virtual reality-based tangent screen test (VTS) to measure subjective ocular deviations including torsion in nine directions of gaze. The test was compared to the analogous Harms tangent screen test (HTS).
Methods: We used an Oculus Go controller and head-mounted-display with rotation sensors to measure patient's head orientation for the VTS.
The morphophonological dimensions of Spanish gender marking: NP processing in Spanish bilinguals.
Front Hum Neurosci
December 2024
Department of Linguistics, University of Toronto, Toronto, ON, Canada.
The processing literature provides some evidence that heritage Spanish speakers process gender like monolinguals, since gender-marking in definite articles facilitates their lexical access to nouns, albeit these effects may be reduced relative to speakers who learned the language as majority language. However, previous studies rely on slowed-down speech, which leaves open the question of how processing occurs under normal conditions. Using naturalistic speech, our study tests bilingual processing of gender in determiners, and in word-final gender vowels.
View Article and Find Full Text PDFLooking, pointing, and talking together: How dyads of differential expertise coordinate attention during conversation.
PLoS One
December 2024
Department of Psychology, University of British, Columbia, BC, Canada.
When people discuss something that they can both see, their attention becomes increasingly coupled. Previous studies have found that this coupling is temporally asymmetric (e.g.
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!