Introducing the NEMO-Lowlands iconic gesture dataset, collected through a gameful human-robot interaction.

Behav Res Methods

Department of Cognitive Science and Artificial Intelligence, Tilburg University, PO Box 90153, 5000LE, Tilburg, Netherlands.

Published: June 2021

This paper describes a novel dataset of iconic gestures, together with a publicly available robot-based elicitation method to record these gestures, which consists of playing a game of charades with a humanoid robot. The game was deployed at a science museum (NEMO) and a large popular music festival (Lowlands) in the Netherlands. This resulted in recordings of 428 participants, both adults and children, performing 3715 silent iconic gestures for 35 different objects in a naturalistic setting. Our dataset adds to existing collections of iconic gesture recordings in two important ways. First, participants were free to choose how they represented the broad concepts using gestures, and they were asked to perform a second attempt if the robot did not recognize their gesture the first time. This provides insight into potential repair strategies that might be used. Second, by making the interactive game available we enable other researchers to collect additional recordings, for different concepts, and in diverse cultures or contexts. This can be done in a consistent manner because a robot is used as a confederate in the elicitation procedure, which ensures that every data collection session plays out in the same way. The current dataset can be used for research into human gesturing behavior, and as input for the gesture recognition and production capabilities of robots and virtual agents.

Download full-text PDF

Source
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC8219587PMC
http://dx.doi.org/10.3758/s13428-020-01487-0DOI Listing

Publication Analysis

Top Keywords

iconic gesture
8
iconic gestures
8
introducing nemo-lowlands
4
iconic
4
nemo-lowlands iconic
4
gesture
4
dataset
4
gesture dataset
4
dataset collected
4
collected gameful
4

Similar Publications

Ever since de Saussure [Course in General Lingustics (Columbia University Press, 1916)], theorists of language have assumed that the relation between form and meaning of words is arbitrary. However, recently, a body of empirical research has established that language is embodied and contains iconicity. Sound symbolism, an intrinsic link language users perceive between word sound and properties of referents, is a representative example of iconicity in language and has offered profound insights into theories of language pertaining to language processing, language acquisition, and evolution.

View Article and Find Full Text PDF

Gestures are essential in early language development. We investigate the use of gestures in children with cochlear implants (CIs), with a particular focus on deictic, iconic, and conventional gestures. The aim is to understand how the use of gestures in everyday interactions relates to age, vocabulary testing results, and language development reported by parents.

View Article and Find Full Text PDF

Forms and Functions of Gestures in Preverbal 12- to 15-Months Old Infants.

Infancy

December 2024

Centre de Recherche en Psychologie et Neurosciences (CRPN), CNRS, Aix-Marseille Université, Marseille, France.

Speech and co-speech gestures always go hand in hand. Whether we find the precursors of these co-speech gestures in infants before they master their native language still remains an open question. Except for deictic gestures, there is little agreement on the existence of iconic, non-referential and conventional gestures before children start producing their first words.

View Article and Find Full Text PDF

Characterizing iconic gesture during narratives in chronic traumatic brain injury recovery.

Front Hum Neurosci

November 2024

Department of Speech, Language and Hearing Sciences, Indiana University Bloomington, Bloomington, IN, United States.

Article Synopsis
  • Co-speech hand gestures can help individuals with language impairments, like those after a brain injury, communicate more effectively, particularly in cases of traumatic brain injury (TBI).
  • A study with 54 individuals with moderate-severe TBI found that they used various iconic gestures while narrating a simple procedure, even though the relationship between their language impairment and gesturing was complex.
  • Longitudinal analysis indicated that while iconic gesturing was consistent over 2 years of recovery, individual differences were observed, suggesting some relationships between language skills and the use of gestures in TBI patients.
View Article and Find Full Text PDF

Want AI Summaries of new PubMed Abstracts delivered to your In-box?

Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!