Conversational facial signals combine into compositional meanings that change the interpretation of speaker intentions.

Human language is extremely versatile, combining a limited set of signals in an unlimited number of ways. However, it is unknown whether conversational visual signals feed into the composite utterances with which speakers communicate their intentions. We assessed whether different combinations of visual signals lead to different intent interpretations of the same spoken utterance.

Information distribution patterns in naturalistic dialogue differ across languages.

The natural ecology of language is conversation, with individuals taking turns speaking to communicate in a back-and-forth fashion. Language in this context involves strings of words that a listener must process while simultaneously planning their own next utterance. It would thus be highly advantageous if language users distributed information within an utterance in a way that may facilitate this processing-planning dynamic.

Conversational Eyebrow Frowns Facilitate Question Identification: An Online Study Using Virtual Avatars.

Conversation is a time-pressured environment. Recognizing a social action (the ''speech act,'' such as a question requesting information) early is crucial in conversation to quickly understand the intended message and plan a timely response. Fast turns between interlocutors are especially relevant for responses to questions since a long gap may be meaningful by itself.

Facilitating question identification through natural intensity eyebrow movements in virtual avatars.

In conversation, recognizing social actions (similar to 'speech acts') early is important to quickly understand the speaker's intended message and to provide a fast response. Fast turns are typical for fundamental social actions like questions, since a long gap can indicate a dispreferred response. In multimodal face-to-face interaction, visual signals may contribute to this fast dynamic.

Embodied Processing at Six Linguistic Granularity Levels: A Consensus Paper.

Language processing is influenced by sensorimotor experiences. Here, we review behavioral evidence for embodied and grounded influences in language processing across six linguistic levels of granularity. We examine (a) sub-word features, discussing grounded influences on iconicity (systematic associations between word form and meaning); (b) words, discussing boundary conditions and generalizations for the simulation of color, sensory modality, and spatial position; (c) sentences, discussing boundary conditions and applications of action direction simulation; (d) texts, discussing how the teaching of simulation can improve comprehension in beginning readers; (e) conversations, discussing how multi-modal cues improve turn taking and alignment; and (f) text corpora, discussing how distributional semantic models can reveal how grounded and embodied knowledge is encoded in texts.

Specific facial signals associate with categories of social actions conveyed through questions.

The early recognition of fundamental social actions, like questions, is crucial for understanding the speaker's intended message and planning a timely response in conversation. Questions themselves may express more than one social action category (e.g.

The Dynamic Interplay of Kinetic and Linguistic Coordination in Danish and Norwegian Conversation.

In conversation, individuals work together to achieve communicative goals, complementing and aligning language and body with each other. An important emerging question is whether interlocutors entrain with one another equally across linguistic levels (e.g.

Interactionally Embedded Gestalt Principles of Multimodal Human Communication.

Natural human interaction requires us to produce and process many different signals, including speech, hand and head gestures, and facial expressions. These communicative signals, which occur in a variety of temporal relations with each other (e.g.

Differences in functional brain organization during gesture recognition between autistic and neurotypical individuals.

Persons with and without autism process sensory information differently. Differences in sensory processing are directly relevant to social functioning and communicative abilities, which are known to be hampered in persons with autism. We collected functional magnetic resonance imaging data from 25 autistic individuals and 25 neurotypical individuals while they performed a silent gesture recognition task.

A multi-scale investigation of the human communication system's response to visual disruption.

In human communication, when the speech is disrupted, the visual channel (e.g. manual gestures) can compensate to ensure successful communication.

Differences in the production and perception of communicative kinematics in autism.

In human communication, social intentions and meaning are often revealed in the way we move. In this study, we investigate the flexibility of human communication in terms of kinematic modulation in a clinical population, namely, autistic individuals. The aim of this study was twofold: to assess (a) whether communicatively relevant kinematic features of gestures differ between autistic and neurotypical individuals, and (b) if autistic individuals use communicative kinematic modulation to support gesture recognition.

Facial Signals and Social Actions in Multimodal Face-to-Face Interaction.

In a conversation, recognising the speaker's social action (e.g., a request) early may help the potential following speakers understand the intended message quickly, and plan a timely response.

The Kinematics of Social Action: Visual Signals Provide Cues for What Interlocutors Do in Conversation.

During natural conversation, people must quickly understand the meaning of what the other speaker is saying. This concerns not just the semantic content of an utterance, but also the social action (i.e.

Translation, Cross-Cultural Adaptation, and Validation of a Dutch Version of the Actions and Feelings Questionnaire in Autistic and Neurotypical Adults.

The actions and feelings questionnaire (AFQ) provides a short, self-report measure of how well someone uses and understands visual communicative signals such as gestures. The objective of this study was to translate and cross-culturally adapt the AFQ into Dutch (AFQ-NL) and validate this new version in neurotypical and autistic populations. Translation and adaptation of the AFQ consisted of forward translation, synthesis, back translation, and expert review.

The quantification of gesture-speech synchrony: A tutorial and validation of multimodal data acquisition using device-based and video-based motion tracking.

There is increasing evidence that hand gestures and speech synchronize their activity on multiple dimensions and timescales. For example, gesture's kinematic peaks (e.g.

Seeing the Unexpected: How Brains Read Communicative Intent through Kinematics.

Social interaction requires us to recognize subtle cues in behavior, such as kinematic differences in actions and gestures produced with different social intentions. Neuroscientific studies indicate that the putative mirror neuron system (pMNS) in the premotor cortex and mentalizing system (MS) in the medial prefrontal cortex support inferences about contextually unusual actions. However, little is known regarding the brain dynamics of these systems when viewing communicatively exaggerated kinematics.

The communicative advantage: how kinematic signaling supports semantic comprehension.

Humans are unique in their ability to communicate information through representational gestures which visually simulate an action (eg. moving hands as if opening a jar). Previous research indicates that the intention to communicate modulates the kinematics (e.

Toward the markerless and automatic analysis of kinematic features: A toolkit for gesture and movement research.

Action, gesture, and sign represent unique aspects of human communication that use form and movement to convey meaning. Researchers typically use manual coding of video data to characterize naturalistic, meaningful movements at various levels of description, but the availability of markerless motion-tracking technology allows for quantification of the kinematic features of gestures or any meaningful human movement. We present a novel protocol for extracting a set of kinematic features from movements recorded with Microsoft Kinect.

Communicative intent modulates production and comprehension of actions and gestures: A Kinect study.

Actions may be used to directly act on the world around us, or as a means of communication. Effective communication requires the addressee to recognize the act as being communicative. Humans are sensitive to ostensive communicative cues, such as direct eye gaze (Csibra & Gergely, 2009).

Impaired planning in Parkinson's disease is reflected by reduced brain activation and connectivity.

Objective: Parkinson's disease (PD) often entails impairments of executive functions, such as planning. Although widely held that these impairments arise from dopaminergic denervation of the striatum, not all executive functions are affected early on, and the underlying neural dynamics are not fully understood. In a combined longitudinal and cross-sectional study, we investigated how planning deficits progress over time in the early stages of PD compared to matched healthy controls.

Reduced neural connectivity but increased task-related activity during working memory in de novo Parkinson patients.

Objective: Patients with Parkinson's disease (PD) often suffer from impairments in executive functions, such as working memory deficits. It is widely held that dopamine depletion in the striatum contributes to these impairments through decreased activity and connectivity between task-related brain networks. We investigated this hypothesis by studying task-related network activity and connectivity within a sample of de novo patients with PD, versus healthy controls, during a visuospatial working memory task.