Human language is extremely versatile, combining a limited set of signals in an unlimited number of ways. However, it is unknown whether conversational visual signals feed into the composite utterances with which speakers communicate their intentions. We assessed whether different combinations of visual signals lead to different intent interpretations of the same spoken utterance.
View Article and Find Full Text PDFPsychon Bull Rev
August 2024
The natural ecology of language is conversation, with individuals taking turns speaking to communicate in a back-and-forth fashion. Language in this context involves strings of words that a listener must process while simultaneously planning their own next utterance. It would thus be highly advantageous if language users distributed information within an utterance in a way that may facilitate this processing-planning dynamic.
View Article and Find Full Text PDFConversation is a time-pressured environment. Recognizing a social action (the ''speech act,'' such as a question requesting information) early is crucial in conversation to quickly understand the intended message and plan a timely response. Fast turns between interlocutors are especially relevant for responses to questions since a long gap may be meaningful by itself.
View Article and Find Full Text PDFIn conversation, recognizing social actions (similar to 'speech acts') early is important to quickly understand the speaker's intended message and to provide a fast response. Fast turns are typical for fundamental social actions like questions, since a long gap can indicate a dispreferred response. In multimodal face-to-face interaction, visual signals may contribute to this fast dynamic.
View Article and Find Full Text PDFLanguage processing is influenced by sensorimotor experiences. Here, we review behavioral evidence for embodied and grounded influences in language processing across six linguistic levels of granularity. We examine (a) sub-word features, discussing grounded influences on iconicity (systematic associations between word form and meaning); (b) words, discussing boundary conditions and generalizations for the simulation of color, sensory modality, and spatial position; (c) sentences, discussing boundary conditions and applications of action direction simulation; (d) texts, discussing how the teaching of simulation can improve comprehension in beginning readers; (e) conversations, discussing how multi-modal cues improve turn taking and alignment; and (f) text corpora, discussing how distributional semantic models can reveal how grounded and embodied knowledge is encoded in texts.
View Article and Find Full Text PDFThe early recognition of fundamental social actions, like questions, is crucial for understanding the speaker's intended message and planning a timely response in conversation. Questions themselves may express more than one social action category (e.g.
View Article and Find Full Text PDFIn conversation, individuals work together to achieve communicative goals, complementing and aligning language and body with each other. An important emerging question is whether interlocutors entrain with one another equally across linguistic levels (e.g.
View Article and Find Full Text PDFPerspect Psychol Sci
September 2023
Natural human interaction requires us to produce and process many different signals, including speech, hand and head gestures, and facial expressions. These communicative signals, which occur in a variety of temporal relations with each other (e.g.
View Article and Find Full Text PDFPersons with and without autism process sensory information differently. Differences in sensory processing are directly relevant to social functioning and communicative abilities, which are known to be hampered in persons with autism. We collected functional magnetic resonance imaging data from 25 autistic individuals and 25 neurotypical individuals while they performed a silent gesture recognition task.
View Article and Find Full Text PDFIn human communication, when the speech is disrupted, the visual channel (e.g. manual gestures) can compensate to ensure successful communication.
View Article and Find Full Text PDFIn human communication, social intentions and meaning are often revealed in the way we move. In this study, we investigate the flexibility of human communication in terms of kinematic modulation in a clinical population, namely, autistic individuals. The aim of this study was twofold: to assess (a) whether communicatively relevant kinematic features of gestures differ between autistic and neurotypical individuals, and (b) if autistic individuals use communicative kinematic modulation to support gesture recognition.
View Article and Find Full Text PDFIn a conversation, recognising the speaker's social action (e.g., a request) early may help the potential following speakers understand the intended message quickly, and plan a timely response.
View Article and Find Full Text PDFDuring natural conversation, people must quickly understand the meaning of what the other speaker is saying. This concerns not just the semantic content of an utterance, but also the social action (i.e.
View Article and Find Full Text PDFThe actions and feelings questionnaire (AFQ) provides a short, self-report measure of how well someone uses and understands visual communicative signals such as gestures. The objective of this study was to translate and cross-culturally adapt the AFQ into Dutch (AFQ-NL) and validate this new version in neurotypical and autistic populations. Translation and adaptation of the AFQ consisted of forward translation, synthesis, back translation, and expert review.
View Article and Find Full Text PDFThere is increasing evidence that hand gestures and speech synchronize their activity on multiple dimensions and timescales. For example, gesture's kinematic peaks (e.g.
View Article and Find Full Text PDFSocial interaction requires us to recognize subtle cues in behavior, such as kinematic differences in actions and gestures produced with different social intentions. Neuroscientific studies indicate that the putative mirror neuron system (pMNS) in the premotor cortex and mentalizing system (MS) in the medial prefrontal cortex support inferences about contextually unusual actions. However, little is known regarding the brain dynamics of these systems when viewing communicatively exaggerated kinematics.
View Article and Find Full Text PDFHumans are unique in their ability to communicate information through representational gestures which visually simulate an action (eg. moving hands as if opening a jar). Previous research indicates that the intention to communicate modulates the kinematics (e.
View Article and Find Full Text PDFAction, gesture, and sign represent unique aspects of human communication that use form and movement to convey meaning. Researchers typically use manual coding of video data to characterize naturalistic, meaningful movements at various levels of description, but the availability of markerless motion-tracking technology allows for quantification of the kinematic features of gestures or any meaningful human movement. We present a novel protocol for extracting a set of kinematic features from movements recorded with Microsoft Kinect.
View Article and Find Full Text PDFActions may be used to directly act on the world around us, or as a means of communication. Effective communication requires the addressee to recognize the act as being communicative. Humans are sensitive to ostensive communicative cues, such as direct eye gaze (Csibra & Gergely, 2009).
View Article and Find Full Text PDFObjective: Parkinson's disease (PD) often entails impairments of executive functions, such as planning. Although widely held that these impairments arise from dopaminergic denervation of the striatum, not all executive functions are affected early on, and the underlying neural dynamics are not fully understood. In a combined longitudinal and cross-sectional study, we investigated how planning deficits progress over time in the early stages of PD compared to matched healthy controls.
View Article and Find Full Text PDFObjective: Patients with Parkinson's disease (PD) often suffer from impairments in executive functions, such as working memory deficits. It is widely held that dopamine depletion in the striatum contributes to these impairments through decreased activity and connectivity between task-related brain networks. We investigated this hypothesis by studying task-related network activity and connectivity within a sample of de novo patients with PD, versus healthy controls, during a visuospatial working memory task.
View Article and Find Full Text PDF