The Processing of Semantic Complexity and Cospeech Gestures in Schizophrenia: A Naturalistic, Multimodal fMRI Study.

Schizophrenia is marked by aberrant processing of complex speech and gesture, which may contribute functionally to its impaired social communication. To date, extant neuroscientific studies of schizophrenia have largely investigated dysfunctional speech and gesture in isolation, and no prior research has examined how the two communicative channels may interact in more natural contexts. Here, we tested if patients with schizophrenia show aberrant neural processing of semantically complex story segments, and if speech-associated gestures (co-speech gestures) might modulate this effect.

Neural Basis of Speech-Gesture Mismatch Detection in Schizophrenia Spectrum Disorders.

Patients with schizophrenia spectrum disorders (SSD) exhibit an aberrant perception and comprehension of abstract speech-gesture combinations associated with dysfunctional activation of the left inferior frontal gyrus (IFG). Recently, a significant deficit of speech-gesture mismatch detection was identified in SSD, but the underlying neural mechanisms have not yet been examined. A novel mismatch-detection fMRI paradigm was implemented manipulating speech-gesture abstractness (abstract/concrete) and relatedness (related/unrelated).

The role of the left and right inferior frontal gyrus in processing metaphoric and unrelated co-speech gestures.

Gestures are an integral part of in-person conversations and complement the meaning of the speech they accompany. The neural processing of co-speech gestures is supported by a mostly left-lateralized network of fronto-temporal regions. However, in contrast to iconic gestures, metaphoric as well as unrelated gestures have been found to more strongly engage the left and right inferior frontal gyrus (IFG), respectively.

Modality-specific dysfunctional neural processing of social-abstract and non-social-concrete information in schizophrenia.

Schizophrenia is characterized by marked communication dysfunctions encompassing potential impairments in the processing of social-abstract and non-social-concrete information, especially in everyday situations where multiple modalities are present in the form of speech and gesture. To date, the neurobiological basis of these deficits remains elusive. In a functional magnetic resonance imaging (fMRI) study, 17 patients with schizophrenia or schizoaffective disorder, and 18 matched controls watched videos of an actor speaking, gesturing (unimodal), and both speaking and gesturing (bimodal) about social or non-social events in a naturalistic way.

Feeling addressed! The neural processing of social communicative cues in patients with major depression.

The feeling of being addressed is the first step in a complex processing stream enabling successful social communication. Social impairments are a relevant characteristic of patients with major depressive disorder (MDD). Here, we investigated a mechanism which-if impaired-might contribute to withdrawal or isolation in MDD, namely, the neural processing of social cues such as body orientation and gesture.

Emotion specific neural activation for the production and perception of facial expressions.

The distinction between different facial emotions is crucial for interpersonal communication. Shared neural circuits for facial emotion production and perception are considered to facilitate the ability to understand other's emotional state via mirror neuron mechanisms. Little is known about how diverse emotions differentially activate the Mirror Neuron System (MNS) during facial expression processing.

Conflicting group memberships modulate neural activation in an emotional production-perception network.

Social group membership modulates the neural processing of emotional facial expressions, which, in turn, recruits part of the neural production system. However, little is known about how mixed - and potentially conflicting - social identity cues affect this mechanism. In this study, we tested the hypothesis that incongruent cues of two group memberships (ethnic and experimentally created minimal groups) elicit conflict processing for neutral and, in particular, angry facial expressions.

The facilitative effect of gestures on the neural processing of semantic complexity in a continuous narrative.

Gestures are elemental components of social communication and aid comprehension of verbal messages; however, little is known about the potential role of gestures in facilitating processing of semantic complexity in an ecologically valid setting. The goal of this study was to investigate whether cognitive load, as indexed by semantic complexity, is modulated by the presentation of gestures accompanying speech. Twenty healthy participants watched 16 video clips of a short narrative while instructed to carefully listen to and watch the narrator while functional magnetic resonance imaging (fMRI) data were acquired.

Intergroup empathy: Enhanced neural resonance for ingroup facial emotion in a shared neural production-perception network.

Introduction: Empathic behavior and related neural processing are strongly modified by group membership. Shared neural circuits for the production and perception of facial emotional expressions represent mirror neuron mechanisms which play a pivotal role for empathy. In this study, we investigate the influence of group membership on mirror neuron mechanisms for emotional facial expressions.

Evidence for gesture-speech mismatch detection impairments in schizophrenia.

Patients with schizophrenia suffer from impairments in the perception and production of gestures. The extent to which patients can access the semantic association between speech and co-verbal gestures in concrete or abstract/metaphorical meaning contexts is unknown. We investigated 1) how patients differ from controls in gesture matching performance, 2) how performance differs in the context of abstract versus concrete meaning, and 3) whether formal thought disorder (FTD) symptom severity predicts task impairment.

Spatial-temporal dynamics of gesture-speech integration: a simultaneous EEG-fMRI study.

The semantic integration between gesture and speech (GSI) is mediated by the left posterior temporal sulcus/middle temporal gyrus (pSTS/MTG) and the left inferior frontal gyrus (IFG). Evidence from electroencephalography (EEG) suggests that oscillations in the alpha and beta bands may support processes at different stages of GSI. In the present study, we investigated the relationship between electrophysiological oscillations and blood-oxygen-level-dependent (BOLD) activity during GSI.

Action-Related Speech Modulates Beta Oscillations During Observation of Tool-Use Gestures.

Language and action have been thought of as closely related. Comprehending words or phrases that are related to actions commonly activates motor and premotor areas, and this comprehension process interacts with action preparation and/or execution. However, it remains unclear whether comprehending action-related language interacts with action observation.

The EEG and fMRI signatures of neural integration: An investigation of meaningful gestures and corresponding speech.

One of the key features of human interpersonal communication is our ability to integrate information communicated by speech and accompanying gestures. However, it is still not fully understood how this essential combinatory process is represented in the human brain. Functional magnetic resonance imaging (fMRI) studies have unanimously attested the relevance of activation in the posterior superior temporal sulcus/middle temporal gyrus (pSTS/MTG), while electroencephalography (EEG) studies have shown oscillatory activity in specific frequency bands to be associated with multisensory integration.

Feeling addressed! The role of body orientation and co-speech gesture in social communication.

During face-to-face communication, body orientation and coverbal gestures influence how information is conveyed. The neural pathways underpinning the comprehension of such nonverbal social cues in everyday interaction are to some part still unknown. During fMRI data acquisition, 37 participants were presented with video clips showing an actor speaking short sentences.

Supramodal neural processing of abstract information conveyed by speech and gesture.

ness and modality of interpersonal communication have a considerable impact on comprehension. They are relevant for determining thoughts and constituting internal models of the environment. Whereas concrete object-related information can be represented in mind irrespective of language, abstract concepts require a representation in speech.