The integration of text and picture is the core of multimedia information processing. Relevant theories suggest that text and picture are processed through different channels in the early stage, and integrated in the late stage of processing. Based on these theories, the current study adopted measures of event-related potentials to examine the cognitive and neural processes of text-picture integration. The results showed that in the early stage of text-picture integration, picture processing evoked a more negative N1 over the occipital area and a N300 over the prefrontal area, which might reflect the discrimination process of visual stimuli and the imagery representation of the picture, respectively; in the late stage, the text-picture induced a N400 in the central area and an LPC over the central, parietal and temporal areas, which might be associated with the semantic activation and integration of text and picture, respectively. These results not only provide support for existing theories, but also further elucidate the dynamic neural processing of text-picture integration in terms of its temporal and spatial characteristics.
Download full-text PDF |
Source |
|---|---|
| http://dx.doi.org/10.1016/j.brainres.2019.146513 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
The Text-Picture Integration Scale for Perspectives on Mental Illness: Development and Validation.
Am J Occup Ther
May 2024
Chia-Hui Hung, PhD, is Associate Professor, Department of Occupational Therapy, College of Medical Science and Technology, Chung Shan Medical University, and Occupational Therapy Room, Chung Shan Medical University Hospital, Taichung City, Taiwan Taichung City, Taiwan;
Importance: Understanding the root cause of mental illness stigma is necessary to adopt effective management strategies.
Objective: To establish a stable and effective text-picture integration rating scale to predict public perspectives on mental illness and to examine its reliability and validity.
Design: Descriptive cross-sectional study using internet survey data.
Using eye-tracking and EEG to study the mental processing demands during learning of text-picture combinations.
Int J Psychophysiol
December 2020
Leibniz-Institut für Wissensmedien Tübingen, Germany; Department of Cognitive Psychology and Media Psychology, University of Tübingen, Germany.
Using and combining eye-tracking and EEG frequency band power as process measures, in the current study we were interested in the mental processing demands during learning of text-picture combinations that either enabled or prohibited text-picture integration (TPI). In the mismatch condition, the textual and pictorial information being dissimilar, TPI was not possible, whereas in the match and the partial-match condition, the textual and pictorial information being identical respective complementary, TPI was possible. We expected mental processing demands to be higher in the mismatch condition, when pictorial and textual information had to be processed and memorized as separate representations, compared to the match and partial-match conditions when TPI was possible.
View Article and Find Full Text PDFDynamic cognitive processes of text-picture integration revealed by event-related potentials.
Brain Res
January 2020
Key Laboratory of Adolescent Cyberpsychology and Behavior (CCNU), Ministry of Education, Central China Normal University, Wuhan 430079, China; Key Laboratory of Human Development and Mental Health of Hubei Province, School of Psychology, Central China Normal University, Wuhan 430079, China. Electronic address:
The integration of text and picture is the core of multimedia information processing. Relevant theories suggest that text and picture are processed through different channels in the early stage, and integrated in the late stage of processing. Based on these theories, the current study adopted measures of event-related potentials to examine the cognitive and neural processes of text-picture integration.
View Article and Find Full Text PDFImplementation of the uterine fibroids Option Grid patient decision aids across five organizational settings: a randomized stepped-wedge study protocol.
Implement Sci
September 2019
The Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine at Dartmouth College, One Medical Center Drive, 5th floor, Lebanon, NH, 03756, USA.
Background: Uterine fibroids are non-cancerous overgrowths of the smooth muscle in the uterus. As they grow, some cause problems such as heavy menstrual bleeding, pelvic pain, discomfort during sexual intercourse, and rarely pregnancy complications or difficulty becoming pregnant. Multiple treatment options are available.
View Article and Find Full Text PDFTexts and pictures serve different functions in conjoint mental model construction and adaptation.
Mem Cognit
January 2020
University of Hagen, Hagen, Germany.
In this study we examined the different functions of text and pictures during text-picture integration in multimedia learning. In Study 1, 144 secondary school students (age = 11 to 14 years; 72 females, 72 males) received six text-picture units under two conditions. In the delayed-question condition, students first read the units without a specific question (no-question phase), to stimulate initial coherence-oriented mental model construction.
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!