Generative AI, short for Generative Artificial Intelligence, a class of artificial intelligence systems, is not currently the choice technology for text analysis, but prior work suggests it may have some utility to assess dynamics like emotion. The current work builds upon this empirical foundation to consider how analytic thinking scores from a large language model chatbot, ChatGPT, were linked to analytic thinking scores from dictionary-based tools like Linguistic Inquiry and Word Count (LIWC). Using over 16,000 texts from four samples and tested against three prompts and two large language models (GPT-3.5, GPT-4), the evidence suggests there were small associations between ChatGPT and LIWC analytic thinking scores (meta-analytic effect sizes: .058 < rs < .304; ps < .001). When given the formula to calculate the LIWC analytic thinking index, ChatGPT performed incorrect mathematical operations in 22% of the cases, suggesting basic word and number processing may be unreliable with large language models. Researchers should be cautious when using AI for text analysis.
Download full-text PDF |
Source |
|---|---|
| http://dx.doi.org/10.3758/s13428-024-02344-0 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
Dynamic switching between brain networks predicts creative ability.
Commun Biol
January 2025
Department of Psychology, Pennsylvania State University, University Park, Pennsylvania, USA.
Creativity is hypothesized to arise from a mental state which balances spontaneous thought and cognitive control, corresponding to functional connectivity between the brain's Default Mode (DMN) and Executive Control (ECN) Networks. Here, we conduct a large-scale, multi-center examination of this hypothesis. Employing a meta-analytic network neuroscience approach, we analyze resting-state fMRI and creative task performance across 10 independent samples from Austria, Canada, China, Japan, and the United States (Nā=ā2433)-constituting the largest and most ethnically diverse creativity neuroscience study to date.
View Article and Find Full Text PDFTargeting audiences' moral values shapes misinformation sharing.
J Exp Psychol Gen
January 2025
Department of Psychology, University of Southern California.
Does aligning misinformation content with individuals' core moral values facilitate its spread? We investigate this question in three behavioral experiments ( = 615; = 505; ā = 533) that examine how the alignment of audience values and misinformation framing affects sharing behavior, in conjunction with analyzing real-world Twitter data ( = 20,235; 809,414 tweets) that explores how aligning the moral values of message senders with misinformation content influences its dissemination in the context of COVID-19 vaccination misinformation. First, we investigate how aligning messages' moral framing with participants' moral values impacts participants' intentions to share true and false news headlines and whether this effect is driven by a lack of analytical thinking. Our results show that framing a post such that it aligns with audiences' moral values leads to increased sharing intentions, independent of headline familiarity, and participants' political ideology but find no effect of analytical thinking.
View Article and Find Full Text PDFQuantifying and Measuring Value in Rehabilitation: A Scoping Review Protocol.
Musculoskeletal Care
March 2025
Rehabilitation Services, The Ohio State University Wexner Medical Center, Columbus, Ohio, USA.
Prioritizing attributes of approaches to analyzing patient-centered outcomes that are truncated due to death in critical care clinical trials: a Delphi study.
Trials
January 2025
Palliative and Advanced Illness Research (PAIR) Center, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
Background: A key challenge for many critical care clinical trials is that some patients will die before their outcome is fully measured. This is referred to as "truncation due to death" and must be accounted for in both the treatment effect definition (i.e.
View Article and Find Full Text PDFResearch on the construction of an urban underground parking space color system from the perspective of psychological perception.
PLoS One
January 2025
College of Architecture and Civil Engineering, Xi'an University of Science and Technology, Xi'an, Shaanxi, P. R. China.
Color, an intuitive element affecting human senses, can adapt to the environment of a space, evoke emotional responses, trigger and accumulate visual experiences, and enhance the effectiveness of color in shaping spatial atmosphere and reinforcing spatial divisions. In the context of rapid urban underground space development, examining the rational application of color in underground parking spaces is crucial for improving guidance, comfort, and aesthetics. This exploration is essential for achieving high-quality development in urban underground parking environments.
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!