Background: ChatGPT may act as a research assistant to help organize the direction of thinking and summarize research findings. However, few studies have examined the quality, similarity (abstracts being similar to the original one), and accuracy of the abstracts generated by ChatGPT when researchers provide full-text basic research papers.
Objective: We aimed to assess the applicability of an artificial intelligence (AI) model in generating abstracts for basic preclinical research.
Methods: We selected 30 basic research papers from Nature, Genome Biology, and Biological Psychiatry. Excluding abstracts, we inputted the full text into ChatPDF, an application of a language model based on ChatGPT, and we prompted it to generate abstracts with the same style as used in the original papers. A total of 8 experts were invited to evaluate the quality of these abstracts (based on a Likert scale of 0-10) and identify which abstracts were generated by ChatPDF, using a blind approach. These abstracts were also evaluated for their similarity to the original abstracts and the accuracy of the AI content.
Results: The quality of ChatGPT-generated abstracts was lower than that of the actual abstracts (10-point Likert scale: mean 4.72, SD 2.09 vs mean 8.09, SD 1.03; P<.001). The difference in quality was significant in the unstructured format (mean difference -4.33; 95% CI -4.79 to -3.86; P<.001) but minimal in the 4-subheading structured format (mean difference -2.33; 95% CI -2.79 to -1.86). Among the 30 ChatGPT-generated abstracts, 3 showed wrong conclusions, and 10 were identified as AI content. The mean percentage of similarity between the original and the generated abstracts was not high (2.10%-4.40%). The blinded reviewers achieved a 93% (224/240) accuracy rate in guessing which abstracts were written using ChatGPT.
Conclusions: Using ChatGPT to generate a scientific abstract may not lead to issues of similarity when using real full texts written by humans. However, the quality of the ChatGPT-generated abstracts was suboptimal, and their accuracy was not 100%.
Download full-text PDF |
Source |
|---|---|
| http://www.ncbi.nlm.nih.gov/pmc/articles/PMC10760418 | PMC |
| http://dx.doi.org/10.2196/51229 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
[Overview of the French Memory Centres healthcare pathway for patients with early Alzheimer disease].
Geriatr Psychol Neuropsychiatr Vieil
September 2024
CMRR CHU Toulouse, IHU Health Age, Toulouse, UMR 1297 CERPOP.
The growing prevalence of the Alzheimer's disease (AD) is an increasing public health concern that led to French recommendations for timely AD diagnosis and patient management as well as a territorial coverage of specialized structures [Memory Centers including Resources and Memory Research Centers (RMRC) and Memory Consultations (MC)]. In view of the potential availability of Disease Modifying Therapies (DMTs), this French observatory aimed to describe the current organization of the Memory Centers, and the care pathway of patients suffering from early AD. Overall, 12 of the 28 RMRC and 44 of the 250 MC solicited by the Federation of Memory Centers participated in this study.
View Article and Find Full Text PDF[Knowing (or not knowing) their own socio-cognitive abilities].
Geriatr Psychol Neuropsychiatr Vieil
September 2024
Metacognition, the ability to monitor and regulate one's own cognitive processes, is subject to varying degrees of modification in patients suffering from neurodegenerative diseases. The literature suggests the existence of dissociations within metacognitive abilities, with some patients exhibiting, for example, specific impairments in self-assessing their memory (and not other cognitive domains). The specific assessment of metacognition in patients' social-cognitive abilities is underdeveloped, although it has significant implications for both clinical and theoretical purposes.
View Article and Find Full Text PDFCarvacrol: Innovative Synthesis Pathways and Overview of its Patented Applications.
Recent Pat Biotechnol
January 2025
Chemical Science and Engineering Research Team (ERSIC), Department of Chemistry, Polydisciplinary Faculty of Beni Mellal (FPBM), Sultan Moulay Slimane University (USMS), P.O. Box 592 Mghila, Beni Mellal 23000, Morocco.
Aim: This research concerns the patentability of carvacrol; it could be helpful for researchers to easily identify any innovation in the biotechnological application of this monoterpene as well as other similar compounds.
Background: Like thyme or oregano, several plants in the Lamiaceae family produce carvacrol. It is one of the secondary metabolites with several biological activities, including the improvement in plants' resistance and their protection.
Quantifying research hotspots and trends in brucella spondylitis: a bibliometric analysis.
Front Surg
January 2025
Department of Orthopedic, General Hospital of Ningxia Medical University, Yinchuan, Ningxia, China.
Background: Human brucellosis is the most common bacterial zoonosis worldwide, with brucella spondylitis (BS) being one of its most severe forms, potentially leading to spinal deformity or paralysis. This study aims to provide a comprehensive overview of the current status and research trends in the BS field using bibliometric methods.
Methods: Publications on BS from January 1, 1980, to March 24, 2024, were retrieved from the Web of Science database.
Logistics Management Information System (LMIS) for Health Commodities at Public Health Facilities in Amhara National Regional State of Ethiopia: A Data Quality Evaluation Survey.
J Multidiscip Healthc
January 2025
Department of Pharmaceutics and Social Pharmacy, Addis Ababa University, Addis Ababa, Ethiopia.
Introduction: Access to essential medicines is limited in developing countries mainly due to inefficiencies in health supply chain management, such as the absence of standard monitoring frameworks and poorly designed Logistics Management Information Systems (LMIS). Health supply chain managers need accurate and timely data for decision-making. However, routine health information systems suffer from poor data quality, reliance on paper-based reports, insufficient logistic formats, inadequate infrastructure, and limited human resources.
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!