Objective: To analyse and compare the assessment and grading of human-written and machine-written formative essays.
Study Design: Quasi-experimental, qualitative cross-sectional study. Place and Duration of the Study: Department of Science of Dental Materials, Hamdard College of Medicine & Dentistry, Hamdard University, Karachi, from February to April 2023.
Methodology: Ten short formative essays of final-year dental students were manually assessed and graded. These essays were then graded using ChatGPT version 3.5. The chatbot responses and prompts were recorded and matched with manually graded essays. Qualitative analysis of the chatbot responses was then performed.
Results: Four different prompts were given to the artificial intelligence (AI) driven platform of ChatGPT to grade the summative essays. These were the chatbot's initial responses without grading, the chatbot's response to grading against criteria, the chatbot's response to criteria-wise grading, and the chatbot's response to questions for the difference in grading. Based on the results, four innovative ways of using AI and machine learning (ML) have been proposed for medical educators: Automated grading, content analysis, plagiarism detection, and formative assessment. ChatGPT provided a comprehensive report with feedback on writing skills, as opposed to manual grading of essays.
Conclusion: The chatbot's responses were fascinating and thought-provoking. AI and ML technologies can potentially supplement human grading in the assessment of essays. Medical educators need to embrace AI and ML technology to enhance the standards and quality of medical education, particularly when assessing long and short essay-type questions. Further empirical research and evaluation are needed to confirm their effectiveness.
Key Words: Machine learning, Artificial intelligence, Essays, ChatGPT, Formative assessment.
Download full-text PDF |
Source |
|---|---|
| http://dx.doi.org/10.29271/jcpsp.2024.05.595 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
Designing Health Recommender Systems to Promote Health Equity: A Socioecological Perspective.
J Med Internet Res
January 2025
Department High-Tech Business and Entrepreneurship Section, Industrial Engineering and Business Information Systems, University of Twente, Enschede, Overijssel, Netherlands.
Health recommender systems (HRS) have the capability to improve human-centered care and prevention by personalizing content, such as health interventions or health information. HRS, an emerging and developing field, can play a unique role in the digital health field as they can offer relevant recommendations, not only based on what users themselves prefer and may be receptive to, but also using data about wider spheres of influence over human behavior, including peers, families, communities, and societies. We identify and discuss how HRS could play a unique role in decreasing health inequities.
View Article and Find Full Text PDFUser-Oriented Requirements for Artificial Intelligence-Based Clinical Decision Support Systems in Sepsis: Protocol for a Multimethod Research Project.
JMIR Res Protoc
January 2025
Institute for Health Care Management and Research, University of Duisburg-Essen, Essen, Germany.
Background: Artificial intelligence (AI)-based clinical decision support systems (CDSS) have been developed for several diseases. However, despite the potential to improve the quality of care and thereby positively impact patient-relevant outcomes, the majority of AI-based CDSS have not been adopted in standard care. Possible reasons for this include barriers in the implementation and a nonuser-oriented development approach, resulting in reduced user acceptance.
View Article and Find Full Text PDFEvolution of Artificial Intelligence in Medical Education From 2000 to 2024: Bibliometric Analysis.
Interact J Med Res
January 2025
Department of Obstetrics and Gynecology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China.
Background: Incorporating artificial intelligence (AI) into medical education has gained significant attention for its potential to enhance teaching and learning outcomes. However, it lacks a comprehensive study depicting the academic performance and status of AI in the medical education domain.
Objective: This study aims to analyze the social patterns, productive contributors, knowledge structure, and clusters since the 21st century.
Identification of Intracranial Germ Cell Tumors Based on Facial Photos: Exploratory Study on the Use of Deep Learning for Software Development.
J Med Internet Res
January 2025
Department of Radiation Oncology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China.
Background: Primary intracranial germ cell tumors (iGCTs) are highly malignant brain tumors that predominantly occur in children and adolescents, with an incidence rate ranking third among primary brain tumors in East Asia (8%-15%). Due to their insidious onset and impact on critical functional areas of the brain, these tumors often result in irreversible abnormalities in growth and development, as well as cognitive and motor impairments in affected children. Therefore, early diagnosis through advanced screening techniques is vital for improving patient outcomes and quality of life.
View Article and Find Full Text PDFRisk Factors for Gastrointestinal Bleeding in Patients With Acute Myocardial Infarction: Multicenter Retrospective Cohort Study.
J Med Internet Res
January 2025
Department of Gastroenterology, Affiliated Hospital of Guangdong Medical University, Zhanjiang, China.
Background: Gastrointestinal bleeding (GIB) is a severe and potentially life-threatening complication in patients with acute myocardial infarction (AMI), significantly affecting prognosis during hospitalization. Early identification of high-risk patients is essential to reduce complications, improve outcomes, and guide clinical decision-making.
Objective: This study aimed to develop and validate a machine learning (ML)-based model for predicting in-hospital GIB in patients with AMI, identify key risk factors, and evaluate the clinical applicability of the model for risk stratification and decision support.
Want AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!