Severity: Warning
Message: fopen(/var/lib/php/sessions/ci_session0crffd5sj31khvoj59lg1obsu1h6tg3m): Failed to open stream: No space left on device
Filename: drivers/Session_files_driver.php
Line Number: 177
Backtrace:
File: /var/www/html/index.php
Line: 316
Function: require_once
Severity: Warning
Message: session_start(): Failed to read session data: user (path: /var/lib/php/sessions)
Filename: Session/Session.php
Line Number: 137
Backtrace:
File: /var/www/html/index.php
Line: 316
Function: require_once
Severity: Warning
Message: Undefined array key "choices"
Filename: controllers/Detail.php
Line Number: 249
Backtrace:
File: /var/www/html/application/controllers/Detail.php
Line: 249
Function: _error_handler
File: /var/www/html/index.php
Line: 316
Function: require_once
Severity: Warning
Message: Trying to access array offset on value of type null
Filename: controllers/Detail.php
Line Number: 249
Backtrace:
File: /var/www/html/application/controllers/Detail.php
Line: 249
Function: _error_handler
File: /var/www/html/index.php
Line: 316
Function: require_once
Severity: Warning
Message: Trying to access array offset on value of type null
Filename: controllers/Detail.php
Line Number: 249
Backtrace:
File: /var/www/html/application/controllers/Detail.php
Line: 249
Function: _error_handler
File: /var/www/html/index.php
Line: 316
Function: require_once
Severity: Warning
Message: Trying to access array offset on value of type null
Filename: controllers/Detail.php
Line Number: 249
Backtrace:
File: /var/www/html/application/controllers/Detail.php
Line: 249
Function: _error_handler
File: /var/www/html/index.php
Line: 316
Function: require_once
Severity: 8192
Message: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated
Filename: models/Detail_model.php
Line Number: 71
Backtrace:
File: /var/www/html/application/models/Detail_model.php
Line: 71
Function: strpos
File: /var/www/html/application/controllers/Detail.php
Line: 252
Function: insertAPISummary
File: /var/www/html/index.php
Line: 316
Function: require_once
Severity: 8192
Message: str_replace(): Passing null to parameter #3 ($subject) of type array|string is deprecated
Filename: helpers/my_audit_helper.php
Line Number: 8919
Backtrace:
File: /var/www/html/application/helpers/my_audit_helper.php
Line: 8919
Function: str_replace
File: /var/www/html/application/controllers/Detail.php
Line: 255
Function: formatAIDetailSummary
File: /var/www/html/index.php
Line: 316
Function: require_once
Severity: Warning
Message: Undefined array key "choices"
Filename: controllers/Detail.php
Line Number: 256
Backtrace:
File: /var/www/html/application/controllers/Detail.php
Line: 256
Function: _error_handler
File: /var/www/html/index.php
Line: 316
Function: require_once
Severity: Warning
Message: Trying to access array offset on value of type null
Filename: controllers/Detail.php
Line Number: 256
Backtrace:
File: /var/www/html/application/controllers/Detail.php
Line: 256
Function: _error_handler
File: /var/www/html/index.php
Line: 316
Function: require_once
Severity: Warning
Message: Trying to access array offset on value of type null
Filename: controllers/Detail.php
Line Number: 256
Backtrace:
File: /var/www/html/application/controllers/Detail.php
Line: 256
Function: _error_handler
File: /var/www/html/index.php
Line: 316
Function: require_once
Severity: Warning
Message: Undefined array key "usage"
Filename: controllers/Detail.php
Line Number: 257
Backtrace:
File: /var/www/html/application/controllers/Detail.php
Line: 257
Function: _error_handler
File: /var/www/html/index.php
Line: 316
Function: require_once
Severity: Warning
Message: Trying to access array offset on value of type null
Filename: controllers/Detail.php
Line Number: 257
Backtrace:
File: /var/www/html/application/controllers/Detail.php
Line: 257
Function: _error_handler
File: /var/www/html/index.php
Line: 316
Function: require_once
Severity: Warning
Message: Undefined array key "usage"
Filename: controllers/Detail.php
Line Number: 258
Backtrace:
File: /var/www/html/application/controllers/Detail.php
Line: 258
Function: _error_handler
File: /var/www/html/index.php
Line: 316
Function: require_once
Severity: Warning
Message: Trying to access array offset on value of type null
Filename: controllers/Detail.php
Line Number: 258
Backtrace:
File: /var/www/html/application/controllers/Detail.php
Line: 258
Function: _error_handler
File: /var/www/html/index.php
Line: 316
Function: require_once
Severity: Warning
Message: Undefined array key "usage"
Filename: controllers/Detail.php
Line Number: 259
Backtrace:
File: /var/www/html/application/controllers/Detail.php
Line: 259
Function: _error_handler
File: /var/www/html/index.php
Line: 316
Function: require_once
Severity: Warning
Message: Trying to access array offset on value of type null
Filename: controllers/Detail.php
Line Number: 259
Backtrace:
File: /var/www/html/application/controllers/Detail.php
Line: 259
Function: _error_handler
File: /var/www/html/index.php
Line: 316
Function: require_once
Severity: Warning
Message: Undefined array key "usage"
Filename: controllers/Detail.php
Line Number: 260
Backtrace:
File: /var/www/html/application/controllers/Detail.php
Line: 260
Function: _error_handler
File: /var/www/html/index.php
Line: 316
Function: require_once
Severity: Warning
Message: Trying to access array offset on value of type null
Filename: controllers/Detail.php
Line Number: 260
Backtrace:
File: /var/www/html/application/controllers/Detail.php
Line: 260
Function: _error_handler
File: /var/www/html/index.php
Line: 316
Function: require_once
Severity: Warning
Message: Trying to access array offset on value of type null
Filename: controllers/Detail.php
Line Number: 260
Backtrace:
File: /var/www/html/application/controllers/Detail.php
Line: 260
Function: _error_handler
File: /var/www/html/index.php
Line: 316
Function: require_once
Severity: Warning
Message: file_get_contents(https://...@gmail.com&api_key=61f08fa0b96a73de8c900d749fcb997acc09): Failed to open stream: HTTP request failed! HTTP/1.1 429 Too Many Requests
Filename: helpers/my_audit_helper.php
Line Number: 143
Backtrace:
File: /var/www/html/application/helpers/my_audit_helper.php
Line: 143
Function: file_get_contents
File: /var/www/html/application/helpers/my_audit_helper.php
Line: 209
Function: simplexml_load_file_from_url
File: /var/www/html/application/helpers/my_audit_helper.php
Line: 3098
Function: getPubMedXML
File: /var/www/html/application/controllers/Detail.php
Line: 574
Function: pubMedSearch_Global
File: /var/www/html/application/controllers/Detail.php
Line: 488
Function: pubMedGetRelatedKeyword
File: /var/www/html/index.php
Line: 316
Function: require_once
Severity: Warning
Message: Attempt to read property "Count" on bool
Filename: helpers/my_audit_helper.php
Line Number: 3100
Backtrace:
File: /var/www/html/application/helpers/my_audit_helper.php
Line: 3100
Function: _error_handler
File: /var/www/html/application/controllers/Detail.php
Line: 574
Function: pubMedSearch_Global
File: /var/www/html/application/controllers/Detail.php
Line: 488
Function: pubMedGetRelatedKeyword
File: /var/www/html/index.php
Line: 316
Function: require_once
Aim Of The Study: Artificial intelligence (AI) chatbots are established as tools for answering medical questions worldwide. Healthcare trainees are increasingly using this cutting-edge technology, although its reliability and accuracy in the context of healthcare remain uncertain. This study evaluated the suitability of Chat-GPT versions 3.5 and 4 for healthcare professionals seeking up-to-date evidence and recommendations for resuscitation by comparing the key messages of the resuscitation guidelines, which methodically set the gold standard of current evidence and recommendations, with the statements of the AI chatbots on this topic.
Methods: This prospective comparative content analysis was conducted between the 2021 European Resuscitation Council (ERC) guidelines and the responses of two freely available ChatGPT versions (ChatGPT-3.5 and the Bing version of the ChatGPT-4) to questions about the key messages of clinically relevant ERC guideline chapters for adults. (1) The content analysis was performed bidirectionally by independent raters. The completeness and actuality of the AI output were assessed by comparing the key message with the AI-generated statements. (2) The conformity of the AI output was evaluated by comparing the statements of the two ChatGPT versions with the content of the ERC guidelines.
Results: In response to inquiries about the five chapters, ChatGPT-3.5 generated a total of 60 statements, whereas ChatGPT-4 produced 32 statements. ChatGPT-3.5 did not address 123 key messages, and ChatGPT-4 did not address 132 of the 172 key messages of the ERC guideline chapters. A total of 77% of the ChatGPT-3.5 statements and 84% of the ChatGPT-4 statements were fully in line with the ERC guidelines. The main reason for nonconformity was superficial and incorrect AI statements. The interrater reliability between the two raters, measured by Cohen's kappa, was greater for ChatGPT-4 (0.56 for completeness and 0.76 for conformity analysis) than for ChatGPT-3.5 (0.48 for completeness and 0.36 for conformity).
Conclusion: We advise healthcare professionals not to rely solely on the tested AI-based chatbots to keep up to date with the latest evidence, as the relevant texts for the task were not part of the training texts of the underlying LLMs, and the lack of conceptual understanding of AI carries a high risk of spreading misconceptions. Original publications should always be considered for comprehensive understanding.
Download full-text PDF |
Source |
---|---|
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC11425874 | PMC |
http://dx.doi.org/10.1186/s13049-024-01266-2 | DOI Listing |
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!