Epilepsy surgery candidate identification with artificial intelligence: An implementation study.

Background: To (a) evaluate the effect of a machine learning algorithm in the identification of patients suitable for epilepsy surgery evaluation, and (b) examine the performance of a large language model (LLM) in the collation of key pieces of information pertaining to epilepsy surgery evaluation referral.

Methods: Artificial intelligence analyses were performed for all patients seen in the epilepsy or first seizure clinic at a tertiary hospital over a 12-month period. This study design was intended to emulate a case review that could subsequently be conducted periodically (e.g., quarterly). The previously derived random forest model was used to stratify all patients by their likelihood of being a candidate for epilepsy surgery evaluation, and the top 5% of cases underwent manual case note review. An open source LLM was utilised to answer 7 prompts summarising and extracting pieces of information from the most recent clinic note, which would be relevant to epilepsy surgery evaluation referral.

Results: 310 patients were included in the study, with 15 undergoing manual review. Of these patients 8/15 (53.3 %) met the prespecified criteria for epilepsy surgery evaluation. 3/15 (20.0 %) of these patients were subsequently referred for further evaluation within 1 month of the study. The LLM had an accuracy ranging between 80 % to 100 % on the different prompts. Errors occurred most often when summarising the management plan. Errors included hallucinations, omissions, and copying erroneous information.

Conclusions: Artificial intelligence may be able to assist with the identification of patients suitable for epilepsy surgery evaluation.