AI Article Synopsis
- NER models using Transformers have become popular for their performance across different languages and fields, but differences in how they evaluate tokens versus entities are often ignored.
- This study focuses on French oncological reports, fine-tuning four BERT-based models to classify tokens and assess their performance at both token and entity levels.
- Results show significant discrepancies in effectiveness between the two evaluation methods and indicate that while BERT outperforms ChatGPT in recognizing complex entities in French, comprehensive evaluation approaches are crucial in NER tasks.
Article Abstract
Named Entity Recognition (NER) models based on Transformers have gained prominence for their impressive performance in various languages and domains. This work delves into the often-overlooked aspect of entity-level metrics and exposes significant discrepancies between token and entity-level evaluations. The study utilizes a corpus of synthetic French oncological reports annotated with entities representing oncological morphologies. Four different French BERT-based models are fine-tuned for token classification, and their performance is rigorously assessed at both token and entity-level. In addition to fine-tuning, we evaluate ChatGPT's ability to perform NER through prompt engineering techniques. The findings reveal a notable disparity in model effectiveness when transitioning from token to entity-level metrics, highlighting the importance of comprehensive evaluation methodologies in NER tasks. Furthermore, in comparison to BERT, ChatGPT remains limited when it comes to detecting advanced entities in French.
Download full-text PDF |
Source |
|---|---|
| http://dx.doi.org/10.3233/SHTI240502 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
Want AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!