Publikation
Evaluating the Robustness of Adverse Drug Event Classification Models using Templates
Dorothea MacPhail; David Harbecke; Lisa Raithel; Sebastian Möller
In: Dina Demner-Fushman; Sophia Ananiadou; Makoto Miwa; Kirk Roberts; Junichi Tsujii (Hrsg.). Proceedings of the 23rd Workshop on Biomedical Natural Language Processing. Workshop on Current Trends in Biomedical Natural Language Processing (BioNLP-2024), located at ACL 2024, Pages 25-38, Association for Computational Linguistics, 8/2024.
Zusammenfassung
An adverse drug effect (ADE) is any harmful event resulting from medical drug treatment. Despite their importance, ADEs are often under-reported in official channels. Some research has therefore turned to detecting discussions of ADEs in social media. Impressive results have been achieved in various attempts to detect ADEs. In a high-stakes domain such as medicine, however, an in-depth evaluation of a model's abilities is crucial. We address the issue of thorough performance evaluation in English-language ADE detection with hand-crafted templates for four capabilities: Temporal order, negation, sentiment, and beneficial effect. We find that models with similar performance on held-out test sets have varying results on these capabilities.
Projekte
- KEEPHA - Mehrsprachige wissensverbesserte Informationsextraktion für die Pharmakovigilanz
- TRAILS - Vertrauenswürdige und integrative Maschinen