TLab at #SMM4H 2024: Retrieval-Augmented Generation for ADE Extraction and Normalization
Published in Proceedings of the 9th Social Media Mining for Health Research and Applications (SMM4H 2024), 2024
SMM4H 2024 Task 1 is focused on the identification and standardization of Adverse Drug Events (ADEs) in tweets. We introduce a novel Retrieval-Augmented Generation (RAG) method, leveraging the capabilities of Llama 3, GPT-4, and the SFR-embedding-mistral model, along with few-shot prompting techniques, to map colloquial tweet language to MedDRA Preferred Terms (PTs) without relying on extensive training datasets. Our method achieved competitive performance, with an F1 score of 0.359 in the normalization task and 0.392 in the named entity recognition (NER) task. Notably, our model demonstrated robustness in identifying previously unseen MedDRA PTs (F1=0.363) greatly surpassing the median taskscore of 0.141 for such terms.
Recommended citation: Jacob Berkowitz, Apoorva Srinivasan, Jose Cortina, and Nicholas Tatonetti1. 2024. TLab at #SMM4H 2024: Retrieval-Augmented Generation for ADE Extraction and Normalization. In Proceedings of the 9th Social Media Mining for Health Research and Applications (SMM4H 2024) Workshop and Shared Tasks, pages 153–157, Bangkok, Thailand. Association for Computational Linguistics.
Read paper | Download paper