Almog Gueta

Google, Technion
LLMs Accelerate Annotation for Medical Information Extraction


Almog Gueta is an MSc student in Data Science and Engineering at the Technion, under the supervision of Professor Roi Reichart. Her research focuses on Computational Social Science, exploring whether LLMs can learn Macroeconomic Narratives from Social Media. She is an intern in Google Research in medical NLP where she tackled real-world healthcare challenges. In addition to her studies, Almog is a lecturer for the Data Science courses at the Technion Continuing Education Unit.


The unstructured nature of clinical notes within electronic health records often conceals vital patient-related information, making it challenging to access or interpret. To uncover this hidden information, specialized Natural Language Processing (NLP) models are required. However, training these models necessitates large amounts of labeled data, a process that is both time-consuming and costly when relying solely on human experts for annotation. In this paper, we propose an approach that combines Large Language Models (LLMs) with human expertise to create an efficient method for generating ground truth labels for medical text annotation. By utilizing LLMs in conjunction with human annotators, we significantly reduce the human annotation burden, enabling the rapid creation of labeled datasets. We rigorously evaluate our method on a medical information extraction task, demonstrating that our approach not only substantially cuts down on human intervention but also maintains high accuracy. The results highlight the potential of using LLMs to improve the utilization of unstructured clinical data, allowing for the swift deployment of tailored NLP solutions in healthcare.

Published in ML4H 2023, the health symposium of NeurIPS.


8:45 Reception
9:30 Opening remarks by WiDS TLV ambassadors Noah Eyal Altman, Or Basson, and Nitzan Gado
9:45 Dr. Aya Soffer, IBM: "Putting Generative AI to Work: What Have We Learned So Far?"
10:15 Prof. Reut Tsarfaty, Bar-llan University: "Will Hebrew Speakers Be Able to Use Generative AI in Their Native Tongue?"
10:45 Poster Pitches
10:55 Break
11:10 Lightning talks
12:30 Lunch & poster session
13:30 Roundtable session & poster session
14:15 Roundtable closing
14:30 Break
14:40 Naomi Ken Korem, Lightricks: "Mastering the Art of Generative Models: Training and Controlling Text-to-Video Models"
15:00 Dr. Yael Mathov, Intuit: "Surviving the AI-pocalypse: Your Guide to LLM Security"
15:20 Closing remarks
15:30 The end