Dr. Mor Geva

What computations underlie the behavior of large language models (LLMs) like GPT-4 and Gemini? This talk will introduce mechanistic interpretability and discuss recent advances in mapping functional components within a model. We will begin with the challenge of automatically describing simple units in LLMs, such as neurons. Current interpretability pipelines explain a neuron primarily by analyzing the inputs that activate it while overlooking its downstream effect on model outputs. We will show how augmenting these pipelines with output-based descriptions leads to substantially more faithful and useful interpretations. Next, we will move beyond neurons to examine attention heads — one of the building blocks of modern LLMs. We will present MAPS, an efficient method for inferring the functionality of attention heads directly from their parameters. MAPS enables comprehensive mapping of predefined operations across attention heads, revealing both cross-model universality patterns and architecture-specific biases. Lastly, we will show how LLMs can automatically interpret MAPS’ estimations, identifying the salient operations of most attention heads within a model.