Hail Hochman

Large Language Models (LLMs) encode vast amounts of factual knowledge, much of which is linked to real-world entities. While previous works have explored the mechanisms behind entity representation, open questions remain about their extent and consistency. This research investigates whether LLMs maintain a shared representation for different descriptions of the same entity, such as “Barack Obama” and “the 44th president of the United States.” Specifically, we examine whether knowledge is retrieved from the same source or transferred through a shared mechanism when answering questions about an entity and its description. Understanding whether and how LLMs internally link different representations of the same entity is crucial for improving interpretability, trust, and targeted model editing.

Our research uses datasets of paired entity descriptions, ensuring that model responses remain consistent across naming variations. We employ interpretability techniques, including vocabulary projection and sparse autoencoders to examine the internal retrieval mechanisms of LLMs. By analyzing activations across transformer layers, we determine whether knowledge about an entity is retrieved from the same parameters when queried through different descriptions. Additionally, we explore whether entity attributes are encoded in a shared manner, independent of the specific phrasing used to reference them.
Our findings indicate that different descriptions of the same entity tend to share representation structures, exhibit consistent knowledge recall patterns, and influence each other in knowledge editing tasks.

These findings provide evidence that entity representations in LLMs exhibit some degree of consistency across different descriptions. The mutual influence of knowledge editing between descriptions of the same entity suggests the presence of a shared component in the model’s fact-retrieval mechanism. These insights help improve trust in LLMs by revealing how real-world entities are internally represented.