词形很重要：LLMs在Typoglycemia下的语义重构
Word Form Matters: LLMs' Semantic Reconstruction under Typoglycemia
AI总结：
本文研究LLMs在Typoglycemia现象下的语义重构机制，提出SemRecScore度量标准，发现词形是核心因素，LLMs依赖专门的注意力头处理词形信息，与人类自适应策略不同，为增强LLM性能提供见解。
摘要：
人类读者能够高效理解打乱的单词，这种现象被称为Typoglycemia，主要依赖于词形；如果仅靠词形不足以理解，他们会进一步利用上下文线索进行解释。尽管先进的大型语言模型（LLMs）展现出类似能力，但其潜在机制仍不清楚。为了调查这一点，我们进行了受控实验，以分析词形和上下文信息在语义重构中的作用，并检查LLM的注意力模式。具体而言，我们首先提出了SemRecScore，一种可靠度量标准来量化语义重构的程度，并验证了其有效性。使用该度量标准，我们研究了词形和上下文信息如何影响LLMs的语义重构能力，发现词形是这一过程的核心因素。此外，我们分析了LLMs如何利用词形，发现它们依赖专门的注意力头来提取和处理词形信息，这一机制在不同程度的单词打乱下保持稳定。LLMs专注于词形的固定注意力模式与人类读者在平衡词形和上下文信息时的自适应策略之间的差异，为通过引入类人、上下文感知机制来增强LLM性能提供了见解。
英文摘要：
Human readers can efficiently comprehend scrambled words, a phenomenon known as Typoglycemia, primarily by relying on word form; if word form alone is insufficient, they further utilize contextual cues for interpretation. While advanced large language models (LLMs) exhibit similar abilities, the underlying mechanisms remain unclear. To investigate this, we conduct controlled experiments to analyze the roles of word form and contextual information in semantic reconstruction and examine LLM attention patterns. Specifically, we first propose SemRecScore, a reliable metric to quantify the degree of semantic reconstruction, and validate its effectiveness. Using this metric, we study how word form and contextual information influence LLMs' semantic reconstruction ability, identifying word form as the core factor in this process. Furthermore, we analyze how LLMs utilize word form and find that they rely on specialized attention heads to extract and process word form information, with this mechanism remaining stable across varying levels of word scrambling. This distinction between LLMs' fixed attention patterns primarily focused on word form and human readers' adaptive strategy in balancing word form and contextual information provides insights into enhancing LLM performance by incorporating human-like, context-aware mechanisms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
