文本恐怖谷：LLM信息检索中的非单调性能退化
The Text Uncanny Valley: Non-Monotonic Performance Degradation in LLM Information Retrieval
AI总结：
研究探讨了文本碎片化对LLM信息检索性能的影响，发现性能呈现U型曲线，提出模式过渡假说解释LLM在不同文本状态下的工作模式变化。
摘要：
现有大语言模型（LLM）基准主要关注语法正确输入，缺乏对不完美文本的评估。本文研究了单词边界破坏对LLM检测目标信息的影响。通过在单词内插入空格字符将其分裂成碎片，LLM的检测准确性随插入率增加呈现U型曲线。我们将其称为文本恐怖谷。为解释此现象，我们提出模式过渡假说：LLM在近正常文本中处于单词级模式，在严重碎片化文本中处于字符级模式，恐怖谷标志着两者过渡时的无序状态。四项实验和一次分析支持此观点：上下文学习无法挽救谷底性能；正则化扰动显著降低U型曲线；数学推理任务在Gemini 3.0 Flash上复制U型曲线但对更强模型无效，表明当任务依赖较少精确词法对齐时效果减弱；并且tokenization熵在F1最小值前达到峰值，与制度冲突解释一致。这些发现揭示了一种在干净文本基准中不可见但直接相关于任何涉及嘈杂或未整理文本输入部署场景的失败模式。
英文摘要：
Existing Large Language Model (LLM) benchmarks primarily focus on syntactically correct inputs, leaving a significant gap in evaluation on imperfect text. In this work, we study how word-boundary corruption affects how LLMs detect targeted information. By inserting whitespace characters within words to break them into fragments, LLMs' detection accuracy follows a U-shaped curve with the increase in insertion rate. We refer to this curve as the Text Uncanny Valley. To explain such observation, we propose a mode transition hypothesis: LLMs operate in a word-level mode for near-normal text and a character-level mode for heavily fragmented text, with the valley marking the disordered transition where neither mode is effective. Four experiments and one analysis are consistent with this account: in-context learning fails to rescue valley-bottom performance; regularizing the perturbation substantially reduces the U-shape; a math reasoning task replicates the U-shape for Gemini 3.0 Flash but not for stronger models, suggesting the effect is attenuated when tasks rely less on exact lexical alignment; and tokenization entropy peaks before the F1 minimum, consistent with a regime-conflict interpretation. These findings reveal a failure mode invisible to clean-text benchmarks yet directly relevant to any deployment scenario involving noisy or uncurated text inputs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
