什么让一个查询良好？测量人类困惑语言特征对LLM性能的影响
What Makes a Good Query? Measuring the Impact of Human-Confusing Linguistic Features on LLM Performance
AI总结：
研究通过分析查询特征对LLM幻觉的影响，揭示了特定语言特征与幻觉风险的关系，为优化查询设计提供依据。
摘要：
大型语言模型（LLM）的幻觉通常被视为模型或其解码策略的缺陷。借鉴经典语言学，我们主张查询的结构也会影响听者（和模型）的回应。我们通过构建一个包含22个维度的查询特征向量来操作这一见解，涵盖从句复杂性、词汇稀有性、指代、否定、可回答性和意图 grounding 等已知影响人类理解的因素。使用369,837个真实世界查询，我们提问：是否存在某些类型的查询更容易导致幻觉？大规模分析揭示了一种一致的“风险景观”：某些特征如深层从句嵌套和不明确性与更高的幻觉倾向相关。相反，清晰的意图 grounding 和可回答性与较低的幻觉率相关。其他因素，包括领域特异性，显示出混合的、数据集和模型依赖的效果。因此，这些发现建立了一种与幻觉风险相关的可观察查询-特征表示，为引导查询重写和未来干预研究铺平了道路。
英文摘要：
Large Language Model (LLM) hallucinations are usually treated as defects of the model or its decoding strategy. Drawing on classical linguistics, we argue that a query's form can also shape a listener's (and model's) response. We operationalize this insight by constructing a 22-dimension query feature vector covering clause complexity, lexical rarity, and anaphora, negation, answerability, and intention grounding, all known to affect human comprehension. Using 369,837 real-world queries, we ask: Are there certain types of queries that make hallucination more likely? A large-scale analysis reveals a consistent "risk landscape": certain features such as deep clause nesting and underspecification align with higher hallucination propensity. In contrast, clear intention grounding and answerability align with lower hallucination rates. Others, including domain specificity, show mixed, dataset- and model-dependent effects. Thus, these findings establish an empirically observable query-feature representation correlated with hallucination risk, paving the way for guided query rewriting and future intervention studies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
