机器蛇会像电子羊一样做梦吗？探究架构归纳偏置对幻觉的影响
Do Robot Snakes Dream like Electric Sheep? Investigating the Effects of Architectural Inductive Biases on Hallucination
AI总结：
本研究通过广泛评估，探究了架构归纳偏置（如循环模型与传统自注意力模型）对大型语言模型幻觉倾向的影响，发现幻觉虽普遍但发生情境与诱导难度因架构而异，强调需结合理解并设计通用处理技术。
摘要：
大型语言模型（LLM）在日常生活中的日益突出，很大程度上归因于其生成能力，但其中一部分也归因于与其使用相关的风险和成本。一方面，它们倾向于产生虚假或误导性的信息，即幻觉，从而限制了其可靠性。另一方面，人们越来越关注基于传统自注意力机制的LLM所面临的计算限制，这催生了新的替代方案，特别是循环模型，旨在克服这些限制。然而，同时考虑这两个问题的情况仍然罕见。架构的改变是否会加剧或缓解关于幻觉的现有担忧？它们是否会影响幻觉发生的方式和位置？通过广泛的评估，我们研究了这些基于架构的归纳偏置如何影响产生幻觉的倾向。虽然幻觉仍然是一个不限于特定架构的普遍现象，但幻觉发生的情境以及诱导特定类型幻觉的难易程度，可能因模型架构的不同而有显著差异。这些发现强调，需要更好地结合理解这两个问题，并考虑如何设计更通用的技术来处理幻觉。
英文摘要：
The growth in prominence of large language models (LLMs) in everyday life can be largely attributed to their generative abilities, yet some of this is also owed to the risks and costs associated with their use. On one front is their tendency to hallucinate false or misleading information, limiting their reliability. On another is the increasing focus on the computational limitations associated with traditional self-attention based LLMs, which has brought about new alternatives, in particular recurrent models, meant to overcome them. Yet it remains uncommon to consider these two concerns simultaneously. Do changes in architecture exacerbate/alleviate existing concerns about hallucinations? Do they affect how and where they occur? Through an extensive evaluation, we study how these architecture-based inductive biases affect the propensity to hallucinate. While hallucination remains a general phenomenon not limited to specific architectures, the situations in which they occur and the ease with which specific types of hallucinations can be induced can significantly differ based on the model architecture. These findings highlight the need for better understanding both these problems in conjunction with each other, as well as consider how to design more universal techniques for handling hallucinations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
