幻象的精通：LLMs将记忆技巧误认为是人工提升的自我认知
Mirage of Mastery: Memorization Tricks LLMs into Artificially Inflated Self-Knowledge
AI总结：
本文研究LLMs将记忆误认为智能的问题，揭示其自我认知的不一致性和缺陷，提出需改进模型自我认知的平衡与一致性以提升AI可信度。
摘要：
当人工智能将记忆误认为是智能时，就会产生危险的推理幻象。现有研究将记忆和自我认知缺陷视为独立问题，并未认识到它们之间交织的联系，这种联系会降低LLM响应的可信度。在本研究中，我们利用一种新的框架来确定LLMs是否真正从训练数据中学习推理模式，还是仅仅记忆了这些模式，以在类似复杂度的STEM领域问题中假扮具备能力。我们的分析显示了一个值得注意的泛化问题：LLMs从记忆中的解决方案中获得信心，从而推断出对自己推理能力的更高自我认知，这在面对经过自我验证的逻辑一致的任务扰动时，会表现出超过45%的一致性不一致。这种效应在科学和医学领域最为明显，这些领域通常具有最大的标准化术语和问题，进一步验证了我们的方法。LLMs自我认知中的显著波动也显示了当前架构和训练模式的缺陷，突显了需要确保模型对自身知识的感知保持平衡和一致性的技术需求，以实现最大化的AI可解释性和可信度。我们的代码和结果可在https://github.com/Sahil-R-Kale/mirage_of_mastery公开获取。
英文摘要：
When artificial intelligence mistakes memorization for intelligence, it creates a dangerous mirage of reasoning. Existing studies treat memorization and self-knowledge deficits in LLMs as separate issues and do not recognize an intertwining link that degrades the trustworthiness of LLM responses. In our study, we utilize a novel framework to ascertain if LLMs genuinely learn reasoning patterns from training data or merely memorize them to assume competence across problems of similar complexity focused on STEM domains. Our analysis shows a noteworthy problem in generalization: LLMs draw confidence from memorized solutions to infer a higher self-knowledge about their reasoning ability, which manifests as an over 45% inconsistency in feasibility assessments when faced with self-validated, logically coherent task perturbations. This effect is most pronounced in science and medicine domains, which tend to have maximal standardized jargon and problems, further confirming our approach. Significant wavering within the self-knowledge of LLMs also shows flaws in current architectures and training patterns, highlighting the need for techniques that ensure a balanced, consistent stance on models' perceptions of their own knowledge for maximum AI explainability and trustworthiness. Our code and results are available publicly at https://github.com/Sahil-R-Kale/mirage_of_mastery
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
