LLM知识范式中的开放问题与一条假设性的前进路径
Open Problems and a Hypothetical Path Forward in LLM Knowledge Paradigms
AI总结：
本文针对LLM知识更新困难、反转诅咒和内部知识冲突三大开放问题，提出基于上下文知识扩展的假设性新范式，并探讨其实现路径，为下一代模型架构提供方向。
摘要：
知识是大语言模型（LLMs）整体能力的基础。模型的知识范式决定了它如何编码和利用知识，并显著影响其性能。尽管LLMs在现有知识范式下持续发展，但这些框架内的问题仍然制约着模型潜力。\n本文重点指出三个限制模型能力的关键开放问题：（1）LLMs知识更新的挑战，（2）反向知识泛化的失败（反转诅咒），以及（3）内部知识冲突。我们回顾了近期在解决这些问题方面取得的进展，并讨论了潜在的通用解决方案。基于这些领域的观察，我们提出了一种基于上下文知识扩展（Contextual Knowledge Scaling）的假设性范式，并进一步勾勒了在当代技术条件下仍具可行性的实现路径。证据表明，该方法有望解决当前缺陷，是我们对未来模型范式的愿景。\n本文旨在为研究人员提供LLM知识系统进展的简要概述，同时为下一代模型架构的发展提供启发。
英文摘要：
Knowledge is fundamental to the overall capabilities of Large Language Models (LLMs). The knowledge paradigm of a model, which dictates how it encodes and utilizes knowledge, significantly affects its performance. Despite the continuous development of LLMs under existing knowledge paradigms, issues within these frameworks continue to constrain model potential. This blog post highlight three critical open problems limiting model capabilities: (1) challenges in knowledge updating for LLMs, (2) the failure of reverse knowledge generalization (the reversal curse), and (3) conflicts in internal knowledge. We review recent progress made in addressing these issues and discuss potential general solutions. Based on observations in these areas, we propose a hypothetical paradigm based on Contextual Knowledge Scaling, and further outline implementation pathways that remain feasible within contemporary techniques. Evidence suggests this approach holds potential to address current shortcomings, serving as our vision for future model paradigms. This blog post aims to provide researchers with a brief overview of progress in LLM knowledge systems, while provide inspiration for the development of next-generation model architectures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
