发表机构
The George Washington University; Johns Hopkins University（乔治·华盛顿大学; 约翰·霍普金斯大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究针对开放世界中因条件冲突等导致的不适定问题，提出FlowEdit框架利用信息论原理控制大语言模型推理流，能生成多样替代响应，实验证明其优于专有模型，提升了准确率和响应信息性。
AI 中文摘要
大语言模型在具有可行答案的明确推理任务上表现出色。然而，开放世界中遇到的问题可能因条件不一致、陈述冲突或要求相互不兼容而变得不适定，没有有效答案。我们认为，此类涉及冲突的不适定问题的推理需要新颖的大语言模型能力，使隐藏冲突显式化，通过多个推理分支维持相互竞争的假设，并一次性生成替代响应，而由于大语言模型中下一个token预测机制的限制，这些都具有挑战性。为此，我们提出了FlowEdit，这是一个利用信息论原理来量化和调节大语言模型内部推理流的新颖框架，用于在有效假设下生成一整套替代响应。FlowEdit可被视为在模型的内部推理表示上使用两个对偶信息论目标来强制执行分支感知推理过程：最大化从每个选定假设到分支结果的信息流，同时最小化兄弟分支之间的重叠和条件依赖性，以提供具有广泛覆盖范围的多样、信息丰富的响应集。我们表明，这是通过在边界嵌入为{\epsilon}-充分的情况下的可处理变分界来实现的，优化大语言模型推理过程中的潜在条件互信息。大量实验表明，FlowEdit优于领先的专有模型，将精确集匹配准确率提高了68%，同时将整体响应信息性提高了24%。我们进一步表明，流调节在token流中表现为下一个token熵的重新分布，该分布集中在每个分支内部，在流边界处放大，并随问题所需的流数量而缩放。
英文摘要
Large Language Models (LLMs) perform strongly on well-specified reasoning tasks with a feasible answer. However, problems encountered in the open world can become ill-posed due to inconsistent conditions, conflicting statements, or mutually incompatible requirements, admitting no valid responses. We argue that reasoning of such ill-posed problems involving conflicts require novel LLM capabilities to make hidden conflicts explicit, maintain competing hypotheses via multiple reasoning branches, and generate alternative responses in a single pass, all of which are challenging due to the limitation of the next-token prediction mechanism in LLMs. To this end, we propose FlowEdit, a novel framework that leverages information-theoretic principles to quantify and regulate internal reasoning flows of LLMs, for generating a full set of alternative responses under valid hypotheses. FlowEdit can be viewed as enforcing a branch-aware reasoning process using two dual information-theoretic objectives on the model's internal reasoning representations: maximizing the information flow from each selected hypothesis to the branch outcome, while minimizing the overlap and conditional dependence across sibling branches, to provide a diverse, informative set of responses with broad coverage. We show that this is achieved through tractable variational bounds under boundary embeddings being ε-sufficient, optimizing the underlying conditional mutual information in LLM reasoning process. Extensive experiments demonstrate that FlowEdit outperforms leading proprietary models, improving exact-set-match accuracy by 68%, while boosting overall response informativeness by 24%. We further show that flow regulation surfaces in the token stream as a redistribution of next-token entropy that concentrates inside each branch, amplifies at flow boundaries, and scales with the number of flows the problem requires.
