Vis-CoT:一种用于LLM思维链推理中交互式可视化和干预的人机协同框架
Vis-CoT: A Human-in-the-Loop Framework for Interactive Visualization and Intervention in LLM Chain-of-Thought Reasoning
AI总结:
提出Vis-CoT框架,将线性思维链文本转换为交互式推理图,允许用户可视化逻辑流程、识别错误步骤并通过剪枝和嫁接进行干预,显著提升推理准确性和用户信任。
摘要:
大型语言模型(LLM)通过思维链(CoT)提示展现出强大的推理能力,但该过程不透明,使得在高风险场景中的验证、调试和控制变得困难。我们提出Vis-CoT,一种人机协同框架,将线性CoT文本转换为交互式推理图。用户可以可视化逻辑流程,识别有缺陷的步骤,并通过剪枝错误路径和嫁接用户定义的新前提进行干预。这将交互从被动观察转变为主动协作,引导模型得出更准确和可信的结论。在GSM8K和StrategyQA上,Vis-CoT相比非交互基线将最终答案准确率提高了最多24个百分点。一项用户研究还显示,在感知可用性和信任度方面有显著提升。Vis-CoT通过将LLM与针对性的人类监督相结合,为更可靠、可理解和协作的推理指明了一条实用路径。
英文摘要:
Large language models (LLMs) show strong reasoning via chain-of-thought (CoT) prompting, but the process is opaque, which makes verification, debugging, and control difficult in high-stakes settings. We present Vis-CoT, a human-in-the-loop framework that converts linear CoT text into an interactive reasoning graph. Users can visualize the logical flow, identify flawed steps, and intervene by pruning incorrect paths and grafting new, user-defined premises. This shifts interaction from passive observation to active collaboration, steering models toward more accurate and trustworthy conclusions. Across GSM8K and StrategyQA, Vis-CoT improves final-answer accuracy by up to 24 percentage points over non-interactive baselines. A user study also shows large gains in perceived usability and trust. Vis-CoT points to a practical path for more reliable, understandable, and collaborative reasoning by combining LLMs with targeted human oversight.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。