arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.01029cs.SEcs.AIcs.CL
收藏

修订还是重新解决?多LLM流水线中第二轮收益的分解

Revision or Re-Solving? Decomposing Second-Pass Gains in Multi-LLM Pipelines

School of Computer Science, Carnegie Mellon University(卡内基梅隆大学计算机科学学院)

Jingjie Ning, Xueqi Li, Chengyu Yu

AI总结:

本文通过控制实验分解多LLM流水线的第二轮收益,发现其收益取决于任务结构、草稿质量和信息类型,不同任务需不同策略。

摘要:

本文通过控制实验分解多LLM流水线的第二轮收益,发现其收益取决于任务结构、草稿质量和信息类型,不同任务需不同策略。

英文摘要:

Multi-LLM revision pipelines, in which a second model reviews and improves a draft produced by a first, are widely assumed to derive their gains from genuine error correction. We question this assumption with a controlled decomposition experiment that uses four matched conditions to separate second-pass gains into three additive components: re-solving, scaffold, and content. We evaluate this design across two model pairs on three benchmarks spanning knowledge-intensive MCQ and competitive programming. Our results show that the gains of multi-LLM revision are not monolithic, but depend on task structure, draft quality, and the type of draft information. On MCQ tasks, where the answer space is constrained and drafts provide little structural guidance, most gains are consistent with stronger-model re-solving, and directly routing queries to the stronger model can be more effective than revising a weak draft. On code generation tasks, however, two-stage prompting remains useful because even semantically null drafts can provide substantial structural scaffolding, while weak draft content can be harmful. Finally, role-reversed experiments show that strong drafts clearly benefit weak reviewers. Ultimately, our findings demonstrate that the utility of multi-LLM revision is dynamically bottlenecked by task structure and draft quality, necessitating more targeted pipeline designs rather than blanket revision strategies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑