arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.08003cs.CLcs.AIcs.MA

LLM Review: 通过盲审同行反馈增强创造性写作

LLM Review: Enhancing Creative Writing via Blind Peer Review Feedback

Harvard University(哈佛大学);Carnegie Mellon University(卡内基梅隆大学);Stanford University(斯坦福大学)

Weiyue Li, Mingxiao Song, Zhenda Shen, Dachuan Zhao, Yunfan Long, Yi Li, Yongce Li, Ruyi Yang, Mengyu Wang

AI总结:

LLM Review通过盲审同行反馈机制提升创造性写作,实验显示其优于多智能体基线,并使小模型超越大模型。

摘要:

大型语言模型(LLMs)在创造性生成方面常常遇到困难,而通过交互改进推理的多智能体框架反而可能通过诱导内容同质化来抑制创造力。我们引入LLM Review,一种受同行评审启发的框架,实施盲审同行评审:代理在独立修订过程中交换针对性反馈,以保持多样化的创造性路径。为了实现严格评估,我们提出SciFi-100,一个包含科学幻想写作数据集的统一框架,结合LLM-as-a-judge评分、人工标注和基于规则的新颖性度量。实验表明,LLM Review在多智能体基线中表现一致,且较小的模型在使用我们的框架后可以超越更大的单智能体模型,这表明交互结构可能取代模型规模。

英文摘要:

Large Language Models (LLMs) often struggle with creative generation, and multi-agent frameworks that improve reasoning through interaction can paradoxically hinder creativity by inducing content homogenization. We introduce LLM Review, a peer-review-inspired framework implementing Blind Peer Review: agents exchange targeted feedback while revising independently, preserving divergent creative trajectories. To enable rigorous evaluation, we propose SciFi-100, a science fiction writing dataset with a unified framework combining LLM-as-a-judge scoring, human annotation, and rule-based novelty metrics. Experiments demonstrate that LLM Review consistently outperforms multi-agent baselines, and smaller models with our framework can surpass larger single-agent models, suggesting interaction structure may substitute for model scale.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑