写作作为开放式智能体的测试平台
Writing as a testbed for open ended agents
AI总结：
本文以写作为测试平台，研究LLMs作为自主协作共同作者的潜力，分析三个模型的动作多样性、人类对齐和迭代改进能力，并建立基准框架，以应对开放式任务的挑战。
摘要：
开放式任务对大型语言模型（LLMs）而言尤为具有挑战性，因为其解决方案空间极为广阔，要求既进行广泛探索又采用适应性策略，尤其是在成功缺乏明确客观定义的情况下。写作因其庞大的解决方案空间和主观的评价标准，为研究此类问题提供了一个引人注目的测试平台。在本文中，我们探讨了LLMs作为协作式共同作者的潜力，即能够自主建议并实施文本改进。我们分析了三个著名的LLMs——Gemini 1.5 Pro、Claude 3.5 Sonnet和GPT-4o——重点关注它们的动作多样性、人类对齐以及迭代改进能力如何影响整体性能。这项工作建立了一个用于基准测试自主写作智能体的框架，并更广泛地强调了在多样化开放式领域中构建卓越系统所面临的基本挑战和潜在解决方案。
英文摘要：
Open-ended tasks are particularly challenging for LLMs due to the vast solution space, demanding both expansive exploration and adaptable strategies, especially when success lacks a clear, objective definition. Writing, with its vast solution space and subjective evaluation criteria, provides a compelling testbed for studying such problems. In this paper, we investigate the potential of LLMs to act as collaborative co-writers, capable of suggesting and implementing text improvements autonomously. We analyse three prominent LLMs - Gemini 1.5 Pro, Claude 3.5 Sonnet, and GPT-4o - focusing on how their action diversity, human alignment, and iterative improvement capabilities impact overall performance. This work establishes a framework for benchmarking autonomous writing agents and, more broadly, highlights fundamental challenges and potential solutions for building systems capable of excelling in diverse open-ended domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
