CollabStory：多LLM协作故事生成与作者分析
CollabStory: Multi-LLM Collaborative Story Generation and Authorship Analysis
AI总结：
本研究构建首个纯LLM生成的协作故事数据集CollabStory，探索多LLM协作创作场景，扩展PAN任务至该场景并提出基线，发现现有基线无法应对，为识别多LLM使用提供资源。
摘要：
使大型语言模型（LLMs）无缝互操作的统一框架兴起，让LLM之间协作完成开放式任务成为可能。尽管如此，尚未有研究探索此类协作写作。我们超越人机LLM协作，通过构建首个纯LLM生成的协作故事数据集CollabStory来探索多LLM场景。我们聚焦单作者到多作者（最多5个LLM）场景，即多个LLM共同创作故事。我们使用开源指令微调LLM生成了超过32000个故事。此外，我们从为人类协作写作任务和分析设定标准的PAN任务中汲取灵感，扩展了其与作者相关的任务以适应多LLM场景，并提出了LLM协作的基线。我们发现现有基线无法应对这一新兴场景。因此，CollabStory是一个有助于推动理解和开发新技术以识别多个LLM使用情况的资源。这在写作任务背景下至关重要，因为LLM协作可能会加剧抄袭检测、学分分配、教育环境中学术诚信维护以及版权侵权问题等现有挑战。我们的数据集和代码可在https://github.com/saranya-venkatraman/CollabStory获取。
英文摘要：
The rise of unifying frameworks that enable seamless interoperability of Large Language Models (LLMs) has made LLM-LLM collaboration for open-ended tasks a possibility. Despite this, there have not been efforts to explore such collaborative writing. We take the next step beyond human-LLM collaboration to explore this multi-LLM scenario by generating the first exclusively LLM-generated collaborative stories dataset called CollabStory. We focus on single-author to multi-author (up to 5 LLMs) scenarios, where multiple LLMs co-author stories. We generate over 32k stories using open-source instruction-tuned LLMs. Further, we take inspiration from the PAN tasks that have set the standard for human-human multi-author writing tasks and analysis. We extend their authorship-related tasks for multi-LLM settings and present baselines for LLM-LLM collaboration. We find that current baselines are not able to handle this emerging scenario. Thus, CollabStory is a resource that could help propel an understanding as well as the development of new techniques to discern the use of multiple LLMs. This is crucial to study in the context of writing tasks since LLM-LLM collaboration could potentially overwhelm ongoing challenges related to plagiarism detection, credit assignment, maintaining academic integrity in educational settings, and addressing copyright infringement concerns. We make our dataset and code available at https://github.com/saranya-venkatraman/CollabStory.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
