STORYWARS:面向协作故事理解与生成的指令调优数据集与基线
STORYWARS: A Dataset and Instruction Tuning Baselines for Collaborative Story Understanding and Generation
AI总结:
针对协作故事理解与生成缺乏语料的问题,提出包含4万余协作故事的STORYWARS数据集及101个多任务基准,并验证指令调优模型INSTRUCTSTORY在零样本、少样本和全监督场景均取得最优性能。
摘要:
协作故事是由多位写作风格和意图各异的作者共同创作而成的文本,给自然语言处理模型带来了独特的挑战。由于缺乏开放领域的语料库,理解和生成此类故事仍是一个研究不足的领域。为解决这一问题,我们引入了STORYWARS,这是一个包含来自在线平台的9,400位不同作者创作的超过40,000个协作故事的新数据集。我们在STORYWARS上设计了12种任务类型,包括7种理解任务和5种生成任务,共衍生出101个多样的故事相关任务,作为覆盖全监督、少样本和零样本场景的多任务基准。此外,我们提出了针对故事任务的指令调优模型INSTRUCTSTORY,结果表明,指令调优除了在零样本和少样本场景中取得优异结果外,还能在STORYWARS的全监督任务上获得最佳性能,从而在STORYWARS上建立了强大的多任务基准性能。
英文摘要:
Collaborative stories, which are texts created through the collaborative efforts of multiple authors with different writing styles and intentions, pose unique challenges for NLP models. Understanding and generating such stories remains an underexplored area due to the lack of open-domain corpora. To address this, we introduce STORYWARS, a new dataset of over 40,000 collaborative stories written by 9,400 different authors from an online platform. We design 12 task types, comprising 7 understanding and 5 generation task types, on STORYWARS, deriving 101 diverse story-related tasks in total as a multi-task benchmark covering all fully-supervised, few-shot, and zero-shot scenarios. Furthermore, we present our instruction-tuned model, INSTRUCTSTORY, for the story tasks showing that instruction tuning, in addition to achieving superior results in zero-shot and few-shot scenarios, can also obtain the best performance on the fully-supervised tasks in STORYWARS, establishing strong multi-task benchmark performances on STORYWARS.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。