CML-Bench：评估与增强大语言模型电影剧本生成的框架
CML-Bench: A Framework for Evaluating and Enhancing LLM-Powered Movie Scripts Generation
AI总结：
针对大语言模型生成电影剧本缺乏叙事深度的问题，提出CML-Bench框架，含数据集、三维度评估指标及CML-Instruction提示策略，有效评估并提升剧本质量。
摘要：
大语言模型（LLMs）在生成高度结构化文本方面已展现出卓越的能力。然而，电影剧本虽然具有高度的结构组织性，却要求额外的细致叙事和情感深度——即引人入胜的电影的“灵魂”——而这往往是LLMs难以捕捉的。为了探究这一缺陷，我们首先构建了CML-Dataset，一个包含电影标记语言（CML）的（摘要，内容）对的数据集，其中“内容”由来自备受赞誉的高质量电影剧本的片段组成，“摘要”是对内容的简洁描述。通过对这些真实剧本中内在的多镜头连续性和叙事结构的深入分析，我们确定了质量评估的三个关键维度：对话连贯性（DC）、角色一致性（CC）和情节合理性（PR）。基于这些发现，我们提出了CML-Bench，它提供了跨这些维度的定量指标。CML-Bench能有效地为精心编写的人工剧本分配高分，同时指出LLMs生成的剧本中的弱点。为进一步验证我们的基准，我们引入了CML-Instruction，一种带有关于角色对话和事件逻辑的详细指令的提示策略，以引导LLMs生成更具结构性和电影感的剧本。大量实验验证了我们基准的有效性，并表明由CML-Instruction引导的LLMs能生成更高质量的剧本，其结果与人类偏好一致。
英文摘要：
Large Language Models (LLMs) have demonstrated remarkable proficiency in generating highly structured texts. However, while exhibiting a high degree of structural organization, movie scripts demand an additional layer of nuanced storytelling and emotional depth-the 'soul' of compelling cinema-that LLMs often fail to capture. To investigate this deficiency, we first curated CML-Dataset, a dataset comprising (summary, content) pairs for Cinematic Markup Language (CML), where 'content' consists of segments from esteemed, high-quality movie scripts and 'summary' is a concise description of the content. Through an in-depth analysis of the intrinsic multi-shot continuity and narrative structures within these authentic scripts, we identified three pivotal dimensions for quality assessment: Dialogue Coherence (DC), Character Consistency (CC), and Plot Reasonableness (PR). Informed by these findings, we propose the CML-Bench, featuring quantitative metrics across these dimensions. CML-Bench effectively assigns high scores to well-crafted, human-written scripts while concurrently pinpointing the weaknesses in screenplays generated by LLMs. To further validate our benchmark, we introduce CML-Instruction, a prompting strategy with detailed instructions on character dialogue and event logic, to guide LLMs to generate more structured and cinematically sound scripts. Extensive experiments validate the effectiveness of our benchmark and demonstrate that LLMs guided by CML-Instruction generate higher-quality screenplays, with results aligned with human preferences.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
