CCSBench：评估大语言模型在科学文档摘要中的组合可控性
CCSBench: Evaluating Compositional Controllability in LLMs for Scientific Document Summarization
AI总结：
CCSBench是首个科学文档组合可控摘要基准，评估LLMs同时控制长度与实证焦点等多属性的能力，发现其在平衡隐性属性上存在显著局限。
摘要：
为了向不同受众广泛传播科学知识，科学文档摘要系统需要同时控制长度和实证焦点等多个属性。然而，现有研究通常侧重于控制单一属性，对多属性组合控制的研究尚不充分。为弥补这一空白，我们提出了CCSBench，这是科学领域首个用于组合可控摘要的评估基准。该基准能够对显性属性（如长度）和隐性属性（如概念或实证焦点）进行细粒度控制，其中显性属性客观且直接，而隐性属性更为主观和抽象。我们使用多种大语言模型（LLMs）在各种设置下进行了广泛实验，包括上下文学习、参数高效微调以及用于平衡不同属性控制的两阶段模块化方法。我们的研究结果揭示了LLMs在平衡控制属性之间的权衡方面存在显著局限性，尤其是那些需要更深层次理解和抽象推理的隐性属性。
英文摘要：
To broaden the dissemination of scientific knowledge to diverse audiences, it is desirable for scientific document summarization systems to simultaneously control multiple attributes such as length and empirical focus. However, existing research typically focuses on controlling single attributes, leaving the compositional control of multiple attributes underexplored. To address this gap, we introduce CCSBench, the first evaluation benchmark for compositional controllable summarization in the scientific domain. Our benchmark enables fine-grained control over both explicit attributes (e.g., length), which are objective and straightforward, and implicit attributes (e.g., conceptual or empirical focus), which are more subjective and abstract. We conduct extensive experiments using various large language models (LLMs) under various settings, including in-context learning, parameter-efficient fine-tuning, and two-stage modular methods for balancing control over different attributes. Our findings reveal significant limitations in LLMs capabilities in balancing trade-offs between control attributes, especially implicit ones that require deeper understanding and abstract reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
