DEFINED: 辩论场景中细粒度创造力评估的数据高效计算框架
DEFINED: A Data-Efficient Computational Framework for Fine-Grained Creativity Assessment in Debate Scenarios
AI总结：
提出DEFINED框架，通过层次化八维指标体系、预训练语言模型和混合粒度训练策略，在辩论场景中实现数据高效的细粒度创造力自动评估，优于现有方法。
摘要：
人类创造力已成为大语言模型时代的关键能力。在复杂、开放环境中评估创造力是数据挖掘领域的一大挑战，目前受限于对标准化简单任务的依赖以及细粒度专家数据的稀缺。作为生态有效的评估场景，辩论反映了创造力的多个维度，涵盖发散思维和收敛思维。此外，辩论是一个数据丰富的领域，拥有大量公开可获取的材料。当前主流的自动评分方法难以适应辩论等复杂场景，因此仍然依赖昂贵的人工评估。为此，本文提出DEFINED，一种数据高效的计算框架，用于辩论场景中的细粒度创造力评估。DEFINED通过层次化的八维指标体系操作化辩论创造力，采用预训练自回归语言模型，并配备支持细粒度和粗粒度评估的层次化评分头。从真实辩论比赛中获取陈述及其相关专家评分，并采用约束数据增强策略以解决原始数据中的精英偏差。DEFINED采用混合粒度训练策略，能够从训练有素的研究生专家提供的有限细粒度监督中实现鲁棒学习。为严格验证超越合成基准的生态效度，我们纳入了一项针对辩论新手参与者的实证研究，利用这些真实数据作为中低水平人群的定性案例研究。在我们的评估协议中，评分模型实现了准确且稳定的评分，优于基于提示的大语言模型评估器和现有的辩论评分方法。
英文摘要：
Human creativity has emerged as a critical competency in the era of large language models. Assessing creativity in complex, open-ended environments is a grand challenge in data mining, currently hindered by a reliance on standardized simple tasks and the scarcity of fine-grained expert data. As an ecologically valid assessment context, debate reflects multiple dimensions of creativity, encompassing both divergent thinking and convergent thinking. Moreover, debate is a data-rich domain, with a large volume of publicly accessible materials. Current mainstream automated scoring methods are poorly suited to complex settings such as debate, and therefore still rely on costly human evaluation. To this end, this paper proposes DEFINED, a data-efficient computational framework for fine-grained creativity assessment in debate scenarios. DEFINED operationalizes debate creativity through a hierarchical eight-dimensional metric system, implemented via a pre-trained autoregressive language model with a hierarchical scoring head that supports both fine-grained and coarse-grained evaluation. Statements and their associated expert scores were obtained from authentic debate competitions, and a constrained data augmentation strategy was employed to address the elite bias inherent in the original data. DEFINED adopts a mixed-granularity training strategy enabling robust learning from limited fine-grained supervision annotated by trained graduate experts. To rigorously validate ecological validity beyond synthetic benchmarks, we incorporate an empirical study with debate-naive participants, utilizing these authentic data to serve as a qualitative case study for mid-to-low proficiency populations. Across our evaluation protocol, our scoring model achieves accurate and stable scoring, outperforming prompt-based large language model evaluators and existing debate scoring methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
