发表机构
Wheeler Magnet High School; Kennesaw State University（惠勒磁石高中; 肯尼索州立大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究针对自动化创造力评估的资源密集问题，采用多编码器（Poly-Encoder）结合小型预训练BERT，在约1.8万份人工评分的科学创造性思维测试数据集上，达到与大语言模型相当的性能，显著降低计算需求，为教育等场景提供实用的创造力评估方案。
AI 中文摘要
自动化创造力评估长期以来一直是一项挑战，传统方法通常资源密集或缺乏实用准确性。我们提出一种新方法，使用多编码器（Poly-Encoder）实现计算高效且准确的自动化创造力评估。我们在来自科学创造性思维测试（Scientific Creative Thinking Test）的公共数据集上对多编码器进行微调，该数据集包含约18000份经人工评分的问题回答。我们的方法利用小型预训练BERT编码器，达到了与微调后的大语言模型（Large Language Models）相当的性能，同时显著降低了计算需求。对BERT家族模型和多代码计数（poly-code counts）的实验显示，与人工评分者的皮尔逊相关系数最高达r=0.74，95%置信区间为[0.73, 0.75]，匹配了资源密集型大语言模型的性能。本研究弥合了高性能与计算效率之间的差距，有望在可访问的消费级硬件上实现自动化创造力评估的广泛应用。尽管存在一些局限性，我们的发现表明，多编码器（Poly-Encoders）是大语言模型的有前景替代方案，适用于各种场景下实用、可扩展的创造力评估，尤其是教育领域。
英文摘要
Automated creativity assessment has been a long standing challenge, with traditional methods often being resource intensive or lacking practical accuracy. We introduce a novel approach by using Poly-Encoder for computationally efficient and accurate automated creativity assessment. We fine-tuned a Poly-Encoder on a public dataset from the Scientific Creative Thinking Test, comprised of approximately 18,000 human-rated question responses. Our method leverages small pre-trained BERT encoders, achieving performance comparable to fine-tuned Large Language Models while significantly reducing computational demands. Experiments with the BERT-family models and poly-code counts achieved Pearson correlations of up to r = 0.74, 95% CI [0.73, 0.75] with human raters, matching the performance of resource intensive LLMs. This study bridges the gap between high performance and computational efficiency, potentially enabling widespread implementation of automated creativity assessment on accessible consumer-grade hardware. With some limitations, our findings suggest that Poly-Encoders are a promising alternative to LLMs for practical, scalable creativity assessment in various contexts, especially educational.
CommentsAccepted at AIED 2026. The final authenticated version is available online at https://doi.org/10.1007/978-3-032-29755-6_32
