MuseScorer：大规模创意原创性评分
MuseScorer: Idea Originality Scoring At Scale
AI总结：
MuseScorer通过大语言模型与检索结合自动构建创意桶，实现无需人工标注的基于频率的原创性评分，在多个数据集上与人类评分高度一致。
摘要：
一种客观且表面有效的创意原创性评分方法是衡量每个创意在群体中的统计罕见性——这一方法在创造力研究中长期使用。然而，计算这些频率需要人工对创意的不同表述进行分桶，这一过程主观、劳动密集、易出错且在大规模下脆弱。我们提出MuseScorer，一个全自动、经过心理测量学验证的基于频率的原创性评分系统。MuseScorer将大语言模型（LLM）与外部编排的检索相结合：给定一个新创意，它检索语义相似的已有创意桶，并以零样本方式提示LLM判断该创意是适合已有桶还是形成新桶。这些桶使得无需人工标注即可进行基于频率的原创性评分。在五个数据集（N_participants=1143，n_ideas=16,294）上，MuseScorer在创意聚类结构（AMI = 0.59）和参与者层面评分（r = 0.89）上与人类标注者相匹配，同时展现出较强的聚合效度和外部效度。该系统为创造力研究提供了可扩展、意图敏感且与人类一致的原创性评估。
英文摘要：
An objective, face-valid method for scoring idea originality is to measure each idea's statistical infrequency within a population -- an approach long used in creativity research. Yet, computing these frequencies requires manually bucketing idea rephrasings, a process that is subjective, labor-intensive, error-prone, and brittle at scale. We introduce MuseScorer, a fully automated, psychometrically validated system for frequency-based originality scoring. MuseScorer integrates a Large Language Model (LLM) with externally orchestrated retrieval: given a new idea, it retrieves semantically similar prior idea-buckets and zero-shot prompts the LLM to judge whether the idea fits an existing bucket or forms a new one. These buckets enable frequency-based originality scoring without human annotation. Across five datasets N_{participants}=1143, n_{ideas}=16,294), MuseScorer matches human annotators in idea clustering structure (AMI = 0.59) and participant-level scoring (r = 0.89), while demonstrating strong convergent and external validity. The system enables scalable, intent-sensitive, and human-aligned originality assessment for creativity research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
