创造性心理测量项目生成器：一个使用大语言模型进行项目生成与验证的框架
The creative psychometric item generator: a framework for item generation and validation using large language models
AI总结：
本文提出CPIG框架，利用大语言模型迭代生成和评估创造性问题解决测试项目，实验证明其能产生有效可靠且不受已知评估偏差影响的创造力测试题。
摘要：
大语言模型（LLMs）正越来越多地被用于自动化需要高度创造力的工作流程。尽管已有大量先前工作考察了LLMs的创造力，但关于它们能否为人类生成有效的创造力评估工具的研究却很少，尽管创造力在现代经济中扮演着日益核心的角色。我们开发了一个受心理测量学启发的框架，用于为经典的自由回答式创造力测试——创造性问题解决（CPS）任务——创建测试项目（问题）。我们的框架，即创造性心理测量项目生成器（CPIG），混合使用基于LLM的项目生成器和评估器，迭代地开发用于编写CPS项目的新提示，使得后续迭代产生的项目能够从受测者那里引发更具创造性的回答。我们发现强有力的经验证据表明，CPIG能够生成有效且可靠的项目，并且这一效果不能归因于评估过程中已知的偏差。我们的发现对于使用LLMs自动为人类和AI生成有效且可靠的创造力测试具有启示意义。
英文摘要：
Increasingly, large language models (LLMs) are being used to automate workplace processes requiring a high degree of creativity. While much prior work has examined the creativity of LLMs, there has been little research on whether they can generate valid creativity assessments for humans despite the increasingly central role of creativity in modern economies. We develop a psychometrically inspired framework for creating test items (questions) for a classic free-response creativity test: the creative problem-solving (CPS) task. Our framework, the creative psychometric item generator (CPIG), uses a mixture of LLM-based item generators and evaluators to iteratively develop new prompts for writing CPS items, such that items from later iterations will elicit more creative responses from test takers. We find strong empirical evidence that CPIG generates valid and reliable items and that this effect is not attributable to known biases in the evaluation process. Our findings have implications for employing LLMs to automatically generate valid and reliable creativity tests for humans and AI.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
