使用大型语言模型创建AI人格以复制、泛化和预测媒体效应：对133项已发表实验研究结果的实证检验
Using Large Language Models to Create AI Personas for Replication, Generalization and Prediction of Media Effects: An Empirical Test of 133 Published Experimental Research Findings
AI总结：
本研究通过复制133项已发表实验，检验了LLM驱动的AI人格在媒体效应研究中的复制能力，主效应复制率达76%，并探讨了其在解决复制危机和加速理论构建方面的潜力。
摘要：
本报告分析了大型语言模型（LLMs）在加速准确复制和泛化已发表的关于营销中信息效应的研究方面的潜力。通过复制《营销杂志》上发表的14篇论文中包含的45项近期研究中的133个实验发现，对由LLM驱动的参与者（人格）进行了测试。对于每项研究，使用测量指标、刺激物和抽样规范来生成提示，使LLM扮演独特的人格。所有研究中总计19,447个AI人格生成了完整的数据集，随后将统计分析结果与原始人类研究结果进行比较。LLM复制成功重现了76%的原始主效应（111个中的84个），显示出AI辅助复制的强大潜力。包括交互效应在内的总体复制率为68%（133个中的90个）。此外，对人类结果如何泛化到不同参与者样本、媒体刺激和测量指标的测试表明，当测试超出原始人类研究的参数范围时，复制结果可能会发生变化。本文讨论了这些发现对社会科学中复制和泛化危机、媒体和营销心理学中理论构建的加速，以及快速信息测试对消费品的实际优势所产生的影响。还针对复杂的交互效应、AI模型中的偏差以及为营销研究中的AI指标建立基准等问题，讨论了AI复制的局限性。
英文摘要：
This report analyzes the potential for large language models (LLMs) to expedite accurate replication and generalization of published research about message effects in marketing. LLM-powered participants (personas) were tested by replicating 133 experimental findings from 14 papers containing 45 recent studies published in the Journal of Marketing. For each study, the measures, stimuli, and sampling specifications were used to generate prompts for LLMs to act as unique personas. The AI personas, 19,447 in total across all of the studies, generated complete datasets and statistical analyses were then compared with the original human study results. The LLM replications successfully reproduced 76% of the original main effects (84 out of 111), demonstrating strong potential for AI-assisted replication. The overall replication rate including interaction effects was 68% (90 out of 133). Furthermore, a test of how human results generalized to different participant samples, media stimuli, and measures showed that replication results can change when tests go beyond the parameters of the original human studies. Implications are discussed for the replication and generalizability crises in social science, the acceleration of theory building in media and marketing psychology, and the practical advantages of rapid message testing for consumer products. Limitations of AI replications are addressed with respect to complex interaction effects, biases in AI models, and establishing benchmarks for AI metrics in marketing research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
