arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.29890cs.HCcs.AI

基于访谈的生成代理在产品发现中的应用:一项验证研究

Interview-Informed Generative Agents for Product Discovery: A Validation Study

Adobe Research(Adobe研究院)

Zichao Wang, Alexa Siu

AI总结:

本文探讨了基于访谈的生成代理在概念测试场景中模拟用户反馈的有效性,发现代理在群体层面响应分布上准确但个体层面不够精准,为产品开发流程中合理整合模拟提供了启示。

摘要:

大型语言模型(LLMs)在标准化社会科学工具上表现出色,但其在产品发现中的价值尚不明确。本文研究了基于访谈的生成代理是否能模拟概念测试场景中的用户响应。通过深入的工作流程访谈与知识工作者,我们创建了个性化代理,并将它们对新颖AI概念的评估与相同参与者的真实响应进行比较。结果显示,代理在分布校准上表现良好,但在身份精确性上存在不足:它们无法复制所基于的特定个体,但能近似群体层面的响应分布。这些发现突显了LLM模拟在设计研究中的潜力与局限。虽然不适合作为个体层面洞察的替代,模拟可能在早期概念筛选和迭代中提供价值,其中分布准确性足够。我们讨论了将模拟负责任地整合到产品开发流程中的意义。

英文摘要:

Large language models (LLMs) have shown strong performance on standardized social science instruments, but their value for product discovery remains unclear. We investigate whether interview-informed generative agents can simulate user responses in concept testing scenarios. Using in-depth workflow interviews with knowledge workers, we created personalized agents and compared their evaluations of novel AI concepts against the same participants' responses. Our results show that agents are distribution-calibrated but identity-imprecise: they fail to replicate the specific individual they are grounded in, yet approximate population-level response distributions. These findings highlight both the potential and the limits of LLM simulation in design research. While unsuitable as a substitute for individual-level insights, simulation may provide value for early-stage concept screening and iteration, where distributional accuracy suffices. We discuss implications for integrating simulation responsibly into product development workflows.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑