arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.16157cs.HC
收藏

提前窥视田野研究:探索VLM人设作为人机交互领域具身研究的支持工具

Peeking Ahead of the Field Study: Exploring VLM Personas as Support Tools for Embodied Studies in HCI

The University of Tokyo(东京大学);UCL Interaction Centre(伦敦大学学院交互中心);Google(谷歌);Kyoto University(京都大学)

Xinyue Gui, Ding Xia, Mark Colley, Yuan Li, Vishal Chauhan, Anubhav Anubhav, Zhongyi Zhou, Ehsan Javanmardi, Stela Hanbyeol Seo, Chia-Ming Chang, Manabu Tsukada, Takeo Igarashi

AI总结:

本文提出利用VLM人设模拟田野研究结果,验证其在具身研究中的应用潜力,发现其在响应模式上接近人类但缺乏多样性。

摘要:

田野研究是不可替代的,但成本高、耗时且容易出错,需要精心准备。受制造业快速原型设计的启发,我们提出了一种快速、低成本的评估方法,利用视觉语言模型(VLM)人设来模拟与田野研究结果相当的成果。尽管LLM展示了类人推理和语言能力,但自动驾驶(AV)与行人交互需要空间意识、情感共情和行为生成。这引发了我们的研究问题:VLM人设能有多大程度地模仿田野研究中的人类响应?我们进行了平行研究:1)一项有20名参与者的真实世界研究;2)一项使用20个VLM人设的视频研究,两者均针对过街任务。我们比较了它们的响应,并采访了五位人机交互(HCI)研究人员探讨潜在应用。结果显示,VLM人设能够模仿人类响应模式(例如平均过街时间为5.25秒 vs. 5.07秒),但缺乏行为的多样性和深度。它们在形成性研究、田野研究准备和人类数据增强方面显示出潜力。

英文摘要:

Field studies are irreplaceable but costly, time-consuming, and error-prone, which need careful preparation. Inspired by rapid-prototyping in manufacturing, we propose a fast, low-cost evaluation method using Vision-Language Model (VLM) personas to simulate outcomes comparable to field results. While LLMs show human-like reasoning and language capabilities, autonomous vehicle (AV)-pedestrian interaction requires spatial awareness, emotional empathy, and behavioral generation. This raises our research question: To what extent can VLM personas mimic human responses in field studies? We conducted parallel studies: 1) one real-world study with 20 participants, and 2) one video-study using 20 VLM personas, both on a street-crossing task. We compared their responses and interviewed five HCI researchers on potential applications. Results show that VLM personas mimic human response patterns (e.g., average crossing times of 5.25 s vs. 5.07 s) lack the behavioral variability and depth. They show promise for formative studies, field study preparation, and human data augmentation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑