AI 中文总结
本研究通过驾驶模拟器收集用户与全自主车辆交互时的多模态数据，分析用户对车载系统意外行为的细微情绪响应，为设计适配乘员行为的车辆提供依据。
AI 中文摘要
现代车辆具备先进的AI语音和自主导航功能，已超出传统驾驶范畴，但与任何自主系统一样，它们可能出现错误或做出用户意料之外的行为。尽管提供实时解释可缓解部分困惑，但持续的信息会使用户不堪重负，并可能造成不必要的分心。某些情况可能需要解释或车辆的纠正行为，因此，识别用户对车辆意外行为的响应至关重要。为研究此类用户响应，本研究专注于收集和分析用户在驾驶模拟器中与全自主车辆交互时对意外事件的行为响应。我们还旨在解决捕捉用户对车载事件的细微响应（面部、口语、生理信号）的数据集匮乏问题，因为现有数据集主要关注传统人类驾驶汽车中的强烈情绪信号，以及用户对外部道路和交通状况的响应。用户在平板电脑上执行次要任务、通过语音命令和车载显示屏与车辆交互时，会接触到旨在引发惊讶、困惑和沮丧的刺激。我们收集了包含视频、音频和心率数据的多模态数据集，并获得了对细微用户响应的见解，这些见解凸显了进一步调查用户细微行为的必要性。这些观察强调了设计能够识别并适应乘员行为的车辆的重要性，这可能会改善用户体验。
英文摘要
Modern vehicles, with advanced AI voice and autonomous navigation features, extend beyond traditional driving but, like any autonomous system, can potentially make mistakes or behave in ways unexpected by users. Although providing real-time explanations can alleviate some confusion, constant information can overwhelm users and potentially cause unnecessary distractions. Some situations may require explanations or corrective vehicle behavior, and thus, recognizing user response to unexpected vehicle behavior is critical. To investigate such user responses, our study focused on collecting and analyzing user behavioral responses to unexpected events while interacting with a fully autonomous vehicle in a driving simulator. We also aimed to address the lack of datasets capturing subtle user responses (facial, spoken language, physiological signals) to in-vehicle events, as existing datasets primarily focus on strong emotional signals in conventional human-driven cars and user response to external road and traffic conditions. Users were exposed to stimuli designed to induce surprise, confusion, and frustration while performing a secondary task on a tablet and interacting with the vehicle through voice commands and in-vehicle displays. We collected a multi-modal dataset with video, audio, and heart rate data and gained insights into subtle user responses that underscored the need for further investigation of nuanced user behaviors. These observations highlight the importance of designing vehicles that recognize and adapt to occupants' behavior, potentially improving their experience.
Comments23 pages, 10 figures
