XR中的视觉到触觉增强：一种用于多模态交互感知接地的可穿戴手套
Visual-to-Haptic Augmentation in XR: A Wearable Glove for Perceptual Grounding in Multimodal Interaction
浏览论文内容
中文总结 AI 辅助
该研究针对XR系统触觉感知利用不足的问题，提出一款可穿戴手套及视觉到触觉映射算法，经20人实验验证，其触觉增强可提升XR交互的真实感与沉浸感，为多模态XR系统提供感知增强层。
中文摘要 AI 辅助
扩展现实（XR）系统日益提供高保真的视觉和听觉体验，但触觉感知作为丰富具身交互的模态仍相对未被充分利用。本研究提出一种视觉到触觉可穿戴手套，以及一种基于特征的视觉到触觉映射算法，该算法将图像和视频中的空间与时间视觉特征转换为分布式振动触觉模式。所提方法提取运动、边缘和亮度线索，并将它们融合为与29个执行器对齐的执行器级强度图，该手套采用5×7布局。系统通过模块化四层架构实现，包括XR环境、媒体内容处理、视觉到触觉处理以及嵌入式触觉硬件。一项被试内用户研究（N=20）在基于纹理和动态视频场景中，对比仅视觉交互与视觉加触觉增强的效果。结果表明，触觉增强在动态视频场景中显著提升感知真实感，并在所有场景中增强沉浸感和视觉-触觉对应关系，对动态视觉事件的效果更强且更一致。尽管当前实现为单用户、离线同步配置，但研究结果表明，视觉驱动的触觉增强可作为多模态XR系统内的感知增强层，该层或为未来社交丰富的XR环境奠定基础，其中连贯的多感官接地支持更高层次的交互与沟通。
英文摘要
Extended Reality (XR) systems increasingly deliver high-fidelity visual and auditory experiences, yet tactile perception remains comparatively underutilized as a modality for enriching embodied interaction. This work presents a visual-to-haptic wearable glove and a feature-based visual-to-haptic mapping algorithm that translates spatial and temporal visual features from images and videos into distributed vibrotactile patterns. The proposed method extracts motion, edge, and brightness cues and fuses them into actuator-level intensity maps aligned with a 29-actuator glove arranged in a five-by-seven layout. The system is implemented through a modular four-layer architecture comprising the XR environment, media content handling, visual-to-haptic processing, and embedded haptic hardware. A within-subject user study (N = 20) compared visual-only interaction with visual-plus-haptic augmentation across texture-based and dynamic video scenarios. Results indicate that tactile augmentation significantly improves perceived realism in dynamic video scenarios and enhances immersion and visual-tactile correspondence across conditions, with stronger and more consistent effects observed for dynamic visual events. While the current implementation operates in a single-user, offline-synchronized configuration, the findings demonstrate that vision-driven tactile augmentation can function as a perceptual enhancement layer within multimodal XR systems. Such a layer may provide a foundation for future socially enriched XR environments where coherent multisensory grounding supports higher-level interaction and communication.
