TouchScribe: 通过自动化实时视觉描述增强非视觉手-物体交互
TouchScribe: Augmenting Non-Visual Hand-Object Interactions with Automated Live Visual Descriptions
AI总结：
TouchScribe通过自动化实时视觉描述增强手-物体交互，帮助视障人士更准确地理解物体特征。
摘要：
视障人士或低视力人群经常通过手部与物理世界互动来获取物体的形状、大小、重量和纹理信息。然而，许多丰富的视觉特征仍然无法通过触觉单独获得，这使得难以区分相似物体、解读视觉可能性，并形成对物体的完整理解。在本工作中，我们提出了TouchScribe系统，该系统通过自动化实时视觉描述来增强手-物体交互。我们训练了一个定制的视角手部交互模型，以识别常见的手势（例如，抓取以检查，并排持以比较）和盲人特有的手势（例如，指向以探索颜色，或滑动以阅读可用文本）。此外，TouchScribe根据手部运动提供实时且自适应的反馈，从手部交互状态到物体标签，再到视觉细节。我们的用户研究和技术评估表明，TouchScribe能够提供丰富且有用的描述以支持物体理解。最后，我们讨论了使实时视觉描述对用户身体触碰做出响应的含义。
英文摘要：
People who are blind or have low vision regularly use their hands to interact with the physical world to gain access to objects' shape, size, weight, and texture. However, many rich visual features remain inaccessible through touch alone, making it difficult to distinguish similar objects, interpret visual affordances, and form a complete understanding of objects. In this work, we present TouchScribe, a system that augments hand-object interactions with automated live visual descriptions. We trained a custom egocentric hand interaction model to recognize both common gestures (e.g., grab to inspect, hold side-by-side to compare) and unique ones by blind people (e.g., point to explore color, or swipe to read available texts). Furthermore, TouchScribe provides real-time and adaptive feedback based on hand movement, from hand interaction states, to object labels, and to visual details. Our user study and technical evaluations demonstrate that TouchScribe can provide rich and useful descriptions to support object understanding. Finally, we discuss the implications of making live visual descriptions responsive to users' physical reach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
