arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.19643cs.RO
收藏

基于手势的视觉学习模型用于声学聚沉交互的声学机器人群

A Gesture-Based Visual Learning Model for Acoustophoretic Interactions using a Swarm of AcoustoBots

Department of Computer Science University College London London United Kingdom（计算机科学系伦敦大学学院伦敦英国）；Department of Artificial Intelligence University of Malaya Kuala Lumpur Malaysia（人工智能系马来大学吉隆坡马来西亚）；University College London（伦敦大学学院）；University of Malaya（马来大学）

Alex Lin, Lei Gao, Narsimlu Kemsaram, Sriram Subramanian

备注：
This paper has been accepted for publication in the Proceedings of the 2026 4th International Conference on Robotics, Control and Vision Engineering (RCVE 2026)
发表信息：
Proceedings of the 2026 ACM International Conference on Robotics, Control and Vision Engineering (RCVE 2026), 10-12 July 2026, Tokyo, Japan
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出基于手势的视觉学习框架，实现无接触人-机器人群交互，通过多模态AcoustoBot平台实现手势识别与触觉、音频、悬浮等模态映射，验证准确率提升至98%。

摘要：

AcoustoBots是能够提供空中触觉、定向音频和声学悬浮的移动声学机器人，但现有实现依赖于脚本命令且缺乏直观的实时人机交互界面。本文提出一种基于手势的视觉学习框架，结合ESP32-CAM手势捕捉、PhaseSpace运动追踪、集中式处理和基于OpenCLIP的视觉学习模型（VLM）进行线性探测，实现三种手势识别并映射到触觉、音频和悬浮模态。验证准确率从小数据集的约67%提升到大数据集的近98%。在两个AcoustoBots的集成实验中，系统在90次试验中实现了87.8%的 gesture-to-modality 切换准确率，平均端到端延迟为3.95秒。这些结果展示了基于视觉-语言模型的手势接口在多模态人-机器人群交互中的可行性。尽管当前系统受限于集中式处理、静态手势集和受控环境评估，但为更生动、可扩展和可访问的机器人群接口奠定了基础。

英文摘要：

AcoustoBots are mobile acoustophoretic robots capable of delivering mid-air haptics, directional audio, and acoustic levitation, but existing implementations rely on scripted commands and lack an intuitive interface for real-time human control. This work presents a gesture-based visual learning framework for contactless human-swarm interaction with a multimodal AcoustoBot platform. The system combines ESP32-CAM gesture capture, PhaseSpace motion tracking, centralized processing, and an OpenCLIP-based visual learning model (VLM) with linear probing to classify three hand gestures and map them to haptics, audio, and levitation modalities. Validation accuracy improved from about 67% with a small dataset to nearly 98% with the largest dataset. In integrated experiments with two AcoustoBots, the system achieved an overall gesture-to-modality switching accuracy of 87.8% across 90 trials, with an average end-to-end latency of 3.95 seconds. These results demonstrate the feasibility of using a vision-language-model-based gesture interface for multimodal human-swarm interaction. While the current system is limited by centralized processing, a static gesture set, and controlled-environment evaluation, it establishes a foundation for more expressive, scalable, and accessible swarm robotic interfaces.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑