发表机构
IRIDIA, Université Libre de Bruxelles; Department of Aerospace Science and Technology, Politecnico di Milano; Télécom Physique Strasbourg, Université de Strasbourg; Polytech Lyon, Lyon 1 Université Claude Bernard; University of Pisa; Scuola Superiore Sant’Anna（布鲁塞尔自由大学IRIDIA研究所; 米兰理工大学航空航天科学与技术系; 斯特拉斯堡大学斯特拉斯堡电信物理学院; 里昂第一大学里昂理工学院; 比萨大学; 圣安娜高等研究学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
Auto-HSI利用LLM根据自然语言和手势自动生成代码，为未经训练的操作员按需创建个性化人-集群交互界面，实现手势控制机器人集群，并在仿真和真实实验中验证了有效性。
AI 中文摘要
本文提出了Auto-HSI，一种按需生成个性化人-集群交互（HSI）界面的方法。其目标是使未经训练的操作员能够使用自然语言描述和手势演示，来解释他们希望机器人如何集体响应其手势。基于这些输入，应自动生成代码，用于构建个性化的状态机，以按期望的方式控制机器人，响应所需的手势输入。在开发的Auto-HSI原型中，生成的代码产生一个用于集中控制的个性化界面，支持单手和双手手势，使用户能够远程操作机器人的运动、队形形状和形状变形。我们针对性能基准测试了Auto-HSI的手势跟踪和代码生成组件。然后，我们在“实时”操作实验中测试了完整的Auto-HSI原型，其中真实的人类操作员在物理仿真器中集中控制50个模拟机器人，分别在标称和噪声条件下进行。在这些实验中，机器人被远程操作以完成：进球、穿越需要形状变形的迷宫，以及通过分成两组同时进两个球。我们还演示了真实操作员在操作过程中（在仿真中）对其个性化Auto-HSI界面进行实时更新。最后，我们演示了真实机器人的实时操作。
英文摘要
This paper presents Auto-HSI, a method for generating personalized human-swarm interaction (HSI) interfaces on demand. The objective is to enable untrained operators to use natural language descriptions and gesture demonstrations to explain how they want the robots to collectively behave in response to their gestures. Based on these inputs, the code should automatically be generated for personalized state machines that will control the robots as desired, in response to the desired gesture inputs. In the developed Auto-HSI prototype, the generated code produces a personalized interface for centralized control using one- and two-handed gestures, enabling a user to teleoperate the robots' motion, formation shape, and shape deformation. We test the gesture tracking and code generation components of Auto-HSI against performance benchmarks. We then test the full Auto-HSI prototype in ``live'' operation experiments, in which real human operators centrally control 50 simulated robots in a physics-based simulator, under nominal and noisy conditions. In these experiments, robots are teleoperated to: score a goal, traverse a maze that requires shape deformation, and score two simultaneous goals by splitting into two groups. We also demonstrate a real human operator making live updates to their personalized Auto-HSI interface during operation (in simulation). Finally, we demonstrate live operation of real robots.
