CLIPSwarm:将文本转换为机器人编队
CLIPSwarm: Converting text into formations of robots
AI总结:
本文提出CLIPSwarm算法,通过Montecarlo粒子滤波变体和CLIP相似度度量,将自然语言描述迭代转化为机器人集群编队,展示了基础模型在多机器人系统中的新应用。
摘要:
我们提出了CLIPSwarm,一种从自然语言描述生成机器人集群编队的算法。CLIPSwarm接收输入文本,并寻找机器人的位置以形成与给定文本相对应的形状。为此,我们实现了Montecarlo粒子滤波的一种变体,以迭代地获得匹配的编队。在每次迭代中,我们生成一组新编队,并评估它们与给定文本的Clip Similarity,根据该指标选择最佳编队。该指标是使用Clip [1]获得的,这是一种现有的基础模型,经过训练可将图像和文本编码为公共潜在空间中的向量。这些向量之间的比较决定了给定文本描述这些形状的可能性。我们最初的概念验证展示了该解决方案仅从自然语言描述生成机器人集群编队的潜力,并展示了诸如CLIP等基础模型在多机器人系统领域的新颖应用。在这第一种方法中,我们使用Convex-Hull方法创建编队。接下来的步骤包括在获得合适集群编队的过程中进行更稳健和通用的表示及优化步骤。
英文摘要:
We present CLIPSwarm, an algorithm to generate robot swarm formations from natural language descriptions. CLIPSwarm receives an input text and finds the position of the robots to form a shape that corresponds to the given text. To do so, we implement a variation of the Montecarlo particle filter to obtain a matching formation iteratively. In every iteration, we generate a set of new formations and evaluate their Clip Similarity with the given text, selecting the best formations according to this metric. This metric is obtained using Clip, [1], an existing foundation model trained to encode images and texts into vectors within a common latent space. The comparison between these vectors determines how likely the given text describes the shapes. Our initial proof of concept shows the potential of this solution to generate robot swarm formations just from natural language descriptions and demonstrates a novel application of foundation models, such as CLIP, in the field of multi-robot systems. In this first approach, we create formations using a Convex-Hull approach. Next steps include more robust and generic representation and optimization steps in the process of obtaining a suitable swarm formation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。