想象一下!利用应急能力进行3D工具合成
Imagine That! Leveraging Emergent Affordances for 3D Tool Synthesis
摘要:
在本文中,我们探讨了丰富的信息捕获的潜在空间的视觉为基础的生成模型。该模型将无监督生成学习与基于任务的性能预测相结合,以学习和利用任务相关的对象启示,从一个达到任务的视觉观察,涉及一个场景和一个棒状工具。虽然生成模型的学习嵌入捕获3D工具几何形状的变化因素(例如,长度、宽度和形状),但性能预测器识别与任务成功相关的嵌入的子流形。在各种情况下,我们证明,通过从性能预测器反向传播遍历潜在空间,使我们能够想象适合手头任务的工具。我们的研究结果表明,像实用程序的可及性编码沿着平滑的轨迹在潜在的空间。通过只考虑高级别的性能标准(如任务成功)来验证这些紧急的启示,使代理能够以有针对性和故意的方式操纵工具几何形状。
英文摘要:
In this paper we explore the richness of information captured by the latent space of a vision-based generative model. The model combines unsupervised generative learning with a task-based performance predictor to learn and to exploit task-relevant object affordances given visual observations from a reaching task, involving a scenario and a stick-like tool. While the learned embedding of the generative model captures factors of variation in 3D tool geometry (e.g. length, width, and shape), the performance predictor identifies sub-manifolds of the embedding that correlate with task success. Within a variety of scenarios, we demonstrate that traversing the latent space via backpropagation from the performance predictor allows us to imagine tools appropriate for the task at hand. Our results indicate that affordances-like the utility for reaching-are encoded along smooth trajectories in latent space. Accessing these emergent affordances by considering only high-level performance criteria (such as task success) enables an agent to manipulate tool geometries in a targeted and deliberate way.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。