Cosmos-Transfer1：基于自适应多模态控制的条件世界生成
Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control
AI总结：
提出条件世界生成模型Cosmos-Transfer，通过自适应多模态空间控制输入实现高度可控的世界生成，适用于Sim2Real等场景，并开源模型与代码以加速Physical AI研究。
摘要：
我们提出了Cosmos-Transfer，一种条件世界生成模型，能够基于分割、深度和边缘等多种模态的多个空间控制输入生成世界模拟。在设计上，空间条件方案是自适应且可定制的。它允许在不同的空间位置对不同的条件输入赋予不同的权重。这使得高度可控的世界生成成为可能，并适用于各种世界到世界转换的用例，包括Sim2Real。我们进行了广泛的评估来分析所提出的模型，并展示了其在Physical AI中的应用，包括机器人Sim2Real和自动驾驶车辆数据增强。我们进一步展示了一种推理扩展策略，以使用NVIDIA GB200 NVL72机架实现实时世界生成。为了加速该领域的研究进展，我们在https://github.com/nvidia-cosmos/cosmos-transfer1开源了我们的模型和代码。
英文摘要：
We introduce Cosmos-Transfer, a conditional world generation model that can generate world simulations based on multiple spatial control inputs of various modalities such as segmentation, depth, and edge. In the design, the spatial conditional scheme is adaptive and customizable. It allows weighting different conditional inputs differently at different spatial locations. This enables highly controllable world generation and finds use in various world-to-world transfer use cases, including Sim2Real. We conduct extensive evaluations to analyze the proposed model and demonstrate its applications for Physical AI, including robotics Sim2Real and autonomous vehicle data enrichment. We further demonstrate an inference scaling strategy to achieve real-time world generation with an NVIDIA GB200 NVL72 rack. To help accelerate research development in the field, we open-source our models and code at https://github.com/nvidia-cosmos/cosmos-transfer1.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
