Kuration SDK：通过数据整理弥合虚拟到现实差距
Kuration SDK: Addressing the Virtual2Real Gap via Data Curation
- AIntropy AI
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文提出Kuration SDK，通过数据整理和诊断属性测量，在训练前弥合动作条件世界模型中的虚拟到现实差距，并揭示特定案例中模型表现差异的根本原因。
AI中文摘要：
用于衡量动作条件世界模型质量的基准仍在不断演变，正从基于视觉相似性的度量转向动作语义和物理基础度量。然而，对于领域和任务无关的动作条件世界模型训练，现有基准提供的信号有限。通过在CounterStrike游戏数据上训练和评估扩散世界模型，我们确认定性可玩性与FVD、LPIPS和JEDi等度量并不对应。我们将此称为虚拟到现实差距。我们认为，在没有可靠基准的情况下，整理原始游戏数据并测量多种诊断属性，可以在训练开始之前提供更稳健的信号来弥合这一差距。我们提出了几种整理策略，并推出了一个用于物理AI数据整理的通用工具包，称为Kuration SDK，随本文开源。该SDK在揭示虚拟到现实差距的根本原因方面发挥了关键作用，具体案例是：尽管两个世界模型在相同的游戏地图、动作和状态分布上训练，且具有非常相似的LPIPS和FVD分数，但在实际游玩时表现却截然不同。因此，Kuration SDK有潜力揭示特定数据集中虚拟到现实差距的根本原因，并加速样本高效训练数据集的开发。
英文摘要：
Benchmarks for measuring the quality of action-conditioned world models are still evolving and shifting away from visual similarity-based metrics to action-semantic and physically-grounded metrics. However, for domain and task-agnostic action-conditioned world model training, existing benchmarks provide a limited signal. By training and evaluating diffusion world models on CounterStrike gameplay data, we confirm that qualitative playability does not correspond with metrics such as FVD, LPIPS, and JEDi. We term this the Virtual2Real gap. We posit that, in lieu of reliable benchmarks, curating raw gameplay data and measuring a variety of diagnostic properties provides a more robust signal to bridge the gap, before the training even begins. We present several curation strategies and a general-purpose kit for physical AI data curation called Kuration SDK, which is being open-sourced with this paper. The SDK was instrumental in uncovering the root cause of the virtual2real gap in a specific case: why two world models trained on identical gameplay map, action and state distribution, behaved very differently when played in spite of having very similar LPIPS and FVD scores. Thus, Kuration SDK has the potential to uncover the root causes of Virtual2Real gap in specific datasets and accelerate development of sample-efficient training datasets.
