基于可泛化高斯泼溅的强化学习
Reinforcement Learning with Generalizable Gaussian Splatting
AI总结：
本文提出GSRL框架，首次以可泛化3D Gaussian Splatting作为强化学习的显式可微场景表示，并在RoboMimic多任务中显著优于基线。
摘要：
优秀的表示对于强化学习（RL）性能至关重要，尤其是在基于视觉的强化学习任务中。环境表示的质量直接影响学习任务的完成情况。以往基于视觉的RL通常使用显式或隐式方式表示环境，例如图像、点、体素和神经辐射场。然而，这些表示存在若干缺陷：它们要么无法描述复杂的局部几何形状，要么不能很好地泛化到未见过的场景，或者需要精确的前景掩码。此外，这些隐式神经表示类似于“黑箱”，严重阻碍了可解释性。3D Gaussian Splatting（3DGS，三维高斯泼溅）凭借其显式场景表示和可微渲染特性，被认为是重建与表示方法的一次革命性变革。本文提出一种新颖的可泛化高斯泼溅框架，用作RL任务的表示，称为GSRL。通过在RoboMimic环境中的验证，我们的方法在多个任务上取得了优于其他基线的结果；在最困难的任务上，相比基线分别提升了10%、44%和15%的性能。本工作是首次尝试将可泛化3DGS用作RL的表示。
英文摘要：
An excellent representation is crucial for reinforcement learning (RL) performance, especially in vision-based reinforcement learning tasks. The quality of the environment representation directly influences the achievement of the learning task. Previous vision-based RL typically uses explicit or implicit ways to represent environments, such as images, points, voxels, and neural radiance fields. However, these representations contain several drawbacks. They cannot either describe complex local geometries or generalize well to unseen scenes, or require precise foreground masks. Moreover, these implicit neural representations are akin to a ``black box", significantly hindering interpretability. 3D Gaussian Splatting (3DGS), with its explicit scene representation and differentiable rendering nature, is considered a revolutionary change for reconstruction and representation methods. In this paper, we propose a novel Generalizable Gaussian Splatting framework to be the representation of RL tasks, called GSRL. Through validation in the RoboMimic environment, our method achieves better results than other baselines in multiple tasks, improving the performance by 10%, 44%, and 15% compared with baselines on the hardest task. This work is the first attempt to leverage generalizable 3DGS as a representation for RL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
