arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.08854cs.CVcs.AIcs.ROeess.IV
收藏

DVI:深度引导的自动驾驶视频修复

DVI: Depth Guided Video Inpainting for Autonomous Driving

Baidu Research(百度研究院);Baidu Inc(百度公司)

Miao Liao, Feixiang Lu, Dingfu Zhou, Sibo Zhang, Wei Li, Ruigang Yang

摘要:

为了在自动驾驶中获得清晰的街景和照片般真实的仿真效果,提出了一种自动视频修复算法,该算法可以在深度/点云的指导下去除视频中的交通代理并合成丢失的区域。通过从拼接的点云构建密集的3D地图,视频中的帧通过这个公共3D地图在几何上相关。为了填充帧中的目标修复区域,可以直接将来自其他帧的像素转换为具有正确遮挡的当前像素。此外,我们能够通过3D点云配准融合多个视频,从而可以使用多个源视频对目标视频进行修补。其动机是解决长时间遮挡问题,其中遮挡区域在整个视频中从未可见。据我们所知,我们是第一个融合多个视频进行视频修复的公司。为了验证我们方法的有效性,我们在真实的城市道路环境中构建了一个大型修复数据集,其中包含同步图像和激光雷达数据,包括许多挑战场景,例如,长时间闭塞。实验结果表明,该方法优于国家的最先进的方法的所有标准,特别是均方根误差(均方根误差)已减少了约13%。

英文摘要:

To get clear street-view and photo-realistic simulation in autonomous driving, we present an automatic video inpainting algorithm that can remove traffic agents from videos and synthesize missing regions with the guidance of depth/point cloud. By building a dense 3D map from stitched point clouds, frames within a video are geometrically correlated via this common 3D map. In order to fill a target inpainting area in a frame, it is straightforward to transform pixels from other frames into the current one with correct occlusion. Furthermore, we are able to fuse multiple videos through 3D point cloud registration, making it possible to inpaint a target video with multiple source videos. The motivation is to solve the long-time occlusion problem where an occluded area has never been visible in the entire video. To our knowledge, we are the first to fuse multiple videos for video inpainting. To verify the effectiveness of our approach, we build a large inpainting dataset in the real urban road environment with synchronized images and Lidar data including many challenge scenes, e.g., long time occlusion. The experimental results show that the proposed approach outperforms the state-of-the-art approaches for all the criteria, especially the RMSE (Root Mean Squared Error) has been reduced by about 13%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑