Tunnel Try-on：挖掘时空隧道以实现高质量视频虚拟试穿
Tunnel Try-on: Excavating Spatial-temporal Tunnels for High-quality Virtual Try-on in Videos
AI总结：
提出基于扩散框架的Tunnel Try-on，通过挖掘聚焦隧道放大服装区域以保留细节，结合Kalman filter平滑裁剪和位置嵌入生成连贯运动，并利用环境编码器提取外部上下文，实现高质量视频虚拟试穿。
摘要：
视频试穿是一项具有挑战性的任务，在以往的工作中尚未得到很好的解决。主要障碍在于同时保留服装细节和建模连贯的运动。面对这些困难，我们通过提出一个名为“Tunnel Try-on”的基于扩散的框架来解决视频试穿问题。核心思想是在输入视频中挖掘一个“聚焦隧道”，提供服装区域的特写镜头。我们放大隧道内的区域以更好地保留服装的精细细节。为了生成连贯的运动，我们首先利用Kalman filter在聚焦隧道中构建平滑裁剪，并将隧道的位置嵌入注入到注意力层中，以提高生成视频的连续性。此外，我们开发了一个环境编码器来提取隧道外的上下文信息作为补充线索。配备了这些技术，Tunnel Try-on保留了服装的精细细节，并合成了稳定平滑的视频。展示了显著的进步，Tunnel Try-on可被视为迈向视频虚拟试穿商业级应用的首次尝试。
英文摘要：
Video try-on is a challenging task and has not been well tackled in previous works. The main obstacle lies in preserving the details of the clothing and modeling the coherent motions simultaneously. Faced with those difficulties, we address video try-on by proposing a diffusion-based framework named "Tunnel Try-on." The core idea is excavating a "focus tunnel" in the input video that gives close-up shots around the clothing regions. We zoom in on the region in the tunnel to better preserve the fine details of the clothing. To generate coherent motions, we first leverage the Kalman filter to construct smooth crops in the focus tunnel and inject the position embedding of the tunnel into attention layers to improve the continuity of the generated videos. In addition, we develop an environment encoder to extract the context information outside the tunnels as supplementary cues. Equipped with these techniques, Tunnel Try-on keeps the fine details of the clothing and synthesizes stable and smooth videos. Demonstrating significant advancements, Tunnel Try-on could be regarded as the first attempt toward the commercial-level application of virtual try-on in videos.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
