AKiRa：面向光学视频生成的光线增强套件
AKiRa: Augmentation Kit on Rays for optical video generation
AI总结：
本文提出AKiRa框架，通过在现有视频生成骨干网络上训练带复杂相机模型的适配器，实现对相机运动及焦距、畸变、光圈等光学参数的精细控制，从而生成具有变焦、鱼眼和散景等电影化效果的视频，并超越现有最先进方法。
摘要：
近年来，基于文本条件的视频扩散模型在视频质量上取得了显著提升。然而，这些方法在相机方面的用户控制能力有限，有时甚至完全缺乏控制，包括动态相机运动、变焦、镜头畸变和焦点偏移等。这些运动与光学特性对于为生成框架增加可控性和电影化元素至关重要，最终能够产生吸引注意力、增强氛围并根据电影制作者的控制引导情感的视觉内容。在本文中，我们旨在弥合可控视频生成与相机光学之间的差距。为此，我们提出了AKiRa（Augmentation Kit on Rays，光线增强套件），这是一种新颖的增强框架，在现有视频生成骨干网络上构建并训练一个带有复杂相机模型的相机适配器。它能够对相机运动以及复杂光学参数（焦距、畸变、光圈）进行精细控制，以实现变焦、鱼眼效果和散景等电影化效果。大量实验表明，AKiRa在组合和合成相机光学效果方面具有有效性，同时优于所有最先进的方法。这项工作为可控且光学增强的视频生成树立了新的里程碑，为未来的光学视频生成方法铺平了道路。
英文摘要：
Recent advances in text-conditioned video diffusion have greatly improved video quality. However, these methods offer limited or sometimes no control to users on camera aspects, including dynamic camera motion, zoom, distorted lens and focus shifts. These motion and optical aspects are crucial for adding controllability and cinematic elements to generation frameworks, ultimately resulting in visual content that draws focus, enhances mood, and guides emotions according to filmmakers' controls. In this paper, we aim to close the gap between controllable video generation and camera optics. To achieve this, we propose AKiRa (Augmentation Kit on Rays), a novel augmentation framework that builds and trains a camera adapter with a complex camera model over an existing video generation backbone. It enables fine-tuned control over camera motion as well as complex optical parameters (focal length, distortion, aperture) to achieve cinematic effects such as zoom, fisheye effect, and bokeh. Extensive experiments demonstrate AKiRa's effectiveness in combining and composing camera optics while outperforming all state-of-the-art methods. This work sets a new landmark in controlled and optically enhanced video generation, paving the way for future optical video generation methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
