通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2503.12077cs.CVcs.AI
V-Stylist:通过MLLM智能体的协作与反思实现视频风格化
V-Stylist: Video Stylization via Collaboration and Reflection of MLLM Agents
Shanghai Jiao Tong University(上海交通大学);Shanghai AI Laboratory(上海人工智能实验室);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University of Chinese Academy of Sciences(中国科学院大学)
Zhengrong Yue, Shaobin Zhuang, Kunchang Li, Yanbo Ding, Yali Wang
尽管视频风格化近期取得了进展,但大多数现有方法在基于用户查询的开放风格描述来渲染包含复杂转场的任意视频时仍面临困难。为填补这一空白,我们提出了一种用于视频风格化的通用多智能体系统V-Stylist,该系统采用多模态大语言模型的新型协作与反思范式。具体而言,V-Stylist是一个包含三个关键角色的系统化工作流:(1)Video Parser将输入视频分解为若干镜头,并生成这些镜头关键内容的文本提示。通过简洁的视频到镜头提示范式,它使V-Stylist能够有效处理具有复杂转场的视频。(2)Style Parser识别用户查询中的风格,并从风格树中逐步搜索匹配的风格模型。通过鲁棒的思维树搜索范式,它使V-Stylist能够精确明确开放用户查询中模糊的风格偏好。(3)Style Artist利用匹配的模型将所有视频镜头渲染为所需风格。通过新颖的多轮自我反思范式,它使V-Stylist能够根据风格要求自适应地调整细节控制。凭借这种模仿人类专业人员的独特设计,V-Stylist在有效且自动的视频风格化所面临的主要挑战上实现了重大突破。此外,我们进一步构建了一个新的基准Text-driven Video Stylization Benchmark(TVSBench),填补了在开放用户查询下评估复杂视频风格化的空白。大量实验表明,V-Stylist达到了最先进水平,例如,V-Stylist在总体平均指标上分别超过FRESCO和ControlVideo 6.05%和4.51%,标志着视频风格化的显著进步。
英文摘要:
Despite the recent advancement in video stylization, most existing methods struggle to render any video with complex transitions, based on an open style description of user query. To fill this gap, we introduce a generic multi-agent system for video stylization, V-Stylist, by a novel collaboration and reflection paradigm of multi-modal large language models. Specifically, our V-Stylist is a systematical workflow with three key roles: (1) Video Parser decomposes the input video into a number of shots and generates their text prompts of key shot content. Via a concise video-to-shot prompting paradigm, it allows our V-Stylist to effectively handle videos with complex transitions. (2) Style Parser identifies the style in the user query and progressively search the matched style model from a style tree. Via a robust tree-of-thought searching paradigm, it allows our V-Stylist to precisely specify vague style preference in the open user query. (3) Style Artist leverages the matched model to render all the video shots into the required style. Via a novel multi-round self-reflection paradigm, it allows our V-Stylist to adaptively adjust detail control, according to the style requirement. With such a distinct design of mimicking human professionals, our V-Stylist achieves a major breakthrough over the primary challenges for effective and automatic video stylization. Moreover,we further construct a new benchmark Text-driven Video Stylization Benchmark (TVSBench), which fills the gap to assess stylization of complex videos on open user queries. Extensive experiments show that, V-Stylist achieves the state-of-the-art, e.g.,V-Stylist surpasses FRESCO and ControlVideo by 6.05% and 4.51% respectively in overall average metrics, marking a significant advance in video stylization.
StylisticBias: A Few Human Visual Cues Drive Most Social Biases in MLLMs
Technical University of Munich(慕尼黑工业大学);Munich Center for Machine Learning(慕尼黑机器学习中心);Princeton Center for Information and Technology Policy(普林斯顿信息与技术政策中心)
Capacity-Controlled Multi-View Stylization of 3D Gaussian Splatting
Guangdong Provincial Key Laboratory of Visual Media and Multidimensional Intelligence, CSSE, Shenzhen University(广东省可视媒体与多维智能重点实验室,计算机科学与软件工程学院,深圳大学);Tencent Games(腾讯游戏);The Hebrew University of Jerusalem(耶路撒冷希伯来大学);Tel Aviv University(特拉维夫大学)
Color and tone stylization strives to enhance unique themes with artistic color and tone adjustments. It has a broad range of applications from professional image postprocessing to photo sharing over social networks. Mainstream photo enhancement softwares provide users with predefined styles, which are often hand-crafted through a trial-and-error process. Such photo adjustment tools lack a semantic understanding of image contents and the resulting global color transform limits the range of artistic styles it can represent. On the other hand, stylistic enhancement needs to apply distinct adjust…
AesTest: Measuring Aesthetic Intelligence from Perception to Production
University of International Business and Economics(国际商务经济大学);University of Science and Technology of China(中国科学技术大学);Huawei Technologies Co., Ltd(华为技术有限公司);Beijing Electronic Science and Technology Institute(北京电子科技研究所);Beijing Institute for General Artificial Intelligence(北京通用人工智能研究院)
Investigating the diversity and stylization of contemporary user generated visual arts in the complexity entropy plane
Graduate School of Culture Technology, KAIST(韩国科学技术院文化科技研究生院);Center for Complex Networks and Systems Research, Luddy School of Informatics, Computing, and Engineering, Indiana University(印第安纳大学复杂网络与系统研究中心,林迪学校);Institute of Basic Science, Sungkyunkwan University(全北国立大学基础科学研究所)
Shenzhen University(深圳大学);Visual Computing Research Center (VCC), College of Computer Science and Software Engineering (CSSE)(视觉计算研究中心(VCC),计算机科学与软件工程学院);Peking University(北京大学)
提出Abstraction in Style (AiS)框架,将结构抽象与视觉风格分离,通过中间抽象代理实现几何保真度放松,从而支持更广泛的非真实感风格迁移。
Hi-GaTA: Hierarchical Gated Temporal Aggregation Adapter for Surgical Video Report Generation
School of Engineering, College of Engineering and Physical Sciences, University of Birmingham, Birmingham, UK(英国伯明翰大学工程学院);School of Computer Science, University of Birmingham, Birmingham, UK(英国伯明翰大学计算机科学学院);Department of Applied Health Sciences, University of Birmingham, Birmingham, UK(英国伯明翰大学应用健康科学系)
A-MAR: Agent-based Multimodal Art Retrieval for Fine-Grained Artwork Understanding
University of Amsterdam(阿姆斯特丹大学);University of Bristol(布里斯托大学);Amazon AGI(亚马逊人工智慧);College of Business and Economics(商学院和经济学学院);University of Johannesburg(约翰内斯堡大学)
OmniStyle2: Learning to Stylize by Learning to Destylize
Jilin University(吉林大学);Nanjing University(南京大学);Shanghai Innovation Institute(上海创新研究院);Adobe;Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, China(中国教育部知识驱动人机智能工程研究中心)