arXiv 2506.176122025-06-21cs.CV

JarvisArt：通过智能照片修饰代理解放人类艺术创造力

JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agent

Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University（中国教育部多媒体可信感知与高效计算重点实验室，厦门大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Chinese University of Hong Kong（香港中文大学）；Bytedance（字节跳动）；National University of Singapore（新加坡国立大学）；Tsinghua University（清华大学）

Yunlong Lin, Zixu Lin, Kunjie Lin, Jinbin Bai, Panwang Pan, Chenxin Li, Haoyu Chen, Zhongdao Wang, Xinghao Ding, Wenbo Li, Shuicheng Yan

备注：: 40 pages, 26 figures

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

JarvisArt是一个由多模态大语言模型驱动的智能照片修饰代理，通过两阶段训练和Agent-to-Lightroom协议，实现全局与局部调整的细粒度控制，在MMArt-Bench上平均像素级指标比GPT-4o提升60%。

摘要：

照片修饰已成为当代视觉叙事中不可或缺的一部分，使用户能够捕捉美学并表达创造力。虽然Adobe Lightroom等专业工具提供了强大的功能，但它们需要大量的专业知识和手动操作。相比之下，现有的基于AI的解决方案提供了自动化，但往往存在可调整性有限和泛化能力差的问题，无法满足多样化和个性化的编辑需求。为了弥合这一差距，我们引入了JarvisArt，一个由多模态大语言模型（MLLM）驱动的代理，它能够理解用户意图，模仿专业艺术家的推理过程，并智能地协调Lightroom中超过200种修饰工具。JarvisArt经历了一个两阶段的训练过程：首先是思维链监督微调，以建立基本的推理和工具使用技能，然后是用于修饰的组相对策略优化（GRPO-R），以进一步增强其决策能力和工具熟练度。我们还提出了代理到Lightroom协议，以促进与Lightroom的无缝集成。为了评估性能，我们开发了MMArt-Bench，一个从真实用户编辑中构建的新基准。JarvisArt展示了用户友好的交互、卓越的泛化能力以及对全局和局部调整的细粒度控制，为智能照片修饰开辟了一条新途径。值得注意的是，它在MMArt-Bench上的内容保真度平均像素级指标上比GPT-4o提升了60%，同时保持了相当的指令跟随能力。项目页面：https://jarvisart.vercel.app/。

英文摘要：

Photo retouching has become integral to contemporary visual storytelling, enabling users to capture aesthetics and express creativity. While professional tools such as Adobe Lightroom offer powerful capabilities, they demand substantial expertise and manual effort. In contrast, existing AI-based solutions provide automation but often suffer from limited adjustability and poor generalization, failing to meet diverse and personalized editing needs. To bridge this gap, we introduce JarvisArt, a multi-modal large language model (MLLM)-driven agent that understands user intent, mimics the reasoning process of professional artists, and intelligently coordinates over 200 retouching tools within Lightroom. JarvisArt undergoes a two-stage training process: an initial Chain-of-Thought supervised fine-tuning to establish basic reasoning and tool-use skills, followed by Group Relative Policy Optimization for Retouching (GRPO-R) to further enhance its decision-making and tool proficiency. We also propose the Agent-to-Lightroom Protocol to facilitate seamless integration with Lightroom. To evaluate performance, we develop MMArt-Bench, a novel benchmark constructed from real-world user edits. JarvisArt demonstrates user-friendly interaction, superior generalization, and fine-grained control over both global and local adjustments, paving a new avenue for intelligent photo retouching. Notably, it outperforms GPT-4o with a 60% improvement in average pixel-level metrics on MMArt-Bench for content fidelity, while maintaining comparable instruction-following capabilities. Project Page: https://jarvisart.vercel.app/.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。