释放美的本质：基于相对-绝对策略优化的高级审美推理
Unlocking the Essence of Beauty: Advanced Aesthetic Reasoning with Relative-Absolute Policy Optimization
AI总结：
针对MLLM审美推理数据稀缺与判断主观的问题，提出Aes-R1框架，通过AesCoT冷启动数据和RAPO强化学习联合优化评分与排序，显著提升审美评估性能。
摘要：
多模态大语言模型（MLLM）非常适合图像审美评估，因为它们能够利用跨模态理解能力捕捉高层级审美特征。然而，多模态审美推理数据的稀缺以及审美判断固有的主观性，使MLLM难以生成带有可解释依据的准确审美判断。为此，我们提出Aes-R1，这是一个采用强化学习（RL）的综合审美推理框架。具体而言，Aes-R1集成了AesCoT流程，用于构建和筛选供冷启动使用的高质量思维链审美推理数据。在教会模型于评分之前生成结构化解释之后，我们采用Relative-Absolute Policy Optimization（RAPO，相对-绝对策略优化），这是一种新型RL算法，联合优化绝对分数回归和相对排序顺序，同时提升单图像准确率和跨图像偏好判断。Aes-R1使MLLM能够生成有依据的解释以及可信的分数，从而在统一框架中增强审美评分与推理。大量实验表明，Aes-R1将骨干模型的平均PLCC/SRCC提升了47.9%/34.8%，超过相近规模的最先进基线。更多消融研究验证了Aes-R1在有限监督和分布外场景下的稳健泛化能力。
英文摘要：
Multimodal large language models (MLLMs) are well suited to image aesthetic assessment, as they can capture high-level aesthetic features leveraging their cross-modal understanding capacity. However, the scarcity of multimodal aesthetic reasoning data and the inherently subjective nature of aesthetic judgment make it difficult for MLLMs to generate accurate aesthetic judgments with interpretable rationales. To this end, we propose Aes-R1, a comprehensive aesthetic reasoning framework with reinforcement learning (RL). Concretely, Aes-R1 integrates a pipeline, AesCoT, to construct and filter high-quality chain-of-thought aesthetic reasoning data used for cold-start. After teaching the model to generate structured explanations prior to scoring, we then employ the Relative-Absolute Policy Optimization (RAPO), a novel RL algorithm that jointly optimizes absolute score regression and relative ranking order, improving both per-image accuracy and cross-image preference judgments. Aes-R1 enables MLLMs to generate grounded explanations alongside faithful scores, thereby enhancing aesthetic scoring and reasoning in a unified framework. Extensive experiments demonstrate that Aes-R1 improves the backbone's average PLCC/SRCC by 47.9%/34.8%, surpassing state-of-the-art baselines of similar size. More ablation studies validate Aes-R1's robust generalization under limited supervision and in out-of-distribution scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
