ReasonX: 基于多模态大语言模型的内在图像分解
ReasonX: MLLM-Guided Intrinsic Image Decomposition
AI总结：
ReasonX通过多模态大语言模型提供相对比较监督，提升内在图像分解的泛化能力与精度。
摘要：
内在图像分解旨在将图像分解为物理成分，如反光率、深度、法线和光照。尽管最近基于扩散和变换器的模型受益于合成数据集的配对监督，但它们在多样、真实世界场景中的泛化能力仍面临挑战。我们提出了ReasonX，一种新的框架，该框架利用多模态大语言模型（MLLM）作为感知法官，提供相对的内在比较，并将这些比较作为GRPO奖励，用于在无标签的真实世界图像上微调内在分解模型。与生成模型的强化学习方法不同，我们的框架通过奖励法官的相对评估与模型输出的分析关系之间的协议来对齐条件内在预测器。ReasonX是模型无关的，可以应用于不同的内在预测器。在多个基础架构和模态上，ReasonX产生了显著的改进，包括在IIW反光率上9-25%的WHDR减少，以及在ETH3D上的深度准确性提升高达46%，这展示了基于MLLM的相对监督在连接低级和高级视觉推理方面的潜力。
英文摘要：
Intrinsic image decomposition aims to separate images into physical components such as albedo, depth, normals, and illumination. While recent diffusion- and transformer-based models benefit from paired supervision from synthetic datasets, their generalization to diverse, real-world scenarios remains challenging. We propose ReasonX, a novel framework that leverages a multimodal large language model (MLLM) as a perceptual judge providing relative intrinsic comparisons, and uses these comparisons as GRPO rewards for fine-tuning intrinsic decomposition models on unlabeled, in-the-wild images. Unlike RL methods for generative models, our framework aligns conditional intrinsic predictors by rewarding agreement between the judge's relational assessments and analytically derived relations from the model's outputs. ReasonX is model-agnostic and can be applied to different intrinsic predictors. Across multiple base architectures and modalities, ReasonX yields significant improvements, including 9-25% WHDR reduction on IIW albedo and up to 46% depth accuracy gains on ETH3D, highlighting the promise of MLLM-guided comparative supervision to bridge low- and high-level vision reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
