Marmot：通过多智能体推理实现对象级自校正
Marmot: Object-Level Self-Correction via Multi-Agent Reasoning
AI总结：
Marmot通过多智能体推理实现对象级自校正，提升图像文本对齐的准确性，解决多对象场景中计数、属性和空间关系的校正问题。
摘要：
尽管扩散模型在生成高质量图像方面表现出色，但它们在复杂多对象场景中准确计数、属性和空间关系方面常有困难。一种潜在的解决方案是利用多模态大语言模型（MLLM）作为AI代理来构建自校正框架。然而，这些方法严重依赖所用MLLM的能力，往往无法涵盖图像中的所有对象，并且在多轮编辑过程中会产生累积畸变。为了解决这些挑战，我们提出了Marmot，一种新颖且可推广的框架，利用多智能体推理进行多对象自校正，以增强图像-文本对齐。首先，我们使用大型语言模型作为对象感知代理，执行对象级的分而治之，根据图像描述自动将自校正任务分解为以对象为中心的子任务。对于每个子任务，我们构建一个对象校正系统，该系统包含决策-执行-验证机制，仅在单个对象的分割掩码或对象对的边界框上运行，有效减轻了对象间干扰并提高编辑可靠性。为了高效整合子任务的校正结果并避免多阶段编辑中的累积畸变，我们提出了一种像素域拼接平滑器，该平滑器采用掩码引导的两阶段潜在空间优化。这一创新使子任务能够并行处理，显著提高了运行时间效率，同时防止了畸变积累。大量实验表明，Marmot在图像生成任务中显著提高了对象计数、属性分配和空间关系的准确性。
英文摘要：
While diffusion models excel at generating high-quality images, they often struggle with accurate counting, attributes, and spatial relationships in complex multi-object scenes. One potential solution involves employing Multimodal Large Language Model (MLLM) as an AI agent to construct a self-correction framework. However, these approaches heavily rely on the capabilities of the MLLMs used, often fail to account for all objects within the image, and suffer from cumulative distortions during multi-round editing processes. To address these challenges, we propose Marmot, a novel and generalizable framework that leverages Multi-Agent Reasoning for Multi-Object Self-Correcting to enhance image-text alignment. First, we employ a large language model as an Object-Aware Agent to perform object-level divide-and-conquer, automatically decomposing self-correction tasks into object-centric subtasks based on image descriptions. For each subtask, we construct an Object Correction System featuring a decision-execution-verification mechanism that operates exclusively on a single object's segmentation mask or the bounding boxes of object pairs, effectively mitigating inter-object interference and enhancing editing reliability. To efficiently integrate correction results from subtasks while avoiding cumulative distortions from multi-stage editing, we propose a Pixel-Domain Stitching Smoother, which employs mask-guided two-stage latent space optimization. This innovation enables parallel processing of subtasks, significantly improving runtime efficiency while preventing distortion accumulation. Extensive experiments demonstrate that Marmot significantly improves accuracy in object counting, attribute assignment, and spatial relationships for image generation tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
