将多模态大语言模型知识整合到无模态补全中
Integrating Multimodal Large Language Model Knowledge into Amodal Completion
AI总结：
本文提出AmodalCG框架，利用多模态大语言模型知识指导无模态补全，通过评估遮挡程度选择性调用MLLM指导，结合视觉生成模型迭代优化补全结果，实验表明优于现有方法。
摘要：
本文提出AmodalCG框架，利用多模态大语言模型知识指导无模态补全，通过评估遮挡程度选择性调用MLLM指导，结合视觉生成模型迭代优化补全结果，实验表明优于现有方法。
英文摘要：
With the widespread adoption of autonomous vehicles and robotics, amodal completion, which reconstructs the occluded parts of people and objects in an image, has become increasingly crucial. Just as humans infer hidden regions based on prior experience and common sense, this task inherently requires physical knowledge about real-world entities. However, existing approaches either depend solely on the image generation ability of visual generative models, which lack such knowledge, or leverage it only during the segmentation stage, preventing it from explicitly guiding the completion process. To address this, we propose AmodalCG, a novel framework that harnesses the real-world knowledge of Multimodal Large Language Models (MLLMs) to guide amodal completion. Our framework first assesses the extent of occlusion to selectively invoke MLLM guidance only when the target object is heavily occluded. If guidance is required, the framework further incorporates MLLMs to reason about both the (1) extent and (2) content of the missing regions. Finally, a visual generative model integrates these guidance and iteratively refines imperfect completions that may arise from inaccurate MLLM guidance. Experimental results on various real-world images show impressive improvements compared to all existing works, suggesting MLLMs as a promising direction for addressing challenging amodal completion.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
