自上而下语义细化用于图像描述生成
Top-Down Semantic Refinement for Image Captioning
AI总结：
自上而下语义细化通过高效MCTS算法提升图像描述生成的性能与质量
摘要：
大型视觉-语言模型（VLMs）在图像描述生成中面临一个固有的矛盾：它们强大的单步生成能力往往导致短视的决策过程。这使得在捕捉丰富细节的同时保持全局叙述连贯性变得困难，尤其是在需要多步骤和复杂场景描述的任务中。为了解决这一根本性挑战，我们将图像描述定义为一个以目标为导向的分层细化规划问题，并进一步提出一种新的框架，称为自上而下语义细化（TDSR），它将生成过程建模为马尔可夫决策过程（MDP）。然而，在VLM的广阔状态空间中进行规划存在显著的计算障碍。因此，我们的核心贡献是设计了一种针对VLMs的高效蒙特卡洛树搜索（MCTS）算法。通过引入视觉引导的并行扩展和轻量级价值网络，我们的TDSR将调用昂贵的VLM的频率减少了一个数量级，而不会牺牲规划质量。此外，一个自适应的提前停止机制动态地将计算开销与图像的复杂性相匹配。在多个基准测试上进行的大量实验表明，我们的TDSR作为插件模块，可以显著提升现有VLMs（如LLaVA-1.5、Qwen2.5-VL）的性能，在细粒度描述、组合泛化和幻觉抑制方面取得最先进的或高度竞争性的结果。
英文摘要：
Large Vision-Language Models (VLMs) face an inherent contradiction in image captioning: their powerful single-step generation capabilities often lead to a myopic decision-making process. This makes it difficult to maintain global narrative coherence while capturing rich details, a limitation that is particularly pronounced in tasks that require multi-step and complex scene description. To overcome this fundamental challenge, we redefine image captioning as a goal-oriented hierarchical refinement planning problem, and further propose a novel framework, named Top-Down Semantic Refinement (TDSR), which models the generation process as a Markov Decision Process (MDP). However, planning within the vast state space of a VLM presents a significant computational hurdle. Our core contribution, therefore, is the design of a highly efficient Monte Carlo Tree Search (MCTS) algorithm tailored for VLMs. By incorporating a visual-guided parallel expansion and a lightweight value network, our TDSR reduces the call frequency to the expensive VLM by an order of magnitude without sacrificing planning quality. Furthermore, an adaptive early stopping mechanism dynamically matches computational overhead to the image's complexity. Extensive experiments on multiple benchmarks, including DetailCaps, COMPOSITIONCAP, and POPE, demonstrate that our TDSR, as a plug-and-play module, can significantly enhance the performance of existing VLMs (e.g., LLaVA-1.5, Qwen2.5-VL) by achieving state-of-the-art or highly competitive results in fine-grained description, compositional generalization, and hallucination suppression.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
