从小处起步，从大处着眼：面向视觉定位的基于课程学习的相对策略优化
Start Small, Think Big: Curriculum-based Relative Policy Optimization for Visual Grounding
AI总结：
针对强化学习微调CoT会降低视觉定位性能、数据规模增大不一定提升效果的问题，提出CuRPO课程训练策略，在多个基准数据集上取得更优性能，少样本场景下也表现出色。
摘要：
思维链（Chain-of-Thought, CoT）提示近期通过显式生成中间推理步骤，在各类自然语言处理与计算机视觉任务中展现出显著潜力。但我们发现，基于强化学习（RL）微调的CoT推理反而会在视觉定位任务中造成性能下降，当CoT输出冗长或复杂时该问题尤为突出。此外我们的分析表明，受数据复杂度差异影响，增大数据集规模并不总能提升性能。基于上述发现，我们提出基于课程学习的相对策略优化（Curriculum-based Relative Policy Optimization, CuRPO），这一新型训练策略将CoT长度与广义交并比（generalized Intersection over Union, gIoU）奖励作为复杂度指标，按照从简单样本到困难样本的顺序渐进组织训练数据。我们在RefCOCO、RefCOCO+、RefCOCOg和LISA数据集上开展了大量实验验证方法有效性，CuRPO性能持续优于包括Visual-RFT在内的现有方法，在RefCOCO上最高实现了+12.52 mAP的显著提升。此外CuRPO具备优异的效率与鲁棒性，即便在少样本学习场景下也能取得出色的定位性能，尤其适用于文本描述模糊复杂的任务。代码已发布于https://github.com/qyoung-yan/CuRPO。
英文摘要：
Chain-of-Thought (CoT) prompting has recently shown significant promise across various NLP and computer vision tasks by explicitly generating intermediate reasoning steps. However, we find that reinforcement learning (RL)-based fine-tuned CoT reasoning can paradoxically degrade performance in Visual Grounding tasks, particularly as CoT outputs become lengthy or complex. Additionally, our analysis reveals that increased dataset size does not always enhance performance due to varying data complexities. Motivated by these findings, we propose Curriculum-based Relative Policy Optimization (CuRPO), a novel training strategy that leverages CoT length and generalized Intersection over Union (gIoU) rewards as complexity indicators to progressively structure training data from simpler to more challenging examples. Extensive experiments on RefCOCO, RefCOCO+, RefCOCOg, and LISA datasets demonstrate the effectiveness of our approach. CuRPO consistently outperforms existing methods, including Visual-RFT, with notable improvements of up to +12.52 mAP on RefCOCO. Moreover, CuRPO exhibits exceptional efficiency and robustness, delivering strong localization performance even in few-shot learning scenarios, particularly benefiting tasks characterized by ambiguous and intricate textual descriptions.The code is released on https://github.com/qyoung-yan/CuRPO.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
