arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.02450cs.CV
收藏

先推理后检索:面向CoVR-R的结构化编辑提示与密集-稀疏融合

Reason-Then-Retrieve for CoVR-R with Structured Edit Prompts and Dense-Sparse Fusion

Beijing University of Posts and Telecommunications(北京邮电大学)

Dongqing Liu, Mengshi Qi, Hongwei Ji

AI总结:

针对CoVR-R任务,提出一种零样本的“先推理后检索”流水线,利用Qwen3.5-27B生成结构化描述和密集嵌入,并结合TF-IDF稀疏分支进行融合排序,在验证集和测试集上取得了领先性能。

摘要:

CoVR-R研究基于推理的组合视频检索:给定一个参考视频和一个编辑指令,系统必须检索满足编辑的目标视频。主要困难在于目标未被直接描述,必须从对象身份、动作顺序、最终状态、手部交互和场景转换的细粒度变化中推断。我们围绕Qwen3.5-27B构建了一个零样本的“先推理后检索”流水线。对于每个图库视频,模型通过池化生成令牌的隐藏状态(使用令牌相关权重)生成面向检索的结构化描述和密集嵌入。对于每个查询,模型首先对参考视频和指令进行编辑推理,然后生成目标视频描述,其隐藏状态作为查询嵌入。我们通过生成文本上的TF-IDF分支补充密集检索,并使用分割特定权重融合两个排序。在验证集上,当前最佳提交在R@1达到80.81,R@5达到94.86,R@10达到97.11,R@50达到98.59。在盲测集上,R@1达到89.73,R@5达到95.79,R@10达到96.63,R@50达到97.98。

英文摘要:

CoVR-R studies reason-aware composed video retrieval: given a reference video and an edit instruction, the system must retrieve the target video that satisfies the edit. The main difficulty is that the target is not described directly; it must be inferred from fine-grained changes in object identity, action order, final state, hand interaction, and scene transition. We build a zero-shot reason-then-retrieve pipeline around Qwen3.5-27B. For each gallery video, the model generates a retrieval-oriented structured description and a dense embedding by pooling generated-token hidden states with token-dependent weights. For each query, the model first performs edit reasoning over the reference video and instruction, then generates a target-video description whose hidden states serve as the query embedding. We complement dense retrieval with a TF-IDF branch over the generated texts and fuse the two rankings with split-specific weights. On validation, the current best submission reaches 80.81 at R@1, 94.86 at R@5, 97.11 at R@10, and 98.59 at R@50. On the blind test split, it reaches 89.73 at R@1, 95.79 at R@5, 96.63 at R@10, and 97.98 at R@50.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑