视频中时间句定位的偏置冲突样本合成与对抗性去除去偏策略
Bias-Conflict Sample Synthesis and Adversarial Removal Debias Strategy for Temporal Sentence Grounding in Video
AI总结：
本文针对视频时间句定位中的数据集偏置问题，提出BSSARD策略，通过动态生成偏置冲突样本并进行对抗训练，同时消除语言和视觉偏置，在多个数据集上验证了其去偏效果。
摘要：
视频中的时间句定位（TSGV）受到数据集偏置问题的困扰，该问题源于输入视频或查询文本中具有相似语义成分的样本其目标时刻在时间分布上不均匀。现有方法借助关于偏置的先验知识人为打破这种不均匀分布，但只能去除有限数量的显著语言偏置。本文提出偏置冲突样本合成与对抗性去除去偏策略（BSSARD），该方法通过显式利用单模态特征与目标时刻时间位置之间潜在的虚假相关性，动态生成偏置冲突样本。通过对抗训练，其偏置生成器不断引入偏置并生成偏置冲突样本来欺骗其定位模型。同时，定位模型不断消除所引入的偏置，这要求它对多模态对齐信息进行建模。BSSARD将覆盖大多数耦合关系，并同时破坏语言和视觉偏置。在Charades-CD和ActivityNet-CD上的大量实验证明了BSSARD良好的去偏能力。源代码可在https://github.com/qzhb/BSSARD获取。
英文摘要：
Temporal Sentence Grounding in Video (TSGV) is troubled by dataset bias issue, which is caused by the uneven temporal distribution of the target moments for samples with similar semantic components in input videos or query texts. Existing methods resort to utilizing prior knowledge about bias to artificially break this uneven distribution, which only removes a limited amount of significant language biases. In this work, we propose the bias-conflict sample synthesis and adversarial removal debias strategy (BSSARD), which dynamically generates bias-conflict samples by explicitly leveraging potentially spurious correlations between single-modality features and the temporal position of the target moments. Through adversarial training, its bias generators continuously introduce biases and generate bias-conflict samples to deceive its grounding model. Meanwhile, the grounding model continuously eliminates the introduced biases, which requires it to model multi-modality alignment information. BSSARD will cover most kinds of coupling relationships and disrupt language and visual biases simultaneously. Extensive experiments on Charades-CD and ActivityNet-CD demonstrate the promising debiasing capability of BSSARD. Source codes are available at https://github.com/qzhb/BSSARD.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
