arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.06709cs.MMcs.AIcs.SDeess.AS
收藏

揭示视听定位基准中的视觉偏差

Unveiling Visual Biases in Audio-Visual Localization Benchmarks

Renmin University of China(中国人民大学)

Liangyu Chen, Zihao Yue, Boshen Xu, Qin Jin

AI总结:

该研究发现现有视听源定位(AVSL)基准存在视觉偏差问题,即发声物体易仅靠视觉线索识别,通过检验VGG-SS、EpicSounding-Object两个基准验证了仅视觉模型优于视听基线,指出现有基准需完善以促进视听学习。

摘要:

视听源定位(Audio-Visual Source Localization,AVSL)旨在定位视频中的声源。本文指出现有基准存在一个重要问题:发声物体往往仅靠视觉线索就能轻易识别,我们将其称为视觉偏差。这类偏差导致这些基准无法有效评估AVSL模型。为进一步验证关于视觉偏差的假设,我们检验了两个代表性AVSL基准——VGG-SS和EpicSounding-Object,发现仅视觉模型的表现优于所有视听基线模型。研究结果表明,现有AVSL基准需进一步完善以推动视听学习发展。

英文摘要:

Audio-Visual Source Localization (AVSL) aims to localize the source of sound within a video. In this paper, we identify a significant issue in existing benchmarks: the sounding objects are often easily recognized based solely on visual cues, which we refer to as visual bias. Such biases hinder these benchmarks from effectively evaluating AVSL models. To further validate our hypothesis regarding visual biases, we examine two representative AVSL benchmarks, VGG-SS and EpicSounding-Object, where the vision-only models outperform all audiovisual baselines. Our findings suggest that existing AVSL benchmarks need further refinement to facilitate audio-visual learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑