The audio-visual event localization task requires identifying concurrent visual and auditory events from unconstrained videos within a network model, locating them, and classifying their category. The efficient extraction and integration of audio and visual modal information have always been challenging in this field. In this paper, we introduce CACE-Net, which differs from most existing methods that solely use audio signals to guide visual information. We propose an audio-visual co-guidance attention mechanism that allows for adaptive bi-directional cross-modal attentional guidance between audio and visual information, thus reducing inconsistencies between modalities. Moreover, we have observed that existing methods have difficulty distinguishing between similar background and event and lack the fine-grained features for event classification. Consequently, we employ background-event contrast enhancement to increase the discrimination of fused feature and fine-tuned pre-trained model to extract more refined and discernible features from complex multimodal inputs. Specifically, we have enhanced the model's ability to discern subtle differences between event and background and improved the accuracy of event classification in our model. Experiments on the AVE dataset demonstrate that CACE-Net sets a new benchmark in the audio-visual event localization task, proving the effectiveness of our proposed methods in handling complex multimodal learning and event localization in unconstrained videos. Code is available at https://github.com/Brain-Cog-Lab/CACE-Net.
MultiSoundGen: Video-to-Audio Generation for Multi-Event Scenarios via SlowFast Contrastive Audio-Visual Pretraining and Direct Preference Optimization
Semantic Noise Reduction via Teacher-Guided Dual-Path Audio-Visual Representation Learning
Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所基础模型研究中心）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Objecteye Inc.（北京眼神科技有限公司）
Learning Audio-Visual Source Localization via False Negative Aware Contrastive Learning
Australian National University（澳大利亚国立大学）；OPPO Research Institute（OPPO研究院）；The University of Oxford（牛津大学）；Shanghai AI Lab（上海人工智能实验室）；Beihang University（北京航空航天大学）
Complementary and Contrastive Learning for Audio-Visual Segmentation
School of Information and Communication Engineering, Dalian University of Technology（信息与通信工程学院，大连理工大学）；School of Future Technology and the School of Artificial Intelligence, Dalian University of Technology（未来技术学院和人工智能学院，大连理工大学）
CCStereo: Audio-Visual Contextual and Contrastive Learning for Binaural Audio Generation
Australian Institute for Machine Learning, University of Adelaide（澳大利亚机器学习研究所，阿德莱德大学）；Sony Group Corporation（索尼集团）；Sony AI, Sony Group Corporation（索尼人工智能，索尼集团）
MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）；UT Austin（德克萨斯大学奥斯汀分校）；IBM Research AI（IBM研究院人工智能部门）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）；Goethe University Frankfurt（法兰克福大学）
Contrastive Conditional Latent Diffusion for Audio-visual Segmentation
School of Electronics and Information, Northwestern Polytechnical University（电子与信息学院，西北工业大学）；Shanghai AI Laboratory（上海人工智能实验室）；Shaanxi Key Laboratory of Information Acquisition and Processing（信息获取与处理陕西省重点实验室）
HiCMAE: Hierarchical Contrastive Masked Autoencoder for Self-Supervised Audio-Visual Emotion Recognition
School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Department of Automation, Tsinghua University（清华大学自动化系）；Beijing National Research Center for Information Science and Technology, Tsinghua University（清华大学北京信息科学与技术国家研究中心）
CAV-MAE Sync: Improving Contrastive Audio-Visual Mask Autoencoders via Fine-Grained Alignment
Goethe University of Frankfurt（法兰克福歌德大学）；MIT（麻省理工学院）；IBM Research（IBM研究院）；MIT-IBM Watson AI Lab（MIT-IBM沃森人工智能实验室）；Tuebingen AI Center/University of Tuebingen（图宾根人工智能中心/图宾根大学）
Unraveling Instance Associations: A Closer Look for Audio-Visual Segmentation
Australian Institute for Machine Learning, University of Adelaide（阿德莱德大学澳大利亚机器学习研究院）；St Vincent’s Hospital Melbourne（墨尔本圣文森特医院）；Centre for Vision, Speech and Signal Processing, University of Surrey（萨里大学视觉、语音与信号处理中心）
Speed Co-Augmentation for Unsupervised Audio-Visual Pre-training
The Chinese University of Hong Kong（香港中文大学）；University of Birmingham（伯明翰大学）；Fudan University（复旦大学）；Dyson Robot Learning Lab（戴森机器人学习实验室）；Tencent AI Lab（腾讯人工智能实验室）；The University of Hong Kong（香港大学）；UC Berkeley（加州大学伯克利分校）
