We examine the problem of joint top-down active search of multiple objects under interaction, e.g., person riding a bicycle, cups held by the table, etc.. Such objects under interaction often can provide contextual cues to each other to facilitate more efficient search. By treating each detector as an agent, we present the first collaborative multi-agent deep reinforcement learning algorithm to learn the optimal policy for joint active object localization, which effectively exploits such beneficial contextual information. We learn inter-agent communication through cross connections with gates between the Q-networks, which is facilitated by a novel multi-agent deep Q-learning algorithm with joint exploitation sampling. We verify our proposed method on multiple object detection benchmarks. Not only does our model help to improve the performance of state-of-the-art active localization models, it also reveals interesting co-detection patterns that are intuitively interpretable.
Nonnegative-Constrained Joint Collaborative Representation with Union Dictionary for Hyperspectral Anomaly Detection
Institute of Advanced Research in Artificial Intelligence (IARAI)(高级人工智能研究所(IARAI));Helmholtz-Zentrum Dresden-Rossendorf(亥姆霍兹德累斯顿罗森多夫研究中心);Helmholtz Institute Freiberg for Resource Technology(亥姆霍兹弗赖贝格资源技术研究所)
指称表达式理解和切分是两个高度相关的任务,它们都是为了根据自然语言表达式识别指称对象。在本文中,我们提出了一种新的多任务协作网络(MCN),以实现联合学习的REC和RES的第一次。在MCN中,RES可以帮助REC实现更好的语言-视觉对齐,而REC可以帮助RES更好地定位所指对象。此外,我们解决了这种多任务设置中的一个关键挑战,即,预测冲突,两个创新的设计,即,一致性能量最大化(CEM)和自适应软非定位抑制(ASNLS)。具体来说,CEM使REC和RES能够通过最大化两个任务之间的一致性能量来关注相似的视觉区域。ASNLS基于REC的预测来抑制RES中不相关区域的响应。为了验证我们的模型,我们在REC和RES的三个基准数据集上进行了广泛的实验,即,RefCOCO、RefCOCO+和RefCOCOg。实验结果报告了MCN在所有现有方法上的显着性能增益,即,最高可达+7.13%for REC和+11.50%for RES over SOTA,这充分证实了我们的模型对于REC和RES联合学习的有效性。
Exposing and Defending the Achilles' Heel of Video Mixture-of-Experts
Nanjing University(南京大学);China Mobile Information Technology Co., Ltd.(中国移动信息科技有限公司);Nanjing University of Information Science and Technology(南京信息科技大学)
Collaborative Temporal Feature Generation via Critic-Free Reinforcement Learning for Cross-User Sensor-Based Activity Recognition
School of Artificial Intelligence (School of Future Technology), Nanjing University of Information Science and Technology(信息科学与技术大学人工智能学院);School of Cyberspace Security, Nanjing University of Information Science and Technology(信息科学与技术大学网络空间安全学院);Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University(南京大学附属医院金陵医院);Department of Electrical, Computer, and Software Engineering, The University of Auckland(奥克兰大学电气、计算机和软件工程系)
VideoChat-M1: Collaborative Policy Planning for Video Understanding via Multi-Agent Reinforcement Learning
Shenzhen Key Lab of Computer Vision and Pattern Recognition(深圳计算机视觉与模式识别重点实验室);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);VIVO AI Lab(VIVO人工智能实验室);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Shenzhen Campus of Sun Yat-sen University(孙逸仙大学深圳校区);Shanghai Jiao Tong University(上海交通大学);Institute for AI Industry Research (AIR), Tsinghua University(清华大学人工智能产业研究院);Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University(清华大学计算机科学与技术系,人工智能研究院)
R&D department ALTECA(阿尔泰卡公司研发部门);LIPN - CNRS UMR 7030 Sorbonne Paris Nord University(巴黎第十三大学(索邦巴黎北)法国国家科学研究中心UMR 7030实验室);ETIS - CNRS UMR 8051 CY Cergy Paris University(CY塞尔吉-巴黎大学法国国家科学研究中心UMR 8051实验室)