We discuss an attentional model for simultaneous object tracking and recognition that is driven by gaze data. Motivated by theories of perception, the model consists of two interacting pathways: identity and control, intended to mirror the what and where pathways in neuroscience models. The identity pathway models object appearance and performs classification using deep (factored)-Restricted Boltzmann Machines. At each point in time the observations consist of foveated images, with decaying resolution toward the periphery of the gaze. The control pathway models the location, orientation, scale and speed of the attended object. The posterior distribution of these states is estimated with particle filtering. Deeper in the control pathway, we encounter an attentional mechanism that learns to select gazes so as to minimize tracking uncertainty. Unlike in our previous work, we introduce gaze selection strategies which operate in the presence of partial information and on a continuous action space. We show that a straightforward extension of the existing approach to the partial information setting results in poor performance, and we propose an alternative method based on modeling the reward surface as a Gaussian Process. This approach gives good performance in the presence of partial information and allows us to expand the action space from a small, discrete set of fixation points to a continuous domain.
Attend, Adapt and Transfer: Attentive Deep Architecture for Adaptive Transfer from multiple sources in the same domain
University of Michigan(密歇根大学);Indian Institute of Technology Madras(印度理工学院马德拉斯分校);McGill University(麦吉尔大学)
在解决新的目标任务时,从先前的源任务转移知识在若干学习应用中可能是有用的。实施转让带来了两个尚未得到充分解决的严重挑战。首先,智能体应该能够避免负迁移,即当迁移阻碍或减缓学习而不是帮助学习时,就会发生负迁移。其次,智能体应该能够选择性迁移,即能够从不同的和多个源任务中选择和迁移目标任务状态空间的不同部分。我们提出了A2T(Attend,Adapt and Transfer),这是一种专注的深度架构,可以从这些源任务进行调整和转移。我们的模型是通用的,足以影响政策或价值函数的转移。对不同学习算法的实证评估表明,A2T是一种有效的迁移架构,能够避免负迁移,同时选择性地从同一领域的多个源任务迁移。
WASE: Learning When to Attend for Speaker Extraction in Cocktail Party Environments
Institute of Automation, Chinese Academy of Sciences (CASIA)(中国科学院自动化研究所);School of Future Technology, University of Chinese Academy of Sciences(中国科学院大学未来技术学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Center for Excellence in Brain Science and Intelligence Technology, CAS(中国科学院脑科学与智能技术卓越创新中心)
A Generative User Simulator with GPT-based Architecture and Goal State Tracking for Reinforced Multi-Domain Dialog Systems
Tsinghua University(清华大学);Tsinghua University-China Mobile Communications Group Co., Ltd. Joint Institute(清华大学-中国移动通信集团有限公司联合研究院);China Mobile Research Institute(中国移动研究院)
GOT-JEPA: Generic Object Tracking with Model Adaptation and Occlusion Handling using Joint-Embedding Predictive Architecture
Department of Computer Science, National Yang Ming Chiao Tung University(国家阳明交通大学计算机科学系);Research Center for Information Technology Innovation, Academia Sinica(中华学术院信息技术创新研究中心);Microsoft AI(微软人工智能)