Over the years various methods have been proposed for the problem of object detection. Recently, we have witnessed great strides in this domain owing to the emergence of powerful deep neural networks. However, there are typically two main assumptions common among these approaches. First, the model is trained on a fixed training set and is evaluated on a pre-recorded test set. Second, the model is kept frozen after the training phase, so no further updates are performed after the training is finished. These two assumptions limit the applicability of these methods to real-world settings. In this paper, we propose Interactron, a method for adaptive object detection in an interactive setting, where the goal is to perform object detection in images observed by an embodied agent navigating in different environments. Our idea is to continue training during inference and adapt the model at test time without any explicit supervision via interacting with the environment. Our adaptive object detection model provides a 7.2 point improvement in AP (and 12.7 points in AP50) over DETR, a recent, high-performance object detector. Moreover, we show that our object detection model adapts to environments with completely different appearance characteristics, and performs well in them. The code is available at: https://github.com/allenai/interactron .
SegmATRon: Embodied Adaptive Semantic Segmentation for Indoor Environment
Moscow Institute of Physics and Technology（莫斯科物理技术学院）；Artificial Intelligence Research Institute（人工智能研究院）；Federal Research Center for Computer Science and Control of Russian Academy of Sciences（俄罗斯科学院计算机科学与控制联邦研究中心）
AdaOcc: Adaptive 3D Occupancy Prediction for Embodied Tasks
Beihang University（北京航空航天大学）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；XYZ Embodied AI（XYZ具身智能公司）；Hebei University of Technology（河北工业大学）；ShanghaiTech University（上海科技大学）；University of Science and Technology Beijing（北京科技大学）
Reinforced Embodied Active Defense: Exploiting Adaptive Interaction for Robust Visual Perception in Adversarial 3D Environments
Dept. of Comp. Sci. & Tech.（计算机科学与技术系）；Institute for AI（人工智能研究院）；BNRist Center（BNRist中心）；THBI Lab（THBI实验室）；Tsinghua-Bosch Joint Center for ML（清华大学-博世联合机器学习中心）；Tsinghua University（清华大学）
Department of Computer Science and Technology, Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing（计算机科学与技术系，北京信息科学与技术国家研究中心，清华大学，北京）
CP-uniGuard: A Unified, Probability-Agnostic, and Adaptive Framework for Malicious Agent Detection and Defense in Multi-Agent Embodied Perception Systems
Hong Kong JC STEM Lab of Smart City and Department of Computer Science, City University of Hong Kong（香港JC STEM实验室及城市大学计算机科学系）；School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；Department of Electrical and Electronic Engineering, The University of Hong Kong（香港大学电子与电气工程系）；School of Data Science, Lingnan University（岭南大学数据科学学院）
pFedNavi: Structure-Aware Personalized Federated Vision-Language Navigation for Embodied AI
Openmind (Wuhu) Intelligent Robot Co., Ltd.（Openmind（芜湖）智能机器人有限公司）；Shanghai Key Laboratory of Scalable Computing and Systems（上海可扩展计算与系统重点实验室）；National Natural Science Foundation of China（国家自然科学基金委员会）；US National Science Foundation（美国国家科学基金会）
Beihang University（北航）；City University of Macau（澳门城市大学）；Peking University（北京大学）；Mohamed bin Zayed University of Artificial Intelligence（马尔代夫 bin Zayed 人工智能大学）；Beijing Normal University（北京师范大学）
