CLEVER: Stream-based Active Learning for Robust Semantic Perception from Human Instructions
Institute of Robotics and Mechatronics, German Aerospace Center (DLR)(机器人与机电研究所,德国航空航天中心);Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology(人机学与机器人研究所,卡尔斯鲁厄技术大学)
Jongseok Lee, Timo Birr, Rudolph Triebel, Tamim Asfour
We propose CLEVER, an active learning system for robust semantic perception with Deep Neural Networks (DNNs). For data arriving in streams, our system seeks human support when encountering failures and adapts DNNs online based on human instructions. In this way, CLEVER can eventually accomplish the given semantic perception tasks. Our main contribution is the design of a system that meets several desiderata of realizing the aforementioned capabilities. The key enabler herein is our Bayesian formulation that encodes domain knowledge through priors. Empirically, we not only motivate CLEVER's design but further demonstrate its capabilities with a user validation study as well as experiments on humanoid and deformable objects. To our knowledge, we are the first to realize stream-based active learning on a real robot, providing evidence that the robustness of the DNN-based semantic perception can be improved in practice. The project website can be accessed at https://sites.google.com/view/thecleversystem.
ST-Booster: An Iterative SpatioTemporal Perception Booster for Vision-and-Language Navigation in Continuous Environments
Robotics and Control Laboratory, the School of Advanced Manufacturing and Robotics, and the State Key Laboratory of Turbulence and Complex Systems, Peking University(机器人与控制实验室、先进制造与机器人学院、湍流与复杂系统国家重点实验室,北京大学);Defense Innovation Institute, Academy of Military Sciences(国防创新研究院,军事科学院);Tianjin Artificial Intelligence Innovation Center(天津人工智能创新中心);Harbin Institute of Technology(哈尔滨工业大学)
OpenMap: Instruction Grounding via Open-Vocabulary Visual-Language Mapping
School of Software, Tsinghua University(清华大学软件学院);School of computer science and engineering, Central South University(中南大学计算机科学与工程学院);Inspur Yunzhou Industrial Internet Co., Ltd(Inspur Yunzhou工业互联网有限公司);Tsinghua University(清华大学)
Memory-Maze: Scenario Driven Visual Language Navigation Benchmark for Guiding Blind People
Waseda University(早稻田大学);Miraikan - The National Museum of Emerging Science and Innovation(Mirai Kan - 国家新兴科学与创新博物馆);Carnegie Mellon University(卡内基梅隆大学)
Look, Zoom, Understand: The Robotic Eyeball for Embodied Perception
School of Artificial Intelligence, Shanghai Jiao Tong University(上海交通大学人工智能学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Dalian University of Technology(大连理工大学)