Robotic systems for retail have gained a lot of attention due to the labor-intensive nature of such business environments. Many tasks have the potential to be automated via intelligent robotic systems that have manipulation capabilities. For example, empty shelves can be replenished, stray products can be picked up or new items can be delivered. However, many challenges make the realization of this vision a challenge. In particular, robots are still too expensive and do not work out of the box. In this paper, we discuss a work-in-progress approach for enabling power-on-and-go robots in retail environments through a combination of active, physical sensors and passive, artificial sensors. In particular, we use low-cost hardware sensors in conjunction with machine learning techniques in order to generate high-quality environmental information. More specifically, we present a setup in which a standard monocular camera and Bluetooth low-energy yield a reliable robot system that can immediately be used after placing a couple of sensors in the environment. The camera information is used to synthesize accurate 3D point clouds, whereas the BLE data is used to integrate the data into a complex map of the environment. The combination of active and passive sensing enables high-quality sensing capabilities at a fraction of the costs traditionally associated with such tasks.
MARLIN: A Cloud Integrated Robotic Solution to Support Intralogistics in Retail
Robotics Innovation Center of German Research Center for Artificial Intelligence GmbH (DFKI)(德国人工智能研究中心机器人创新中心);Institute for Artificial Intelligence of the University of Bremen(不来梅大学人工智能研究所);Robotics group of the University of Bremen(不来梅大学机器人组)
本文提出服务机器人 MARLIN 与零售 AI 云平台 K4R 集成,利用语义数字孪生增强感知、导航与任务规划,并通过新型铰接式牵引-拖车导航算法提升真实补货场景表现。
From Pixels to Shelf: An Integrated Robotic System for Autonomous Supermarket Stocking with a Mobile Manipulator
Department of Information Engineering, University of Padova(帕多瓦大学信息工程系);Division of Decision and Control Systems, School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology(皇家理工学院电气工程与计算机科学学院决策与控制系统系);Animum AB(Animum公司)
Assessing the Value of Visual Input: A Benchmark of Multimodal Large Language Models for Robotic Path Planning
Department of Micro-Nano Mechanical Science and Engineering, Nagoya University, Nagoya, Aichi, Japan(Nagoya大学微纳米机械科学与工程系);Institutes of Innovation for Future Society, Nagoya University, Nagoya, Aichi, Japan(Nagoya大学未来社会创新研究所)
RoboMP$^2$: A Robotic Multimodal Perception-Planning Framework with Multimodal Large Language Models
School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳)计算机科学与技术学院);School of Engineering, Great Bay University(大湾区大学工程学院);School of Computing and Information Technology, Great Bay University(大湾区大学计算与信息技术学院)
OpenRC: An Open-Source Robotic Colonoscopy Framework for Multimodal Data Acquisition and Autonomy Research
Walker Department of Mechanical Engineering, The University of Texas at Austin(德克萨斯大学奥斯汀分校沃克机械工程系);Department of Surgical Oncology, Division of Surgery, The University of Texas MD Anderson Cancer Center(德克萨斯大学MD安德森癌症中心外科肿瘤学系);School of Medicine and Health, Technical University of Munich(慕尼黑工业大学医学与健康学院)
M3ET: Efficient Vision-Language Learning for Robotics based on Multimodal Mamba-Enhanced Transformer
School of Software Northwestern Polytechnical University Xi'an, China(软件学院 西安理工大学 西安);Laboratoire L2Tl University Sorbonne Paris Nord Paris, France(L2Tl实验室 索邦巴黎北大学 巴黎)
Preventing Robotic Jailbreaking via Multimodal Domain Adaptation
University of Padova(帕多瓦大学);Stanford University(斯坦福大学);Carnegie Mellon University(卡内基梅隆大学);University of Pennsylvania(宾夕法尼亚大学);Örebro University(奥雷布罗大学);NVIDIA Research(NVIDIA研究)
A General One-Shot Multimodal Active Perception Framework for Robotic Manipulation: Learning to Predict Optimal Viewpoint
Institute of Robotics and Automatic Information Systems, College of Artificial Intelligence, Nankai University(机器人与自动信息系统研究所,人工智能学院,南开大学);Tianjin Key Laboratory of Intelligent Robotics, Nankai University(智能机器人重点实验室,南开大学);College of Artificial Intelligence, Nankai University(人工智能学院,南开大学);Department of Computing, The Hong Kong Polytechnic University(computing 部门,香港理工大学)
BronchoCopilot: Towards Autonomous Robotic Bronchoscopy via Multimodal Reinforcement Learning
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
WheelArm-Sim: A Manipulation and Navigation Combined Multimodal Synthetic Data Generation Simulator for Unified Control in Assistive Robotics
Aerospace and Mechanical Engineering Department, Saint Louis University(圣路易斯大学航空航天与机械工程系);Computer Science Department, Saint Louis University(圣路易斯大学计算机科学系)
AssemLM: A Spatial Reasoning Multimodal Large Language Model for Robotic Assembly
Fudan University(复旦大学);Institute of Artificial Intelligence (TeleAI), China Telecom(人工智能研究所(TeleAI),中国电信);Tianjin University(天津大学);Northwestern Polytechnical University(西北工业大学);Tsinghua University(清华大学);City University of Hong Kong(香港城市大学)
STREAMS: An Assistive Multimodal AI Framework for Empowering Biosignal Based Robotic Controls
Dept. of Electrical Engineering University of Rhode Island Kingston, RI, USA(电气工程系 罗德岛大学 罗得岛州金斯顿);Computer Engineering Northeastern University Boston, Massachusetts, USA(计算机工程 美国东北大学 波士顿)
MLA: A Multisensory Language-Action Model for Multimodal Understanding and Forecasting in Robotic Manipulation
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(多媒体信息处理国家重点实验室,计算机科学学院,北京大学);Beijing Innovation Center of Humanoid Robotics (X-Humanoid)(北京类人机器人创新中心(X-Humanoid));The Chinese University of Hong Kong (CUHK)(香港中文大学(CUHK))
REASSEMBLE: A Multimodal Dataset for Contact-rich Robotic Assembly and Disassembly
Autonomous Systems Lab, Institute of Computer Technology, TU Wien(自主系统实验室,计算机技术学院,维也纳技术大学);Institute of Robotics and Mechatronics (DLR), German Aerospace Center(机器人与机电系统研究所(DLR),德国航空航天中心)
Learning Physics from Pretrained Video Models: A Multimodal Continuous and Sequential World Interaction Models for Robotic Manipulation
Sun Yat-sen University(中山大学);Guangdong Key Laboratory of Big Data Analysis(广东大数据分析与处理重点实验室);X-Era AI Lab(X-Era人工智能实验室);Guangdong University of Technology(广东工业大学)
Robo-MUTUAL: Robotic Multimodal Task Specification via Unimodal Learning
Institute for AI Industry Research (AIR), Tsinghua University(清华大学人工智能产业研究院(AIR));College of Engineering, Peking University(北京大学工学院);Sensetime Research(商汤科技研究院);University of Toronto(多伦多大学);University College London(伦敦大学学院);Shanghai AI Lab(上海人工智能实验室)
M2R2: MultiModal Robotic Representation for Temporal Action Segmentation
Autonomous Systems Lab, Technische Universität Wien (TU Wien)(自主系统实验室,维也纳技术大学);Institute of Robotics and Mechatronics, German Aerospace Center (DLR)(机器人与机电研究所,德国航空航天中心)
Advantages of Multimodal versus Verbal-Only Robot-to-Human Communication with an Anthropomorphic Robotic Mock Driver
Centre for Applied Autonomous Sensor Systems (AASS)(应用自主传感器系统中心(AASS));Örebro University(厄勒布鲁大学);Robert Bosch GmbH(罗伯特·博世有限公司);University of Skövde(舍夫德大学);TU Munich(慕尼黑工业大学)
RoboAfford++: A Generative AI-Enhanced Dataset for Multimodal Affordance Learning in Robotic Manipulation and Navigation
Xiaomi EV(小米电动车);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学 Gallagher人工智能学院);School of Computer Science and Information Engineering, Hefei University of Technology(合肥工业大学计算机科学与信息工程学院)
RoboCodeX: Multimodal Code Generation for Robotic Behavior Synthesis
The University of Hong Kong(香港大学);Shanghai AI Laboratory(上海人工智能实验室);ETH Zurich(苏黎世联邦理工学院);Shanghai Jiao Tong University(上海交通大学);The Chinese University of Hong Kong(香港中文大学);Tsinghua University(清华大学)