Towards Language-Based Modulation of Assistive Robots through Multimodal Models
Center for Information and Language Processing (CIS)（信息与语言处理中心（CIS））；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心（MCML））；Munich Institute of Robotics and Machine Intelligence (MIRMI)（慕尼黑机器人与机器智能研究所（MIRMI））；Technical University Munich (TUM)（慕尼黑工业大学（TUM））
Philipp Wicke, Lüfti Kerem Şenel, Shengqiang Zhang, Luis Figueredo, Abdeldjallil Naceri, Sami Haddadin, Hinrich Schütze
In the field of Geriatronics, enabling effective and transparent communication between humans and robots is crucial for enhancing the acceptance and performance of assistive robots. Our early-stage research project investigates the potential of language-based modulation as a means to improve human-robot interaction. We propose to explore real-time modulation during task execution, leveraging language cues, visual references, and multimodal inputs. By developing transparent and interpretable methods, we aim to enable robots to adapt and respond to language commands, enhancing their usability and flexibility. Through the exchange of insights and knowledge at the workshop, we seek to gather valuable feedback to advance our research and contribute to the development of interactive robotic systems for Geriatronics and beyond.
STREAMS: An Assistive Multimodal AI Framework for Empowering Biosignal Based Robotic Controls
Dept. of Electrical Engineering University of Rhode Island Kingston, RI, USA（电气工程系 罗德岛大学 罗得岛州金斯顿）；Computer Engineering Northeastern University Boston, Massachusetts, USA（计算机工程 美国东北大学 波士顿）
WheelArm-Sim: A Manipulation and Navigation Combined Multimodal Synthetic Data Generation Simulator for Unified Control in Assistive Robotics
Aerospace and Mechanical Engineering Department, Saint Louis University（圣路易斯大学航空航天与机械工程系）；Computer Science Department, Saint Louis University（圣路易斯大学计算机科学系）
HARMONI: Multimodal Personalization of Multi-User Human-Robot Interactions with LLMs
Institut Curie, Université Paris-Saclay（巴黎-萨克勒大学Curie研究所）；Institute of Intelligent Systems and Robotics (ISIR), Sorbonne University（索邦大学智能系统与机器人研究所）；Assistance Publique – Hôpitaux de Paris (AP-HP), Université Paris Cité（巴黎公共医院（AP-HP）与巴黎城市大学）
AnyUser: Translating Sketched User Intent into Domestic Robots
College of Computer Science and Technology, National University of Defense Technology（国防科技大学计算机科学与技术学院）；National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University（湖南大学国家机器人视觉感知与控制技术工程研究中心）
Bidirectional Intent Communication: A Role for Large Foundation Models
TU Chemnitz（开姆尼茨工业大学）；Robert Bosch GmbH（罗伯特·博世有限公司）；Technical University of Munich（慕尼黑工业大学）；Centre for Applied Autonomous Sensor Systems (AASS), Örebro University（厄勒布鲁大学应用自主传感系统中心）
Learning Multimodal AI Algorithms for Amplifying Limited User Input into High-dimensional Control Space
Department of Electrical, Computer, and Biomedical Engineering, University of Rhode Island（电气、计算机与生物医学工程系，罗德岛大学）；Department of Electrical and Computer Engineering, Northeastern University（电气与计算机工程系，东北大学）；Department of Neurology, Harvard Medical School（神经病学系，哈佛医学院）
EgoTraj: Real-World Egocentric Human Trajectory Dataset for Multimodal Prediction
Department of Civil, Architectural, and Environmental Engineering, The University of Texas at Austin（土木、建筑与环境工程系，德克萨斯大学奥斯汀分校）；Meta Reality Labs（Meta现实实验室）；School of Architecture, The University of Texas at Austin（建筑学院，德克萨斯大学奥斯汀分校）
