Understanding Large-Language Model (LLM)-powered Human-Robot Interaction
University of Wisconsin–Madison(威斯康星大学麦迪逊分校)
Callie Y. Kim, Christine P. Lee, Bilge Mutlu
备注:
10 pages, 4 figures. Callie Y. Kim and Christine P. Lee contributed equally to the work. To be published in Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction (HRI '24), March 11--14, 2024, Boulder, CO, USA
Large-language models (LLMs) hold significant promise in improving human-robot interaction, offering advanced conversational skills and versatility in managing diverse, open-ended user requests in various tasks and domains. Despite the potential to transform human-robot interaction, very little is known about the distinctive design requirements for utilizing LLMs in robots, which may differ from text and voice interaction and vary by task and context. To better understand these requirements, we conducted a user study (n = 32) comparing an LLM-powered social robot against text- and voice-based agents, analyzing task-based requirements in conversational tasks, including choose, generate, execute, and negotiate. Our findings show that LLM-powered robots elevate expectations for sophisticated non-verbal cues and excel in connection-building and deliberation, but fall short in logical communication and may induce anxiety. We provide design implications both for robots integrating LLMs and for fine-tuning LLMs for use with robots.
Towards Emotion Co-regulation with LLM-powered Socially Assistive Robots: Integrating LLM Prompts and Robotic Behaviors to Support Parent-Neurodivergent Child Dyads
Department of Industrial Design, Eindhoven University of Technology(工业设计系,埃因霍温理工大学);Department of Biomedical Engineering, Eindhoven University of Technology(生物医学工程系,埃因霍温理工大学);Department of Engineering, Institute of Science Tokyo(工程系,东京科学研究所);Department of Engineering, Utrecht University(工程系,乌得勒支大学)
Engagement and Disclosures in LLM-Powered Cognitive Behavioral Therapy Exercises: A Factorial Design Comparing the Influence of a Robot vs. Chatbot Over Time
Would you let a humanoid play storytelling with your child? A usability study on LLM-powered narrative Human-Robot Interaction
Humanoid Sensing and Perception, Italian Institute of Technology (IIT)(人形感知与感知,意大利技术研究院);Social Cognition in Human-Robot Interaction, IIT(人机交互中的社会认知,IIT)
Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);ETH Zurich(苏黎世联邦理工学院);Max Planck and ETH Center for Learning Systems(马克斯·普朗克-苏黎世联邦理工学院学习系统中心);Koç University(科奇大学)
Magnetic resonance imaging (MRI)-guided robotic systems offer great potential for new minimally invasive medical tools, including MRI-powered miniature robots. By re-purposing the imaging hardware of an MRI scanner, the magnetic miniature robot could be navigated into the remote part of the patient's body without needing tethered endoscopic tools. However, the state-of-art MRI-powered magnetic miniature robots have limited functionality besides navigation. Here, we propose an MRI-powered magnetic miniature capsule robot benefiting from acoustic streaming forces generated by MRI-guided high-int…
Artificial and Mechanical Intelligence, Istituto Italiano di Tecnologia(人工智能与机械智能,意大利技术研究院);School of Computer Science, University of Manchester(计算机科学学院,曼彻斯特大学);Department of Industrial Engineering, University of Naples Federico II(工业工程系,那不勒斯费德里克二世大学)
Robots have inherently limited onboard processing, storage, and power capabilities. Cloud computing resources have the potential to provide significant advantages for robots in many applications. However, to make use of these resources, frameworks must be developed that facilitate robot interactions with cloud services. In this paper, we propose a cloud-based architecture called Smart Cloud that intends to overcome the physical limitations of single- or multi-robot systems through massively parallel computation, provided on demand by cloud services. Smart Cloud is implemented on Amazon Web Ser…
Unified Multi-Rate Model Predictive Control for a Jet-Powered Humanoid Robot
Artificial and Mechanical Intelligence, Istituto Italiano di Tecnologia(人工智能与机械智能研究所);School of Computer Science, University of Manchester(曼彻斯特大学计算机科学学院)
Manual spraying of pesticides and herbicides to crops and weed inhibitors onto the field are quite laborious work to humans. Manual trimming of selected unwanted plants or harvested crops from the field is also difficult. Our project proposes a multipurpose solar powered, flexible, Remote Controlled, semi-automated spraying robot with 4 Degrees of Freedom (DoF) in spatial movement, with an additional plant mowing equipment. The robot is designed to spray pesticide/insecticide directly onto individual lesions minimizing wastage or excess chemical spraying, hence making the system cost effective…
Delft University of Technology(代尔夫特理工大学);Prince Sultan University(苏丹王子大学)
This paper introduces a compound vision system that enables robots to localize people up to 15m away using a cheap camera. And, it proposes a robust navigation stack that combines Deep Reinforcement Learning (DRL) and a probabilistic localization method. To test the efficacy of these systems, we prototyped a low-cost mobile robot that we call Covy. Covy can be used for applications such as promoting social distancing during pandemics or estimating the density of a crowd. We evaluated Covy's performance through extensive sets of experiments both in simulated and realistic environments. Our resu…
Even though artificial muscles have gained popularity due to their compliant, flexible, and compact properties, there currently does not exist an easy way of making informed decisions on the appropriate actuation strategy when designing a muscle-powered robot; thus limiting the transition of such technologies into broader applications. What's more, when a new muscle actuation technology is developed, it is difficult to compare it against existing robot muscles. To accelerate the development of artificial muscle applications, we propose a data driven approach for robot muscle actuator selection…
The arrangement of an in-pipe climbing robot that works using a sharp transmission part to explore complex relationship of lines. Standard wheeled/continued in-pipe climbing robots are leaned to slip and take while researching in pipe turns. The instrument helps in achieving the really unavoidable consequence of getting out slip and drag in the robot tracks during progression. The proposed transmission likes the useful uttermost scopes of the standard two-yield transmission, which is fostered the fundamental time for a transmission with three outcomes. The instrument decisively changes the tra…
SwiftBot: A Decentralized Platform for LLM-Powered Federated Robotic Task Execution
California State University, Long Beach, USA(加州州立大学长滩分校);Case Western Reserve University, USA(凯斯西储大学);University of Colorado Denver, USA(科罗拉多大学丹佛分校);Montana State University, USA(蒙塔那州立大学);University of Houston Clear Lake, USA(休斯顿大学clear lake分校)
We present the design of an insect-sized jumping microrobot measuring 17mm$\times$6mm$\times$14mm and weighing 75 milligrams. The microrobot consumes 6.4mW of power to jump up by 8mm in height. The tethered version of the robot can jump 6 times per minute each time landing perfectly on its feet. The untethered version of the robot is powered using onboard photovoltaic cells illuminated by an external infrared laser source. It is, to the best of our knowledge, the lightest untethered jumping microrobot with onboard power source that has been reported yet.
SoK: Security and Privacy of Foundation-Model-Powered Robots
College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院);School of Cyber Science and Engineering, Wuhan University(武汉大学网络空间安全学院)
One of the key challenges in the collaboration within heterogeneous multi-robot systems is the optimization of the amount and type of data to be shared between robots with different sensing capabilities and computational resources. In this paper, we present a novel approach to managing collaboration terms in heterogeneous multi-robot systems with blockchain technology. Leveraging the extensive research of consensus algorithms in the blockchain domain, we exploit key technologies in this field to be integrated for consensus in robotic systems. We propose the utilization of proof of work systems…
DaDu-Corki: Algorithm-Architecture Co-Design for Embodied AI-powered Robotic Manipulation
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Shenzhen Institute of Artificial Intelligence and Robotics for Society(深圳人工智能与机器人研究院);Meituan(美团);Tianjin University(天津大学)