Amir Aly
This study proposes different approaches for static and dynamic gesture analysis and imitation with the social robot Nao
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
With an increasing need for elderly and disability care, there is an increasing opportunity for intelligent and mobile devices such as robots to provide care and support solutions. In order to naturally assist and interact with humans, a robot must possess effective conversational capabilities. Gestures accompanying spoken sentences are an important factor in human-to-human conversational communication. Humanoid robots must also use gestures if they are to be capable of the rich interactions implied and afforded by their humanlike appearance. However, present systems for gesture generation do …
Bio-inspired sensorimotor control systems may be appealing to roboticists who try to solve problems of multiDOF humanoids and human-robot interactions. This paper presents a simple posture control concept from neuroscience, called disturbance estimation and compensation, DEC concept [1]. It provides human-like mechanical compliance due to low loop gain, tolerance of time delays, and automatic adjustment to changes in external disturbance scenarios. Its outstanding feature is that it uses feedback of multisensory disturbance estimates rather than 'raw' sensory signals for disturbance compensati…
Gesture Generation from Trimodal Context for Humanoid Robots
本文复现并扩展 Yoon 等人的三模态协同语音手势生成方法，将仿真动作迁移到人形机器人，并通过 FGD、motion variance 与人类评估验证其自然性、语音相关性、风格多样性及偏好差异。
Imitation of Arm Gestures by the Semi-Humanoid Robot NICO
研究半人形机器人NICO模仿人类手臂手势的方法，基于解析几何和预训练模型，利用MediaPipe获取人体地标3D坐标计算关节角度并映射执行，实验表明该方法能从单目RGB输入产生模仿动作，也揭示了复杂姿势和手腕动作的局限。
WaveSync: Constrained Wavefront Optimization for Synchronized Co-Speech Gestures in Humanoid Robots
提出WaveSync框架，利用大语言模型分解语义并构建重要性波，通过动态运动基元生成手势轨迹，再经波前优化实现手势与语音的峰值同步，同时满足运动学约束，在五组对话场景中优于基线方法。
Purpose - Most industrial robots are still programmed using the typical teaching process, through the use of the robot teach pendant. This is a tedious and time-consuming task that requires some technical expertise, and hence new approaches to robot programming are required. The purpose of this paper is to present a robotic system that allows users to instruct and program a robot with a high-level of abstraction from the robot language. Design/methodology/approach - The paper presents in detail a robotic system that allows users, especially non-expert programmers, to instruct and program a rob…
The advancement of technology such as artificial intelligence, machine learning and internet of things it became easy to develop more humanoid robots and automate different processes. An interactive robot must have high social behavior so that it can be easily accepted by the people using it. In this study we designed a traffic police robot (TRAPROB) to automate the traffic control at intersection. The human police officer experiences high stress because of long duty hours as well as pose the risk of accidents. The digital electronic signals are automatic but we want to create a system which i…
Integrating emotional intelligence, memory architecture, and gestures to achieve empathetic humanoid robot interaction in an educational setting
本研究将情商、记忆架构和手势同步整合到基于LLaMa 3.2的AI-Agent教育机器人中，并通过Engagement Vector模型验证其能显著提升学生参与度和学习成果。
EMOTION: Expressive Motion Sequence Generation for Humanoid Robots with In-Context Learning
针对现有机器人非语言交流方法表现力不足的问题，提出基于大语言模型上下文学习的EMOTION框架，可生成符合社交场景的手势动作，实验显示其效果在特定场景下媲美甚至超越人类操作，并给出相关设计启示。
Body language such as conversational gesture is a powerful way to ease communication. Conversational gestures do not only make a speech more lively but also contain semantic meaning that helps to stress important information in the discussion. In the field of robotics, giving conversational agents (humanoid robots or virtual avatars) the ability to properly use gestures is critical, yet remain a task of extraordinary difficulty. This is because given only a text as input, there are many possibilities and ambiguities to generate an appropriate gesture. Different to previous works we propose a n…
Real-Time Synchronized Interaction Framework for Emotion-Aware Humanoid Robots
本文提出一种实时框架，通过双通道情感引擎、动态时间规整和闭环验证，实现人形机器人的情感同步交互，提升在医疗、教育等动态场景中的应用能力。
This study proposes a novel imitation learning approach for the stochastic generation of human-like rhythmic wave gestures and their modulation for effective non-verbal communication through a probabilistic formulation using joint angle data from human demonstrations. This is achieved by learning and modulating the overall expression characteristics of the gesture (e.g., arm posture, waving frequency and amplitude) in the frequency domain. The method was evaluated on simulated robot experiments involving a robot with a manipulator of 6 degrees of freedom. The results show that the method provi…
Energy Efficient Personalized Hand-Gesture Recognition with Neuromorphic Computing
针对人机交互的手势识别需求，提出基于神经形态计算的脉冲卷积神经网络训练新方法，用于低功耗个性化手势识别，给出初步结果并提及性能优化方向。
Spotting signal patterns with varying lengths has been still an open problem in the literature. In this study, we describe a signal pattern recognition approach for continuous and simultaneous classification of a tracked hand's posture and gestures and map them to steering commands for control of a robotic wheelchair. The developed methodology not only affords 100\% recognition accuracy on a streaming signal for continuous recognition, but also brings about a new perspective for building a training dictionary which eliminates human intervention to spot the gesture or postures on a training sig…
Whenever we are addressing a specific object or refer to a certain spatial location, we are using referential or deictic gestures usually accompanied by some verbal description. Especially pointing gestures are necessary to dissolve ambiguities in a scene and they are of crucial importance when verbal communication may fail due to environmental conditions or when two persons simply do not speak the same language. With the currently increasing advances of humanoid robots and their future integration in domestic domains, the development of gesture interfaces complementing human-robot interaction…
Empathetic Motion Generation for Humanoid Educational Robots via Reasoning-Guided Vision--Language--Motion Diffusion Architecture
本文提出RG-VLMD框架，通过多模态情感估计、教学推理和教学-动作条件运动合成，生成适应性且语义一致的机器人行为。实验表明，推理引导的指令条件生成提升了教育人机交互中的动作可控性和教学表达性。
Generating Natural and Expressive Robot Gestures through Iterative Reinforcement Learning with Human Feedback using LLMs
针对社交机器人手势生成僵硬问题，提出将ChatGPT集成到Pepper机器人中生成共语手势，并引入基于人类反馈的迭代强化学习（RLHF）优化手势，实验表明RLHF提升了手势的表现力、相关性和流畅性。
NVP-HRI: Zero Shot Natural Voice and Posture-based Human-Robot Interaction via Large Language Model
本文提出NVP-HRI多模态人机交互范式，结合语音命令与指示性姿态，利用SAM实现零样本物体识别，并通过大语言模型实时生成无碰撞轨迹，较传统手势控制效率提升59.2%。
Sound-Based Recognition of Touch Gestures and Emotions for Enhanced Human-Robot Interaction
本文提出一种仅用触摸声音的轻量级模型，在保护隐私的同时识别触觉手势和情感唤醒度与效价，以极低参数量和计算量达到与大型预训练音频网络相近的效果。
Development of a Humanoid Robot Prototype for Multimodal Human-Robot Interaction
本文开发了一款集成手势识别、目标检测、语音命令处理等AI模块的人形机器人原型，经实验验证其定位精度高、任务准确率超90%，可作为人机交互研究的易用平台。
Hierarchical Intention-Aware Expressive Motion Generation for Humanoid Robots
针对人机交互中意图识别与实时表达性运动生成问题，提出融合ICL意图推理和扩散模型的分层框架，并通过物理平台验证其鲁棒性与社交一致性。
Human-Robot Collaboration: A Non-Verbal Approach with the NAO Humanoid Robot
本研究探讨NAO仿人机器人在厨房备餐等场景下，通过手势、肢体动作和视觉提示等非语言方式提供反馈以增强人机交互与安全性，发现多提示结合可提升交流效果，为未来机器人设计提供参考。
Posture control and balance are basic requirements for a humanoid robot performing motor tasks like walking and interacting with the environment. For this reason, posture control is one of the elements taken into account when evaluating the performance of humanoids. In this work, we describe and analyze a performance indicator based on the comparison between the body sway of a robot standing on a moving surface and the one of healthy subjects performing the same experiment. This approach is here oriented to the evaluation of human likeness. The measure is tested with three human-inspired human…
Natural Multimodal Fusion-Based Human-Robot Interaction: Application With Voice and Deictic Posture via Large Language Model
本文提出一种结合语音与指示姿态的多模态人机交互框架，利用目标检测与大型语言模型生成机器人动作序列，并通过控制语法约束避免幻觉，在UR3e机械臂上验证了更高的准确性与鲁棒性。
This work presents a system to benchmark humanoid posture control and balance performances under perturbed conditions. The specific benchmarking scenario consists, for example, of balancing upright stance while performing voluntary movements on moving surfaces. The system includes a motion platform used to provide the perturbation, an innovative body-tracking system suitable for robots, humans and exoskeletons, control software and a set of predefined perturbations, a humanoid robot used to test algorithms, and analysis software providing state of the art data analysis used to provide quantita…
Similarly to humans, humanoid robots require posture control and balance to walk and interact with the environment. In this work posture control in perturbed conditions is evaluated as a performance test for humanoid control. A specific performance indicator is proposed: the score is based on the comparison between the body sway of the tested humanoid standing on a moving surface and the sway produced by healthy subjects performing the same experiment. This approach is here oriented to the evaluation of a human-likeness. The measure is tested using a humanoid robot in order to demonstrate a ty…
Dataset-Free Compliant Humanoid Loco-Manipulation with Dynamic Online Posture
提出OCLO系统，无需人类数据，仅通过末端执行器目标实现人形机器人移动操作，利用解析可达性先验和策略在环细化生成姿态，并学习全身顺从性，显著提高成功率并保持平衡。
Bridging Speech, Emotion, and Motion: a VLM-based Multimodal Edge-deployable Framework for Humanoid Robots
本文提出SeM$^2$框架，通过多模态感知、思维链推理和语义序列对齐机制，实现情感一致的多模态交互，提升人形机器人在现实环境中的社交表达能力。
Co-speech gestures enhance interaction experiences between humans as well as between humans and robots. Existing robots use rule-based speech-gesture association, but this requires human labor and prior knowledge of experts to be implemented. We present a learning-based co-speech gesture generation that is learned from 52 h of TED talks. The proposed end-to-end neural network model consists of an encoder for speech text understanding and a decoder to generate a sequence of gestures. The model successfully produces various gestures including iconic, metaphoric, deictic, and beat gestures. In a …
RoboGesture: Real-Time Semantic-aligned Co-Speech Gestures Generation for Humanoid Interaction
本研究提出 RoboGesture 框架，构建专属数据集并设计相关算法，使人形机器人可实时生成语义对齐的伴随语音手势，在 Unitree G1 机器人上的实验表明其性能优于现有最优方法。
Social Gesture Recognition in spHRI: Leveraging Fabric-Based Tactile Sensing on Humanoid Robots
针对人机社交触觉交互需求，研究提出搭载于人形机器人手臂的织物基触觉社交手势识别系统，构建多参与者数据集并提取时序特征分类，为spHRI系统发展提供新思路。
Semantic Co-Speech Gesture Synthesis and Real-Time Control for Humanoid Robots
本文提出了一种基于语义的手势生成与实时控制框架，通过整合生成检索机制和模仿学习策略，实现人形机器人自然流畅的非语言交流。
Humanoid Robot Whole-body Geometric Calibration with Embedded Sensors and a Single Plane
本文针对人形机器人全身几何标定耗时费力的问题，提出利用单一平面、嵌入式力传感器和导纳控制器的无人工干预标定方法，并设计 IROC 算法自动选择最小最优位姿集，在 TALOS 机器人上仅用 31 个位姿即将 RMS 误差降低 2.3 倍。
Adapting Gait Frequency for Posture-regulating Humanoid Push-recovery via Hierarchical Model Predictive Control
本文提出一种基于分层MPC的仿人机器人推力恢复方法，通过适应步态频率和姿态感知规划器，在未知干扰下快速恢复并调节身体姿态，仿真显示最大可恢复冲量平均提升131%，硬件实验中恢复步态时机提前125毫秒。
SignBot: Learning Human-to-Humanoid Sign Language Interaction
SignBot通过整合运动重定向、运动控制和生成交互模块，实现人与人形机器人之间自然的手语交流，提升聋人和听力障碍者的沟通可及性。
This work presents a system identification procedure based on Convolutional Neural Networks (CNN) for human posture control using the DEC (Disturbance Estimation and Compensation) parametric model. The modular structure of the proposed control model inspired the design of a modular identification procedure, in the sense that the same neural network is used to identify the parameters of the modules controlling different degrees of freedom. In this way the presented examples of body sway induced by external stimuli provide several training samples at once.
Talk, Render, Act: Integrating Social Gesture and Digital Face with Synchronized Speech for Conversational Humanoid Robot
针对人形机器人交互中语音、面部和手势不协调的问题，提出TRABot框架，通过语义运动原子和组合规划器实现同步，实验证明其在自然性、表现力和多模态连贯性上最优。
Learning Humanoid Standing-up Control across Diverse Postures
提出HoST强化学习框架，从零学习人形机器人跨姿态站立控制，通过多评论家架构和课程训练实现稳健的仿真到现实迁移，并在Unitree G1上验证了平滑稳定的站立性能。
Adaptive Whole-Body MPC with Optimal Posture Planning for Humanoid Robots under Unknown Payloads
本文针对人形机器人未知负载下的平衡问题，提出整合质心MPC、自适应模糊平衡控制器与最优姿态规划器的自适应全身控制架构，经实验验证可提升平衡鲁棒性并降低跟踪退化。
Self-Body Image Acquisition and Posture Generation with Redundancy using Musculoskeletal Humanoid Shoulder Complex for Object Manipulation
提出一种利用肩复合体冗余的逆运动学方法学习肌骨人形机器人自身体像，以实现姿态生成和物体操作，并在Kengoro机器人上验证了汽车方向盘操作的有效性。
Requirement-Driven Design of Whole-Body Social Tactile Sensing via Virtual Human-Robot Interaction
该研究针对社交物理人机交互中触觉传感的硬件驱动局限，提出需求驱动框架，利用VR平台收集数据，识别社交触摸手势并构建数据集，分析得出定量基线，其方法可转移，能为特定形态机器人得出传感需求。
To enable humanoid robots to share our social space we need to develop technology for easy interaction with the robots using multiple modes such as speech, gestures and share our emotions with them. We have targeted this research towards addressing the core issue of emotion recognition problem which would require less computation resources and much lesser number of network hyperparameters which will be more adaptive to be computed on low resourced social robots for real time communication. More specifically, here we have proposed an Inception module based Convolutional Neural Network Architect…
Pose estimation commonly refers to computer vision methods that recognize people's body postures in images or videos. With recent advancements in deep learning, we now have compelling models to tackle the problem in real-time. Since these models are usually designed for human images, one needs to adapt existing models to work on other creatures, including robots. This paper examines different state-of-the-art pose estimation models and proposes a lightweight model that can work in real-time on humanoid robots in the RoboCup Humanoid League environment. Additionally, we present a novel dataset …
Currently, most social robots interact with their surroundings and humans through sensors that are integral parts of the robots, which limits the usability of the sensors, human-robot interaction, and interchangeability. A wearable sensor garment that fits many robots is needed in many applications. This article presents an affordable wearable sensor vest, and an open-source software architecture with the Internet of Things (IoT) for social humanoid robots. The vest consists of touch, temperature, gesture, distance, vision sensors, and a wireless communication module. The IoT feature allows th…
Most of industrial robots are still programmed using the typical teaching process, through the use of the robot teach pendant. In this paper is proposed an accelerometer-based system to control an industrial robot using two low-cost and small 3-axis wireless accelerometers. These accelerometers are attached to the human arms, capturing its behavior (gestures and postures). An Artificial Neural Network (ANN) trained with a back-propagation algorithm was used to recognize arm gestures and postures, which then will be used as input in the control of the robot. The aim is that the robot starts the…
The gesture-determined-dynamic function (GDDF) offers an effective way to handle the control problems of humanoid robots. Specifically, GDDF is utilized to constrain the movements of dual arms of humanoid robots and steer specific gestures to conduct demanding tasks under certain conditions. However, there is still a deficiency in this scheme. Through experiments, we found that the joints of the dual arms, which can be regarded as the redundant manipulators, could exceed their limits slightly at the joint angle level. The performance straightly depends on the parameters designed beforehand for…
Enhancing Human-Robot Interaction in Healthcare: A Study on Nonverbal Communication Cues and Trust Dynamics with NAO Robot Caregivers
本研究通过混合方法和被试内因子设计，探究NAO机器人以触觉、手势和LED模式进行非语言沟通对老年人健康照护中人机交互与信任的影响，发现触觉监测和LED模式效果更佳，且更长交互能提升信任与共情。
Development of a Whole-body Work Imitation Learning System by a Biped and Bi-armed Humanoid
针对带浮动连杆的人形机器人行为模仿难题，研究结合TABLIS全身控制设备与JAXON机器人，搭建基于姿态优化的集成模仿学习系统，实现多类复杂全身任务并验证有效性。
A Sign Language Recognition System with Pepper, Lightweight-Transformer, and LLM
本文面向 Pepper 机器人构建 ASL 识别与交互系统，结合轻量 Transformer、LLM 提示工程及协同语音手势生成，验证其在真实非语言人机交互场景中的有效性。
GestAdapt: Workspace-Conditioned Co-Speech Gesture Generation for Humanoid Robots
GestAdapt提出工作空间条件化框架，通过共享运动表示从多语料库学习，生成适应手腕工作空间的共语手势，并在用户研究和真实机器人评估中优于无约束基线。
EmoPose: Vision-Language Model Guided Emotion-Aware Gesture Generation for Humanoid Robots
EmoPose提出VLM引导的仿人机器人情感手势生成框架，通过可执行语义接口和动作库实现表达性与可控性，在基准上显著优于直接提示方法。
Translating Cultural Choreography from Humanoid Forms to Robotic Arm
本研究提出ROPERA流程，通过符号化姿态转移在六自由度机械臂上实现文化编舞的语义保真与跨形态可移植性。
Physics-Guided Biomechanical Gait Adaptation for Humanoid Locomotion on Extreme Sloped Terrains
研究类人机器人在极端倾斜地形的运动控制问题，提出HumoSlope两阶段物理引导框架，第一阶段建立平衡先验，第二阶段引入BSGA调节奖励，实现对不同坡度地形的适应，实验验证了该方法的有效性。
From Social Reasoning to Embodied Interaction: An Agentic Framework for Social Robots
提出ARISE框架，在Sophia人形机器人上统一社会推理与具身交互，通过多模态理解、记忆和流式执行实现自然协调的社交行为，显著提升交互质量并降低延迟。
Project Report: Requirements for a Social Robot as an Information Provider in the Public Sector
本研究面向市政公共服务场景设计人形社交机器人信息提供应用，验证其相较平板浏览器方案更受偏好，并提出接入 ACT-R 认知架构以增强人机对话。
Contact-Constrained Lower-Limb Joint-Offset Calibration for Humanoid Robots
提出仅用机载关节编码器与骨盆IMU的自包含校准框架，通过非线性最小二乘求解12维偏移向量，校准后A3、A2脚高均方根残差显著降低，可实现人形机器人下肢接触约束的关节偏移校准。
Touched by ChatGPT: Using an LLM to Drive Affective Tactile Interaction
针对人形机器人触觉表达能力有限的问题，研究开发集成5×5振动马达的可穿戴袖套，借助LLM链式提示生成对应10种情绪和6种手势的10秒振动模式，实验验证人类可准确识别情绪，证明LLM可增强人机物理交互。
Humanoid robots are increasingly being integrated into learning contexts to assist teaching and learning. However, challenges remain how to design and incorporate such robots in an educational context. As an important part of teaching includes monitoring the motivational and emotional state of the learner and adapting the interaction style and learning content accordingly, in this paper, we discuss the role of gestures displayed by a humanoid robot (i.e., Pepper robot) in a learning and teaching context and present our ongoing research on designing and developing a teacher robot.
Heavy lifting tasks via haptic teleoperation of a wheeled humanoid
本文提出一种用于轮式人形机器人重物搬运的触觉遥操作框架，通过全身运动重定向和实时力反馈实现动态移动操作，实验验证了最大2.5公斤载荷的协调控制。
$λ_0$: Scaling Open-World Egocentric Human Data for Humanoid Loco-Manipulation
本研究推出HumanVerse数据集，开发λ₀类人机器人基础模型，通过人-类人机器人流水线实现零样本迁移，在SIMPLE任务获90.0%成功率，现实任务达62.5%，为类人机器人移动操作提供可扩展数据方案。
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。