arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2101.11101cs.HCcs.GR
收藏

Text2Gestures:一种基于Transformer的为虚拟智能体生成情感化身体手势的网络

Text2Gestures: A Transformer-Based Network for Generating Emotive Body Gestures for Virtual Agents

Uttaran Bhattacharya, Nicholas Rewkowski, Abhishek Banerjee, Pooja Guhan, Aniket Bera, Dinesh Manocha

AI总结:

本文提出基于Transformer的Text2Gestures方法,结合生物力学情感特征、任务、性别和惯用手信息,为虚拟智能体实时生成与文本对齐的情感手势,并在MPI情感身体表达数据库上取得最先进性能,用户研究中约91%的参与者认为生成手势合理。

摘要:

我们提出了Text2Gestures,一种基于Transformer的学习方法,用于交互式地为虚拟智能体生成与自然语言文本输入对齐的情感化全身手势。我们的方法通过利用身体表达的相关生物力学特征(也称为情感特征)来生成情感表达性手势。在生成流程中,我们还考虑了与文本对应的预期任务以及目标虚拟智能体的预期性别和惯用手。我们在MPI Emotional Body Expressions Database上训练和评估我们的网络,并观察到我们的网络在为虚拟智能体生成与叙述或对话文本对齐的手势方面达到了最先进的性能。我们的网络可以在普通GPU上以交互速率生成这些手势。我们进行了一项基于网络的用户研究,观察到约91%的参与者在五点李克特量表上表示我们生成的手势至少是合理的。参与者从手势中感知到的情绪也与相应的预期情绪呈强正相关,在效价维度上的最小皮尔逊系数为0.77。

英文摘要:

We present Text2Gestures, a transformer-based learning method to interactively generate emotive full-body gestures for virtual agents aligned with natural language text inputs. Our method generates emotionally expressive gestures by utilizing the relevant biomechanical features for body expressions, also known as affective features. We also consider the intended task corresponding to the text and the target virtual agents' intended gender and handedness in our generation pipeline. We train and evaluate our network on the MPI Emotional Body Expressions Database and observe that our network produces state-of-the-art performance in generating gestures for virtual agents aligned with the text for narration or conversation. Our network can generate these gestures at interactive rates on a commodity GPU. We conduct a web-based user study and observe that around 91% of participants indicated our generated gestures to be at least plausible on a five-point Likert Scale. The emotions perceived by the participants from the gestures are also strongly positively correlated with the corresponding intended emotions, with a minimum Pearson coefficient of 0.77 in the valence dimension.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑