通过语义相似度标注句子中的象征性和指示性手势
Labeling Sentences with Symbolic and Deictic Gestures via Semantic Similarity
AI总结：
本文提出基于RoBerta语义相似度的规则算法，用于在人工代理话语中自动标注象征性和指示性手势，并通过30人标注实验验证其有效性。
摘要：
人工智能体上的协同语音手势生成近年来受到关注，尤其是基于数据驱动模型的方法。然而，端到端方法往往无法生成与语义相关的特定形式协同语音手势，即象征性手势和指示性手势。在这项工作中，我们识别句子中哪些词在语境上与象征性手势和指示性手势相关。首先，我们适当选择了12个被意大利文化人群所识别的手势，这些手势可由不同的人形机器人复现。然后，我们实现了两种基于规则的算法，用象征性手势和指示性手势标注句子。这些规则依赖于使用RoBerta模型计算的语义相似度分数，该分数是在启发式表示手势的句子与人工代理必须说出的目标句子内部的子句之间计算得到的。我们还实现了一个基线算法，该算法在不计算相似度分数的情况下分配手势。最后，为了验证结果，我们请30个人通过图形用户界面（GUI）用指示性手势和象征性手势标注一组句子，并将这些标注与我们算法生成的标注进行比较。为此，我们计算了平均精度（AP）和交并比（IOU）分数，并评估了平均计算时间（ACT）。我们的结果表明，语义相似度分数有助于在话语中发现象征性手势和指示性手势。
英文摘要：
Co-speech gesture generation on artificial agents has gained attention recently, mainly when it is based on data-driven models. However, end-to-end methods often fail to generate co-speech gestures related to semantics with specific forms, i.e., Symbolic and Deictic gestures. In this work, we identify which words in a sentence are contextually related to Symbolic and Deictic gestures. Firstly, we appropriately chose 12 gestures recognized by people from the Italian culture, which different humanoid robots can reproduce. Then, we implemented two rule-based algorithms to label sentences with Symbolic and Deictic gestures. The rules depend on the semantic similarity scores computed with the RoBerta model between sentences that heuristically represent gestures and sub-sentences inside an objective sentence that artificial agents have to pronounce. We also implemented a baseline algorithm that assigns gestures without computing similarity scores. Finally, to validate the results, we asked 30 persons to label a set of sentences with Deictic and Symbolic gestures through a Graphical User Interface (GUI), and we compared the labels with the ones produced by our algorithms. For this scope, we computed Average Precision (AP) and Intersection Over Union (IOU) scores, and we evaluated the Average Computational Time (ACT). Our results show that semantic similarity scores are useful for finding Symbolic and Deictic gestures in utterances.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
