TranSTYLer：面向面部与身体姿态生成的多模态行为风格迁移
TranSTYLer: Multimodal Behavioral Style Transfer for Facial and Body Gestures Generation
AI总结：
针对虚拟智能体行为风格迁移需保留行为语义的问题，提出多模态Transformer模型TranSTYLer，通过风格内容解耦实现无标签风格迁移与未见风格泛化，在PATS语料库上的表现优于现有最优模型，并提出泄露评估方法。
摘要：
本文致力于解决在保留承载交际意义的行为形态的前提下，将虚拟智能体的行为表现力风格迁移至另一智能体的挑战。本文将行为表现力风格定义为行为的定性属性。我们提出TranSTYLer，这是一种基于Transformer的多模态模型，能够以目标说话者的风格合成源说话者的多模态行为。我们假设行为表现力风格编码于文本、语音、身体姿态、面部表情等多种交际模态中。该模型采用风格与内容解耦机制，确保迁移后的风格不会干扰源行为所传递的含义。我们的方法无需风格标签，且能够泛化至训练阶段未见过的风格。我们在PATS语料库上训练模型，并对该语料库进行了扩展，新增了对话行为与2D面部关键点数据。客观与主观评估均表明，在训练中见过与未见过的风格的迁移任务上，我们的模型表现均优于现有最优模型。为应对可能出现的风格与内容泄露问题，我们提出了一种评估方法，用于衡量与目标风格相关的行为和姿态的成功迁移程度，同时确保与源内容相关的行为和姿态得到保留。
英文摘要：
This paper addresses the challenge of transferring the behavior expressivity style of a virtual agent to another one while preserving behaviors shape as they carry communicative meaning. Behavior expressivity style is viewed here as the qualitative properties of behaviors. We propose TranSTYLer, a multimodal transformer based model that synthesizes the multimodal behaviors of a source speaker with the style of a target speaker. We assume that behavior expressivity style is encoded across various modalities of communication, including text, speech, body gestures, and facial expressions. The model employs a style and content disentanglement schema to ensure that the transferred style does not interfere with the meaning conveyed by the source behaviors. Our approach eliminates the need for style labels and allows the generalization to styles that have not been seen during the training phase. We train our model on the PATS corpus, which we extended to include dialog acts and 2D facial landmarks. Objective and subjective evaluations show that our model outperforms state of the art models in style transfer for both seen and unseen styles during training. To tackle the issues of style and content leakage that may arise, we propose a methodology to assess the degree to which behavior and gestures associated with the target style are successfully transferred, while ensuring the preservation of the ones related to the source content.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
