AI 中文总结
针对手语生成中线性插值偏离关节旋转流形的问题，提出自然度引导的流形流匹配框架SignNMFlow，结合闭式测地线与可学习偏差，提升生成动作的保真度。
AI 中文摘要
手语生成（SLP）旨在从文本生成手语动作。条件流匹配方法通过构建将源分布转换为目标分布的条件路径，在手语生成中取得了强劲性能。然而，现有方法通过线性插值构建这些路径，而人体关节的旋转几何将有效关节旋转限制在嵌入欧几里得空间的流形上。因此，两个手语动作之间的线性插值会偏离该流形，并忽略其上的动作分布。在本文中，我们从流形传输的角度重新审视手语生成，并提出一个名为\textbf{SignNMFlow}的自然度引导的流形流匹配框架，该框架直接在动作流形上构建条件路径，同时考虑几何效率和动作分布。具体而言，我们利用流形的内在几何，并引入一个动作自然度度量来表征动作分布。通过在该度量下最小化动能，我们学习一种自然度引导的插值，该插值将提供几何高效传输的闭式测地线与结合动作分布的可学习偏差耦合，从而显著提高生成手语动作的保真度。广泛的定性和定量评估证明了该工作的有效性。
英文摘要
Sign Language Production (SLP) aims to generate sign motions from text. Conditional Flow Matching methods have achieved strong performance in SLP by constructing conditional paths that transform a source distribution into a target distribution. However, existing methods construct these paths via linear interpolation, whereas the rotational geometry of human joints confines valid joint rotations to a manifold embedded in Euclidean space. Consequently, linear interpolation between two sign motions leaves this manifold and ignores the motion distribution on it. In this paper, we revisit SLP from the perspective of manifold transport and propose a Naturalness-guided Manifold Flow Matching framework, termed \textbf{SignNMFlow}, which constructs conditional paths directly on the motion manifold by jointly considering geometric efficiency and the motion distribution. Specifically, we exploit the intrinsic geometry of the manifold and introduce a motion naturalness measure to characterize the motion distribution. By minimizing the kinetic energy under this measure, we learn a naturalness-guided interpolation that couples a closed-form geodesic, which provides geometrically efficient transport, with a learnable deviation that incorporates the motion distribution, thereby significantly improving the fidelity of generated sign motions. Extensive qualitative and quantitative evaluations demonstrate the effectiveness of this work.
Comments25 pages, 4 figures
