ArtLang：面向铰接3D驱动的结构化语言到运动学的绑定

中文摘要 AI 辅助

铰接物体重建可恢复显式几何与运动学信息，但其部件常保持语义匿名，需通过部件索引与数值关节参数进行控制。我们提出ArtLang，一种用于持久化重建的铰接资产的开放词汇语言控制框架。ArtLang将资产表示为语义-运动学铰接图，并用语言特征与图约束运动增强其表面。开放词汇提案绑定至重建部件，同时允许不确定部件保持未命名。类型化解析器将命令转换为包含指代表达式、动作、幅度、参考帧与关系的指令图。随后我们求解全局图到图的绑定问题，该问题联合推理语义、空间、关系与运动学兼容性，支持空分配及歧义下的弃权（不执行）。被接受的指令在观测运动范围内转换为连续关节目标，并通过正向运动学执行。在合成重建、基于网格的资产及真实捕获数据上的实验，展示了在重复部件、空间参考、关系命令及歧义指令下的可靠语言绑定与连续铰接控制。

英文摘要

Articulated-object reconstructions recover explicit geometry and kinematics, but their parts often remain semantically anonymous and must be controlled through part indices and numerical joint parameters. We present ArtLang, a framework for open-vocabulary language control of persistent reconstructed articulated assets. ArtLang represents an asset as a semantic-kinematic articulation graph and augments its surface with language features and graph-constrained motion. Open-vocabulary proposals are bound to reconstructed parts while allowing uncertain parts to remain unnamed. A typed parser converts a command into a directive graph containing referring expressions, actions, magnitudes, reference frames, and relations. We then solve a global graph-to-graph grounding problem that jointly reasons about semantic, spatial, relational, and kinematic compatibility, with support for null assignments and abstention under ambiguity. Accepted directives are converted into continuous joint targets within the observed motion range and executed through forward kinematics. Experiments on synthetic reconstructions, mesh-based assets, and real captures demonstrate reliable language grounding and continuous articulated control across repeated parts, spatial references, relational commands, and ambiguous instructions.

发表机构

• University of California San Diego（加利福尼亚大学圣迭戈分校）
• University of North Texas（北得克萨斯大学）

机构由 AI 辅助整理，请以论文原文为准。