发表机构
Sogang University（西江大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究提出基于模拟器的3DTongueQA框架，从3D舌网格构建可验证肌肉驱动问答，验证其可移植性，表明构建的监督信号支持结构化预测与异构自然语言问答。
AI 中文摘要
现有基于实时MRI和电磁 articulography（EMA）的发音语料库可捕捉舌形与运动，但未为生成观测构型的肌肉驱动过程提供可追溯标签。本文提出一种基于模拟器的数据构建框架，并将其实例化为3DTongueQA。通过ArtiSynth Badin有限元模型，将受控的11维肌肉激活映射到拓扑固定的舌网格，转换为结构化生物力学记录，并渲染为关于肌肉状态、几何形态和目标导向变化的确定性问答（QA）。我们筛选了295,157个构型，保留295,115个有效网格，每种语言构建891,156条QA记录。语言本土化仅改变表面形式，并与源记录验证；英语和韩语实例证明了该框架在构建层面的可移植性。可替换的SpiralNet++--Qwen3-8B基线模型达到62.9±9.2的肌肉EM、74.0±0.2的数值准确率和65.9±4.7的方向EM，而不匹配配对网格会使肌肉EM降至2.2；控制数据集泄漏的锚点保留模型在未见过的锚点上保留了全库存分数的80.4%至98.6%。特定任务的结构化读出进一步达到88.7±0.7的肌肉EM和93.3±1.0的方向EM。这些互补结果表明，所构建的监督信号支持高效的结构化预测和异构自然语言问答，而非局限于特定解码器架构。
英文摘要
Articulatory data describe what the tongue looks like, but geometry alone does not explain the muscle-level why behind a configuration or how to move it toward a target posture. We present a simulator-based framework that constructs this supervision from controlled biomechanical inputs. Each simulated configuration is stored with its generating muscle state and geometric properties in a structured fact record. Gold answers are derived before linguistic realization, and every naturalized output is checked against its source record, so the same records support new languages and question types without re-simulation. We instantiate the framework as 3DTongueQA using the ArtiSynth Badin tongue model. From 295,115 meshes, we construct 891,156 record-checked QA instances per language, and the checker detects over 96.9% of injected corruptions. Korean and Spanish realizations and a new question type are added from the same records, confirming portability. A SpiralNet++-Qwen3 probe reaches 62.9 Muscle EM, above 44.0 for nearest-neighbor transfer and 7.2 for a zero-shot commercial LLM given the same mesh, while mesh shuffling drops it to near zero. The supervision is learnable, useful beyond retrieval, and grounded in paired geometry. It reflects simulator-defined states rather than measured physiology. Code, templates, and QA sets are available at https://github.com/esh0504/muscle-grounded-qa.
Comments17 pages, 5 figures, 16 tables,
