Cross2StrA：基于跨语言跨模态结构枢轴对齐的无配对跨语言图像描述
Cross2StrA: Unpaired Cross-lingual Image Captioning with Cross-lingual Cross-modal Structure-pivoted Alignment
AI总结：
本文提出Cross2StrA模型，利用场景图和句法成分树作为枢轴结构，结合跨模态语义对齐、跨语言句法对齐及回译训练，显著提升了无配对跨语言图像描述的相关性和流畅性。
摘要：
无配对跨语言图像描述长期受到不相关和不流畅问题的困扰，这源于迁移过程中语义场景和句法属性的不一致。在本工作中，我们提出通过引入场景图（SG）结构和句法成分（SC）树来解决上述问题。我们的描述器包含语义结构引导的图像到枢轴描述和句法结构引导的枢轴到目标翻译，这两个阶段通过枢轴语言连接。然后，我们以SG和SC结构作为枢轴，执行跨模态语义结构对齐和跨语言句法结构对齐学习。我们进一步引入跨语言与跨模态回译训练，以充分对齐描述和翻译阶段。在英中迁移上的实验表明，我们的模型在提升描述相关性和流畅性方面表现出显著优势。
英文摘要：
Unpaired cross-lingual image captioning has long suffered from irrelevancy and disfluency issues, due to the inconsistencies of the semantic scene and syntax attributes during transfer. In this work, we propose to address the above problems by incorporating the scene graph (SG) structures and the syntactic constituency (SC) trees. Our captioner contains the semantic structure-guided image-to-pivot captioning and the syntactic structure-guided pivot-to-target translation, two of which are joined via pivot language. We then take the SG and SC structures as pivoting, performing cross-modal semantic structure alignment and cross-lingual syntactic structure alignment learning. We further introduce cross-lingual&cross-modal back-translation training to fully align the captioning and translation stages. Experiments on English-Chinese transfers show that our model shows great superiority in improving captioning relevancy and fluency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
