迷失在翻译中？多语言概念下文生图模型公平评估中的翻译错误与挑战
Lost in Translation? Translation Errors and Challenges for Fair Assessment of Text-to-Image Models on Multilingual Concepts
AI总结：
本文发现CoCo-CroLa多语言文生图基准存在翻译错误，提供修正并评估其影响，证明文本域相似度可预测图像域结果变化，为多语言度量开发提供分析工具。
摘要：
文生图（T2I）模型的多语言能力基准测试，是将以测试语言提示生成的图像与概念集上的预期图像分布进行比较。其中一项基准测试“跨语言概念覆盖度”（CoCo-CroLa）通过提示模型从翻译成七种语言的概念列表中生成图片，并比较输出图像群体，来评估T2I模型的实体名词库存。不幸的是，我们发现该基准在西班牙语、日语和中文中包含严重程度不一的翻译错误。我们为这些错误提供了修正，并分析了它们对CoCo-CroLa作为基准的实用性和有效性的影响程度。我们使用修订后的翻译重新评估了多个基线T2I模型，将新翻译下产生的输出与旧翻译条件下的输出进行比较，并表明在图像域基准结果中修正的影响力可以通过文本域中的相似度分数来预测。我们的发现将为T2I多语言度量标准的未来发展提供指导，通过提供实用翻译决策的分析工具。
英文摘要：
Benchmarks of the multilingual capabilities of text-to-image (T2I) models compare generated images prompted in a test language to an expected image distribution over a concept set. One such benchmark, "Conceptual Coverage Across Languages" (CoCo-CroLa), assesses the tangible noun inventory of T2I models by prompting them to generate pictures from a concept list translated to seven languages and comparing the output image populations. Unfortunately, we find that this benchmark contains translation errors of varying severity in Spanish, Japanese, and Chinese. We provide corrections for these errors and analyze how impactful they are on the utility and validity of CoCo-CroLa as a benchmark. We reassess multiple baseline T2I models with the revisions, compare the outputs elicited under the new translations to those conditioned on the old, and show that a correction's impactfulness on the image-domain benchmark results can be predicted in the text domain with similarity scores. Our findings will guide the future development of T2I multilinguality metrics by providing analytical tools for practical translation decisions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
