一图胜千言，但人人都能“听懂”吗？关于面向文化适配性的图像转译（image transcreation）研究
An image speaks a thousand words, but can everyone listen? On image transcreation for cultural relevance
AI总结：
本研究聚焦图像文化适配转译这一新任务，构建了三条生成模型流水线及包含700张图像的评估数据集，经人工评估发现现有图像编辑模型表现不佳，引入LLM和检索器可优化，凸显任务难度并开源了相关资源。
摘要：
随着多媒体内容的兴起，人工译者越来越注重不仅对文字，还对图像等其他模态进行文化适配，以传递相同的含义。尽管多个应用领域都能从中受益，但机器翻译系统仍局限于处理语音和文本形式的语言。在本研究中，我们朝着实现图像的文化适配性翻译迈出了第一步。首先，我们构建了三条融合前沿生成模型的流水线来完成该任务。其次，我们构建了一个由两部分组成的评估数据集：i）概念集：包含600张跨文化语义连贯的图像，每张图像聚焦一个单一概念；ii）应用集：包含100张从真实应用中筛选的图像。我们对转译后的图像开展了多维度人工评估，以衡量其文化适配性和含义保留度。我们发现，目前图像编辑模型无法完成这项任务，但通过在流程中引入大语言模型（LLM）和检索器可以提升效果。在难度较低的概念数据集中，表现最优的流水线针对部分国家仅能完成5%的图像转译；而在应用数据集中，针对部分国家的转译成功率为零，这凸显了该任务的挑战性。我们的代码和数据已发布于：https://github.com/simran-khanuja/image-transcreation。
英文摘要：
Given the rise of multimedia content, human translators increasingly focus on culturally adapting not only words but also other modalities such as images to convey the same meaning. While several applications stand to benefit from this, machine translation systems remain confined to dealing with language in speech and text. In this work, we take a first step towards translating images to make them culturally relevant. First, we build three pipelines comprising state-of-the-art generative models to do the task. Next, we build a two-part evaluation dataset: i) concept: comprising 600 images that are cross-culturally coherent, focusing on a single concept per image, and ii) application: comprising 100 images curated from real-world applications. We conduct a multi-faceted human evaluation of translated images to assess for cultural relevance and meaning preservation. We find that as of today, image-editing models fail at this task, but can be improved by leveraging LLMs and retrievers in the loop. Best pipelines can only translate 5% of images for some countries in the easier concept dataset and no translation is successful for some countries in the application dataset, highlighting the challenging nature of the task. Our code and data is released here: https://github.com/simran-khanuja/image-transcreation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
