跨越边界：面向印度诗歌翻译与图像生成的多模态挑战
Crossing Borders: A Multimodal Challenge for Indian Poetry Translation and Image Generation
AI总结：
针对印度语言诗歌翻译与理解资源匮乏的问题，本文提出融合LLM与潜在扩散模型的TAI多模态框架，构建MorphoVerse数据集，实验验证其图像生成性能优于基线，可提升印度诗歌全球可及性。
摘要：
印度诗歌以其语言复杂性和深厚的文化共鸣著称，拥有跨越数千年的丰富多样遗产。然而，其多层含义、文化典故与复杂语法结构往往给理解带来挑战，对非母语使用者或不熟悉其语境与语言的读者而言尤其如此。尽管印度诗歌具有重要文化价值，现有诗歌研究大多忽略了印度语言诗歌。本文提出翻译与图像生成（Translation and Image Generation, TAI）框架，通过恰当的提示微调利用大语言模型（LLMs）与潜在扩散模型。该框架通过提升全球受众对文化内涵丰富的印度语言诗歌的可及性，助力联合国可持续发展目标中的优质教育（SDG 4）和减少不平等（SDG 10）。框架包含两个模块：（1）翻译模块，采用比值比偏好对齐算法，将形态丰富的诗歌准确翻译为英语；（2）图像生成模块，利用语义图捕捉词元、依存关系以及隐喻与其含义间的语义关联，生成印度诗歌的具有视觉意义的表征。我们开展了包含人工评估与定量评估的综合实验，结果表明TAI Diffusion在诗歌图像生成任务中表现优异，性能优于强基线模型。为进一步解决印度语言诗歌资源稀缺问题，我们构建了形态丰富印度语言诗歌MorphoVerse数据集，涵盖21种低资源印度语言的1570首诗歌。本研究通过填补诗歌翻译与视觉理解的空白，旨在拓宽诗歌可及性、丰富读者体验。
英文摘要：
Indian poetry, known for its linguistic complexity and deep cultural resonance, has a rich and varied heritage spanning thousands of years. However, its layered meanings, cultural allusions, and sophisticated grammatical constructions often pose challenges for comprehension, especially for non-native speakers or readers unfamiliar with its context and language. Despite its cultural significance, existing works on poetry have largely overlooked Indian language poems. In this paper, we propose the Translation and Image Generation (TAI) framework, leveraging Large Language Models (LLMs) and Latent Diffusion Models through appropriate prompt tuning. Our framework supports the United Nations Sustainable Development Goals of Quality Education (SDG 4) and Reduced Inequalities (SDG 10) by enhancing the accessibility of culturally rich Indian-language poetry to a global audience. It includes (1) a translation module that uses an Odds Ratio Preference Alignment Algorithm to accurately translate morphologically rich poetry into English, and (2) an image generation module that employs a semantic graph to capture tokens, dependencies, and semantic relationships between metaphors and their meanings, to create visually meaningful representations of Indian poems. Our comprehensive experimental evaluation, including both human and quantitative assessments, demonstrates the superiority of TAI Diffusion in poem image generation tasks, outperforming strong baselines. To further address the scarcity of resources for Indian-language poetry, we introduce the Morphologically Rich Indian Language Poems MorphoVerse Dataset, comprising 1,570 poems across 21 low-resource Indian languages. By addressing the gap in poetry translation and visual comprehension, this work aims to broaden accessibility and enrich the reader's experience.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
