对齐词语所在之处：面向孟加拉语图像描述生成的跨注意力引导补丁对齐与对比及传输正则化
Align Where the Words Look: Cross-Attention-Guided Patch Alignment with Contrastive and Transport Regularization for Bengali Captioning
AI总结：
本文提出一种面向低资源孟加拉语图像描述生成的跨注意力引导补丁对齐方法，通过PAL、InfoNCE与最优传输三损失协同，在Flickr30k和MSCOCO上显著提升接地能力并缩小真实—合成差距。
摘要：
将视觉—语言模型落地到低资源语言中仍然具有挑战性，因为这些模型常常围绕错误的对象生成流畅文本。这源于稀缺的成对数据、破坏对齐的翻译中转，以及忽略目标语言语义的以英语为中心的预训练。我们通过一个计算感知的孟加拉语图像描述生成流程来解决这一问题，该流程在经LaBSE验证的英—孟语对和11万张双语提示合成图像上训练。冻结的MaxViT产生稳定的视觉补丁，孟加拉语原生mBART-50负责解码，一个轻量级桥接模块连接两种模态。我们的核心创新是一个三损失目标：补丁对齐损失（PAL）利用解码器跨注意力对齐真实与合成补丁描述符，InfoNCE强制全局真实—合成分离，基于Sinkhorn的最优传输（OT）确保均衡的细粒度补丁对应。这种PAL+InfoNCE+OT协同作用改善了接地能力，减少了虚假匹配，并在Flickr30k-1k（BLEU-4 12.29，METEOR 27.98，BERTScore-F1 71.20）和MSCOCO-1k（BLEU-4 12.00，METEOR 28.14，BERTScore-F1 75.40）上带来显著提升，优于强交叉熵基线，并将真实—合成质心差距缩小了41%。
英文摘要：
Grounding vision--language models in low-resource languages remains challenging, as they often produce fluent text about the wrong objects. This stems from scarce paired data, translation pivots that break alignment, and English-centric pretraining that ignores target-language semantics. We address this with a compute-aware Bengali captioning pipeline trained on LaBSE-verified EN--BN pairs and 110k bilingual-prompted synthetic images. A frozen MaxViT yields stable visual patches, a Bengali-native mBART-50 decodes, and a lightweight bridge links the modalities. Our core novelty is a tri-loss objective: Patch-Alignment Loss (PAL) aligns real and synthetic patch descriptors using decoder cross-attention, InfoNCE enforces global real--synthetic separation, and Sinkhorn-based OT ensures balanced fine-grained patch correspondence. This PAL+InfoNCE+OT synergy improves grounding, reduces spurious matches, and drives strong gains on Flickr30k-1k (BLEU-4 12.29, METEOR 27.98, BERTScore-F1 71.20) and MSCOCO-1k (BLEU-4 12.00, METEOR 28.14, BERTScore-F1 75.40), outperforming strong CE baselines and narrowing the real--synthetic centroid gap by 41%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
