告知而非展示！：语言引导简化图像与视频中的跨域迁移
Tell, Don't Show!: Language Guidance Eases Transfer Across Domains in Images and Videos
AI总结：
本文提出 LaGTran 框架，利用文本监督引导跨域判别知识迁移，在图像数据集上显著超越现有方法，并引入 Ego2Exo 视频基准验证其在 ego-exo 迁移中的有效性。
摘要：
我们提出 LaGTran，一个利用文本监督来引导判别性知识从有标签源域稳健迁移到存在域差距的无标签目标域的新框架。尽管无监督自适应方法已被用于解决该问题，但由于它们仅在像素空间内运行，在处理具有挑战性的域偏移时表现出局限性。受我们观察到语义更丰富的文本模态具有更有利的迁移特性的启发，我们设计了一种迁移机制：使用在源域上训练的文本分类器对目标文本描述生成预测，并将这些预测用作对应图像的监督。我们这种由语言引导驱动的方法出人意料地简单易行，但在 GeoNet 和 DomainNet 等具有挑战性的数据集上显著优于所有先前方法，验证了其极高的有效性。为了将研究范围进一步扩展到图像之外，我们引入了一个名为 Ego2Exo 的新基准，用于研究视频中的 ego-exo 迁移，并发现我们的语言辅助方法 LaGTran 在这一极具挑战性且非平凡的迁移设置中取得了显著增益。代码、模型和所提出的数据集已在 https://tarun005.github.io/lagtran/ 公开。
英文摘要：
We introduce LaGTran, a novel framework that utilizes text supervision to guide robust transfer of discriminative knowledge from labeled source to unlabeled target data with domain gaps. While unsupervised adaptation methods have been established to address this problem, they show limitations in handling challenging domain shifts due to their exclusive operation within the pixel-space. Motivated by our observation that semantically richer text modality has more favorable transfer properties, we devise a transfer mechanism to use a source-trained text-classifier to generate predictions on the target text descriptions, and utilize these predictions as supervision for the corresponding images. Our approach driven by language guidance is surprisingly easy and simple, yet significantly outperforms all prior approaches on challenging datasets like GeoNet and DomainNet, validating its extreme effectiveness. To further extend the scope of our study beyond images, we introduce a new benchmark called Ego2Exo to study ego-exo transfer in videos and find that our language-aided approach LaGTran yields significant gains in this highly challenging and non-trivial transfer setting. Code, models, and proposed datasets are publicly available at https://tarun005.github.io/lagtran/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
