改进的视觉提示关键词在现实低资源环境中的本地化
Improved Visually Prompted Keyword Localisation in Real Low-Resource Settings
AI总结：
本文提出了一种无需转录的少样本学习方法，用于在低资源语言中实现视觉提示关键词本地化，并在约鲁巴语言上验证了其有效性。
摘要：
给定一张图像查询，视觉提示关键词本地化（VPKL）旨在在一个语音集合中找到所描绘的词的出现。这在语音转录不可用的低资源语言（例如，如果未写成文字）时非常有用。先前的工作表明，可以通过在配对图像和无标签语音上训练一个视觉基础的语音模型来执行VPKL。但所有实验都是在英语上进行的。此外，使用转录来获得正负对用于对比损失。本文介绍了一种少样本学习方案，以自动挖掘配对，而无需转录。在英语上，这导致性能仅有小幅下降。我们还首次考虑了在现实低资源语言约鲁巴上执行VPKL。虽然分数合理，但在这里，与使用地面真实配对相比，由于挖掘的准确性较低，性能下降更大。
英文摘要：
Given an image query, visually prompted keyword localisation (VPKL) aims to find occurrences of the depicted word in a speech collection. This can be useful when transcriptions are not available for a low-resource language (e.g. if it is unwritten). Previous work showed that VPKL can be performed with a visually grounded speech model trained on paired images and unlabelled speech. But all experiments were done on English. Moreover, transcriptions were used to get positive and negative pairs for the contrastive loss. This paper introduces a few-shot learning scheme to mine pairs automatically without transcriptions. On English, this results in only a small drop in performance. We also - for the first time - consider VPKL on a real low-resource language, Yoruba. While scores are reasonable, here we see a bigger drop in performance compared to using ground truth pairs because the mining is less accurate in Yoruba.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
