去噪余弦相似度：一种理论驱动的用于高效表示学习的方法
Denoising Cosine Similarity: A Theory-Driven Approach for Efficient Representation Learning
AI总结：
本文针对表示学习中数据含噪问题，提出具有去噪特性的dCS损失函数，并构造了有统计保证的估计量，在视觉和语音任务上验证了其优于基线目标函数。
摘要：
表示学习对机器学习的研究和实践的影响日益增大，因为它能够学习到可高效应用于各种下游任务的表示。然而，近期的工作很少关注这样一个事实：在表示学习阶段使用的现实世界数据集通常受到噪声污染，这会降低所学表示的质量。本文解决在原始数据集中学习对噪声具有鲁棒性的表示的问题。为此，受近期去噪工作以及表示学习中基于余弦相似度的目标函数所取得成功的启发，我们提出了去噪余弦相似度（dCS）损失。dCS损失是一种改进的余弦相似度损失，并融入了去噪特性，这一特性得到了我们的理论和实证发现的支持。为了使dCS损失可实现，我们还构造了具有统计保证的dCS损失估计量。最后，我们通过实验证明了dCS损失在视觉和语音领域相对于基线目标函数的效率。
英文摘要：
Representation learning has been increasing its impact on the research and practice of machine learning, since it enables to learn representations that can apply to various downstream tasks efficiently. However, recent works pay little attention to the fact that real-world datasets used during the stage of representation learning are commonly contaminated by noise, which can degrade the quality of learned representations. This paper tackles the problem to learn robust representations against noise in a raw dataset. To this end, inspired by recent works on denoising and the success of the cosine-similarity-based objective functions in representation learning, we propose the denoising Cosine-Similarity (dCS) loss. The dCS loss is a modified cosine-similarity loss and incorporates a denoising property, which is supported by both our theoretical and empirical findings. To make the dCS loss implementable, we also construct the estimators of the dCS loss with statistical guarantees. Finally, we empirically show the efficiency of the dCS loss over the baseline objective functions in vision and speech domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
