Generalizing knowledge to unseen domains, where data and labels are unavailable, is crucial for machine learning models. We tackle the domain generalization problem to learn from multiple source domains and generalize to a target domain with unknown statistics. The crucial idea is to extract the underlying invariant features across all the domains. Previous domain generalization approaches mainly focused on learning invariant features and stacking the learned features from each source domain to generalize to a new target domain while ignoring the label information, which will lead to indistinguishable features with an ambiguous classification boundary. For this, one possible solution is to constrain the label-similarity when extracting the invariant features and to take advantage of the label similarities for class-specific cohesion and separation of features across domains. Therefore we adopt optimal transport with Wasserstein distance, which could constrain the class label similarity, for adversarial training and also further deploy a metric learning objective to leverage the label information for achieving distinguishable classification boundary. Empirical results show that our proposed method could outperform most of the baselines. Furthermore, ablation studies also demonstrate the effectiveness of each component of our method.
Structural-Aware Sentence Similarity with Recursive Optimal Transport
Research Institute of Information Technology(信息技术研究所);Beijing National Research Center for Information Science and Technology (BNRist)(北京信息科学与技术国家研究中心);Tsinghua University(清华大学)
Discriminative Similarity for Clustering and Semi-Supervised Learning
Snap Research(Snap 研究院);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Microsoft Research(微软研究院);National University of Singapore(新加坡国立大学);Beckman Institute, University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校贝克曼研究所)
Department of Computer Science and Technology(计算机科学与技术系);MIIT Key Laboratory of Pattern Analysis and Machine Intelligence(工业和信息化部模式分析与机器智能重点实验室);Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Jiangsu Province(江苏省)
我们推导出上界的泛化误差的学习算法的基础上,他们的\emph{algorithmic transport cost}:预期的瓦瑟斯坦距离之间的输出假设和输出假设的输入示例为条件。边界提供了一种新的方法来研究学习算法的推广,从最优传输的观点,并施加更少的限制,如亚高斯或有界的损失函数。我们进一步提供了几个上界的算法运输成本的全变差距离,相对熵(或KL发散),和VC维,从而进一步桥接最优运输理论和信息理论与统计学习理论。此外,我们还研究了损失函数的不同条件下,学习算法的泛化误差可以由不同的概率距离分布之间的上限与输出假设和/或输入数据。最后,在我们建立的框架下,我们分析了深度学习中的泛化,并得出结论,随着层数的增加,深度神经网络(DNN)中的泛化误差呈指数下降到零。我们对深度学习泛化错误的分析主要利用了DNN的层次结构和$f$ -发散的收缩特性,这可能对分析其他具有层次结构的学习模型具有独立的意义。
School of Computer Science, Guangdong University of Technology(广东工业大学计算机科学学院);Pazhou Laboratory (Huangpu)(黄埔实验室);College of Science, Shantou University(汕头大学理学院);Didi Chuxing(滴滴出行)
Does CLIP's Generalization Performance Mainly Stem from High Train-Test Similarity?
University of Tübingen(蒂宾根大学);Tübingen AI Center(蒂宾根人工智能中心);Max-Planck-Institute for Intelligent Systems, Tübingen(马克斯·普朗克智能系统研究所蒂宾根分所);ELLIS Institute Tübingen(ELLIS研究所蒂宾根分所)
Variational Adapter for Cross-modal Similarity Representation
School of Remote Sensing and Information Engineering(遥感与信息工程学院);Wuhan University(武汉大学);School of Data Science and Engineering(数据科学与工程学院);East China Normal University(华东师范大学);State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing(测绘遥感信息工程国家重点实验室)
A Similarity Paradigm Through Textual Regularization Without Forgetting
Shanghai Jiao Tong University(上海交通大学);Hong Kong Baptist University(香港浸会大学);Northeastern University(东北大学);University of Science and Technology of China(中国科学技术大学);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳))
Peking University(北京大学);Cornell University(康奈尔大学);Huazhong University of Science and Technology(华中科技大学);MIT(麻省理工学院);Beijing Institute for General Artificial Intelligence(北京通用人工智能研究院);UCLA(加州大学洛杉矶分校)
Self-similarity-based super-resolution of photoacoustic angiography from hand-drawn doodles
Tsinghua University(清华大学);Tsinghua Shenzhen International Graduate School(清华大学深圳国际研究生院);Tsinghua-Berkeley Shenzhen Institute(清华-伯克利深圳学院);Xidian University(西安电子科技大学);South China Normal University(华南师范大学);College of Biophotonics(生物光子学学院);Institute of Laser Life Science(激光生命科学研究所)