In this paper, we provide a new neural-network based perspective on multi-task learning (MTL) and multi-domain learning (MDL). By introducing the concept of a semantic descriptor, this framework unifies MDL and MTL as well as encompassing various classic and recent MTL/MDL algorithms by interpreting them as different ways of constructing semantic descriptors. Our interpretation provides an alternative pipeline for zero-shot learning (ZSL), where a model for a novel class can be constructed without training data. Moreover, it leads to a new and practically relevant problem setting of zero-shot domain adaptation (ZSDA), which is the analogous to ZSL but for novel domains: A model for an unseen domain can be generated by its semantic descriptor. Experiments across this range of problems demonstrate that our framework outperforms a variety of alternatives.
A Shift in Perspective on Causality in Domain Generalization
CHAI Hub(CHAI中心);The University of Edinburgh(爱丁堡大学);University of Manchester(曼彻斯特大学);Archimedes, Athena Research Center(阿奇米德与雅典研究所);King's College London(伦敦国王学院);University College London(伦敦大学学院);National and Kapodistrian University of Athens(雅典国家与卡波迪斯蒂亚纳大学)
Noise May Contain Transferable Knowledge: Understanding Semi-supervised Heterogeneous Domain Adaptation from an Empirical Perspective
Beijing Teleinfo Technology Company Ltd.(北京 Teleinfo 科技有限公司);China Academy of Information and Communications Technology(中国信息通信研究院);Inspur Computer Technology Co., Ltd.(浪潮计算机技术有限公司);Southern University of Science and Technology(南方科技大学);Research Institute of Industrial Internet of Things(工业互联网物联网研究院);Hong Kong University of Science and Technology(香港科技大学);WeBank(微众银行)
On the Hardness of Robustness Transfer: A Perspective from Rademacher Complexity over Symmetric Difference Hypothesis Space
The Pennsylvania State University(宾夕法尼亚州立大学);Michigan State University(密歇根州立大学);Sony AI(索尼人工智能研究院)
最近的研究表明,在$\ell_\infty$攻击下的对抗鲁棒学习比标准的领域自适应更难推广到不同的领域。如何在不同领域之间传递鲁棒性是领域自适应研究的一个关键问题。为了研究对抗性鲁棒领域自适应(或鲁棒性转移)背后的根本困难,我们提出分析控制跨领域泛化的关键复杂性度量:{ \em对称差异假设空间} $\mathcal{H} Δ\mathcal{H}$上的对抗性Rademacher复杂性。对于线性模型,我们表明,这种复杂性的对抗版本总是大于非对抗版本,这揭示了对抗鲁棒领域自适应的内在硬度。我们还建立了这个复杂性措施的上限。然后,我们将它们扩展到ReLU神经网络类,通过在二进制分类设置中上界对抗Rademacher复杂度。最后,尽管鲁棒领域自适应被证明是困难的,我们确实发现鲁棒学习和标准领域自适应之间的正相关关系。我们解释\emph{how adversarial training helps domain adaptation in terms of standard risk}。我们相信,我们的研究结果启动了对抗鲁棒领域自适应的泛化理论研究,并可能揭示分布式对抗鲁棒学习的异构来源,例如,联邦学习场景。