We propose Distribution Embedding Networks (DEN) for classification with small data. In the same spirit of meta-learning, DEN learns from a diverse set of training tasks with the goal to generalize to unseen target tasks. Unlike existing approaches which require the inputs of training and target tasks to have the same dimension with possibly similar distributions, DEN allows training and target tasks to live in heterogeneous input spaces. This is especially useful for tabular-data tasks where labeled data from related tasks are scarce. DEN uses a three-block architecture: a covariate transformation block followed by a distribution embedding block and then a classification block. We provide theoretical insights to show that this architecture allows the embedding and classification blocks to be fixed after pre-training on a diverse set of tasks; only the covariate transformation block with relatively few parameters needs to be fine-tuned for each new task. To facilitate training, we also propose an approach to synthesize binary classification tasks, and demonstrate that DEN outperforms existing methods in a number of synthetic and real tasks in numerical studies.
Explore and Exploit the Diverse Knowledge in Model Zoo for Domain Generalization
Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）
SHED: Style-Homogenized Embedding Alignment for Domain Generalization
School of Computer Science and Engineering, Southeast University, Nanjing 210096, China（1 东南大学计算机科学与工程学院，南京 210096，中国）；Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, China（2 教育部计算机网络与信息集成重点实验室（东南大学），中国）
Finding Diverse and Predictable Subgraphs for Graph Domain Generalization
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Center for Excellence in Brain Science and Intelligence Technology, CAS（中国科学院脑科学与智能技术卓越创新中心）
Less but Better: Generalization Enhancement of Ordinal Embedding via Distributional Margin
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
