National Key Laboratory of Space Integrated Information System（空间信息集成国家重点实验室）；Institute of Software Chinese Academy of Sciences（中国科学院软件研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Thrust of Artificial Intelligence（人工智能研究所）；Hong Kong University of Science and Technology（香港科技大学）；Department of Computer Science and Engineering（计算机科学与工程系）；Amazon.com, Inc.（亚马逊公司）
Jingyao Wang, Wenwen Qiang, Changwen Zheng, Hui Xiong, Gang Hua
Meta-learning seeks to learn a well-generalized model initialization from training tasks to solve unseen tasks. From the "learning to learn" perspective, the quality of the initialization is modeled with one-step gradient decent in the inner loop. However, contrary to theoretical expectations, our empirical analysis reveals that this may expose meta-learning to underfitting. To bridge the gap between theoretical understanding and practical implementation, we reconsider meta-learning from the "Learning" lens. We propose that the meta-learning model comprises two interrelated components: parameters for model initialization and a meta-layer for task-specific fine-tuning. These components will lead to the risks of overfitting and underfitting depending on tasks, and their solutions, fewer parameters vs. more meta-layer, are often in conflict. To address this, we aim to regulate the task information the model receives without modifying the data or model structure. Our theoretical analysis indicates that models adapted to different tasks can mutually reinforce each other, highlighting the effective information. Based on this insight, we propose TRLearner, a plug-and-play method that leverages task relation to calibrate meta-learning. It first extracts task relation matrices and then applies relation-aware consistency regularization to guide optimization. Extensive theoretical and empirical evaluations demonstrate its effectiveness.
Is Meta-Learning Out? Rethinking Unsupervised Few-Shot Classification with Limited Entropy
Huazhong University of Science and Technology（华中科技大学）；Nanyang Technological University（南洋理工大学）；University of Washington（华盛顿大学）；University of Copenhagen（哥本哈根大学）
CMoS: Rethinking Time Series Prediction Through the Lens of Chunk-wise Spatial Correlations
Computer Network Information Center, CAS（中国科学院计算机网络信息中心）；University of Chinese Academy of Sciences（中国科学院大学）；Nanjing University（南京大学）；Tsinghua University（清华大学）
