Existing gradient-based meta-learning approaches to few-shot learning assume that all tasks have the same input feature space. However, in the real world scenarios, there are many cases that the input structures of tasks can be different, that is, different tasks may vary in the number of input modalities or data types. Existing meta-learners cannot handle the heterogeneous task distribution (HTD) as there is not only global meta-knowledge shared across tasks but also type-specific knowledge that distinguishes each type of tasks. To deal with task heterogeneity and promote fast within-task adaptions for each type of tasks, in this paper, we propose HetMAML, a task-heterogeneous model-agnostic meta-learning framework, which can capture both the type-specific and globally shared knowledge and can achieve the balance between knowledge customization and generalization. Specifically, we design a multi-channel backbone module that encodes the input of each type of tasks into the same length sequence of modality-specific embeddings. Then, we propose a task-aware iterative feature aggregation network which can automatically take into account the context of task-specific input structures and adaptively project the heterogeneous input spaces to the same lower-dimensional embedding space of concepts. Our experiments on six task-heterogeneous datasets demonstrate that HetMAML successfully leverages type-specific and globally shared meta-parameters for heterogeneous tasks and achieves fast within-task adaptions for each type of tasks.
Modality-Agnostic Input Channels Enable Segmentation of Brain lesions in Multimodal MRI with Sequences Unavailable During Training
Department of Engineering Science, University of Oxford, Oxford, UK(工程科学系,牛津大学,牛津,英国);Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK(牛津大学临床神经科学系,牛津,英国)
FreeSliders: Training-Free, Modality-Agnostic Concept Sliders for Fine-Grained Diffusion Control in Images, Audio, and Video
Faculty of Computer and Information Science, Ben-Gurion University of the Negev(计算机与信息科学学院,贝内-约尔大学纳格尔分校);Lightricks(Lightricks公司);School of Electrical and Computer Engineering, Ben-Gurion University of the Negev(电气与计算机工程学院,贝内-约尔大学纳格尔分校)
Predicting Crop Yield With Machine Learning: An Extensive Analysis Of Input Modalities And Models On a Field and sub-field Level
University of Kaiserslautern-Landau (RPTU)(凯泽斯劳滕-兰道大学);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);Vision Impulse GmbH(Vision Impulse 有限责任公司)
Sparse-by-Design Cross-Modality Prediction: L0-Gated Representations for Reliable and Efficient Learning
School of Computing, Macquarie University(麦考瑞大学计算学院);Institute for Clarity in Documentation(文档清晰研究所);Inria Paris-Rocquencourt(法国国家信息与自动化研究所巴黎-罗康库尔中心);Rajiv Gandhi University(拉吉夫·甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕尔默研究实验室)
UniAlign: A Model-Agnostic Framework for Robust Network Traffic Classification under Distribution Shifts
Institute for Network Sciences and Cyberspace, Tsinghua University(网络科学与网络空间研究院,清华大学);Department of Computer Science and Technology, Tsinghua University(计算机科学与技术系,清华大学)
CITYREP: A Unified Benchmark for Urban Representations Across Cities, Tasks, and Modalities
SpaceTimeLab, University College London, UK(伦敦大学空间时间实验室);DIMPact, University College London, UK(伦敦大学3DIMPact实验室);School of Resource and Environmental Sciences, Wuhan University, China(武汉大学资源与环境科学学院);State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, China(武汉大学测绘遥感信息工程国家重点实验室)
CARL: Camera-Agnostic Representation Learning for Spectral Image Analysis
Siemens AG(西门子股份公司);Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ)(德国癌症研究中心智能医疗系统部门);National Center for Tumor Diseases (NCT), NCT Heidelberg(肿瘤国家研究中心(NCT)海德堡分部);HIDSS4Health, Heidelberg(海德堡HIDSS4Health);Department of Urology and Urosurgery, University Medical Center Mannheim(曼海姆大学医学中心泌尿外科与泌尿外科部门);Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital(海德堡大学医院普通外科、 visceral外科和移植外科部门);Division of Intelligent Systems and Robotics in Urology, DKFZ Heidelberg(泌尿外科智能系统和机器人技术部门,德国癌症研究中心海德堡分部);DKFZ Hector Cancer Institute, University Medical Center Mannheim(DKFZ海德堡癌症研究所,曼海姆大学医学中心)
Actor-agnostic Multi-label Action Recognition with Multi-modal Query
University of Surrey(萨里大学);CVSSP(视觉、语音与信号处理中心);Surrey Institute for People-Centred AI(萨里以人为本人工智能研究所);iFlyTek-Surrey Joint Research Center on AI(科大讯飞-萨里人工智能联合研究中心);School of Veterinary Medicine(兽医学院)