arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.04727cs.LGcs.AI
收藏

使用表示学习的任务感知调制:一种用于环境系统中少样本学习的方法

Task Aware Modulation using Representation Learning: An Approach for Few Shot Learning in Environmental Systems

University of Minnesota(明尼苏达大学)

Arvind Renganathan, Rahul Ghosh, Ankush Khandelwal, Vipin Kumar

AI总结:

本文提出多模态元学习框架TAM-RL,通过调制网络和基础网络学习任务特定参数,在有限数据下高效适应环境系统异构任务,在GPP和径流预测上显著优于基线且训练速度提升3倍以上。

摘要:

我们引入了 TAM-RL(使用表示学习的任务感知调制),这是一种用于异构系统少样本学习的新型多模态元学习框架,专为实体共享相同底层正向模型但因实体特定特征而表现出异质性的科学与工程问题而设计。TAM-RL 利用包含调制网络和基础网络的摊销训练过程来学习任务特定的调制参数,从而能够利用有限数据高效适应新任务。我们在两个真实环境数据集上评估了 TAM-RL:总初级生产力(GPP)预测和径流预测,证明其相比现有元学习方法有显著提升。在 FLUXNET 数据集上,TAM-RL 仅用一个月的少样本数据就将 RMSE 较 MMAML 提高了 18.9%,而在径流预测中,使用一年数据实现了 8.21% 的提升。合成数据实验进一步验证了 TAM-RL 在异构任务分布中的优越性能,在最异构的设置中优于基线方法。值得注意的是,TAM-RL 具有极高的计算效率,与基于梯度的元学习方法相比,训练时间至少快 3 倍,同时由于复杂度降低,训练过程也更为简单。消融研究突显了预训练和适应机制在 TAM-RL 性能中的重要性。

英文摘要:

We introduce TAM-RL (Task Aware Modulation using Representation Learning), a novel multimodal meta-learning framework for few-shot learning in heterogeneous systems, designed for science and engineering problems where entities share a common underlying forward model but exhibit heterogeneity due to entity-specific characteristics. TAM-RL leverages an amortized training process with a modulation network and a base network to learn task-specific modulation parameters, enabling efficient adaptation to new tasks with limited data. We evaluate TAM-RL on two real-world environmental datasets: Gross Primary Product (GPP) prediction and streamflow forecasting, demonstrating significant improvements over existing meta-learning methods. On the FLUXNET dataset, TAM-RL improves RMSE by 18.9\% over MMAML with just one month of few-shot data, while for streamflow prediction, it achieves an 8.21\% improvement with one year of data. Synthetic data experiments further validate TAM-RL's superior performance in heterogeneous task distributions, outperforming the baselines in the most heterogeneous setting. Notably, TAM-RL offers substantial computational efficiency, with at least 3x faster training times compared to gradient-based meta-learning approaches while being much simpler to train due to reduced complexity. Ablation studies highlight the importance of pretraining and adaptation mechanisms in TAM-RL's performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑