arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2305.16360cs.LGcs.CEstat.AP
收藏

基于均衡专家混合模型的多变量软测量中的任务关系建模

Modeling Task Relationships in Multi-variate Soft Sensor with Balanced Mixture-of-Experts

Zhejiang University(浙江大学)

Yuxin Huang, Hao Wang, Zhaoran Liu, Licheng Pan, Haozhe Li, Xinggao Liu

AI总结:

本文提出均衡专家混合模型BMoE,结合MMoE与任务梯度均衡模块,以缓解多变量软测量中的负迁移问题,并在硫磺回收装置上取得优于基线的性能。

摘要:

对多个质量变量的准确估计对于构建工业软测量模型至关重要,而这类模型长期以来一直面临数据效率和负迁移问题的困扰。在任务之间共享骨干参数的方法解决了数据效率问题;然而,它们仍未能缓解负迁移问题。为解决这一问题,本文提出了一种均衡专家混合模型(BMoE),该模型由一个多门控专家混合(MMoE)模块和一个任务梯度均衡(TGB)模块组成。MoE模块旨在刻画任务关系,而TGB模块则动态地平衡任务之间的梯度。两者协同工作以缓解负迁移问题。在典型的硫磺回收装置上进行的实验表明,BMoE能够有效建模任务关系并平衡训练过程,且性能显著优于基线模型。

英文摘要:

Accurate estimation of multiple quality variables is critical for building industrial soft sensor models, which have long been confronted with data efficiency and negative transfer issues. Methods sharing backbone parameters among tasks address the data efficiency issue; however, they still fail to mitigate the negative transfer problem. To address this issue, a balanced Mixture-of-Experts (BMoE) is proposed in this work, which consists of a multi-gate mixture of experts (MMoE) module and a task gradient balancing (TGB) module. The MoE module aims to portray task relationships, while the TGB module balances the gradients among tasks dynamically. Both of them cooperate to mitigate the negative transfer problem. Experiments on the typical sulfur recovery unit demonstrate that BMoE models task relationship and balances the training process effectively, and achieves better performance than baseline models significantly.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑