arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2104.02317cs.LG
收藏

一种结合负相关学习的混合集成回归方法

A hybrid ensemble method with negative correlation learning for regression

Beihang University(北京航空航天大学);IMT Atlantique(大西洋国立高等工业技术与矿业学校)

Yun Bai, Ganglin Tian, Yanfei Kang, Suling Jia

AI总结:

本研究提出一种混合集成回归方法,通过内点滤波线性搜索优化目标函数,引入负相关学习惩罚项自动选择并加权异构子模型,构建NCL集成,兼顾多样性与准确性,性能优于现有加权方法。

摘要:

混合集成作为集成学习的一个重要分支,在回归领域蓬勃发展,已有研究证实了多样性的重要性。然而,以往的集成方法在子模型训练阶段考虑多样性,与单一模型相比改进有限。相比之下,本研究从异构模型池中自动选择并加权子模型。它使用内点滤波线性搜索算法求解一个优化问题。目标函数创新性地将负相关学习作为惩罚项纳入其中,从而能够选择出多样化的子模型集合。从每个模型类别中选出最佳子模型来构建NCL集成,其性能优于简单平均和其他最先进的加权方法。通过在目标函数中加入正则化项,也可以改进NCL集成。在实践中,由于模型不确定性,事先难以确定某个数据集的最优子模型。尽管如此,我们的方法仍能达到与潜在最优子模型相当的精度。总之,本研究的价值在于其易用性和有效性,使混合集成能够兼顾多样性与准确性。

英文摘要:

Hybrid ensemble, an essential branch of ensembles, has flourished in the regression field, with studies confirming diversity's importance. However, previous ensembles consider diversity in the sub-model training stage, with limited improvement compared to single models. In contrast, this study automatically selects and weights sub-models from a heterogeneous model pool. It solves an optimization problem using an interior-point filtering linear-search algorithm. The objective function innovatively incorporates negative correlation learning as a penalty term, with which a diverse model subset can be selected. The best sub-models from each model class are selected to build the NCL ensemble, which performance is better than the simple average and other state-of-the-art weighting methods. It is also possible to improve the NCL ensemble with a regularization term in the objective function. In practice, it is difficult to conclude the optimal sub-model for a dataset prior due to the model uncertainty. Regardless, our method would achieve comparable accuracy as the potential optimal sub-models. In conclusion, the value of this study lies in its ease of use and effectiveness, allowing the hybrid ensemble to embrace diversity and accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑