arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2306.02808cs.LGcs.AI
收藏

基于结构化神经深度搜索的深度主动学习

Deep Active Learning with Structured Neural Depth Search

Sichuan University（四川大学）

Xiaoyun Zhang, Xieyi Ping, Jianwei Zhang

备注：
10 pages, 8 figures, prepare for TNNLS
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对Active-iNAS重复独立训练导致计算开销高的问题，提出基于SVI/SNDS的深度主动学习策略，在深度搜索中使用梯度下降，并在三种查询技术和三个数据集上验证其优于现有方法。

摘要：

先前工作利用基于数据复杂度变化的增量神经网络架构搜索（Active-iNAS）来优化传统主动学习（AL）流程，从而提升了准确率和学习效率。然而，Active-iNAS会训练多个模型，并在每个主动学习周期之后选择泛化性能最佳的模型来查询后续样本。这些相互独立的训练过程导致难以承受的计算预算，效率极低，并限制了搜索灵活性和最终性能。为解决该问题，我们提出一种新颖的主动策略，其方法称为结构化变分推断（SVI）或结构化神经深度搜索（SNDS），借此可在AL过程中的神经网络深度搜索里使用梯度下降方法。同时，我们从理论上证明，当前基于均值场假设的VI方法可能导致较差性能。我们使用三种查询技术和三个数据集应用所提策略，并表明该策略优于当前方法。

英文摘要：

Previous work optimizes traditional active learning (AL) processes with incremental neural network architecture search (Active-iNAS) based on data complexity change, which improves the accuracy and learning efficiency. However, Active-iNAS trains several models and selects the model with the best generalization performance for querying the subsequent samples after each active learning cycle. The independent training processes lead to an insufferable computational budget, which is significantly inefficient and limits search flexibility and final performance. To address this issue, we propose a novel active strategy with the method called structured variational inference (SVI) or structured neural depth search (SNDS) whereby we could use the gradient descent method in neural network depth search during AL processes. At the same time, we theoretically demonstrate that the current VI-based methods based on the mean-field assumption could lead to poor performance. We apply our strategy using three querying techniques and three datasets and show that our strategy outperforms current methods.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑