arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.12255cs.LGstat.ML
收藏

高效数据子集选择以跨模型泛化训练:直推式与归纳式网络

Efficient Data Subset Selection to Generalize Training Across Models: Transductive and Inductive Networks

IIT Bombay(印度理工学院孟买分校);Google(谷歌公司);UT Dallas(德克萨斯大学达拉斯分校)

Eeshaan Jain, Tushar Nandy, Gaurav Aggarwal, Ashish Tendulkar, Rishabh Iyer, Abir De

AI总结:

提出SubSelNet框架,利用注意力机制和架构图结构近似模型预测,实现跨架构的数据子集选择,包含直推式和归纳式两种变体,实验表明在多个真实数据集上优于现有方法。

摘要:

现有的用于高效学习的子集选择方法主要采用离散组合和特定模型的方法,这些方法缺乏泛化能力。对于未见过的架构,无法使用为不同模型选择的子集。为解决此问题,我们提出了SubSelNet,一个可训练的子集选择框架,能够跨架构泛化。首先,我们引入一个基于注意力的神经组件,利用架构的图结构,并作为已训练深度神经网络的替代品,用于快速模型预测。然后,我们利用这些预测来构建子集采样器。这自然为我们提供了SubSelNet的两种变体。第一种变体是直推式的(称为Transductive-SubSelNet),它通过求解一个小型优化问题,为每个模型单独计算子集。由于用模型近似器替代了显式模型训练,这种优化仍然非常快速。第二种变体是归纳式的(称为Inductive-SubSelNet),它使用训练好的子集选择器计算子集,无需任何优化。我们的实验表明,我们的模型在多个真实数据集上优于多种方法。

英文摘要:

Existing subset selection methods for efficient learning predominantly employ discrete combinatorial and model-specific approaches which lack generalizability. For an unseen architecture, one cannot use the subset chosen for a different model. To tackle this problem, we propose $\texttt{SubSelNet}$, a trainable subset selection framework, that generalizes across architectures. Here, we first introduce an attention-based neural gadget that leverages the graph structure of architectures and acts as a surrogate to trained deep neural networks for quick model prediction. Then, we use these predictions to build subset samplers. This naturally provides us two variants of $\texttt{SubSelNet}$. The first variant is transductive (called as Transductive-$\texttt{SubSelNet}$) which computes the subset separately for each model by solving a small optimization problem. Such an optimization is still super fast, thanks to the replacement of explicit model training by the model approximator. The second variant is inductive (called as Inductive-$\texttt{SubSelNet}$) which computes the subset using a trained subset selector, without any optimization. Our experiments show that our model outperforms several methods across several real datasets

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑