arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.02720cs.CV
收藏

TenExp: 基于专家混合的张量分解结构搜索框架

Tensor Decomposition Structure Search Framework from an Interaction Perspective

School of Mathematical Sciences/Research Center for Image and Vision Computing, University of Electronic Science and Technology of China(数学科学学院/图像与视觉计算研究中心,电子科技大学);School of Information Science and Technology, Southwest Jiaotong University(信息科学与技术学院,西南交通大学);School of Mathematics and Statistics and Ministry of Education Key Lab of Intelligent Networks and Network Security, Xi’an Jiaotong University(数学与统计学学院及教育部智能网络与网络安全重点实验室,西安交通大学);Pazhou Laboratory (Huangpu), Guangzhou, Guangdong(琶洲实验室(黄埔),广州,广东)

Ting-Wei Zhou, Xi-Le Zhao, Sheng Liu, Wei-Hao Wu, Yu-Bang Zheng, Deyu Meng

AI总结:

TenExp通过基于专家混合的方法,实现了张量分解结构的动态选择和混合,提升了张量分解的准确性和灵活性。

摘要:

最近,张量分解继续涌现并受到越来越多的关注。选择一个合适的张量分解来精确捕捉数据背后的低秩结构是张量分解领域的核心,这仍然是一个具有挑战性且相对未被深入探索的问题。当前的张量分解结构搜索方法仍然受限于固定的因子交互家族(例如张量收缩),无法提供分解的混合。为了解决这个问题,我们精心设计了一个基于专家混合的张量分解结构搜索框架(称为TenExp),该框架允许我们在无监督的方式下动态选择和激活合适的张量分解。该框架相比最先进的张量分解结构搜索方法有两个独特的优点。首先,TenExp可以在固定因子交互家族之外提供合适的单一分解。其次,TenExp可以在单一分解之外提供合适的分解混合。理论上,我们还提供了TenExp的近似误差界,这揭示了TenExp的近似能力。在合成和现实数据集上的广泛实验表明,所提出的TenExp相比最先进的基于张量分解的方法具有优越性。

英文摘要:

Recently, tensor decompositions have attracted increasing attention. Fundamentally, different interactions among factors induce distinct tensor decomposition structures (i.e., tensor decomposition). Identifying an appropriate interaction-induced tensor decomposition structure for given data is a fundamental yet challenging problem in tensor modeling, and remains largely under-explored. Existing tensor decomposition structure search methods are typically restricted to a predefined interaction family, such as tensor contraction. To address this problem, we suggest a tensor decomposition structure search framework (I-TSS) from an interaction perspective that can identify either a single structure beyond a predefined interaction family or a mixture of structures involving heterogeneous interaction families. Specifically, we first systematically review existing tensor decomposition structures from an interaction perspective and construct a heterogeneous interaction-induced candidate structure set. Based on it, we propose a unified energy-based rank estimation scheme for heterogeneous interaction-induced candidate structure set. We then introduce the top-$k$ gating mechanism with learnable gating scores to dynamically select a suitable single-candidate structure or combine suitable multi-candidate structures, thereby identifying the tensor decomposition structure in a data-adaptive manner. Theoretically, we derive an approximation error bound for I-TSS, thereby establishing its approximation capability. Extensive experiments on synthetic and real-world datasets demonstrate that I-TSS consistently outperforms state-of-the-art tensor decomposition methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑