张量数据字典学习的Minimax下界
Minimax Lower Bounds on Dictionary Learning for Tensor Data
摘要:
本文提供了基本限制的样本复杂性估计字典张量数据。这项工作的具体重点是在$K$第阶张量数据和情况下,底层字典可以表示在$K$较小的字典。假设数据由这些结构化字典原子的线性组合产生,并通过高斯白噪声观察。这项工作首先提供了一个一般的下界的极小极大风险的字典学习这样的张量数据,然后适应证明技术的情况下,稀疏和稀疏高斯线性组合的专门结果。结果表明,张量数据的字典学习的样本复杂度可以显着低于非结构化数据:对于非结构化数据,它与字典维度的乘积线性缩放,而对于张量结构数据,边界与(较小)组件字典的维度乘积之和线性缩放。对于二阶张量数据的情况,给出了一个部分逆,以表明本文中的界可以是紧的。这涉及到开发一种算法,用于从嘈杂的张量数据中学习高度结构化的字典。最后,数值实验突出了与在字典学习期间明确考虑张量数据结构相关的优势。
英文摘要:
This paper provides fundamental limits on the sample complexity of estimating dictionaries for tensor data. The specific focus of this work is on $K$th-order tensor data and the case where the underlying dictionary can be expressed in terms of $K$ smaller dictionaries. It is assumed the data are generated by linear combinations of these structured dictionary atoms and observed through white Gaussian noise. This work first provides a general lower bound on the minimax risk of dictionary learning for such tensor data and then adapts the proof techniques for specialized results in the case of sparse and sparse-Gaussian linear combinations. The results suggest the sample complexity of dictionary learning for tensor data can be significantly lower than that for unstructured data: for unstructured data it scales linearly with the product of the dictionary dimensions, whereas for tensor-structured data the bound scales linearly with the sum of the product of the dimensions of the (smaller) component dictionaries. A partial converse is provided for the case of 2nd-order tensor data to show that the bounds in this paper can be tight. This involves developing an algorithm for learning highly-structured dictionaries from noisy tensor data. Finally, numerical experiments highlight the advantages associated with explicitly accounting for tensor data structure during dictionary learning.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。