We present a solution to scale spectral algorithms for learning sequence functions. We are interested in the case where these functions are sparse (that is, for most sequences they return 0). Spectral algorithms reduce the learning problem to the task of computing an SVD decomposition over a special type of matrix called the Hankel matrix. This matrix is designed to capture the relevant statistics of the training sequences. What is crucial is that to capture long range dependencies we must consider very large Hankel matrices. Thus the computation of the SVD becomes a critical bottleneck. Our solution finds a subset of rows and columns of the Hankel that realizes a compact and informative Hankel submatrix. The novelty lies in the way that this subset is selected: we exploit a maximal bipartite matching combinatorial algorithm to look for a sub-block with full structural rank, and show how computation of this sub-block can be further improved by exploiting the specific structure of Hankel matrices.
Dept. Computing Science & Alberta Machine Intelligence Institute (Amii), University of Alberta(计算科学系及阿尔伯塔人工智能研究所(Amii),阿尔伯塔大学);Canada CIFAR AI Chair, Amii(加拿大CIFAR人工智能主席,Amii)
FAST: Topology-Aware Frequency-Domain Distribution Matching for Coreset Selection
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University(人机混合增强智能国家重点实验室,人工智能与机器人研究院,西安交通大学);School of Software Engineering, State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University(软件工程学院,人机混合增强智能国家重点实验室,人工智能与机器人研究院,西安交通大学);School of Mathematical Sciences, Nankai University(数学科学学院,南开大学)
Minibatch Selection for Language Models via Partition Matroid Constrained Gradient Matching
Indian Institute of Technology Bombay(印度理工学院班加罗尔);Department of Computer Science and Engineering(计算机科学与工程系);Centre for Machine Intelligence and Data Science(机器智能与数据科学中心);Microsoft Research India(微软印度研究院);Microsoft India(微软印度)
Matched sample selection with GANs for mitigating attribute confounding
University of California at Berkeley(加州大学伯克利分校);Massachusetts Institute of Technology(麻省理工学院);California Institute of Technology(加州理工学院);Amazon Web Services(亚马逊网络服务)
Data Collaboration Analysis with Orthonormal Basis Selection and Alignment
Graduate School of Science and Technology, University of Tsukuba(筑波大学科学技术研究生院);Institute of Systems and Information Engineering, and the Center for Artificial Intelligence Research, Tsukuba Institute for Advanced Research (TIAR), University of Tsukuba(筑波大学系统与信息工程研究所、人工智能研究中心、筑波先进研究所(TIAR))
本文提出Orthonormal Data Collaboration (ODC),通过强制正交的私有和目标基,将对齐问题转化为经典的正交Procrustes问题,提升准确性和稳定性,实现高效的多方协作训练。
Dictionary Learning with Equiprobable Matching Pursuit
EISLAB, Luleå University of Technology, Sweden(埃斯实验室,吕勒奥理工大学,瑞典);SKF–LTU University Technology Center and EISLAB, Luleå University of Technology, Sweden(斯凯孚-吕勒奥理工大学技术中心及埃斯实验室,吕勒奥理工大学,瑞典);Luleå University of Technology, Sweden(吕勒奥理工大学,瑞典);EISLAB, Luleå University of Technology(埃斯实验室,吕勒奥理工大学);SKF–LTU University Technology Center and EISLAB, Luleå University of Technology(斯凯孚-吕勒奥理工大学技术中心及埃斯实验室,吕勒奥理工大学);Luleå University of Technology(吕勒奥理工大学)