The era of Big Data has spawned unprecedented interests in developing hashing algorithms for efficient storage and fast nearest neighbor search. Most existing work learn hash functions that are numeric quantizations of feature values in projected feature space. In this work, we propose a novel hash learning framework that encodes feature's rank orders instead of numeric values in a number of optimal low-dimensional ranking subspaces. We formulate the ranking subspace learning problem as the optimization of a piece-wise linear convex-concave function and present two versions of our algorithm: one with independent optimization of each hash bit and the other exploiting a sequential learning framework. Our work is a generalization of the Winner-Take-All (WTA) hash family and naturally enjoys all the numeric stability benefits of rank correlation measures while being optimized to achieve high precision at very short code length. We compare with several state-of-the-art hashing algorithms in both supervised and unsupervised domain, showing superior performance in a number of data sets.
快速相似性搜索是大规模信息检索中的关键组成部分，其中语义哈希已经成为将文档表示为二进制哈希码的流行策略。这一领域的最新进展是通过基于神经网络的模型获得的：通过学习重建原始文档来训练生成模型。我们提出了一种新的无监督生成语义哈希方法，\textit{Ranking based Semantic Hashing}（RBSH），它由一个变分和排名为基础的组件。类似于变分自编码器，变分组件被训练以根据其生成的哈希码重建原始文档，并且与先前的工作一样，它只考虑单独的文档。排名组件通过将文档间相似性并入哈希代码生成中、通过铰链损失对文档排序进行建模来解决此限制。为了避免需要标记数据来计算铰链损失，我们使用弱标记器，从而保持该方法完全无监督。针对传统基线和最近最先进的语义哈希方法对四个公开可用的数据集进行了广泛的实验评估，结果表明RBSH在所有评估的哈希码长度上都显着优于所有其他方法。事实上，RBSH哈希码能够执行类似于最先进的哈希码，同时使用2- 4倍的比特。
Robust Hashing for Multi-View Data: Jointly Learning Low-Rank Kernelized Similarity Consensus and Hash Functions
The University of Adelaide（阿德莱德大学）；The University of New South Wales（新南威尔士大学）
学习散列函数/代码用于多视图数据的相似性搜索正吸引越来越多的关注，其中相似的散列代码被分配给表征跨视图的一致邻域关系的数据对象。这类传统方法存在三个固有的局限性：1）它们通常采用两阶段方案，首先构造相似性矩阵，然后是随后的哈希函数学习; 2）这些方法通常是在假设具有多种表示的数据样本是无噪声的基础上开发的，这在实际应用中是不切实际的; 3）由于使用数据库（$O(N)$）中的所有$N$点构建邻域图，往往会导致训练模型繁琐。在本文中，我们激发的问题，联合和有效地训练强大的哈希函数的数据对象与多特征表示，可能是噪声破坏。为了实现鲁棒性和训练效率，我们提出了一种有效和高效地学习低秩核化\footnote{We use kernelized similarity rather than kernel, as it is not a squared symmetric matrix for data-landmark affinity matrix.}散列函数的方法。具体地说，我们利用地标图来构造易处理的多视图相似性矩阵，以自动发现数据中的邻域结构。为了学习鲁棒的哈希函数，使用潜在的低秩核函数来构造哈希函数，以适应线性不可分的数据。特别是，潜在的核相似性矩阵恢复多个基于核的相似性矩阵的秩最小化。在真实世界的多视图数据集上进行的大量实验验证了我们的方法在存在错误损坏的情况下的有效性。
Southwest University（西南大学）；China University of Geosciences（中国地质大学）；George Mason University（乔治梅森大学）；University of Electronic Science and Technology of China（电子科技大学）；Beijing University of Civil Engineering and Architecture（北京建筑大学）
Breaking the Frozen Subspace: Importance Sampling for Low-Rank Optimization in LLM Pretraining
Rice University（Rice大学）；University of North Carolina at Charlotte（北卡罗来纳州立大学）；University of Minnesota Twin Cities（明尼苏达大学双城分校）；University of California, Los Angeles（加州大学洛杉矶分校）；Johns Hopkins University（约翰霍普金斯大学）
Cross-modal retrieval aims to search for data with similar semantic meanings across different content modalities. However, cross-modal retrieval requires huge amounts of storage and retrieval time since it needs to process data in multiple modalities. Existing works focused on learning single-source compact features such as binary hash codes that preserve similarities between different modalities. In this work, we propose a jointly learned deep hashing and quantization network (HQ) for cross-modal retrieval. We simultaneously learn binary hash codes and quantization codes to preserve semantic …
Entity linking is an important problem with many applications. Most previous solutions were designed for settings where annotated training data is available, which is, however, not the case in numerous domains. We propose a light-weight and scalable entity linking method, Eigenthemes, that relies solely on the availability of entity names and a referent knowledge base. Eigenthemes exploits the fact that the entities that are truly mentioned in a document (the "gold entities") tend to form a semantically dense subset of the set of all candidate entities in the document. Geometrically speaking, …
In this paper, we extend the recently proposed multivariate rank energy distance, based on the theory of optimal transport, for statistical testing of distributional similarity, to soft rank energy distance. Being differentiable, this in turn allows us to extend the rank energy to a subspace robust rank energy distance, dubbed Projected soft-Rank Energy distance, which can be computed via optimization over the Stiefel manifold. We show via experiments that using projected soft rank energy one can trade-off the detection power vs the false alarm via projections onto an appropriately selected lo…
Multi-label feature selection based on binary hashing learning and dynamic graph constraints
Key Laboratory of Intelligent Education Technology and Application of Zhejiang Province Zhejiang Normal University（浙江省智能教育技术与应用重点实验室，浙江师范大学）；School of Computing, Mathematics and Engineering, Charles Sturt University（查尔斯·斯特劳特大学计算机、数学与工程学院）
Wuhan University（武汉大学）；Sun Yat-Sen University（中山大学）；University of South Carolina（南卡罗来纳大学）
Hash coding has been widely used in approximate nearest neighbor search for large-scale image retrieval. Given semantic annotations such as class labels and pairwise similarities of the training data, hashing methods can learn and generate effective and compact binary codes. While some newly introduced images may contain undefined semantic labels, which we call unseen images, zeor-shot hashing techniques have been studied. However, existing zeor-shot hashing methods focus on the retrieval of single-label images, and cannot handle multi-label images. In this paper, for the first time, a novel t…
From Low Rank Gradient Subspace Stabilization to Low-Rank Weights: Observations, Theories, and Applications
University of Texas at Austin（德克萨斯大学奥斯汀分校）；Eindhoven University of Technology（埃因霍温理工大学）；University of Surrey（萨里大学）；University of Oxford（牛津大学）；California Institute of Technology（加州理工学院）
Southeast University（东南大学）；Caritas Institute of Higher Education（明爱专上学院）；City University of Hong Kong（香港城市大学）
In this letter, we propose a novel semi-supervised subspace clustering method, which is able to simultaneously augment the initial supervisory information and construct a discriminative affinity matrix. By representing the limited amount of supervisory information as a pairwise constraint matrix, we observe that the ideal affinity matrix for clustering shares the same low-rank structure as the ideal pairwise constraint matrix. Thus, we stack the two matrices into a 3-D tensor, where a global low-rank constraint is imposed to promote the affinity matrix construction and augment the initial pair…
University for Applied Sciences Würzburg-Schweinfurt（维尔茨堡-施韦因富特应用科学大学）
Current supervised learning models cannot generalize well across domain boundaries, which is a known problem in many applications, such as robotics or visual classification. Domain adaptation methods are used to improve these generalization properties. However, these techniques suffer either from being restricted to a particular task, such as visual adaptation, require a lot of computational time and data, which is not always guaranteed, have complex parameterization, or expensive optimization procedures. In this work, we present an approach that requires only a well-chosen snapshot of data to…
Low-Rank Representation (LRR) highly suffers from discarding the locality information of data points in subspace clustering, as it may not incorporate the data structure nonlinearity and the non-uniform distribution of observations over the ambient space. Thus, the information of the observational density is lost by the state-of-art LRR models, as they take a constant number of adjacent neighbors into account. This, as a result, degrades the subspace clustering accuracy in such situations. To cope with deficiency, in this paper, we propose to consider a hypergraph model to facilitate having a …
University of Bristol（布里斯托大学）；University of Cambridge（剑桥大学）
Multivariate time series can often have a large number of dimensions, whether it is due to the vast amount of collected features or due to how the data sources are processed. Frequently, the main structure of the high-dimensional time series can be well represented by a lower dimensional subspace. As vast quantities of data are being collected over long periods of time, it is reasonable to assume that the underlying subspace structure would change over time. In this work, we propose a change-point detection method based on low-rank matrix factorisation that can detect multiple changes in the u…
Robust Recovery of Subspace Structures by Low-Rank Representation
Shanghai Jiao Tong University（上海交通大学）；Microsoft Research Asia（微软亚洲研究院）；Peking University（北京大学）；National University of Singapore（新加坡国立大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Columbia University（哥伦比亚大学）
A low-rank transformation learning framework for subspace clustering and classification is here proposed. Many high-dimensional data, such as face images and motion sequences, approximately lie in a union of low-dimensional subspaces. The corresponding subspace clustering problem has been extensively studied in the literature to partition such high-dimensional data into clusters corresponding to their underlying low-dimensional subspaces. However, low-dimensional intrinsic structures are often violated for real-world observations, as they can be corrupted by errors or deviate from ideal models…
