面向表征学习的量子启发嵌入投影与相似性度量
Quantum-inspired Embeddings Projection and Similarity Metrics for Representation Learning
AI总结：
本文提出含量子启发相似性度量的量子电路投影头，将经典嵌入映射至 Hilbert 空间并压缩 BERT 嵌入，在 TREC 基准上以少 32 倍参数取得竞争力性能。
摘要：
在过去十年中，表征学习已成为机器学习中的一项关键技术，它将从大量数据中提取的复杂信息嵌入到稠密向量空间中。在众多应用中，表征学习一直是大型语言模型以及基于对比学习的先进计算机视觉系统的关键构建模块。表征学习系统的一个核心组件是投影头，它将原始嵌入映射到不同的、通常是压缩后的空间，同时保留向量之间的相似性关系。在本文中，我们提出了一种量子启发的投影头，其中包含相应的量子启发相似性度量。具体而言，我们将经典嵌入映射到 Hilbert 空间中的量子态，并引入一种基于量子电路的投影头以降低嵌入维度。为评估该方法的有效性，我们扩展了 BERT 语言模型，将我们的投影头集成进去以进行嵌入压缩。我们在使用 TREC 2019 和 TREC 2020 Deep Learning 基准的信息检索任务上，比较了采用我们的量子启发投影头压缩的嵌入与采用经典投影头压缩的嵌入的性能。结果表明，我们的量子启发方法在使用参数减少 32 倍的情况下，取得了与经典方法相比具有竞争力的性能。此外，在从头开始训练时，它表现尤其出色，特别是在较小的数据集上。本工作不仅凸显了量子启发方法的有效性，还强调了在神经网络中使用高效、专用的低纠缠电路仿真作为一种强大的量子启发技术的实用性。
英文摘要：
Over the last decade, representation learning, which embeds complex information extracted from large amounts of data into dense vector spaces, has emerged as a key technique in machine learning. Among other applications, it has been a key building block for large language models and advanced computer vision systems based on contrastive learning. A core component of representation learning systems is the projection head, which maps the original embeddings into different, often compressed spaces, while preserving the similarity relationship between vectors. In this paper, we propose a quantum-inspired projection head that includes a corresponding quantum-inspired similarity metric. Specifically, we map classical embeddings onto quantum states in Hilbert space and introduce a quantum circuit-based projection head to reduce embedding dimensionality. To evaluate the effectiveness of this approach, we extended the BERT language model by integrating our projection head for embedding compression. We compared the performance of embeddings, which were compressed using our quantum-inspired projection head, with those compressed using a classical projection head on information retrieval tasks using the TREC 2019 and TREC 2020 Deep Learning benchmarks. The results demonstrate that our quantum-inspired method achieves competitive performance relative to the classical method while utilizing 32 times fewer parameters. Furthermore, when trained from scratch, it notably excels, particularly on smaller datasets. This work not only highlights the effectiveness of the quantum-inspired approach but also emphasizes the utility of efficient, ad hoc low-entanglement circuit simulations within neural networks as a powerful quantum-inspired technique.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
