arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.06699quant-phcs.CLcs.LG
收藏

通过重叠干涉实现量子注意力:从经典和多体量子数据中预测序列

Quantum Attention by Overlap Interference: Predicting Classical and Many-Body Quantum Sequences

Dipartimento di Ingegneria Civile, Informatica e delle Tecnologie Aeronautiche, Università degli Studi Roma Tre(土木工程、计算机科学和航空技术系,罗马三大学)

Alessio Pecilli, Matteo Rosati

AI总结:

本文提出了一种基于量子干涉的自注意力机制,通过重叠干涉实现序列预测,展示了其在经典和量子数据中的有效性。

摘要:

我们提出了一种变分量子实现的自注意力(QSA),这是Transformer和大语言模型的核心操作,通过形成重叠加权的过去数据组合来预测序列的未来元素。与以往的方法不同,我们的QSA通过状态重叠的干涉实现所需的非线性,并直接返回Renyi-1/2交叉熵损失作为可观测量的期望值,避免了将幅度编码的预测解码为经典对数几率的需要。此外,QSA自然地容纳了一个受约束、可训练的数据嵌入,将量子态重叠与数据层面的相似性联系起来。我们发现QSA的门复杂度主导缩放为O(T d²),与经典方法的O(T² d)相比,表明在序列长度T远大于嵌入大小d的实用情况下具有优势。在模拟中,我们展示了基于QSA的量子变压器在经典数据和多体横向场Ising量子轨迹上学习序列预测,确立了可训练的注意力作为量子动力学建模的实用原始构件。

英文摘要:

We propose a variational quantum implementation of self-attention (QSA)-the core operation in transformers and large language models-which predicts future elements of a sequence by forming overlap-weighted combinations of past data. At variance with previous approaches, our QSA realizes the required nonlinearity through interference of state overlaps and a degree-$k$ polynomial kernel, and estimates a loss based on Rényi-$1/2$ entropic functionals via two observables' expectation values, avoiding the decoding of amplitude-encoded predictions into classical probabilities. QSA also accommodates a constrained, trainable data-embedding tying state overlaps to data-level similarities. Its dominant end-to-end training complexity scales as $O\left(μ^{-1}k^2Td\right)$, versus $O\left(T d^{k+1}\right)$ of the fairest classical comparison, with $μ$ a training signal; we show numerically that this allows a complexity advantage in the regime where sequence length $T$ dominates the embedding size $d$. In simulations, our QSA-based quantum transformer learns sequence prediction on classical data and on many-body transverse-field Ising trajectories-establishing trainable attention as a practical primitive for quantum dynamical modeling.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑