发表机构
Korea University; Sookmyung Women’s University; Ulsan National Institute of Science and Technology; Purdue University; Seoul National University Hospital（高丽大学; 淑明女子大学; 蔚山国立科学技术研究院; 普渡大学; 首尔国立大学医院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
提出SQUARE，一种结构化量子表示适配器，通过幅度编码和参数化量子电路在冻结语言模型的低维瓶颈中实现二次特征交互，在八个任务上平均准确率达0.7565，优于经典基线。
AI 中文摘要
冻结语言模型（LM）越来越多地被用作下游重排序、评分和偏好建模的固定特征提取器，这引发了一个实际问题：一个紧凑模块应如何在固定的低维瓶颈中表示特征之间的交互？常见的线性和低秩适配器在适配模块本身保持线性，而显式的二阶替代方案通过直接参数化或预定义分解引入成对交互。我们提出SQUARE，一种结构化量子表示适配器，它将瓶颈向量进行幅度编码，应用参数化量子电路，并测量所得状态。我们证明每个基概率特征恰好是瓶颈坐标中的归一化二次形式，而额外的Pauli-$Z$读出是这些概率的带符号线性组合。因此，测量的映射可以通过一小组共享电路参数对$O(d^2)$个坐标对上的交互进行参数化，其中$d$是瓶颈维度。它在经典归一化二次特征类内部（而非外部）提供了一种结构化参数化。在八个源自GLUE的受控交互任务和五个共享种子的不相交同流水线评估中，SQUARE实现了0.7565的平均测试准确率，而仿射归一化二次预测器为0.7355，评估的参数匹配Givens混合模型为0.7271，MLP为0.6817，冻结电路控制为0.6155。在减少监督的情况下，它也显示出相对于评估的最强经典比较器的一致增益，在多个冻结LM骨干上具有相同的定性模式。所有电路实验使用模拟，而学习到的特征映射可以在批量PyTorch中精确评估，无需量子硬件。
英文摘要
Frozen language models (LMs) are increasingly used as fixed feature extractors for downstream reranking, scoring, and preference modeling, raising a practical question: how should a compact module represent interactions among features in a fixed low-dimensional bottleneck? Common linear and low-rank adapters remain linear at the adaptation module itself, whereas explicit second-order alternatives introduce pairwise interactions through direct parameterization or predefined factorizations. We propose SQUARE, a Structured QUAntum REpresentation adapter that amplitude-encodes the bottleneck vector, applies a parameterized quantum circuit, and measures the resulting state. We show that each basis-probability feature is exactly a normalized quadratic form in the bottleneck coordinates, while the additional Pauli-$Z$ readouts are signed linear combinations of these probabilities. The measured map can therefore parameterize interactions over $O(d^2)$ coordinate pairs through a small set of shared circuit parameters, where $d$ is the bottleneck dimension. It provides a structured parameterization within, rather than beyond, the classical normalized-quadratic feature class. In a disjoint same-pipeline evaluation over eight GLUE-derived controlled interaction tasks and five shared seeds, SQUARE achieves an average test accuracy of $0.7565$, compared with $0.7355$ for an affine normalized-quadratic predictor, $0.7271$ for the evaluated parameter-matched Givens mixing model, $0.6817$ for an MLP, and $0.6155$ for a frozen-circuit control. Under reduced supervision, it also shows consistent gains over the strongest evaluated classical comparator, with the same qualitative pattern across multiple frozen LM backbones. All circuit experiments use simulation, while the learned feature map can be evaluated exactly in batched PyTorch without quantum hardware.
Comments39 pages, 5 figures; includes supplementary appendices
