HodgePhaseNet:Transformer注意力中语义流的离散霍奇分解
HodgePhaseNet: Discrete Hodge Decomposition of Semantic Flows in Transformer Attention
AI总结:
HodgePhaseNet提出将语义流表示为1-上链,通过正交分解分离Transformer注意力中的语义方向与环流,设计了规范不变的注意力接口,为语义关系建模提供了数学架构方案。
摘要:
WavePhaseNet引入了频率分辨的语义表示,QuantumPhaseNet引入了规范协变相位连接,ModularPhaseNet实现了有限循环相位通道及其循环完整度在普通Transformer中的可计算性。这些构造互为补充,但它们本身无法将观测到的成对关系集合分离为全局一致的方向、局部组合的环流以及围绕更大未闭合循环持续存在的环流。HodgePhaseNet通过将语义转移表示为有限定向单纯复形上的反对称实值1-上链来解决该分离问题。利用顶点-边关联矩阵B₁和边-面关联矩阵B₂,每个此类场都具有唯一的正交分解Ω = B₁ᵀΦ + B₂Ψ + H,其中B₁H=0且B₂ᵀH=0。即使其势不唯一,三个分量场仍是唯一的。我们给出了显式投影器、稀疏实现、可积性和规范结果、经验证的数值示例以及重新设计的注意力接口。注意力偏置定义在允许的有序查询-键对上,在边定向的任意选择下保持一致,并通过从可见前缀构建其复形来保留解码器的因果信息边界。注意力输出的规范不变性取决于使用不变或适当补偿的特征。顶点嵌入的简单差值没有共精确或调和分量;因此,真正的关系编码器是不可或缺的。局部环流并非自动矛盾,调和流也并非自动缺失知识。这些解释以及事实错误的任何减少都需要受控的实证评估。本文提出的是数学和架构层面的方案,而非外部自然语言性能结果。
英文摘要:
WavePhaseNet introduced frequency-resolved semantic representations, QuantumPhaseNet introduced gauge-covariant phase connections, and ModularPhaseNet made a finite cyclic phase channel and its cycle holonomy computable in an ordinary Transformer. These constructions are complementary, yet they do not by themselves separate an observed collection of pairwise relations into a globally consistent direction, circulation around local compositions, and circulation that persists around a larger unfilled cycle. HodgePhaseNet addresses this separation by representing semantic transitions as an antisymmetric real-valued 1-cochain on a finite oriented simplicial complex. With vertex-edge incidence matrix $B_1$ and edge-face incidence matrix $B_2$, every such field has a unique orthogonal decomposition $Ω= B_1^TΦ+ B_2Ψ+ H$, where $B_1H=0$ and $B_2^TH=0$. The three component fields are unique even when their potentials are not. We give explicit projectors, a sparse implementation, integrability and gauge results, a verified numerical example, and a redesigned attention interface. The attention bias is defined on allowed ordered query-key pairs, remains consistent under arbitrary choices of edge orientation, and preserves a decoder's causal information boundary by constructing its complex from the visible prefix. Gauge invariance of the attention output is conditional on using invariant or appropriately compensated features. A simple difference of vertex embeddings has no coexact or harmonic component; a genuine relation encoder is therefore indispensable. Local circulation is not automatically contradiction, and harmonic flow is not automatically missing knowledge. These interpretations and any reduction in factual error require controlled empirical evaluation. This paper presents a mathematical and architectural proposal, not external natural-language performance results.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。