arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2608.10416cs.DScs.AIcs.CLcs.LG

Riemann GeoResolver:从欧氏Resolver到双曲-球面几何的非欧注意力框架

Riemann GeoResolver: A Non-Euclidean Attention Framework from Euclidean Resolver to Hyperbolic-Spherical Geometry

Liangchen Ge

首次发表
浏览论文内容

中文总结 AI 辅助

该研究提出逆距离注意力的理论基础,构建了Riemann GeoResolver非欧注意力框架,涵盖欧氏原型与双曲-球面扩展,证明了相关定理,建立从欧氏到非欧注意力的理论脉络。

中文摘要 AI 辅助

我们提出了逆距离注意力(inverse-distance attention)的理论基础,涵盖从其欧氏原型(Resolver)到非欧实现(Riemann GeoResolver)的完整脉络。欧氏部分确立了三项核心定理:(1)电路分离性——逆距离注意力(IDA)以O(1)资源实现精确检索,而softmax需要Ω((log n)²)的宽度;(2)Polyak-Lojasiewicz不等式的常数比softmax强Ω(e^(Δ²/√d)/Δ²),这意味着线性收敛性、在低秩/聚类假设下的O(log n) Lipschitz缩放、Θ(1)的Hessian扩散,且不存在虚假局部极小值;(3)与宽度无关的有效秩边界,可限制噪声记忆——当d_h≥n时,softmax会记忆任意标签,而IDA将测试误差限制在O(η²)。非欧扩展则基于该原型构建,用双曲测地距离替代欧氏距离用于存储,用球面测地距离用于路由。Riemann GeoResolver框架包含十个集成模块:四个HIDA算子,每个token的计算量从Θ(n²)到Θ(1);带可证误差边界的双曲曲率压缩(HCC);带有梯度下界定理的HyperGate;具有球面类比PL不等式的球面逆距离注意力(SIDA);带有O(log T)遗憾边界的动态记忆生成(DMG);以及带有质量和通信边界的测地稀疏路由(GSR)。欧氏定理已完整证明,非欧扩展定理通过类似论证证明。本研究构建了一条理论脉络:从作为特例的欧氏注意力,到双曲记忆,再到球面检索。

英文摘要

We present a theoretical foundation for inverse-distance attention, from its Euclidean prototype (Resolver) to its non-Euclidean realization (Riemann GeoResolver). The Euclidean part establishes three core theorems: (1) circuit separation---IDA achieves exact retrieval with $\mathcal{O}(1)$ resources while softmax requires $Ω((\log n)^2)$ width; (2) a Polyak--Lojasiewicz inequality with $Ω(e^{Δ^2/\sqrt{d}}/Δ^2)$ stronger constant than softmax, implying linear convergence, $\mathcal{O}(\log n)$ Lipschitz scaling under a low-rank/clustering assumption, $Θ(1)$ Hessian spread, and absence of spurious local minima; (3) a width-independent effective rank bound that limits noise memorization---softmax memorizes arbitrary labels when $d_h\ge n$, while IDA limits test error to $\mathcal{O}(η^2)$. The non-Euclidean extension then builds upon this prototype, replacing Euclidean distance with hyperbolic geodesic distance for storage and spherical geodesic distance for routing. The Riemann GeoResolver framework comprises ten integrated modules: four HIDA operators spanning $Θ(n^2)$ to $Θ(1)$ per token; Hyperbolic Curvature Compression (HCC) with provable error bounds; HyperGate with gradient lower-bound theorem; Spherical Inverse Distance Attention (SIDA) with sphere-analog PL inequalities; Dynamic Memory Genesis (DMG) with $\mathcal{O}(\log T)$ regret bounds; and Geodesic Sparse Routing (GSR) with quality and communication bounds. The Euclidean theorems are proved in full; the non-Euclidean extension theorems are proved with analogous arguments. This work establishes a theoretical arc: from Euclidean attention as a special case, to hyperbolic memory, to spherical retrieval.

补充信息

↑