arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.17848cs.LG
收藏

持久态位置编码:一种用于持久态图的极简Transformer方案

Persistence Positional Encoding: A Minimal Transformer Recipe for Persistence Diagrams

Ulsan National Institute of Science and Technology(蔚山科学技术院);Graduate School of Artificial Intelligence(人工智能学院)

Eunwoo Heo, Ganghyun Kim, Jaejun Yoo

AI总结:

该研究提出极简Transformer方案PPE,通过固定随机傅里叶特征映射持久态图的出生-死亡对,在多图基准上表现优异,还证明其通用性并提供统一比较协议。

摘要:

持久态图(PDs)将多尺度拓扑信息表示为出生-死亡对(b,d)的多重集。然而,如何将其与Transformer结合使用仍不明确。将PDs与Transformer关联需要选择如何对拓扑信息进行分词并将其暴露给自注意力机制。现有方法针对分词、聚合、Transformer块或注意力机制采用了专门设计,但由于报告的结果很少共享PD输入、数据划分或模型选择规则,因此难以进行比较。我们转而将与图相关的设计限制在点表示上,通过固定随机傅里叶特征映射每个出生-死亡对,同时保持Transformer主干不变,并将该方案称为持久态位置编码(PPE)。在8个TU图基准(TU8)的匹配对照实验中,仅改变点表示就使宏观准确率变化了5.49至6.47个百分点(pp),而改变下游模块仅使准确率变化了0.47至1.45个百分点,这表明点表示是所测试的主要设计变量。我们在共享PD输入、外层折叠和仅验证选择的统一协议下,进一步重新评估了向量化、核函数、学习表示和基于Transformer的模型。在所有评估的PD方法和下游配置中,PPE在大多数基准(包括TU8、ORBIT5K和MCB-C)上取得了最高均值。最后,我们证明网格平均在任何分辨率下都会留下重构误差,而具有固定随机傅里叶频率的Transformer几乎肯定是通用的。因此,PPE为持久态图提供了一种极简的、原生的Transformer基线,也为未来的比较提供了统一协议。

英文摘要:

Persistence diagrams (PDs) represent multiscale topological information as multisets of birth-death pairs $(b,d)$. How best to use them with Transformers, however, remains unclear. Connecting PDs to Transformers requires choosing how topological information is tokenized and exposed to self-attention. Prior methods use specialized designs for tokenization, aggregation, Transformer blocks, or attention, but are hard to compare since reported results rarely share PD inputs, splits, or model-selection rules. We instead confine the diagram-specific design to point representation, mapping each birth-death pair through fixed random Fourier features while leaving the Transformer backbone unchanged, and call this recipe Persistence Positional Encoding (PPE). In matched controls on eight TU graph benchmarks (TU8), changing only point representation shifts macro accuracy by 5.49 to 6.47 percentage points (pp), against 0.47 to 1.45 pp from changing the downstream module, identifying point representation as the dominant design variable tested. We further reevaluate vectorizations, kernels, learned representations, and Transformer-based models under one protocol with shared PD inputs, outer folds, and validation-only selection. Among all evaluated PD methods and downstream configurations, PPE achieves the highest mean on most benchmarks, including TU8, ORBIT5K, and MCB-C. Finally, we prove that grid averaging leaves a reconstruction error at any resolution, whereas a Transformer with fixed random Fourier frequencies is almost surely universal. PPE thus offers a minimal, Transformer-native baseline for persistence diagrams, and a common protocol for future comparison.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑