名称无关：面向开放词汇学习的符号不变Transformer
Names Don't Matter: Symbol-Invariant Transformer for Open-Vocabulary Learning
AI总结：
提出一种符号不变Transformer机制，通过并行嵌入流和聚合注意力实现可互换令牌的重命名不变性，在开放词汇任务上取得显著性能提升。
摘要：
当前的神经架构缺乏处理可互换令牌（即语义等价但可区分的符号，如绑定变量）的原则性方法。因此，在固定词汇表上训练的模型通常难以泛化到未见过的符号，即使底层语义保持不变。我们提出了一种新颖的基于Transformer的机制，该机制对可互换令牌的重命名具有可证明的不变性。我们的方法采用并行嵌入流来隔离输入中每个可互换令牌的贡献，并结合聚合注意力机制实现跨流的结构化信息共享。实验结果证实了我们方法的理论保证，并在需要泛化到新符号的开放词汇任务上展示了显著的性能提升。项目页面：https://bu-depend-lab.github.io/Symbol-Invariant-Transformer/
英文摘要：
Current neural architectures lack a principled way to handle interchangeable tokens, i.e., symbols that are semantically equivalent yet distinguishable, such as bound variables. As a result, models trained on fixed vocabularies often struggle to generalize to unseen symbols, even when the underlying semantics remain unchanged. We propose a novel Transformer-based mechanism that is provably invariant to the renaming of interchangeable tokens. Our approach employs parallel embedding streams to isolate the contribution of each interchangeable token in the input, combined with an aggregated attention mechanism that enables structured information sharing across streams. Experimental results confirm the theoretical guarantees of our method and demonstrate substantial performance gains on open-vocabulary tasks that require generalization to novel symbols. Project page: https://bu-depend-lab.github.io/Symbol-Invariant-Transformer/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
