Spectral graph neural networks (GNNs) learn graph representations via spectral-domain graph convolutions. However, most existing spectral graph filters are scalar-to-scalar functions, i.e., mapping a single eigenvalue to a single filtered value, thus ignoring the global pattern of the spectrum. Furthermore, these filters are often constructed based on some fixed-order polynomials, which have limited expressiveness and flexibility. To tackle these issues, we introduce Specformer, which effectively encodes the set of all eigenvalues and performs self-attention in the spectral domain, leading to a learnable set-to-set spectral filter. We also design a decoder with learnable bases to enable non-local graph convolution. Importantly, Specformer is equivariant to permutation. By stacking multiple Specformer layers, one can build a powerful spectral GNN. On synthetic datasets, we show that our Specformer can better recover ground-truth spectral filters than other spectral GNNs. Extensive experiments of both node-level and graph-level tasks on real-world graph datasets show that our Specformer outperforms state-of-the-art GNNs and learns meaningful spectrum patterns. Code and data are available at https://github.com/bdy9527/Specformer.
SpecFormer: Guarding Vision Transformer Robustness via Maximum Singular Value Penalization
City University of Hong Kong（香港城市大学）；Microsoft Research Asia（微软亚洲研究院）；The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；William & Mary（威廉玛丽学院）
Indian Institute of Information Technology Kottayam (IIITK)（印度信息技术学院科塔亚姆分校（IIITK））；International Institute of Information Technology Hyderabad (IIITH)（印度信息技术学院海得拉巴分校（IIITH））
Fast and scalable retrosynthetic planning with a transformer neural network and speculative beam search
SUPSI, IDSIA（SUPSI、IDSIA）；Pfizer Research and Development（辉瑞研发）；Institute for Digital Technologies for Personalized Healthcare（个性化医疗数字技术研究所）；KAUST（王国立科技大学）；Center for Generative AI（生成人工智能中心）
Bayesian Inverse Problems Meet Flow Matching: Efficient and Flexible Inference via Transformers
Skolkovo Institute of Science and Technology（斯克洛夫学院科学与技术研究所）；Sberbank（储蓄银行）；AI4S Center（AI4S中心）；Artificial Intelligence Research Institute（人工智能研究 institute）
Entropy Meets Importance: A Unified Head Importance-Entropy Score for Stable and Efficient Transformer Pruning
Department of Computer Science and Engineering, Korea University, Seoul, South Korea（计算机科学与工程系，韩国大学，首尔，韩国）；School of Software, Soongsil University, Seoul, South Korea（软件学院，顺天大学，首尔，韩国）
Tree Meets Transformer: A Hybrid Architecture for Scalable Power Allocation in Cell-Free Networks
Smart Networks and Services Joint Undertaking (SNS JU)（智能网络与服务联合体）；Italian Ministry of Education and Research (MUR)（意大利教育与研究部）；HORIZON-JU-SNS-2022 EU project TIMES（HORIZON-JU-SNS-2022 EU项目TIMES）；European Union under the Italian National Recovery and Resilience Plan (NRRP) of NextGenerationEU（欧盟在意大利国家复苏与韧性计划（NRRP）下的NextGenerationEU）
Transformers Meet In-Context Learning: A Universal Approximation Theory
Department of Statistics and Data Science, Chinese University of Hong Kong（统计与数据科学系，香港中文大学）；the Wharton School, University of Pennsylvania（宾夕法尼亚大学沃顿商学院）；University of Pennsylvania（宾夕法尼亚大学）
University of Electronic Science and Technology of China（电子科技大学）；Clinical Hospital of Chengdu Brain Science Institute（成都脑科学医院）；MOE-K Lab for NeuroInformation（神经信息MOE-K实验室）；Int Institute of Brain-Apparatus Communication（脑机通讯国际研究所）；School of Life Science and Technology（生命科学与技术学院）
Scalable Peptide Design via Memory-Efficient Equivariant Transformer
Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）；Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）；Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）；Department of Chemistry, Tsinghua University（清华大学化学系）
CLAReSNet: When Convolution Meets Latent Attention for Hyperspectral Image Classification
Institute of Engineering and Management (IEM), University of Engineering and Management Kolkata, IEM Centre of Excellence for InnovAI, Department of Computer Science & Engineering（工程管理学院（IEM）、工程管理大学科塔卡尔，创新AI研究中心、计算机科学与工程系）；Institute of Engineering and Management (IEM), University of Engineering and Management Kolkata, Department of Computer Science & Engineering（工程管理学院（IEM）、工程管理大学科塔卡尔，计算机科学与工程系）
Spiking Point Transformer for Point Cloud Classification
University of Science and Technology of China（中国科学技术大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Tsingmao Intelligence（清猫智能）；School of Software, Northwestern Polytechnical University（西北工业大学软件学院）
Meitu（美图）；The Hang Seng University of Hong Kong（香港恒生大学）
汉语分词是汉语语言理解的一项基本任务。近年来，基于神经网络的模型在解决领域内的CWS任务方面取得了优异的性能。去年，一种新的语言表示模型BERT（Bidirectional Encoder Representation from Transformers）被提出作为许多自然语言任务的骨干模型，并重新定义了相应的性能。BERT的优良性能激励我们将其应用于CWS任务的解决。通过对第二届国际汉语分词大赛的基准数据集进行深入的实验，我们得到了一些敏锐的观察。BERT可以稍微提高性能，即使数据集包含标签不一致的问题。当应用充分学习的特征时，Softmax，一个更简单的分类器，可以达到与更复杂的分类器相同的性能，例如，条件随机场（CRF）。BERT的性能通常随着模型大小的增加而增加。BERT提取的特征也可以作为其他神经网络模型的良好候选。
