A Transformer-based deep direct sampling method is proposed for electrical impedance tomography, a well-known severely ill-posed nonlinear boundary value inverse problem. A real-time reconstruction is achieved by evaluating the learned inverse operator between carefully designed data and the reconstructed images. An effort is made to give a specific example to a fundamental question: whether and how one can benefit from the theoretical structure of a mathematical problem to develop task-oriented and structure-conforming deep neural networks? Specifically, inspired by direct sampling methods for inverse problems, the 1D boundary data in different frequencies are preprocessed by a partial differential equation-based feature map to yield 2D harmonic extensions as different input channels. Then, by introducing learnable non-local kernels, the direct sampling is recast to a modified attention mechanism. The new method achieves superior accuracy over its predecessors and contemporary operator learners and shows robustness to noises in benchmarks. This research shall strengthen the insights that, despite being invented for natural language processing tasks, the attention mechanism offers great flexibility to be modified in conformity with the a priori mathematical knowledge, which ultimately leads to the design of more physics-compatible neural architectures.
Tree Meets Transformer: A Hybrid Architecture for Scalable Power Allocation in Cell-Free Networks
Smart Networks and Services Joint Undertaking (SNS JU)（智能网络与服务联合体）；Italian Ministry of Education and Research (MUR)（意大利教育与研究部）；HORIZON-JU-SNS-2022 EU project TIMES（HORIZON-JU-SNS-2022 EU项目TIMES）；European Union under the Italian National Recovery and Resilience Plan (NRRP) of NextGenerationEU（欧盟在意大利国家复苏与韧性计划（NRRP）下的NextGenerationEU）
EGMOF: Efficient Generation of Metal-Organic Frameworks Using a Hybrid Diffusion-Transformer Architecture
Department of Chemical and Biomolecular Engineering, Korea Advanced Institute of Science and Technology（韩国科学技术院化学与生物分子工程系）；Department of Chemistry, University of Toronto（多伦多大学化学系）；Vector Institute for Artificial Intelligence（人工智能矢量研究所）；Department of Chemistry, Sungkyunkwan University（成均馆大学化学系）；Acceleration Consortium（加速联盟）；Department of Computer Science, University of Toronto（多伦多大学计算机科学系）；Department of Materials Science & Engineering, University of Toronto（多伦多大学材料科学与工程系）；Department of Chemical Engineering & Applied Chemistry, University of Toronto（多伦多大学化学工程与应用化学系）；Canadian Institute for Advanced Research (CIFAR)（加拿大高级研究 institute (CIFAR)）；NVIDIA
Bayesian Inverse Problems Meet Flow Matching: Efficient and Flexible Inference via Transformers
Skolkovo Institute of Science and Technology（斯克洛夫学院科学与技术研究所）；Sberbank（储蓄银行）；AI4S Center（AI4S中心）；Artificial Intelligence Research Institute（人工智能研究 institute）
Transformers Meet In-Context Learning: A Universal Approximation Theory
Department of Statistics and Data Science, Chinese University of Hong Kong（统计与数据科学系，香港中文大学）；the Wharton School, University of Pennsylvania（宾夕法尼亚大学沃顿商学院）；University of Pennsylvania（宾夕法尼亚大学）
Entropy Meets Importance: A Unified Head Importance-Entropy Score for Stable and Efficient Transformer Pruning
Department of Computer Science and Engineering, Korea University, Seoul, South Korea（计算机科学与工程系，韩国大学，首尔，韩国）；School of Software, Soongsil University, Seoul, South Korea（软件学院，顺天大学，首尔，韩国）
Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering
NVIDIA（英伟达）；University of Toronto（多伦多大学）；Vector Institute（向量研究所）；University of Waterloo（滑铁卢大学）；Stanford University（斯坦福大学）；MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）
University of Electronic Science and Technology of China（电子科技大学）；Clinical Hospital of Chengdu Brain Science Institute（成都脑科学医院）；MOE-K Lab for NeuroInformation（神经信息MOE-K实验室）；Int Institute of Brain-Apparatus Communication（脑机通讯国际研究所）；School of Life Science and Technology（生命科学与技术学院）
Scalable Peptide Design via Memory-Efficient Equivariant Transformer
Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）；Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）；Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）；Department of Chemistry, Tsinghua University（清华大学化学系）
Indian Institute of Information Technology Kottayam (IIITK)（印度信息技术学院科塔亚姆分校（IIITK））；International Institute of Information Technology Hyderabad (IIITH)（印度信息技术学院海得拉巴分校（IIITH））
Transformer-Based Inverse Microrheology for Experimental Mechanics at Ultra-High Strain Rates
Materials Science Graduate Program, Texas Materials Institute, The University of Texas at Austin（材料科学研究生项目，德克萨斯材料研究所，德克萨斯大学奥斯汀分校）；Department of Mechanical Engineering, Michigan State University（机械工程系，密歇根州立大学）；Department of Aerospace Engineering and Engineering Mechanics, The University of Texas at Austin（航空航天工程与工程力学系，德克萨斯大学奥斯汀分校）；The Oden Institute of Computational Science and Engineering, University of Texas at Austin（奥登计算科学与工程研究所，德克萨斯大学奥斯汀分校）；Department of Mechanical Engineering, University of Wisconsin-Madison（机械工程系，威斯康星大学麦迪逊分校）
Public Health School, Fujian Medical University（福建医科大学公共卫生学院）；School of Mathematics and Statistics, Fuzhou University（福州大学数学与统计学院）；Center for Applied Mathematics of Fujian Province（福建省应用数学中心）；National Center for Applied Mathematics Shenzhen, SUSTech（南方科技大学深圳国家应用数学中心）；Department of Mathematics, SUSTech（南方科技大学数学系）；School of Science, BUPT（北京邮电大学理学院）；The Smart City Research Institute of CETC（中国电子科技集团公司智慧城市研究院）
