The success of vision transformers-especially for generative modeling-is limited by the quadratic cost and weak spatial inductive bias of self-attention. We propose PDE-SSM, a spatial state-space block that replaces attention with a learnable convection-diffusion-reaction partial differential equation. This operator encodes a strong spatial prior by modeling information flow via physically grounded dynamics rather than all-to-all token interactions. Solving the PDE in the Fourier domain yields global coupling with near-linear complexity of $O(N \log N)$, delivering a principled and scalable alternative to attention. We integrate PDE-SSM into a flow-matching generative model to obtain the PDE-based Diffusion Transformer PDE-SSM-DiT. Empirically, PDE-SSM-DiT matches or exceeds the performance of state-of-the-art Diffusion Transformers while substantially reducing compute. Our results show that, analogous to 1D settings where SSMs supplant attention, multi-dimensional PDE operators provide an efficient, inductive-bias-rich foundation for next-generation vision models.
Latent Mamba Operator for Partial Differential Equations
Department of Electrical Communication Engineering, Indian Institute of Science, Bangalore, India（电子通信工程系，印度科学研究院，班加罗尔，印度）；Yardi School of AI, Indian Institute of Technology, New Delhi, India（AI学院，印度理工学院，新德里，印度）；Department of Civil Engineering, Indian Institute of Technology, New Delhi, India（土木工程系，印度理工学院，新德里，印度）；Department of Computer Science and Automation, Indian Institute of Science, Bangalore, India（计算机科学与自动化系，印度科学研究院，班加罗尔，印度）
针对神经算子求解偏微分方程时面临的高维扩展性差、计算成本高及难以捕捉长程依赖等问题，提出 Latent Mamba Operator (LaMO)，结合潜空间状态空间模型与核积分公式，并在多种 PDE 基准上取得 SOTA 性能。
Fixed-Point RNNs: Interpolating from Diagonal to Dense
ELLIS Institute Tuebingen（图宾根ELLIS研究所）；Max Planck Institute for Intelligent Systems（智能系统马克斯·普朗克研究所）；Department of Mathematics, Imperial College London（伦敦帝国学院数学系）
NVIDIA（英伟达）；California Institute of Technology（加州理工学院）；Stanford University（斯坦福大学）
Vision transformers have delivered tremendous success in representation learning. This is primarily due to effective token mixing through self attention. However, this scales quadratically with the number of pixels, which becomes infeasible for high-resolution inputs. To cope with this challenge, we propose Adaptive Fourier Neural Operator (AFNO) as an efficient token mixer that learns to mix in the Fourier domain. AFNO is based on a principled foundation of operator learning which allows us to frame token mixing as a continuous global convolution without any dependence on the input resolution…
