MeshTok：可扩展PDE Transformer的高效多尺度分词化
MeshTok: Efficient Multi-Scale Tokenization for Scalable PDE Transformers
AI总结：
提出受自适应网格细化启发的MeshTok框架，通过多尺度分词化在统一Transformer中同时捕获粗粒度全局上下文和细粒度局部细节，改善PDE建模的效率-准确率权衡。
摘要：
传统的分块Transformer在均匀空间分区上运行，将计算工作量均匀分布在整个域中，而不考虑局部特征。这种不灵活的分词化方案在有效表示和处理复杂PDE解方面具有固有的局限性。为了解决这个问题，我们提出了MeshTok，一种受自适应网格细化（AMR）启发的分词化和序列建模框架。该方法选择性地细化具有陡峭梯度、瞬态特征或多尺度结构的空间区域，在固定模拟网格上生成一组异质的多尺度令牌。这些令牌在统一的Transformer序列中处理，使模型能够同时捕获粗粒度的全局上下文和细粒度的局部细节，而无需专门的架构组件。尽管自适应细化适度增加了令牌数量，但它促进了计算资源向物理信息区域的更有针对性的分配，我们将其视为一种实用的归纳偏置，而不是形式上的最优性保证。跨多个PDE族和基准数据集的实验评估表明，与均匀网格基线相比，MeshTok持续改善了效率-准确率权衡。这表明自适应多尺度分词化是神经PDE建模的一种可扩展且可推广的设计原则。代码可在https://github.com/SCAILab-USTC/MeshTok获取。
英文摘要：
Conventional patchified Transformers operate on uniform spatial partitions, distributing computational effort evenly across the domain irrespective of local features. This inflexible tokenization scheme is inherently limited in its ability to efficiently represent and process solutions to complex PDEs. To address this, we propose MeshTok, an adaptive mesh refinement (AMR)-inspired tokenization and sequence modeling framework. This method selectively refines spatial regions exhibiting sharp gradients, transient features, or multiscale structures, generating a heterogeneous set of multiscale tokens defined on a fixed simulation grid. These tokens are processed within a unified Transformer sequence, enabling the model to simultaneously capture coarse-grained global context and fine-grained local details without requiring specialized architectural components. Although adaptive refinement moderately increases token count, it promotes a more targeted allocation of computational resources to physically informative regions, which we view as a practical inductive bias rather than a formal optimality guarantee. Experimental evaluations across multiple PDE families and benchmark datasets demonstrate that MeshTok consistently improves the efficiency-accuracy trade-off compared to uniform-grid baselines. This suggests adaptive multiscale tokenization as a scalable and generalizable design principle for neural PDE modeling. Code is available at https://github.com/SCAILab-USTC/MeshTok.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
