Graph neural networks (GNNs) have achieved state-of-the-art performance in various graph-based tasks. However, as mainstream GNNs are designed based on the neural message passing mechanism, they do not scale well to data size and message passing steps. Although there has been an emerging interest in the design of scalable GNNs, current researches focus on specific GNN design, rather than the general design space, limiting the discovery of potential scalable GNN models. This paper proposes PasCa, a new paradigm and system that offers a principled approach to systemically construct and explore the design space for scalable GNNs, rather than studying individual designs. Through deconstructing the message passing mechanism, PasCa presents a novel Scalable Graph Neural Architecture Paradigm (SGAP), together with a general architecture design space consisting of 150k different designs. Following the paradigm, we implement an auto-search engine that can automatically search well-performing and scalable GNN architectures to balance the trade-off between multiple criteria (e.g., accuracy and efficiency) via multi-objective optimization. Empirical studies on ten benchmark datasets demonstrate that the representative instances (i.e., PasCa-V1, V2, and V3) discovered by our system achieve consistent performance among competitive baselines. Concretely, PasCa-V3 outperforms the state-of-the-art GNN method JK-Net by 0.4\% in terms of predictive accuracy on our large industry dataset while achieving up to $28.3\times$ training speedups.
SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation
Shanghai AI Laboratory（上海人工智能实验室）；Zhejiang University（浙江大学）；University of Maryland, College Park（马里兰大学 College Park 分校）；Shanghai Jiao Tong University（上海交通大学）；Tsinghua University（清华大学）
本文提出 SDAR 范式，将训练好的 AR 模型低成本转换为分块离散扩散模型，兼顾全局连贯与块内并行解码，并在密集及 MoE 模型上实现无损加速与推理提升。
Scalable Learning Paradigms for Data-Driven Wireless Communication
Beijing University of Posts and Telecommunications（北京邮电大学）；National Chiao Tung University（国立交通大学）；Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Research Institute of Big Data（深圳大数据研究院）
CLEAN-MI: A Scalable and Efficient Pipeline for Constructing High-Quality Neurodata in Motor Imagery Paradigm
Ministry of Education Key Laboratory of Image Processing and Intelligent Control, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（教育部长图像处理与智能控制重点实验室，人工智能与自动化学院，华中科技大学）；Zhongguancun Academy（中关村学院）
Spike Agreement Dependent Plasticity: A scalable Bio-Inspired learning paradigm for Spiking Neural Networks
School of Data Science, Indian Institute of Science Education and Research, Thiruvananthapuram, India（数据科学学院，印度科学教育与研究学院，特里布文纳纳特帕尔姆，印度）；School of Data Science, Indian Institute of Science Education（数据科学学院，印度科学教育与研究学院）；School of Physics, Indian Institute of Science Education（物理学院，印度科学教育与研究学院）
Rank Minimization on Tensor Ring: A New Paradigm in Scalable Tensor Decomposition and Completion
Graduate School of Engineering, Saitama Institute of Technology, Japan（日本萨它马工学院工程研究生院）；Tensor Learning Unit, RIKEN Center for Advanced Intelligence Project (AIP), Japan（日本RIKEN高级智能项目（AIP）张量学习单元）；School of Automation, Guangdong University of Technology, China（中国广东技术大学自动化学院）；School of Computer Science and Technology, Hangzhou Dianzi University, China（中国杭州电子科技大学计算机科学与技术学院）；Department of Electrical and Electronic Engineering, Imperial College London, United Kingdom（英国伦敦帝国理工学院电子与电气工程系）
Constraint-based Pre-training: From Structured Constraints to Scalable Model Initialization
School of Computer Science and Engineering, Southeast University, Nanjing, China（东南大学计算机科学与工程学院，南京，中国）；Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China（新一代人工智能技术及其跨学科应用重点实验室（东南大学），教育部，中国）
Interlaced R2D2 DNN Series for Scalable Non-Cartesian MRI with Sensitivity Self-calibration
Institute of Sensors, Signals and Systems, Heriot-Watt University Edinburgh, United Kingdom（传感器、信号与系统研究所，赫里奥特-沃斯大学爱丁堡分校，英国）；Department of Diagnostic Imaging and Interventional Radiology, Lausanne University and Hospital, Switzerland（诊断影像与介入放射学系，洛桑大学及医院，瑞士）
