MeanFlow promises high-quality generative modeling in few steps, by jointly learning instantaneous and average velocity fields. Yet, the underlying training dynamics remain unclear. We analyze the interaction between the two velocities and find: (i) well-established instantaneous velocity is a prerequisite for learning average velocity; (ii) learning of instantaneous velocity benefits from average velocity when the temporal gap is small, but degrades as the gap increases; and (iii) task-affinity analysis indicates that smooth learning of large-gap average velocities, essential for one-step generation, depends on the prior formation of accurate instantaneous and small-gap average velocities. Guided by these observations, we design an effective training scheme that accelerates the formation of instantaneous velocity, then shifts emphasis from short- to long-interval average velocity. Our enhanced MeanFlow training yields faster convergence and significantly better few-step generation: With the same DiT-XL backbone, our method reaches an impressive FID of 2.87 on 1-NFE ImageNet 256x256, compared to 3.43 for the conventional MeanFlow baseline. Alternatively, our method matches the performance of the MeanFlow baseline with 2.5x shorter training time, or with a smaller DiT-L backbone.
Recent advances in Neural Architecture Search (NAS) such as one-shot NAS offer the ability to extract specialized hardware-aware sub-network configurations from a task-specific super-network. While considerable effort has been employed towards improving the first stage, namely, the training of the super-network, the search for derivative high-performing sub-networks is still under-explored. Popular methods decouple the super-network training from the sub-network search and use performance predictors to reduce the computational burden of searching on different hardware platforms. We propose a f…
IBM Research India（IBM印度研究院）；IBM Research USA（IBM美国研究院）
We develop a novel method, called PoWER-BERT, for improving the inference time of the popular BERT model, while maintaining the accuracy. It works by: a) exploiting redundancy pertaining to word-vectors (intermediate encoder outputs) and eliminating the redundant vectors. b) determining which word-vectors to eliminate by developing a strategy for measuring their significance, based on the self-attention mechanism. c) learning how many word-vectors to eliminate by augmenting the BERT model and the loss function. Experiments on the standard GLUE benchmark shows that PoWER-BERT achieves up to 4.5…
MeanFuser: Fast One-Step Multi-Modal Trajectory Generation and Adaptive Reconstruction via MeanFlow for End-to-End Autonomous Driving
SKL-MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所模式识别国家重点实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Xiaomi EV（小米汽车）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学智能产业研究院）
GPAS: Accelerating Convergence of LLM Pretraining via Gradient-Preserving Activation Scaling
The Hong Kong University of Science and Technology（香港科技大学）；International Digital Economy Academy（国际数字经济学院）；Dalian University of Technology（大连理工大学）；Emory University（埃默里大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；NVIDIA（英伟达）；University of Oxford（牛津大学）；University of Surrey（萨里大学）
On the Stability and Convergence of Flow Map Training
Inria, ENS de Lyon, CNRS, Université Claude Bernard Lyon 1（法国国家信息与自动化研究所，里昂高等师范学院，法国国家科学研究中心，克洛德·贝尔纳里昂大学）；Université de Montréal（蒙特利尔大学）；Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；Imperial College London（伦敦帝国理工学院）；CIFAR AI Chair（加拿大联邦信息研究基金会人工智能主席）
Despite the potential of reinforcement learning (RL) for building general-purpose robotic systems, training RL agents to solve robotics tasks still remains challenging due to the difficulty of exploration in purely continuous action spaces. Addressing this problem is an active area of research with the majority of focus on improving RL methods via better optimization or more efficient exploration. An alternate but important component to consider improving is the interface of the RL algorithm with the robot. In this work, we manually specify a library of robot action primitives (RAPS), paramete…
Politecnico di Torino（都灵理工大学）；Technische Universität Wien (TU Wien)（维也纳工业大学（维也纳技术大学））；New York University Abu Dhabi（纽约大学阿布扎比分校）
Adversarial training is exploited to develop a robust Deep Neural Network (DNN) model against the malicious altered data. These attacks may have catastrophic effects on DNN models but are indistinguishable for a human being. For example, an external attack can modify an image adding noises invisible for a human eye, but a DNN model misclassified the image. A key objective for developing robust DNN models is to use a learning algorithm that is fast but can also give model that is robust against different types of adversarial attacks. Especially for adversarial training, enormously long training…
