Diffusion Large Language Models (DLLMs) have emerged as a powerful alternative to autoregressive models, enabling parallel token generation across multiple positions. However, preference alignment of DLLMs remains challenging due to high variance introduced by Evidence Lower Bound (ELBO)-based likelihood estimation. In this work, we propose AR-MAP, a novel transfer learning framework that leverages preference-aligned autoregressive LLMs (AR-LLMs) as implicit teachers for DLLM alignment. We reveal that DLLMs can effectively absorb alignment knowledge from AR-LLMs through simple weight scaling, exploiting the shared architectural structure between these divergent generation paradigms. Crucially, our approach circumvents the high variance and computational overhead of direct DLLM alignment and comprehensive experiments across diverse preference alignment tasks demonstrate that AR-MAP achieves competitive or superior performance compared to existing DLLM-specific alignment methods, achieving 69.08\% average score across all tasks and models. Our Code is available at https://github.com/AMAP-ML/AR-MAP.
RLKD: Distilling LLMs' Reasoning via Reinforcement Learning
State Key Laboratory of AI Safety, Institute of Computing Technology, CAS(人工智能安全国家重点实验室,计算技术研究所,中国科学院);University of Chinese Academy of Sciences(中国科学院大学);Huawei Inc.(华为公司)
When the Strongest Teacher Is Not the Best Teacher: Student-Centric Answer Selection
University of Washington(华盛顿大学);University of Texas at Austin(德克萨斯大学奥斯汀分校);University of Southern California(南加州大学);Independent Researcher(独立研究者);National University of Singapore(新加坡国立大学);Microsoft(微软);Google(谷歌);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Northwestern University(西北大学);Allen Institute for AI (AI2)(人工智能研究院(AI2))
Fuzzy Alignments in Directed Acyclic Graph for Non-Autoregressive Machine Translation
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳))
Guided by the Plan: Enhancing Faithful Autoregressive Text-to-Audio Generation with Guided Decoding
The Hong Kong Polytechnic University(香港理工大学);IROOTECH TECHNOLOGY(IROOTECH技术公司);Wolf 1069 b Lab, Sany Group(三一集团沃尔夫1069实验室);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Zhejiang University(浙江大学)