The Stop-Think-AutoRegress Language Diffusion Model (STAR-LDM) integrates latent diffusion planning with autoregressive generation. Unlike conventional autoregressive language models limited to token-by-token decisions, STAR-LDM incorporates a "thinking" phase that pauses generation to refine a semantic plan through diffusion before continuing. This enables global planning in continuous space prior to committing to discrete tokens. Evaluations show STAR-LDM significantly outperforms similar-sized models on language understanding benchmarks and achieves $>70\%$ win rates in LLM-as-judge comparisons for narrative coherence and commonsense reasoning. The architecture also allows straightforward control through lightweight classifiers, enabling fine-grained steering of attributes without model retraining while maintaining better fluency-control trade-offs than specialized approaches.
Think Big, Generate Quick: LLM-to-SLM for Fast Autoregressive Decoding
Hasso Plattner Institute for Digital Engineering, University of Potsdam(波茨坦大学哈索·普拉特纳数字工程研究所);Qualcomm AI Research(高通人工智能研究);University of Amsterdam(阿姆斯特丹大学)
Distilling Reasoning Ability from Large Language Models with Adaptive Thinking
College of Computer Science and Technology, National University of Defense Technology(计算机科学与技术学院,国防科技大学);College of Intelligence Science and Technology, National University of Defense Technology(智能科学与技术学院,国防科技大学)
Adaptive Look, Flexible Thought: Learning How and How Long to Think in Multimodal Reasoning
Sun Yat-sen University(中山大学);Shenzhen Loop Area Institute(深圳河套学院);Beijing Zhongguancun Academy(北京中关村研究院);Huawei Technologies Co., Ltd(华为技术有限公司);Harbin Institute of Technology(哈尔滨工业大学)
BayLing-Duplex: Native Full-Duplex Speech Dialogue with a Single Autoregressive LLM
Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences (ICT/CAS)(中国科学院计算技术研究所智能信息处理重点实验室);Key Laboratory of AI Safety, Chinese Academy of Sciences(中国科学院人工智能安全重点实验室);University of Chinese Academy of Sciences(中国科学院大学)
SoftCoT++: Test-Time Scaling with Soft Chain-of-Thought Reasoning
Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly(联合NTU-UBC老龄化积极生活卓越研究中心);Alibaba-NTU Global e-Sustainability CorpLab (ANGEL)(阿里巴巴-NTU全球可持续性企业实验室(ANGEL));College of Computing and Data Science, Nanyang Technological University, Singapore(南洋理工大学计算与数据科学学院);KTH Royal Institute of Technology, Sweden(瑞典皇家理工学院)
Reviewing the Reviewer: LLM-Assisted Reviewer Feedback Generation for Guideline Compliance
Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science, Technical University of Darmstadt and National Research Center for Applied Cybersecurity ATHENE, Germany(技术大学达姆施塔特计算机科学系 Ubiquitous Knowledge Processing Lab(UKP Lab)及国家应用网络安全研究中心 ATHENE,德国);Department of Data Science & AI, Monash University, Australia(墨尔本大学数据科学与人工智能系,澳大利亚);Data Science Group, University of Hamburg(汉堡大学数据科学组)
Beyond Surface Reasoning: Unveiling the True Long Chain-of-Thought Capacity of Diffusion Large Language Models
LARG, Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology(LARG,社会计算与交互机器人研究中心,哈尔滨工业大学);School of Computer Science and Engineering, Central South University(计算机科学与工程学院,中南大学);The University of Hong Kong(香港大学);Shanghai Jiao Tong University(上海交通大学);ByteDance Seed (China)(字节跳动种子(中国))
CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think
LUMIA Lab(LUMIA实验室);School of Artificial Intelligence(人工智能学院);Shanghai Jiao Tong University(上海交通大学);Shanghai Innovation Institute(上海创新研究院);School of Computer Science(计算机科学学院);Shanghai AI Laboratory(上海人工智能实验室)
Single Headed Attention RNN: Stop Thinking With Your Head
Times Labs(时代实验室)
语言建模的主要方法都是痴迷于我年轻时的电视节目-即Transformers和芝麻街。Transformers this,Transformers that,and over here a bonfire worth of GPU-TPU-neuromorphic wafer scale silicon.我们选择了旧的和经过验证的技术的懒惰路径,并使用了一个受加密启发的首字母缩略词:单头注意力RNN(SHA-RNN)。作者唯一的目标是表明,如果我们不是痴迷于一个略有不同的缩写词和略有不同的结果,整个领域可能会朝着不同的方向发展。我们采用了一个以前只基于无聊的LSTM的强大语言模型,并在enwik 8上将其与最先进的字节级语言模型结果进行了一箭之遥的比较。这项工作没有经过密集的超参数优化,完全生活在一个商品台式机,使作者的小工作室公寓太温暖,在旧金山的夏天。最终的结果是可以实现的正负24小时内在一个单一的GPU作为作者是不耐烦的。注意力机制也很容易扩展到大的上下文,以最小的计算。就拿芝麻街来说
Diffusion Language Models Know the Answer Before Decoding
The Hong Kong Polytechnic University(香港理工大学);Dartmouth College(达特茅斯学院);University of Surrey(萨里大学);Sun Yat-sen University(中山大学);ELLIS Institute Tübingen(ELLIS 图宾根研究所);MPI for Intelligent Systems(马克斯·普朗克智能系统研究所);Tübingen AI Center(图宾根人工智能中心)