Constrained decoding guarantees that Large Language Models (LLMs) produce outputs conforming to predefined formats, but is often assumed to inherently degrade reasoning. We examine this assumption through a taxonomy of three constraint types: \emph{format constraints}, which enforce output structure; \emph{validity constraints}, which mask invalid answer; and \emph{reasoning constraints}, which restrict reasoning trajectories. We theoretically show that appropriate constraints can preserve or improve performance by eliminating invalid paths. Across four model families and eleven open-source models, causal ablations on diverse benchmarks using state-of-the-art constrained decoding strategies reveal that apparent performance degradation largely stems from non-equivalent prompting and output formats rather than from decoder-level masking. Meanwhile, validity constraints improve reliability and extractability by eliminating invalid answers, while reasoning constraints can improve reasoning and, consequently, accuracy when they enforce beneficial trajectories. These findings show that the impact of constrained decoding depends on the type of constraint and the output space it removes. Our code is available.
Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院,清华大学);Department of Software Engineering, Harbin Institute of Technology(哈尔滨工业大学软件工程系);School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen(哈尔滨工业大学深圳校区计算机科学与技术学院);Institute for Network Sciences and Cyberspace, Tsinghua University(清华大学网络科学与空间研究院)
It's the Thought that Counts: Evaluating the Attempts of Frontier LLMs to Persuade on Harmful Topics
Université de Montréal, MILA(蒙特利尔大学,MILA);Carnegie Mellon University(卡内基梅隆大学);MIT, Center for Research and Teaching in Economics(麻省理工学院,经济研究与教学中心);Cornell University, University of Regina(康奈尔大学, Regina大学);Cornell University, MIT(康奈尔大学,麻省理工学院)
Conversational Complexity for Assessing Risk in Large Language Models
University of Cambridge(剑桥大学);Spanish National Research Council(西班牙国家研究委员会);Universitat Politècnica de València(瓦伦西亚理工大学);Leverhulme Centre for the Future of Intelligence(勒沃休姆未来智能中心);Center for Automation and Robotics(自动化与机器人中心);Valencian Research Institute for Artificial Intelligence(瓦伦西亚人工智能研究所)
Fine-Grained Activation Steering: Steering Less, Achieving More
School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore(电子与电气工程学院,南洋理工大学,新加坡);Home Team Science and Technology Agency (HTX), Singapore(家庭团队科技局(HTX),新加坡)
Encourage or Inhibit Monosemanticity? Revisit Monosemanticity from a Feature Decorrelation Perspective
King’s College London(伦敦国王学院);Carnegie Mellon University(卡内基梅隆大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);MIT CSAIL(麻省理工学院计算机科学与人工智能实验室);The Alan Turing Institute(艾伦·图灵研究所)
SafetyAnalyst: Interpretable, Transparent, and Steerable Safety Moderation for AI Behavior
University of California, Berkeley(加州大学伯克利分校);University of Washington(华盛顿大学);Duke University(杜克大学);Carnegie Mellon University(卡内基梅隆大学);Stanford University(斯坦福大学)
Revisiting End-to-End Speech-to-Text Translation From Scratch
University of Edinburgh(爱丁堡大学);University of Zurich(苏黎世大学)
端到端(E2 E)语音到文本翻译(ST)通常依赖于通过语音识别或文本翻译任务使用源转录来预训练其编码器和/或解码器,没有这些任务,翻译性能会大幅下降。然而,成绩单并不总是可用的,以及如何显着这样的预训练是E2 E ST很少在文献中研究。在本文中,我们重新审视这个问题,并探讨在何种程度上的语音翻译对单独训练的E2 E ST的质量可以提高。我们重新审视了之前被证明对ST有益的几种技术,并提供了一组最佳实践,使基于Transformer的E2 E ST系统偏向于从头开始训练。此外,我们提出了参数化的距离惩罚,以促进局部性的语音自注意模型的建模。在涵盖23种语言的四个基准上,我们的实验表明,在不使用任何成绩单或预训练的情况下,所提出的系统达到甚至超过了以前采用预训练的研究,尽管在(极)低资源环境中仍然存在差距。最后,我们讨论了神经声学特征建模,其中神经模型被设计为直接从原始语音信号中提取声学特征,其目标是简化归纳偏差并在描述语音时增加模型的自由度。我们第一次证明了它的可行性,并在ST任务上显示出令人鼓舞的结果。