Aligning Large Language Models (LLMs) with human preferences through finetuning is resource-intensive, motivating lightweight alternatives at test time. We address test-time alignment through the lens of sequential decision making, a perspective that reveals two fundamental challenges. When actions are defined at the token level, as in guided decoding, alignment suffers from the curse of horizon. Conversely, when actions are at the response level, as in traditional iterative refinement, the curse of dimensionality emerges. To resolve this trade-off, we draw inspiration from Model Predictive Control (MPC) in control theory to propose Textual Model Predictive Control (TMPC), a novel predictive planning framework adapted for aligning LLMs at inference time. A key limitation of standard MPC is its reliance on predefined, hard segment boundaries, which are often absent in text generation. TMPC overcomes this by introducing two principles inspired by hierarchical reinforcement learning: (1) Hindsight Subgoal Identification, where TMPC analyzes generation subgoals to retrospectively identify high-reward intermediate outputs as subgoals. This allows the framework to discover meaningful, task-specific planning steps (e.g., a sentence in machine translation or a bug fix in code generation.). (2) Subgoal-Conditioned Re-Generation, where these identified subgoals are used to guide subsequent planning iterations. By conditioning on these proven, high-quality subgoals, TMPC ensures stable improvement by building upon previously validated successes. TMPC is evaluated on three tasks with distinct segmentation properties: discourse-level translation, long-form response generation, and program synthesis. The results demonstrate that TMPC consistently improves performance, highlighting the generality.
Single-to-mix Modality Alignment with Multimodal Large Language Model for Document Image Machine Translation
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Zhongke Fanyu Technology Co., Ltd(中科泛语科技有限公司)
CodeRL+: Improving Code Generation via Reinforcement with Execution Semantics Alignment
School of Computer Science, Peking University(北京大学计算机科学系);Tongyi Lab, Alibaba Group(阿里集团通义实验室);School of Computer Science, Wuhan University(武汉大学计算机科学系)
Textual Entailment with Structured Attentions and Composition
Oregon State University(俄勒冈州立大学)
深度学习技术在文本蕴涵任务中越来越受欢迎,克服了传统离散模型的脆弱性,具有硬对齐和逻辑。特别是,最近提出的注意力模型(Rocktäschel等人,2015; Wang and Jiang,2015)通过计算前提和假设句子之间的软词对齐来实现最先进的准确性。然而,仍然存在一个主要的限制:这条工作路线完全忽略了语法和递归,这在许多传统的工作中是有帮助的。我们表明,将注意力模型扩展到前提和假设之间的树节点是有益的。更重要的是,这种子树层次的注意揭示了蕴涵关系的信息。我们研究了这种子树级蕴涵关系的递归组合,它可以被视为自然逻辑框架的软版本(MacCartney和Manning,2009)。实验结果表明,本文提出的结构化注意和蕴涵组合模型能够自自底向上的正确识别和推理蕴涵关系,在准确率上有显著提高。
ITA: Image-Text Alignments for Multi-Modal Named Entity Recognition
Shopee(虾皮);DAMO Academy, Alibaba Group(阿里巴巴集团达摩院);School of Information Science and Technology, ShanghaiTech University(上海科技大学信息科学与技术学院);Shanghai Engineering Research Center of Intelligent Vision and Imaging(上海智能视觉成像工程技术研究中心);Microsoft(微软);Alibaba Group(阿里巴巴集团)
UReason: Benchmarking Reasoning-to-Generation Alignment in Unified Multimodal Models
University of California San Diego(加利福尼亚大学圣迭戈分校);University of Southern California(南加利福尼亚大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Carnegie Mellon University(卡内基梅隆大学)
Large Vision-Language Model Alignment and Misalignment: A Survey Through the Lens of Explainability
Northwestern University(西北大学);New Jersey Institute of Technology(新泽西理工学院);University of Bristol(布里斯托大学);Cisco Research(思科研究);University of Illinois Chicago(伊利诺伊大学芝加哥分校)
Radiology Report Generation with a Learned Knowledge Base and Multi-modal Alignment
Institute of Computing Technology, CAS(中国科学院计算技术研究所);University of Science and Technology of China(中国科学技术大学);Tencent Medical AI Lab(腾讯医疗人工智能实验室);University of Chinese Academy of Sciences(中国科学院大学)