Recent work shows that preference alignment objectives can be interpreted as divergence estimators between aligned (preferred) & unaligned (less-preferred) distributions, yielding a principled recipe for designing alignment losses. However, this view has so far been limited to preference-based supervision. We extend it to general LLM alignment, including reinforcement learning with verifiable rewards (RLVR), where alignment feedback is given only as scalar rewards. We introduce $f$-Group Relative Policy Optimization ($f$-GRPO), a class of on-policy RL objectives, and $f$-Hybrid Alignment Loss ($f$-HAL), which combines on-policy reward optimization with off-policy preference supervision. We show that these objectives estimate $f$-divergences between reward-aligned & reward-unaligned distributions induced by above- & below-average reward responses, and prove expected reward improvement after alignment. Empirically, $f$-GRPO improves over GRPO on math-reasoning RLVR tasks, while hybrid $f$-HAL mitigates reward hacking in on-policy safety alignment when verifiable rewards are unavailable and learned reward models must be used.
Fine-Grained GRPO for Precise Preference Alignment in Flow Models
Shanghai Jiao Tong University（上海交通大学）；University of Science and Technology of China（中国科学技术大学）；Fudan University（复旦大学）；Shanghai AI Laboratory（上海人工智能实验室）；Shanghai Innovation Institute（上海创新研究院）
DRA-GRPO: Your GRPO Needs to Know Diverse Reasoning Paths for Mathematical Reasoning
Morgan Stanley（摩根士丹利）；Clemson University（克莱姆森大学）；Arizona State University（亚利桑那州立大学）；Washington University in St. Louis（圣路易斯华盛顿大学）；University of Notre Dame（圣母大学）；University of Arizona（亚利桑那大学）
Hidden States Know Where Reasoning Diverges: Credit Assignment via Span-Level Wasserstein Distance
Beijing University of Posts and Telecommunications（北京邮电大学）；Meituan（美团）；Tianjin University（天津大学）；Renmin University of China（中国人民大学）；The University of Melbourne（墨尔本大学）；University of International Business and Economics（国际经贸大学）
Baseline-Free Policy Optimization for Neural Combinatorial Optimization
Facultad de Ingeniería y Ciencias, Universidad Adolfo Ibáñez（阿道夫·伊瓦涅斯大学工程与科学学院）；Dirección de Programas, Investigación y Desarrollo, Armada de Chile（智利海军计划、研究与发展局）；Millennium Nucleus for Social Data Science (SODAS)（千禧年社会数据科学核心（SODAS））；Millennium Nucleus in Data Science for Plant Resilience (PhytoLearning)（千禧年植物韧性数据科学核心（PhytoLearning））
RLHF in an SFT Way: From Optimal Solution to Reward-Weighted Alignment
Shenzhen Research Institute of Big Data（深圳大数据研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Qwen LLM Application Team, Alibaba（阿里巴巴Qwen大模型应用团队）；Stanford University（斯坦福大学）
Hindsight-Privileged RLVR: Single-Rollout Token Credit Without Group Sampling
South China University of Technology（华南理工大学）；Tsinghua University（清华大学）；The Hong Kong University of Science and Technology（香港科技大学）；Huawei 2012 Laboratories（华为2012实验室）；Shenzhen Technology University（深圳技术大学）
CheXPO-v2: Preference Optimization for Chest X-ray VLMs with Knowledge Graph Consistency
School of Computer Science and Technology, Xidian University（西安电子科技大学计算机科学与技术学院）；Warren Alpert Medical School of Brown University（布朗大学沃伦·阿尔珀特医学学院）；School of Biomedical Engineering, Capital Medical University（首都医科大学生物医学工程学院）
LVRPO: Language-Visual Alignment with GRPO for Multimodal Understanding and Generation
Department of Machine Learning, CMU, USA（卡内基梅隆大学机器学习系，美国）；Department of Machine Learning, MBZUAI, UAE（穆罕默德·本·扎耶德人工智能大学机器学习系，阿联酋）；Department of Artificial Intelligence, Hanyang University ERICA, South Korea（汉阳大学ERICA校区人工智能系，韩国）
The Alignment Paradox of Medical Large Language Models in Infertility Care: Decoupling Algorithmic Improvement from Clinical Decision-making Quality
Department of Obstetrics and Gynecology, West China Second University Hospital（妇产科部门，西昌第二大学医院）；Reproductive Medical Center, Department of Obstetrics and Gynecology, West China Second University Hospital（生殖医学中心，妇产科部门，西昌第二大学医院）
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Transfer learning has been successfully applied across many high-impact applications. However, most existing work focuses on the static transfer learning setting, and very little is devoted to modeling the time evolving target domain, such as the online reviews for movies. To bridge this gap, in this paper, we study a novel continuous transfer learning setting with a time evolving target domain. One major challenge associated with continuous transfer learning is the potential occurrence of negative transfer as the target domain evolves over time. To address this challenge, we propose a novel l…
