Multi-reward reinforcement learning offers a promising framework for aligning large language models (LLMs) with diverse objectives, but conflicting reward signals can induce ambiguous update directions and hinder stable optimization. Existing approaches address such conflicts through reward reweighting, gradient coordination, or rollout filtering, yet they primarily operate on per-response or group-level optimization signals and underexploit relational information among sampled responses. As a result, conflict-filtered rollouts may still contain useful supervision that is discarded during training. We propose Group Pareto-Dominance Policy Optimization (GPDPO), a multi-reward policy optimization method that recovers pairwise supervision from conflict-filtered rollouts. GPDPO identifies Pareto-dominance relations among sampled responses and converts them into auxiliary preference signals, which are jointly optimized with the original policy objective. This enables more effective reuse of existing rollouts without additional sampling or reward evaluation. Experiments on API-Bank across multiple model scales and reward configurations show that GPDPO consistently outperforms competitive multi-reward optimization baselines. Further analysis demonstrates that GPDPO reduces both group-relative and pairwise reward conflicts, supporting the effectiveness of exploiting relational supervision for multi-reward alignment. Code will be released.
Pareto-wise Ranking Classifier for Multi-objective Evolutionary Neural Architecture Search
College of Software, Northeastern University(东北大学软件学院);Institute of Intelligent Manufacturing, Nanjing Tech University(南京工业大学智能制造研究院);School of Computer Science, Shaanxi Normal University(陕西师范大学计算机学院);College of Computer Science, Northeastern University(东北大学计算机科学与工程学院);Faculty of Technology, Bielefeld University(比勒费尔德大学技术学院)
Ranking-Aware Prompt Optimization for Multimodal Clinical Diagnosis
Harvard University(哈佛大学);University of California, Santa Cruz(加州大学圣克鲁兹分校);Brown University(布朗大学);Johns Hopkins University(约翰斯·霍普金斯大学);Georgia Institute of Technology(佐治亚理工学院)
Beyond the Dirac Delta: Mitigating Diversity Collapse in Reinforcement Fine-Tuning for Versatile Image Generation
Nanjing University(南京大学);Australia National University(澳大利亚国立大学);Wuhan University(武汉大学);National University of Singapore(新加坡国立大学);University of Technology Sydney(技术科技大学)
M-HOF-Opt: Multi-Objective Hierarchical Output Feedback Optimization via Multiplier Induced Loss Landscape Scheduling
Institute of AI for Health, Computational Health Center, Helmholtz Munich(亥姆霍兹慕尼黑人工智能健康研究所(计算健康中心));Machine Learning Research Lab, Volkswagen Group(大众集团机器学习研究实验室);KTH Royal Institute of Technology(瑞典皇家理工学院);Technical University Munich(慕尼黑工业大学);U.S. FDA/CDRH(美国食品药品监督管理局医疗器械和辐射健康中心);Applied Geology, GeoZentrum Nordbayern, Friedrich-Alexander-Universität(弗里德里希-亚历山大-埃尔朗根-纽伦堡大学北巴伐利亚地理中心应用地质学系)
Not All Objectives Are Born Equal: Priority-Constrained Descent for Hierarchical Multi-Objective Optimization
Department of Computer Science & Engineering, American University of Sharjah(美国沙迦大学计算机科学与工程系);Senseable City Laboratory, Massachusetts Institute of Technology(麻省理工学院可感知城市实验室)
On the Fairness ROAD: Robust Optimization for Adversarial Debiasing
AXA Group Operations(安盛集团运营部);Stanford University(斯坦福大学);TRAIL, Sorbonne Université(索邦大学TRAIL);LERIA, Université d’Angers(昂热大学LERIA);Polish Academy of Science(波兰科学院)