语言大模型 / LLM

Scaling Synthetic Task Generation for Agents via Exploration

Ram Ramrakhya, Andrew Szot, Omar Attia, Yuhao Yang, Anh Nguyen, Bogdan Mazoure, Zhe Gan, Harsh Agrawal, Alexander Toshev

机构 * Apple（苹果公司） 纠错

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；post-training(abstract)；prompting(abstract)

Scaling Agents via Continual Pre-training

Liangcai Su, Zhen Zhang, Guangyu Li, Zhuo Chen, Chenxi Wang, Maojia Song, Xinyu Wang, Kuan Li, Jialong Wu, Xuanzhong Chen, Zile Qiao, Zhongwang Zhang, Huifeng Yin, Shihao Cai, Runnan Fang, Zhengwei Tao, Wenbiao Yin, Chenxiong Qian, Yong Jiang, Pengjun Xie, Fei Huang, Jingren Zhou

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；foundation model(abstract)；post-training(abstract)

Comments https://tongyi-agent.github.io/blog/introducing-tongyi-deep-research/

RIVAL: Reinforcement Learning with Iterative and Adversarial Optimization for Machine Translation

Tianjiao Li, Mengran Yu, Chenyu Shi, Yanjun Zhao, Xiaojing Liu, Qiang Zhang, Qi Zhang, Xuanjing Huang, Jiayin Wang

机构 * Bilibili Inc.（哔哩哔哩公司） ； Xi’an Jiaotong University（西安交通大学） ； School of Computer Science, Fudan University（复旦大学计算机科学学院） 纠错

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Activation Reward Models for Few-Shot Model Alignment

Tianning Chai, Chancharik Mitra, Brandon Huang, Gautam Rajendrakumar Gare, Zhiqiu Lin, Assaf Arbelle, Leonid Karlinsky, Rogerio Feris, Trevor Darrell, Deva Ramanan, Roei Herzig

机构 * University of California, Berkeley（加州大学伯克利分校） ； Carnegie Mellon University（卡内基梅隆大学） ； IBM Research（IBM研究院） ； MIT-IBM Watson AI Lab（MIT-IBM沃森人工智能实验室） 纠错

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；post-training(abstract)

Mind the Inconspicuous: Revealing the Hidden Weakness in Aligned LLMs' Refusal Boundaries

Jiahao Yu, Haozheng Luo, Jerry Yao-Chieh Hu, Wenbo Guo, Han Liu, Xinyu Xing

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；SFT(abstract)；RLHF(abstract)

Comments published at USENIX Security 25

Mitigating Reward Over-optimization in Direct Alignment Algorithms with Importance Sampling

Phuc Minh Nguyen, Ngoc-Hieu Nguyen, Duy H. M. Nguyen, Anji Liu, An Mai, Binh T. Nguyen, Daniel Sonntag, Khoa D. Doan

机构 * VinUniversity（文大学） ； VinUni-Illinois Smart Health Center（VinUni-伊利诺伊智能健康中心） ； University of Stuttgart（斯图加特大学） ； University of California Los Angeles (UCLA)（加州大学洛杉矶分校） ； International University - VNUHCM（国际大学 - VNUHCM） ； University of Science - VNUHCM（科学大学 - VNUHCM） ； German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心） ； Oldenburg University（奥尔登堡大学） ； Max Planck Research School for Intelligent Systems (IMPRS-IS)（马克斯·普朗克智能系统研究学校） 纠错

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；preference optimization(abstract)

Comments First version

MCP Safety Training: Learning to Refuse Falsely Benign MCP Exploits using Improved Preference Alignment

John Halloran

机构 * Leidos（莱迪斯公司） 纠错

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；post-training(abstract)；preference optimization(abstract)

Comments 27 pages, 19 figures, 4 tables

The Climb Carves Wisdom Deeper Than the Summit: On the Noisy Rewards in Learning to Reason

Ang Lv, Ruobing Xie, Xingwu Sun, Zhanhui Kang, Rui Yan

机构 * GSAI, Renmin University of China（清华大学） ； Large Language Model Department, Tencent（腾讯大语言模型部门） ； University of Macau（澳门大学） ； School of Computer Science, Wuhan University（武汉大学计算机学院） 纠错

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；post-training(abstract)

Comments Preprint

Divide-Then-Align: Honest Alignment based on the Knowledge Boundary of RAG

Xin Sun, Jianan Xie, Zhongqi Chen, Qiang Liu, Shu Wu, Yuehe Chen, Bowen Song, Weiqiang Wang, Zilei Wang, Liang Wang

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；post-training(abstract)；preference optimization(abstract)

Comments ACL 2025 main

Reinforcement Learning from User Feedback

Eric Han, Jun Chen, Karthik Abinav Sankararaman, Xiaoliang Peng, Tengyu Xu, Eryk Helenowski, Kaiyan Peng, Mrinal Kumar, Sinong Wang, Han Fang, Arya Talebzadeh

机构 * Meta GenAI（Meta 生成式人工智能） 纠错

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Learning Explainable Dense Reward Shapes via Bayesian Optimization

Ryan Koo, Ian Yang, Vipul Raheja, Mingyi Hong, Kwang-Sung Jun, Dongyeop Kang

机构 * University of Minnesota（明尼苏达大学） ； Georgia Institute of Technology（佐治亚理工学院） ； Grammarly ； University of Arizona（亚利桑那大学） 纠错

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Exploring Data Scaling Trends and Effects in Reinforcement Learning from Human Feedback

Wei Shen, Guanlin Liu, Zheng Wu, Ruofei Zhu, Qingping Yang, Chao Xin, Yu Yue, Lin Yan

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；SFT(abstract)；RLHF(abstract)

Optimizing Safe and Aligned Language Generation: A Multi-Objective GRPO Approach

Xuying Li, Zhuo Li, Yuji Kosuga, Victor Bian

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；preference optimization(abstract)

Empowering LLMs in Decision Games through Algorithmic Data Synthesis

Haolin Wang, Xueyan Li, Yazhe Niu, Shuai Hu, Hongsheng Li

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；post-training(abstract)

Comparative Analysis Based on DeepSeek, ChatGPT, and Google Gemini: Features, Techniques, Performance, Future Prospects

Anichur Rahman, Shahariar Hossain Mahir, Md Tanjum An Tashrif, Airin Afroj Aishi, Md Ahsan Karim, Dipanjali Kundu, Tanoy Debnath, Md. Abul Ala Moududi, MD. Zunead Abedin Eidmum

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Mask-DPO: Generalizable Fine-grained Factuality Alignment of LLMs

Yuzhe Gu, Wenwei Zhang, Chengqi Lyu, Dahua Lin, Kai Chen

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

Comments Accepted by ICLR 2025. Code is available at https://github.com/open-compass/ANAH

CARMO: Dynamic Criteria Generation for Context-Aware Reward Modelling

Taneesh Gupta, Shivam Shandilya, Xuchao Zhang, Rahul Madhavan, Supriyo Ghosh, Chetan Bansal, Huaxiu Yao, Saravan Rajmohan

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；post-training(abstract)；RLHF(abstract)

Uncertainty-aware Reward Model: Teaching Reward Models to Know What is Unknown

Xingzhou Lou, Dong Yan, Wei Shen, Yuzi Yan, Jian Xie, Junge Zhang

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

Optimizing Knowledge Integration in Retrieval-Augmented Generation with Self-Selection

Yan Weng, Fengbin Zhu, Tong Ye, Haoyan Liu, Fuli Feng, Tat-Seng Chua

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

Comments 12 pages, 6 figures

Reveal the Mystery of DPO: The Connection between DPO and RL Algorithms

Xuerui Su, Yue Wang, Jinhua Zhu, Mingyang Yi, Feng Xu, Zhiming Ma, Yuting Liu

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；preference optimization(abstract)

Reinforcement Learning from Human Feedback without Reward Inference: Model-Free Algorithm and Instance-Dependent Analysis

Qining Zhang, Honghao Wei, Lei Ying

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Improving the Validity of Automatically Generated Feedback via Reinforcement Learning

Alexander Scarlatos, Digory Smith, Simon Woodhead, Andrew Lan

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

Comments Best student paper award, Published in AIED 2024: The 25th International Conference on Artificial Intelligence in Education

Journal ref In International Conference on Artificial Intelligence in Education (pp. 280-294). Cham: Springer Nature Switzerland (2024)

UltraMedical: Building Specialized Generalists in Biomedicine

Kaiyan Zhang, Sihang Zeng, Ermo Hua, Ning Ding, Zhang-Ren Chen, Zhiyuan Ma, Haoxin Li, Ganqu Cui, Biqing Qi, Xuekai Zhu, Xingtai Lv, Hu Jinfang, Zhiyuan Liu, Bowen Zhou

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

Comments Camera ready version for NeurIPS 2024 D&B Track

AlignCap: Aligning Speech Emotion Captioning to Human Preferences

Ziqi Liang, Haoxiang Shi, Hanhui Chen

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；preference optimization(abstract)

Comments Accepted to EMNLP2024 main conference

Generative Reward Models

Dakota Mahan, Duy Van Phung, Rafael Rafailov, Chase Blagden, Nathan Lile, Louis Castricato, Jan-Philipp Fränken, Chelsea Finn, Alon Albalak

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Minor DPO reject penalty to increase training robustness

Shiming Xie, Hong Chen, Fred Yu, Zeye Sun, Xiuyu Wu, Yingfan Hu

专题命中 后训练与偏好优化 ：LLM(abstract)；language model(abstract)；RLHF(abstract)；preference optimization(abstract)

Comments 8 pages, 19 figures

Critique-out-Loud Reward Models

Zachary Ankner, Mansheej Paul, Brandon Cui, Jonathan D. Chang, Prithviraj Ammanabrolu

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；RLHF(abstract)

Robust Reinforcement Learning from Corrupted Human Feedback

Alexander Bukharin, Ilgee Hong, Haoming Jiang, Zichong Li, Qingru Zhang, Zixuan Zhang, Tuo Zhao

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；RLHF(abstract)；preference optimization(abstract)

Comments 22 pages, 7 figures

LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedback

Wen Lai, Mohsen Mesgar, Alexander Fraser

专题命中 后训练与偏好优化 ：LLM(abstract)；large language model(abstract)；language model(abstract)；instruction tuning(abstract)

Comments Accepted to Findings of ACL 2024. The code, datasets, and models are publicly available at https://github.com/boschresearch/ACL24-MLLM

Xwin-LM: Strong and Scalable Alignment Practice for LLMs

Bolin Ni, JingCheng Hu, Yixuan Wei, Houwen Peng, Zheng Zhang, Gaofeng Meng, Han Hu

专题命中 后训练与偏好优化 ：large language model(abstract)；language model(abstract)；SFT(abstract)；preference optimization(abstract)