Personalized alignment of large language models seeks to adapt responses to individual user preferences, typically via reinforcement learning. A key challenge is obtaining accurate, user-specific reward signals in open-ended scenarios. Existing personalized reward models face two persistent limitations: (1) oversimplifying diverse, scenario-specific preferences into a small, fixed set of evaluation principles, and (2) struggling with generalization to new users with limited feedback. To this end, we propose P-GenRM, the first Personalized Generative Reward Model with test-time user-based scaling. P-GenRM transforms preference signals into structured evaluation chains that derive adaptive personas and scoring rubrics across various scenarios. It further clusters users into User Prototypes and introduces a dual-granularity scaling mechanism: at the individual level, it adaptively scales and aggregates each user's scoring scheme; at the prototype level, it incorporates preferences from similar users. This design mitigates noise in inferred preferences and enhances generalization to unseen users through prototype-based transfer. Empirical results show that P-GenRM achieves state-of-the-art results on widely-used personalized reward model benchmarks, with an average improvement of 2.31%, and demonstrates strong generalization on an out-of-distribution dataset. Notably, Test-time User-based scaling provides an additional 3% boost, demonstrating stronger personalized alignment with test-time scalability.
VL-GenRM: Enhancing Vision-Language Verification via Vision Experts and Iterative Training
The Hong Kong University of Science and Technology(香港科学与技术大学);Nanyang Technological University(南洋理工大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
One Adapts to Any: Meta Reward Modeling for Personalized LLM Alignment
The Hong Kong Polytechnic University(香港理工大学);Huawei Technologies Ltd.(华为技术有限公司);National University of Singapore(新加坡国立大学);University of Science and Technology of China(中国科学技术大学)
SarcasmMiner: A Dual-Track Post-Training Framework for Robust Audio-Visual Sarcasm Reasoning
Speech Technology Lab, University of Groningen, The Netherlands(格罗宁根大学语音技术实验室,荷兰);Center for Language and Cognition, University of Groningen, The Netherlands(格罗宁根大学语言与认知中心,荷兰)
Interpretability Is Not Just a Tax: Sparsity and Diversity Yield Legible and Composable Reward Experts for Personalization
Saarland University(萨尔兰大学);Independent Researcher(独立研究者);Bielefeld University(比勒菲尔德大学);Max Planck Institute for Software Systems(马克斯·普朗克软件系统研究所);Max Planck Institute for Informatics(马克斯·普朗克信息研究所)
FedPDPO: Federated Personalized Direct Preference Optimization for Large Language Model Alignment
College of Intelligence and Computing, Tianjin University, Tianjin, China(智能与计算学院,天津大学,天津,中国);College of Computing and Data Science, Nanyang Technological University, Singapore(计算与数据科学学院,南洋理工大学,新加坡)
Me-Agent: A Personalized Mobile Agent with Two-Level User Habit Learning for Enhanced Interaction
Yunnan University(云南大学);Hong Kong Polytechnic University(香港理工大学);University of Electronic Science and Technology of China(电子科技大学);Southeast University(东南大学);Chongqing University(重庆大学);Tsinghua University(清华大学);Kuaishou Technology(快手科技)
Towards Faithful and Controllable Personalization via Critique-Post-Edit Reinforcement Learning
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));University of Electronic Science and Technology of China(电子科技大学);South China Agricultural University(华南农业大学)
Learning What to Keep: Transfer-Aware Personal Skill Learning from User Feedback
Beihang University(北京航空航天大学);Chengdu University of Information Technology(成都信息工程大学);Nanyang Technological University(南洋理工大学);Hangzhou International Innovation Institute(杭州国际创新研究院);Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing(未来区块链与隐私计算北京高精尖创新中心)
Personalized Learning Path Planning with Goal-Driven Learner State Modeling
Department of Computer Science and Technology, Beijing National Research Center for Information Science and Technology(计算机科学与技术系,信息科学国家研究中心);Tsinghua University(清华大学);Institution of Education(教育研究所);Department of Statistics and Data Science(统计与数据科学系)
Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferences
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
Multi-Objective Alignment of Language Models for Personalized Psychotherapy
Department of Computer Science, University of California, Los Angeles, CA, USA(计算机科学系,加州大学洛杉矶分校);Department of Psychology, The University of Texas at Austin, Austin, TX, USA(心理学系,德克萨斯大学奥斯汀分校);Department of Psychiatry, NYU Grossman School of Medicine, New York, NY, USA(精神病学系,纽约大学格罗斯曼医学院)
PrLM: Learning Explicit Reasoning for Personalized RAG via Contrastive Reward Optimization
Gaoling School of Artificial Intelligence\ University of China Beijing China;School of Information Technology;Gaoling School of Artificial Intelligence\ University of China
ICG: Improving Cover Image Generation via MLLM-based Prompting and Personalized Preference Alignment
Huazhong University of Science and Technology(华中科技大学);Huawei Noah’s Ark Lab(华为诺亚实验室);Hong Kong Polytechnic University(香港理工大学);Zhejiang University(浙江大学)
Inside Out: Evolving User-Centric Core Memory Trees for Long-Term Personalized Dialogue Systems
School of Information, Renmin University of China(中国人民大学信息学院);MemTensor (Shanghai) Technology Co., Ltd.(MemTensor(上海)科技有限公司);Institute for Advanced Algorithms Research, Shanghai(上海先进算法研究所);China Telecom Research Institute(中国电信研究院);Beijing University of Aeronautics and Astronautics(北京航空航天大学);Nankai University(南开大学)