Aligning language models with human preferences presents significant challenges, particularly in achieving personalization without incurring excessive computational costs. Existing methods rely on reward signals and additional annotated data, limiting their scalability and adaptability to diverse human values. To address these challenges, we introduce Persona-judge, a novel discriminative paradigm that enables training-free personalized alignment with unseen preferences. Instead of optimizing policy parameters through external reward feedback, Persona-judge leverages the intrinsic preference judgment capabilities of the model. Specifically, a draft model generates candidate tokens conditioned on a given preference, while a judge model, embodying another preference, cross-validates the predicted tokens whether to be accepted. Experimental results demonstrate that Persona-judge, using the inherent preference evaluation mechanisms of the model, offers a scalable and computationally efficient solution to personalized alignment, paving the way for more adaptive customized alignment. Our code is available here.
From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge
Arizona State University（亚利桑那州立大学）；University of Illinois Chicago（伊利诺伊大学芝加哥分校）；University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）；Northwestern University（西北大学）；University of California, Berkeley（加州大学伯克利分校）；Emory University（埃默里大学）
Evaluating the Evaluator: Summarization Metrics and LLM-Judges beyond English
HiTZ Center, University of the Basque Country (UPV/EHU)（巴斯克大学（UPV/EHU）HiTZ中心）；University of Alicante (UA)（阿利坎特大学（UA））；GOI Institute, Basque Summer University (UEU)（巴斯克夏季大学（UEU）GOI研究所）
How Long Reasoning Chains Influence LLMs' Judgment of Answer Factuality
State Key Laboratory of AI Safety（人工智能安全国家重点实验室）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Beijing University of Post and Telecommunications（北京邮电大学）
From Calculation to Adjudication: Examining LLM judges on Mathematical Reasoning Tasks
Faculty of Computer Science（计算机科学系）；UniVie Doctoral School Computer Science（维也纳大学计算机科学博士学院）；Faculty of Philological and Cultural Studies（文学与文化研究系）；University of Vienna（维也纳大学）；Saarland University（萨尔兰大学）；Eastern Institute of Technology（东部技术研究所）；Department of Statistics（统计系）；LMU Munich（慕尼黑莱茵河大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）
Label Effects: Shared Heuristic Reliance in Trust Assessment by Humans and LLM-as-a-Judge
National Institute of Informatics (NII)（国立信息学研究所）；University of Amsterdam（阿姆斯特丹大学）；University of Tokyo（东京大学）；Hitotsubashi University（一桥大学）；Centrum Wiskunde & Informatica (CWI)（荷兰数学与计算机科学研究中心）；Utrecht University（乌得勒支大学）
State Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University（信息处理国家重点实验室，计算机学院，北京大学）；StepFun；Xi’an Jiaotong University（西安交通大学）
Do Before You Judge: Self-Reference as a Pathway to Better LLM Evaluation
Department of Computer Science and Information Engineering（计算机科学与信息工程系）；National Taiwan University（国立台湾大学）；Institute of Information Science, Academia Sinica（学术院信息研究所）；AI Research Center (AINTU)（人工智能研究中心）
