While Reinforcement Learning with Verifiable Rewards (RLVR) has shown strong effectiveness in reasoning tasks, it cannot be directly applied to non-verifiable domains lacking ground-truth verifiers, such as LLM alignment. In this work, we investigate whether reference-guided LLM-evaluators can bridge this gap by serving as soft "verifiers". First, we design evaluation protocols that enhance LLM-based evaluators for LLM alignment using reference outputs. Through comprehensive experiments, we show that a reference-guided approach substantially improves the accuracy of less capable LLM-judges using references from frontier models; stronger LLM-judges can also be enhanced by high-quality (i.e., human-written) references. Building on these improved judges, we demonstrate the utility of high-quality references in alignment tuning, where LLMs guided with references are used as judges to self-improve. We show that reference-guided self-improvement yields clear gains over both direct SFT on reference outputs and self-improvement with reference-free judges, achieving performance comparable to training with ArmoRM, a strong finetuned reward model. Specifically, our method achieves 73.1% and 58.7% on AlpacaEval and Arena-Hard with Llama-3-8B-Instruct, and 70.0% and 74.1% with Qwen2.5-7B, corresponding to average absolute gains of +20.2 / +17.1 points over SFT distillation and +5.3 / +3.6 points over reference-free self-improvement on AlpacaEval / Arena-Hard. These results highlight the potential of using reference-guided LLM-evaluators to enable effective LLM post-training in non-verifiable domains.
Biomedical Reference Generation Remains Unreliable across 26 Large Language Models
Columbia University（哥伦比亚大学）；VNS Health（VNS健康机构）；Tel Aviv Sourasky Medical Center（特拉维夫苏拉斯基医疗中心）；University of Eastern Finland（东芬兰大学）；Wellbeing Services County of North Savo（北萨沃福祉服务县）；Wellbeing Services County of Southwest Finland（西南芬兰福祉服务县）
FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoning
MBZUAI；Syllogia；The University of Tokyo（东京大学）；The Fin AI（Fin AI）；Cornell University（康奈尔大学）；The University of Melbourne（墨尔本大学）；IIT Delhi（德里理工学院）；The University of Manchester（曼彻斯特大学）
BeamPERL: Parameter-Efficient RL with Verifiable Rewards Specializes Compact LLMs for Structured Beam Mechanics Reasoning
Department of Mechanical Engineering（机械工程系）；Massachusetts Institute of Technology（麻省理工学院）；Department of Civil and Environmental Engineering（土木与环境工程系）；Schwarzman College of Computing（施瓦茨曼计算学院）
Open Rubric System: Scaling Reinforcement Learning with Pairwise Adaptive Rubric
Qwen Large Model Application Team, Alibaba（阿里巴巴文勤大模型应用团队）；Beijing University Of Posts and Telecommunications（北京邮电大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
AI Alignment through a Game-theoretic Lens: A Survey
James Cook University（詹姆斯库克大学）；Western Sydney University（西悉尼大学）；Adelaide University（阿德莱德大学）；CSIRO（联邦科学与工业研究组织）；The University of Osaka（大阪大学）；Nanjing University of Science and Technology（南京理工大学）；Macquarie University（麦考瑞大学）；La Trobe University（拉筹伯大学）
CSPF: A Constrained Shared-Private Fusion Method for Non-Verifiable Preference Evaluation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Marginal Alignment Does Not Guarantee Joint-Distribution Fidelity: An Official-Reference Audit of Nemotron-Personas-Korea with Cross-Locale Replication
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
Fuzzy Alignments in Directed Acyclic Graph for Non-Autoregressive Machine Translation
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））
