Support Vector Rubrics: Closing the Gap Between Self-Generated and Human Rubrics
National Engineering Research Center for Software Engineering, Peking University（北京大学软件工程国家工程研究中心）；University of Science and Technology of China（中国科学技术大学）
Mengyuan Sun, Yu Li, Zhuohao Yu, Shikun Zhang, Wei Ye
Rubric-based evaluation is a promising paradigm for judging large language model (LLM) outputs, yet self-generated rubrics lag human-annotated criteria on hard instances. We argue this discriminative gap reflects an objective mismatch: self-generated rubrics describe good responses, whereas effective criteria must discriminate between close candidates. To close this gap, we introduce SVR (Support Vector Rubrics), a framework that recasts rubric construction as max-margin boundary learning over preference data. SVR mines contrastive features from preference pairs into a rubric bank, learns a prompt-conditioned selector together with global rubric weights, and iteratively refines the bank through support-pair selection and adversarial probing of hard negatives. At inference, given only the prompt, SVR retrieves the top-rubrics from the bank and scores responses. On RubricBench, SVR narrows the gap to human reference rubrics from 24.1 to 0.3 points and outperforms strong self-rubric and judge baselines, and the learned bank transfers across judges without retraining. On RewardBench 1&2, and RM-Bench, it remains competitive with dedicated reward models, demonstrating broader reward modeling capability. Overall, boundary-defining rubrics offer a principled route to closing the discriminative gap in LLM evaluation.
Beyond Relevance-Centric Retrieval: Rubric-Oriented Document Set Selection and Ranking
University of Science and Technology of China（中国科学技术大学）；Yuanbao Team, Tencent（腾讯元宝团队）；University of Chinese Academy of Sciences（中国科学院大学）；Shandong University（山东大学）
AMARIS: A Memory-Augmented Rubric Improvement System for Rubric-Based Reinforcement Learning
The University of Texas at Dallas（德克萨斯大学达拉斯分校）；Adobe Inc.（Adobe公司）；Department of Computer Science, University of California, Santa Barbara（加州大学圣芭芭拉分校计算机科学系）
Case-Specific Rubrics for Clinical AI Evaluation: Methodology, Validation, and LLM-Clinician Agreement Across 823 Encounters
Canvas Medical；Department of Biomedical Data Science, Stanford University（斯坦福大学生物医学数据科学系）；XPC (X Primary Care)（XPC（X初级保健））；FCA Consulting（FCA咨询）；Department of Pediatrics, University of Nevada, Reno, NV, USA（内华达大学拉斯维加斯分校儿科系）
RubRIX: Rubric-Driven Risk Mitigation in Caregiver-AI Interactions
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；OSF HealthCare（OSF医疗集团）；Indiana University Indianapolis（印第安纳大学印第安纳波利斯分校）
CLR-voyance: Reinforcing Open-Ended Reasoning for Inpatient Clinical Decision Support with Outcome-Aware Rubrics
ASUS Intelligent Cloud Services (AICS)（ASUS智能云服务（AICS））；Rehabilitation Research Institute of Singapore, Nanyang Technological University（新加坡康复研究院，南洋理工大学）；Department of Family Medicine, Taipei Veterans General Hospital（台北荣民总医院家庭医学部）；School of Medicine, National Yang Ming Chiao Tung University（国家阳明交通大学医学院）；Yong Loo Lin School of Medicine, National University of Singapore（新加坡国立大学 Yong Loo Lin 医学院）
AI Rater Discrimination Depends on Scoring Protocol in Complex Clinical Decision-Making
Asclep Korea Inc.（Asclep韩国公司）；Center for Data Science, New York University（纽约大学数据科学中心）；Division of Endocrinology and Metabolism, Department of Medicine, Samsung Medical Center, Sungkyunkwan University School of Medicine（成均馆大学医学院内分泌与代谢科，三星医疗中心）；Biomedical Statistics Center, Samsung Medical Center（三星医疗中心生物医学统计中心）；Department of Digital Health, SAIHST（SAIHST数字健康科）；Department of Data Convergence & Future Medicine, Sungkyunkwan University（成均馆大学数据融合与未来医学科）
Learning to Evaluate Before Improving: Automatic Rubric Induction for Automatic Research Agents
Zhejiang University（浙江大学）；University of Electronic Science and Technology of China（电子科技大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Zhejiang University of Technology（浙江工业大学）
PRISM: Calibrating Black-Box Judges with Counterfactual Probes
Massachusetts Institute of Technology（麻省理工学院）；Sakana AI；Lab for Information and Decision Systems（信息与决策系统实验室）；Statistics and Data Science Center（统计与数据科学中心）；EECS（电气工程与计算机科学系）
Agentic systems for breast cancer treatment recommendations
Spesia（斯佩西亚）；Faculdade de Medicina, Universidade de São Paulo（圣保罗大学医学院）；Laboratory of Artificial Intelligence Applied to Bioinformatics, SEPT, Universidade Federal do Paraná (UFPR)（巴拉那联邦大学人工智能应用于生物信息学实验室，SEPT）；Pontifícia Universidade Católica do Paraná (PUCPR)（巴拉那天主大学）
From Diagnosis to Redesign: Using Quantitative Ethnography to Improve Multi-Agent LLM Reasoning
University of California, Irvine（加利福尼亚大学欧文分校）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；Monash University（莫纳什大学）；Columbia University（哥伦比亚大学）；University of Pennsylvania（宾夕法尼亚大学）
Using Large Language Models to Support High Volume Application Review for an Undergraduate Research Program
Engineering Undergraduate Research Office, Purdue University（普渡大学本科生研究办公室）；Elmore School of Electrical and Computer Engineering, Purdue University（普渡大学电子与计算机工程学院）；School of Materials Engineering, Purdue University（材料工程学院）
Dr.Credit: Rubric-Grounded Process Credit Assignment for Deep Research Agents
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Alibaba Group（阿里巴巴集团）；Peking University（北京大学）
Improving Heart-Focused Medical Question Answering in LLMs via Variance-Aware Rubric Rewards with GRPO
School of Electrical and Computer Engineering, University of Oklahoma, Norman, OK, USA（电气与计算机工程学院，俄克拉荷马大学，诺曼，OK，USA）；Intelligent Neuromorphic and Quantum Understanding for Innovative Research and Engineering (INQUIRE) Laboratory, University of Oklahoma, Norman, OK, USA（创新研究与工程智能神经形态与量子理解实验室，俄克拉荷马大学，诺曼，OK，USA）；Khiabani Data Science and Analytics Institute, University of Oklahoma, Norman, OK, USA（Khiabani数据科学与分析研究所，俄克拉荷马大学，诺曼，OK，USA）；Data Institute for Societal Challenges (DISC), University of Oklahoma, Norman, OK, USA（社会挑战数据研究所（DISC），俄克拉荷马大学，诺曼，OK，USA）；School of Industrial and Systems Engineering, University of Oklahoma, Norman, OK, USA（工业与系统工程学院，俄克拉荷马大学，诺曼，OK，USA）；Office of Responsible Artificial Intelligence (ORAI), University of Arizona, Tucson, AZ, USA（负责任人工智能办公室（ORAI），亚利桑那大学，图森，AZ，USA）
Stanford University（斯坦福大学）；Boston University（波士顿大学）；Columbia University（哥伦比亚大学）；University of Toronto（多伦多大学）；Institut Teknologi Bandung（Bandung 工程技术大学）；Monash University Indonesia（墨尔本大学印尼分校）；Capital One
