University of Washington(华盛顿大学);University of California, Santa Barbara(加州大学圣芭芭拉分校);Stanford University(斯坦福大学);Carnegie Mellon University(卡内基梅隆大学);Northwestern University(西北大学);University of Notre Dame(圣母大学);University of California, Berkeley(加州大学伯克利分校);Michigan State University(密歇根州立大学);MIT-IBM Watson AI Lab(MIT-IBM沃森人工智能实验室);Bake AI;King Abdulaziz City for Science and Technology(国王阿卜杜勒阿齐兹科技城);Western Washington University(西雅图华盛顿大学);University of Chicago(芝加哥大学)
Current benchmarks for occupational AI agents are scoped primarily by economic values, telling a replacement story. We introduce JobBench, which evaluates AI agents on the workflows that experts identify as high-priority for delegation, empowering humans based on their needs instead of replacing them with GDP value. JobBench covers 130 agentic tasks across 35 occupations. Each task is packaged as a workspace of heterogeneous reference files, requiring the agent to reason through the cluttered information streams of real professional work. Outputs are graded by a fact-anchored chain of rubrics, averaging 35.6 binary criteria per task. We evaluate 36 models; the strongest, Claude Opus~4.7 under Claude Code, reaches only 45.9 %. We hope JobBench shifts the community's target labour-market effect from replacement to enhancement: building agents that do what humans actually want delegated, not only what is most economically valuable.
Aligning Agents via Planning: A Benchmark for Trajectory-Level Reward Modeling
State Key Laboratory of Novel Software Technology, Nanjing University(南京大学新型软件技术国家重点实验室);School of Intelligence Science and Technology, Nanjing University(南京大学智能科学与技术学院);AMAP, Alibaba Group(阿里集团AMAP)
Aligning to Social Norms and Values in Interactive Narratives
Allen Institute for Artificial Intelligence(艾伦人工智能研究所);University of Washington(华盛顿大学)
我们专注于创建代理,在互动叙事或基于文本的游戏中与社会有益的规范和价值观保持一致-环境中,代理通过自然语言感知并与世界互动。这种交互式代理通常通过强化学习进行训练,以优化任务性能,即使这种奖励可能导致违反社会规范的代理行为-对代理本身或环境中的其他实体造成伤害。社会价值对齐是指创造行为符合特定背景和人群预期道德和社会规范的代理人-在我们的情况下,它意味着代理人的行为方式对自己和他人的危害较小,更有益。我们建立在Jiminy Cricket基准(Hendrycks et al. 2021)的基础上,这是一组25个注释的交互式叙述,包含数千个道德上突出的场景,涵盖了从盗窃和身体伤害到利他主义的一切。我们引入GALAD(游戏-价值对齐通过行动蒸馏)代理,它使用专门训练的语言模型中存在的社会常识知识,根据上下文将其动作空间限制为仅与社会有益价值对齐的那些动作。一项实验研究表明,GALAD代理有效地做出决策,足以将最先进的任务性能提高4%,同时与强大的当代价值对齐方法相比,将社会有害行为的频率降低25%。
WorkRover: Scaling Long-Horizon Cowork Agent Training Via Recursive Environment Evolution
Fudan University(复旦大学);East China Normal University(华东师范大学);Renmin University of China(中国人民大学);Shanghai Qiji Zhifeng Co., Ltd.(上海齐骥智峰有限公司);Shanghai Innovation Institute(上海创新研究院)
ResearchEnvBench: Benchmarking Agents on Environment Synthesis for Research Code Execution
Institute of Trustworthy Embodied AI(可信具身AI研究院);Shanghai Innovation Institution(上海创新机构);Shanghai Key Laboratory of Multimodal Embodied AI(上海多模态具身AI重点实验室);College of Computer Science and Artificial Intelligence(计算机科学与人工智能学院);OpenMOSS;Wuhan University(武汉大学);Nanjing University(南京大学);Jilin University(吉林大学)
SkillAlign: Aligning Skill Interfaces for LLM-based Agents
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Large Language Models Meet Harry Potter: A Bilingual Dataset for Aligning Dialogue Agents with Characters
Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Hong Kong University of Science and Technology(香港科技大学);Tencent AI Lab(腾讯人工智能实验室)
Aligning LLM agents with human learning and adjustment behavior: a dual agent approach
Department of Civil and Environmental Engineering, University of Michigan, Ann Arbor, United States(美国密歇根大学土木与环境工程系);Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, United States(美国密歇根大学电气工程与计算机科学系);College of Civil Engineering and Architecture, Zhejiang University, Hangzhou, China(浙江大学建筑工程学院)
GROW: Aligning GRPO with State-Action Modeling for Open-World VLM Agents
Shanghai Jiao Tong University(上海交通大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);East China Normal University(华东师范大学);Zhejiang Normal University(浙江师范大学);Shandong Normal University(山东省师范大学)
MIRA: A Musical Intent Refinement Agent for Aligning Text-to-Music Generation with User Intent
Shandong University(山东大学);University of Science and Technology of China(中国科学技术大学);Central Conservatory of Music(中央音乐学院);Kunlun Tech Co. Ltd.(昆仑万维科技股份有限公司);Shanghai Jiao Tong University(上海交通大学)
DARE: Aligning LLM Agents with the R Statistical Ecosystem via Distribution-Aware Retrieval
Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hong Kong SAR, China(数据科学与人工智能系,香港理工大学,香港特别行政区,中国);Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong SAR, China(应用数学系,香港理工大学,香港特别行政区,中国)
Harness Policy Distillation: Aligning Agent Policies with Runtime Semantics
National Key Laboratory for Novel Software Technology, Nanjing University(南京大学新型软件技术国家重点实验室);School of Artificial Intelligence, Nanjing University(南京大学人工智能学院)
Inferring the Unspoken: Aligning Embodied Agents with Implicit Preferences
Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院);School of Computer Science & Technology, Beijing Institute of Technology(北京理工大学计算机科学与技术学院);Yangtze Delta Region Academy of Beijing Institute of Technology, Jiaxing, China(北京理工大学扬子江地区研究院);National Key Laboratory of General Artificial Intelligence, BIGAI(通用人工智能国家重点实验室,BIGAI)
An Irreducible Quantum Advantage in Aligning World Models with Reality
Nanyang Technological University(南洋理工大学);Centre for Quantum Technologies(量子技术中心);Nanyang Quantum Hub(南洋量子中心);School of Physical and Mathematical Sciences(物理与数学科学学院);College of Computing and Data Science(计算与数据科学学院)