大模型对齐与安全

WebGuard: Building a Generalizable Guardrail for Web Agents

Boyuan Zheng, Zeyi Liao, Scott Salisbury, Zeyuan Liu, Michael Lin, Qinyuan Zheng, Zifan Wang, Xiang Deng, Dawn Song, Huan Sun, Yu Su

机构 * The Ohio State University（俄亥俄州立大学） ； Scale AI ； University of California, Berkeley（加州大学伯克利分校） 纠错

专题命中 安全训练 ：safety(abstract)；分类 cs.CL、cs.AI

Comments We publicly release WebGuard, along with its annotation tools and fine-tuned models, to facilitate open-source research on monitoring and safeguarding web agents. All resources are available at https://github.com/OSU-NLP-Group/WebGuard

Mitigation of Camouflaged Adversarial Attacks in Autonomous Vehicles--A Case Study Using CARLA Simulator

Yago Romano Martinez, Brady Carter, Abhijeet Solanki, Wesam Al Amiri, Syed Rafay Hasan, Terry N. Guo

机构 * Department of Computer Engineering Tennessee Technological University（计算机工程系田纳西技术大学） ； Department of Computer Science Tennessee Technological University（计算机科学系田纳西技术大学） 纠错

专题命中 安全训练 ：safety(abstract)；分类 cs.AI、cs.LG

Variational OOD State Correction for Offline Reinforcement Learning

Ke Jiang, Wen Jiang, Xiaoyang Tan

机构 * College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, MIIT Key Laboratory of Pattern Analysis and Machine Intelligence（计算机科学与技术学院，南京航空航天大学，信息科技部模式分析与机器智能重点实验室） 纠错

专题命中 安全训练 ：safety(abstract)；分类 cs.AI、cs.LG

Identification of Potentially Misclassified Crash Narratives using Machine Learning (ML) and Deep Learning (DL)

Sudesh Bhagat, Ibne Farabi Shihab, Jonathan Wood

机构 * Department of Civil, Construction and Environmental Engineering（土木、建设与环境工程系） ； Iowa State University（爱荷华州立大学） ； Department of Computer Science（计算机科学系） 纠错

专题命中 安全训练 ：safety(abstract)；分类 cs.CL、cs.AI

Leveraging the Potential of Prompt Engineering for Hate Speech Detection in Low-Resource Languages

Ruhina Tabasshum Prome, Tarikul Islam Tamiti, Anomadarshi Barua

机构 * Bangladesh Institute of Governance and Management (BIGM)（孟加拉国治理与管理研究所） ； Department of Cyber Security Engineering, George Mason University（网络安全工程系，乔治·梅森大学） 纠错

专题命中 安全训练 ：safety(abstract)；分类 cs.CL、cs.AI

Leveraging Edge Intelligence and LLMs to Advance 6G-Enabled Internet of Automated Defense Vehicles

Murat Arda Onsu, Poonam Lohan, Burak Kantarci

机构 * School of Electrical Engineering and Computer Science University of Ottawa（电气工程与计算机科学学院渥太华大学） 纠错

专题命中 安全训练 ：safety(abstract)；分类 cs.CL、cs.AI

Comments 8 pages, 5 figures, accepted to IEEE Internet of Things Magazine

Rethinking Explainability in the Era of Multimodal AI

Chirag Agarwal

机构 * University of Virginia（弗吉尼亚大学） 纠错

专题命中 安全训练 ：safety(abstract)；分类 cs.AI、cs.LG

Compositional Shielding and Reinforcement Learning for Multi-Agent Systems

Asger Horn Brorholt, Kim Guldstrand Larsen, Christian Schilling

机构 * Aalborg University（奥胡斯大学） 纠错

专题命中 安全训练 ：safety(abstract)；分类 cs.AI、cs.LG

Journal ref AAMAS 2025

On the Generalization of Data-Assisted Control in port-Hamiltonian Systems (DAC-pH)

Mostafa Eslami, Maryam Babazadeh

机构 * Sharif University of Technology（谢里夫大学） 纠错

专题命中 安全训练 ：safety(abstract)；分类 cs.AI、cs.LG

Comments This paper presents an early investigation of Data-Assisted Control (DAC) with reinforcement learning, showcasing its potential through a simple example. Theoretical analysis is ongoing to establish formal support and guarantees for the proposed approach

Enhancing Decision-Making of Large Language Models via Actor-Critic

Heng Dong, Kefei Duan, Chongjie Zhang

专题命中 安全训练 ：alignment(abstract)；分类 cs.CL、cs.AI

Comments Forty-second International Conference on Machine Learning (ICML 2025)

LlavaGuard: An Open VLM-based Framework for Safeguarding Vision Datasets and Models

Lukas Helff, Felix Friedrich, Manuel Brack, Kristian Kersting, Patrick Schramowski

专题命中 安全训练 ：safety(abstract)；分类 cs.AI、cs.LG

Comments In Proceedings of the 42st International Conference on Machine Learning (ICML 2025), Project page at https://ml-research.github.io/human-centered-genai/projects/llavaguard/index.html

Comparative performance of ensemble models in predicting dental provider types: insights from fee-for-service data

Mohammad Subhi Al-Batah, Muhyeeddin Alqaraleh, Mowafaq Salem Alzboon, Abdullah Alourani

专题命中 安全训练 ：safety(abstract)；分类 cs.AI、cs.LG

Journal ref Data and Metadata [Internet]. 2025 Mar. 29 [cited 2025 Jun. 4];4:750

VLMs Can Aggregate Scattered Training Patches

Zhanhui Zhou, Lingjie Chen, Chao Yang, Chaochao Lu

机构 * Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） 纠错

专题命中 安全训练 ：safety(abstract)；分类 cs.CL、cs.AI

On-Policy RL with Optimal Reward Baseline

Yaru Hao, Li Dong, Xun Wu, Shaohan Huang, Zewen Chi, Furu Wei

机构 * Microsoft Research（微软研究院） 纠错

专题命中 安全训练 ：alignment(abstract)；分类 cs.CL、cs.LG

The Hidden Language of Harm: Examining the Role of Emojis in Harmful Online Communication and Content Moderation

Yuhang Zhou, Yimin Xiao, Wei Ai, Ge Gao

机构 * University of Maryland, College Park（马里兰大学学院公园分校） 纠错

专题命中 安全训练 ：safety(abstract)；分类 cs.CL、cs.CY

Comments 18 pages, 3 figures

COSMIC: Generalized Refusal Direction Identification in LLM Activations

Vincent Siu, Nicholas Crispino, Zihao Yu, Sam Pan, Zhun Wang, Yang Liu, Dawn Song, Chenguang Wang

机构 * Washington University in St. Louis（华盛顿大学圣路易斯分校） ； University of California, Berkeley（加州大学伯克利分校） ； University of California, Santa Cruz（加州大学圣克ruz分校） 纠错

专题命中 安全训练 ：alignment(abstract)；分类 cs.CL、cs.AI

Comments 9 pages, Accepted to ACL 2025 Findings

An Optimistic Algorithm for online CMDPS with Anytime Adversarial Constraints

Jiahui Zhu, Kihyun Yu, Dabeen Lee, Xin Liu, Honghao Wei

专题命中 安全训练 ：safety(abstract)；分类 cs.AI、cs.LG

Comments Proceedings of the 41 st International Conference on Machine Learning

BIRD: Behavior Induction via Representation-structure Distillation

Galen Pogoncheff, Michael Beyeler

机构 * Department of Computer Science（计算机科学系） ； UC Santa Barbara（圣巴巴拉大学） ； Department of Psychological & Brain Sciences（心理学与脑科学系） 纠错

专题命中 安全训练 ：alignment(abstract)；分类 cs.AI、cs.LG

Sentiment Simulation using Generative AI Agents

Melrose Tia, Jezreel Sophia Lanuzo, Lei Rigi Baltazar, Marie Joy Lopez-Relente, Diwa Malaya Quiñones, Jason Albia

机构 * Netopia AI, Inc.（Netopia AI公司） ； Institute of Statistics, University of the Philippines Los Baños（菲律宾大学Los Baños统计研究所） ； Department of Psychology, University of the Philippines Diliman（菲律宾大学Diliman心理学系） 纠错

专题命中 安全训练 ：alignment(abstract)；分类 cs.AI、cs.CY

Comments 18 pages, 10 figures

Bridging the Domain Gap in Equation Distillation with Reinforcement Feedback

Wangyang Ying, Haoyue Bai, Nanxu Gong, Xinyuan Wang, Sixun Dong, Haifeng Chen, Yanjie Fu

机构 * Arizona State University（亚利桑那州立大学） ； NEC Laboratories America, Inc.（NEC美国实验室） 纠错

专题命中 安全训练 ：alignment(abstract)；分类 cs.AI、cs.LG

Is LLM an Overconfident Judge? Unveiling the Capabilities of LLMs in Detecting Offensive Language with Annotation Disagreement

Junyu Lu, Kai Ma, Kaichun Wang, Kelaiti Xiao, Roy Ka-Wei Lee, Bo Xu, Liang Yang, Hongfei Lin

机构 * Key Laboratory of Social Computing and Cognitive Intelligence, Dalian University of Technology（大连理工大学社会计算与认知智能重点实验室） ； School of Computer Science and Technology, Xinjiang Normal University（新疆师范大学计算机科学与技术学院） ； Social AI Studio, Singapore University of Technology and Design（新加坡科技设计大学社会AI工作室） 纠错

专题命中 安全训练 ：alignment(abstract)；分类 cs.CL、cs.AI

Comments 18 pages, accepted at the ACL 2025

Performance Gains of LLMs With Humans in a World of LLMs Versus Humans

Lucas McCullum, Pelagie Ami Agassi, Leo Anthony Celi, Daniel K. Ebner, Chrystinne Oliveira Fernandes, Rachel S. Hicklen, Mkliwa Koumbia, Lisa Soleymani Lehmann, David Restrepo

专题命中 安全训练 ：safety(abstract)；分类 cs.CL、cs.AI

Public Constitutional AI

Gilad Abiri

专题命中 安全训练 ：alignment(abstract)；分类 cs.AI、cs.CY

Combining Bayesian Inference and Reinforcement Learning for Agent Decision Making: A Review

Chengmin Zhou, Ville Kyrki, Pasi Fränti, Laura Ruotsalainen

专题命中 安全训练 ：safety(abstract)；分类 cs.AI、cs.LG

Skill-based Safe Reinforcement Learning with Risk Planning

Hanping Zhang, Yuhong Guo

专题命中 安全训练 ：safety(abstract)；分类 cs.AI、cs.LG

Latent Adversarial Training Improves the Representation of Refusal

Alexandra Abbas, Nora Petrova, Helios Ael Lyons, Natalia Perez-Campanero

机构 * Apart Research（Apart研究） 纠错

专题命中 安全训练 ：safety(abstract)；分类 cs.CL、cs.LG

Enhance Exploration in Safe Reinforcement Learning with Contrastive Representation Learning

Duc Kien Doan, Bang Giang Le, Viet Cuong Ta

机构 * HMI Laboratory VNU University of Engineering and Technology（VNU工程技术大学HMI实验室） 纠错

专题命中 安全训练 ：safety(abstract)；分类 cs.AI、cs.LG

Comments Accepted at ACIIDS 2025

Tinker Tales: Interactive Storytelling Framework for Early Childhood Narrative Development and AI Literacy

Nayoung Choi, Peace Cyebukayire, Jinho D. Choi

专题命中 安全训练 ：safety(abstract)；分类 cs.AI、cs.CY

Solving Multi-Agent Safe Optimal Control with Distributed Epigraph Form MARL

Songyuan Zhang, Oswin So, Mitchell Black, Zachary Serlin, Chuchu Fan

机构 * MIT（麻省理工学院） ； MIT Lincoln Laboratory（麻省理工学院林肯实验室） 纠错

专题命中 安全训练 ：safety(abstract)；分类 cs.AI、cs.LG

Comments 28 pages, 16 figures; Accepted by Robotics: Science and Systems 2025

Solving New Tasks by Adapting Internet Video Knowledge

Calvin Luo, Zilai Zeng, Yilun Du, Chen Sun

专题命中 安全训练 ：alignment(abstract)；分类 cs.AI、cs.LG

Comments ICLR 2025. Project Webpage: https://diffusion-supervision.github.io/adapt2act/