大模型对齐与安全

Where Did It Go Wrong? Attributing Undesirable LLM Behaviors via Representation Gradient Tracing

Zhe Li, Wei Zhao, Yige Li, Jun Sun

机构 * Singapore Management University（新加坡管理大学） 纠错

专题命中 安全评测 ：safety(abstract)；AI safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 16 pages, 4 figures

Secure Multi-Modal Data Fusion in Federated Digital Health Systems via MCP

Aueaphum Aueawatthanaphisut

机构 * School of Information, Computer, and Communication Technology（信息、计算机与通信技术学院） ； Sirindhorn International Institute of Technology, Thammasat University（泰国朱拉安吞国际技术学院，泰国 Thammasat 大学） 纠错

专题命中 安全评测 ：alignment(abstract)；trustworthy(abstract)；分类 cs.AI、cs.CY、cs.LG

Comments 6 pages, 8 figures, 7 equations, 1 algorithm

A Comprehensive Survey on the Trustworthiness of Large Language Models in Healthcare

Manar Aljohani, Jun Hou, Sindhura Kommu, Xuan Wang

机构 * Virginia Tech（维吉尼亚理工大学） 纠错

专题命中 安全评测 ：safety(abstract)；trustworthy(abstract)；分类 cs.CL、cs.AI、cs.CY

Safe and Certifiable AI Systems: Concepts, Challenges, and Lessons Learned

Kajetan Schweighofer, Barbara Brune, Lukas Gruber, Simon Schmid, Alexander Aufreiter, Andreas Gruber, Thomas Doms, Sebastian Eder, Florian Mayer, Xaver-Paul Stadlbauer, Christoph Schwald, Werner Zellinger, Bernhard Nessler, Sepp Hochreiter

机构 * TRUSTIFAI GMBH（TRUSTIFAI公司） ； TÜV AUSTRIA HOLDING AG（TÜV奥地利控股公司） ； Software Competence Center Hagenberg（哈根贝格软件竞争力中心） ； Johannes Kepler University Linz - Institute for Machine Learning（林茨约瑟夫·约瑟夫斯大学-机器学习研究所） 纠错

专题命中 安全评测 ：safety(abstract)；trustworthy(abstract)；分类 cs.AI、cs.CY、cs.LG

Comments 63 pages, 27 figures

Shared Autonomy through LLMs and Reinforcement Learning for Applications to Ship Hull Inspections

Cristiano Caissutti, Estelle Gerbier, Ehsan Khorrambakht, Paolo Marinelli, Andrea Munafo', Andrea Caiti

机构 * Dept. of Information Engineering University of Pisa, Italy Pisa, Italy（信息工程系 乌迪内大学 乌迪内，意大利） 纠错

专题命中 安全评测 ：alignment(abstract)；safety(abstract)；trustworthy(abstract)

LUMA: A Benchmark Dataset for Learning from Uncertain and Multimodal Data

Grigor Bezirganyan, Sana Sellami, Laure Berti-Équille, Sébastien Fournier

机构 * Aix Marseille Univ, CNRS, LIS（阿维尼翁-马赛大学、国家科学研究中心、LIS） 纠错

专题命中 安全评测 ：safety(abstract)；trustworthy(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments SIGIR 2025

Journal ref Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2025

Model-Based Soft Maximization of Suitable Metrics of Long-Term Human Power

Jobst Heitzig, Ram Potham

专题命中 安全评测 ：safety(abstract)；AI safety(abstract)；分类 cs.AI、cs.CY、cs.LG

Evaluation of LLM Vulnerabilities to Being Misused for Personalized Disinformation Generation

Aneta Zugecova, Dominik Macko, Ivan Srba, Robert Moro, Jakub Kopal, Katarina Marcincinova, Matus Mesarcik

机构 * Kempelen Institute of Intelligent Technologies（凯普勒智能技术研究所） ； University of Copenhagen（哥本哈根大学） ； Comenius University in Bratislava（布拉迪斯拉瓦科เมนius大学） 纠错

专题命中 安全评测 ：safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI、cs.CY

Comments ACL 2025 main

Journal ref Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025 Volume 1: Long Papers)

SciFi-Benchmark: Leveraging Science Fiction To Improve Robot Behavior

Pierre Sermanet, Anirudha Majumdar, Vikas Sindhwani

专题命中 安全评测 ：alignment(abstract)；safety(abstract)；分类 cs.CL、cs.AI、cs.CY

Comments Minor improvements over previous version

Multimodal Machine Learning in Mental Health: A Survey of Data, Algorithms, and Challenges

Zahraa Al Sahili, Ioannis Patras, Matthew Purver

机构 * Queen Mary University of London United Kingdom ； Queen Mary University of London \& Jo z ef Stefan Institute United Kingdom \& Slovenia ； Queen Mary University of London ； Queen Mary University of London \& Jo z ef Stefan Institute 纠错

专题命中 安全评测 ：alignment(abstract)；trustworthy(abstract)；分类 cs.AI、cs.CY、cs.LG

Scaling Trends in Language Model Robustness

Nikolaus Howe, Ian McKenzie, Oskar Hollinsworth, Michał Zajac, Tom Tseng, Aaron Tucker, Pierre-Luc Bacon, Adam Gleave

专题命中 安全评测 ：safety(abstract)；prompt injection(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 59 pages; updated to ICML version

Faster, Cheaper, Better: Multi-Objective Hyperparameter Optimization for LLM and RAG Systems

Matthew Barker, Andrew Bell, Evan Thomas, James Carr, Thomas Andrews, Umang Bhatt

机构 * Trustwise AI ； New York University（纽约大学） ； The Alan Turing Institute（艾伦·图灵研究所） 纠错

专题命中 安全评测 ：alignment(abstract)；safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Humanizing LLMs: A Survey of Psychological Measurements with Tools, Datasets, and Human-Agent Applications

Wenhan Dong, Yuemeng Zhao, Zhen Sun, Yule Liu, Zifan Peng, Jingyi Zheng, Zongmin Zhang, Ziyi Zhang, Jun Wu, Ruiming Wang, Shengmin Xu, Xinyi Huang, Xinlei He

机构 * Information Hub, Hong Kong University of Science and Technology (Guangzhou)（香港理工大学（广州）信息中心） ； School of Psychology, South China Normal University（华南师范大学心理学学院） ； College of Computer and Cyber Security, Fujian Normal University（福建师范大学计算机与网络安全学院） ； College of Cyber Security, Jinan University（济南大学网络安全学院） 纠错

专题命中 安全评测 ：alignment(abstract)；trustworthy(abstract)；分类 cs.CL、cs.CY、cs.LG

Comments 26 pages,7 figures

AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents

Maksym Andriushchenko, Alexandra Souly, Mateusz Dziemian, Derek Duenas, Maxwell Lin, Justin Wang, Dan Hendrycks, Andy Zou, Zico Kolter, Matt Fredrikson, Eric Winsor, Jerome Wynne, Yarin Gal, Xander Davies

专题命中 安全评测 ：safety(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at ICLR 2025

RECALL-MM: A Multimodal Dataset of Consumer Product Recalls for Risk Analysis using Computational Methods and Large Language Models

Diana Bolanos, Mohammadmehdi Ataei, Daniele Grandi, Kosa Goucher-Lambert

专题命中 安全评测 ：alignment(abstract)；safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Intolerable Risk Threshold Recommendations for Artificial Intelligence

Deepika Raman, Nada Madkour, Evan R. Murphy, Krystal Jackson, Jessica Newman

专题命中 安全评测 ：safety(abstract)；AI safety(abstract)；分类 cs.AI、cs.CY、cs.LG

Comments 79 pages

Mind the Gesture: Evaluating AI Sensitivity to Culturally Offensive Non-Verbal Gestures

Akhila Yerukola, Saadia Gabriel, Nanyun Peng, Maarten Sap

专题命中 安全评测 ：safety(abstract)；AI safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 40 pages, 49 figures

Tell me about yourself: LLMs are aware of their learned behaviors

Jan Betley, Xuchan Bao, Martín Soto, Anna Sztyber-Betley, James Chua, Owain Evans

专题命中 安全评测 ：safety(abstract)；AI safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Submitted to ICLR 2025. 17 pages, 13 figures

Two Types of AI Existential Risk: Decisive and Accumulative

Atoosa Kasirzadeh

专题命中 安全评测 ：safety(abstract)；AI safety(abstract)；分类 cs.AI、cs.CY、cs.LG

Comments Journal article for Philosophical Studies

A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy

Huandong Wang, Wenjie Fu, Yingzhou Tang, Zhilong Chen, Yuxi Huang, Jinghua Piao, Chen Gao, Fengli Xu, Tao Jiang, Yong Li

专题命中 安全评测 ：alignment(abstract)；jailbreak(abstract)；分类 cs.CL、cs.AI、cs.CY

Language-Guided World Models: A Model-Based Approach to AI Control

Alex Zhang, Khanh Nguyen, Jens Tuyls, Albert Lin, Karthik Narasimhan

专题命中 安全评测 ：safety(abstract)；AI safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments SpLU-RoboNLP workshop at ACL 2024

Me, Myself, and AI: The Situational Awareness Dataset (SAD) for LLMs

Rudolf Laine, Bilal Chughtai, Jan Betley, Kaivalya Hariharan, Jeremy Scheurer, Mikita Balesni, Marius Hobbhahn, Alexander Meinke, Owain Evans

专题命中 安全评测 ：safety(abstract)；AI safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 11 page main body, 98 page appendix, 58 figures

Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation

Sahar Abdelnabi, Amr Gomaa, Sarath Sivaprasad, Lea Schönherr, Mario Fritz

专题命中 安全评测 ：alignment(abstract)；safety(abstract)；分类 cs.CL、cs.CY、cs.LG

Comments Updated version with major additions (new experiments, evaluation, and attacks)

Unlocking the Potential of Large Language Models for Clinical Text Anonymization: A Comparative Study

David Pissarra, Isabel Curioso, João Alveira, Duarte Pereira, Bruno Ribeiro, Tomás Souper, Vasco Gomes, André V. Carreiro, Vitor Rolla

专题命中 安全评测 ：safety(abstract)；trustworthy(abstract)；分类 cs.CL、cs.AI、cs.LG

Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models

Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Satheesh Katipomu, Haonan Li, Fajri Koto, William Marshall, Gurpreet Gosal, Cynthia Liu, Zhiming Chen, Osama Mohammed Afzal, Samta Kamboj, Onkar Pandit, Rahul Pal, Lalit Pradhan, Zain Muhammad Mujahid, Massa Baali, Xudong Han, Sondos Mahmoud Bsharat, Alham Fikri Aji, Zhiqiang Shen, Zhengzhong Liu, Natalia Vassilieva, Joel Hestness, Andy Hock, Andrew Feldman, Jonathan Lee, Andrew Jackson, Hector Xuguang Ren, Preslav Nakov, Timothy Baldwin, Eric Xing

专题命中 安全评测 ：alignment(abstract)；safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Arabic-centric, foundation model, large-language model, LLM, generative model, instruction-tuned, Jais, Jais-chat

AAAI FSS-19: Human-Centered AI: Trustworthiness of AI Models and Data Proceedings

Florian Buettner, John Piorkowski, Ian McCulloh, Ulli Waltinger

专题命中 安全评测 ：safety(abstract)；trustworthy(abstract)；分类 cs.AI、cs.CY、cs.LG

Comments Proceedings for AAAI 2019 Fall Symposium Series - Human-centered AI: Trustworthiness of AI Models & Data

Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security

自适应对手：用于大语言模型智能体安全的多轮、多大语言模型基准测试

Devina Jain, David Hartmann, Chuan Li

机构 * Lambda 纠错

专题命中 安全评测 ：safety(abstract,comments)；prompt injection(abstract)；分类 cs.AI、cs.LG

AI总结 该研究提出针对无记忆大语言模型防御者的自适应多轮攻击的21场景基准测试，通过观察防御者响应灵活调整攻击。介绍了不同攻击轮次成功率，汇集多个前沿攻击者大语言模型的情况，还指出不同防御者弱点差异及场景排名不一致性，并发布了相关基准测试及数据集。

Comments Second Workshop on Agents in the Wild: Safety, Security, and Beyond

LieCraft: A Multi-Agent Framework for Evaluating Deceptive Capabilities in Language Models

LieCraft：一种用于评估语言模型欺骗能力的多智能体框架

Matthew Lyle Olson, Neale Ratzlaff, Musashi Hinck, Tri Nguyen, Vasudev Lal, Joseph Campbell, Simon Stepputtis, Shao-Yen Tseng

机构 * Intel Labs（英特尔实验室） 纠错

专题命中 安全评测 ：alignment(abstract,comments)；safety(abstract)；分类 cs.CL、cs.AI

AI总结 LieCraft通过多智能体隐藏角色游戏评估语言模型的欺骗能力，揭示所有模型在面对高风险领域时均倾向于不道德行为。

Comments AAAI 2026 Alignment track. Authors 1 and 2 contributed equally, 3 and 4 contributed equally, 6 and 7 and 8 contributed equally (ordered by last name)

Prediction certification cannot replace explanation certification: a competence envelope for trustworthy AI under compound stress

预测认证无法替代解释认证：复合压力下可信人工智能的能力边界

Nataliya Shakhovska, Ivan Izonin, Stergios-Aristoteles Mitoulis

专题命中 安全评测 ：trustworthy(title)；分类 cs.AI

AI总结 研究表明仅预测侧认证无法确保AI可信，提出结合预测与解释认证的能力边界框架，可捕捉预测侧认证遗漏的失效模式，为可信AI认证提供新方法。

ConspirED: A Dataset for Cognitive Traits of Conspiracy Theories and Large Language Model Safety

ConspirED：一个用于研究阴谋论认知特质与大语言模型安全性的数据集

Luke Bates, Max Glockner, Preslav Nakov, Iryna Gurevych

专题命中 安全评测 ：safety(title)；分类 cs.CL

AI总结 该研究推出首个标注阴谋内容通用认知特质的CONSPIRED数据集，利用其开发识别阴谋特质的计算模型，并发现大语言模型易被阴谋框架误导而复制其修辞模式。

Comments Accepted at TACL