大模型对齐与安全

Quantifying the Importance of Data Alignment in Downstream Model Performance

Krrish Chawla, Aryan Sahai, Mario DePavia, Sudharsan Sundar, Brando Miranda, Elyas Obbad, Sanmi Koyejo

机构 * stanford（斯坦福大学） 纠错

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Journal ref ICLR DMLR Data-centric Machine Learning Research (2024), ICML DataWorld (2025)

VITAL: A New Dataset for Benchmarking Pluralistic Alignment in Healthcare

Anudeex Shetty, Amin Beheshti, Mark Dras, Usman Naseem

机构 * School of Computing and Information System, the University of Melbourne, Australia（墨尔本大学计算与信息学院） ； School of Computing, FSE, Macquarie University, Australia（麦考瑞大学计算学院） 纠错

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to ACL 2025 (Main Proceedings)

SweEval: Do LLMs Really Swear? A Safety Benchmark for Testing Limits for Enterprise Use

Hitesh Laxmichand Patel, Amit Agarwal, Arion Das, Bhargava Kumar, Srikant Panda, Priyaranjan Pattnayak, Taki Hasan Rafi, Tejaswini Kumar, Dong-Kyu Chae

机构 * Oracle AI ； Indian Institute of Information Technology Ranchi（印度拉奇信息与技术学院） ； TD Securities（TD证券） ； Columbia University（哥伦比亚大学） ； Hanyang University（翰阳大学） 纠错

专题命中 安全评测 ：safety(title)；alignment(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published in the Proceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2025), Industry Track, pages 558-582

MORALISE: A Structured Benchmark for Moral Alignment in Visual Language Models

Xiao Lin, Zhining Liu, Ze Yang, Gaotang Li, Ruizhong Qiu, Shuke Wang, Hui Liu, Haotian Li, Sumit Keswani, Vishwa Pardeshi, Huijun Zhao, Wei Fan, Hanghang Tong

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） ； Amazon（亚马逊） ； Fidelity Investments（富达投资） 纠错

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.CY

Comments 21 pages, 11 figures, 7 tables

Understanding Gen Alpha Digital Language: Evaluation of LLM Safety Systems for Content Moderation

Manisha Mehta, Fausto Giunchiglia

机构 * Warren E Hyde Middle School（沃伦·E·海德中学） ； University of Trento（特伦托大学） 纠错

专题命中 安全评测 ：safety(title,abstract)；分类 cs.CL、cs.AI、cs.CY

Comments Accepted to ACM FAccT 2025. To be presented in Athens, June 2025, and published in the conference proceedings. Preprint version; final version will appear in the ACM Digital Library

Safety Evaluation and Enhancement of DeepSeek Models in Chinese Contexts

Wenjing Zhang, Xuejiao Lei, Zhaoxiang Liu, Limin Han, Jiaojiao Zhao, Junting Guo, Zhenhong Long, Shu Yang, Meijuan An, Beibei Huang, Rongjia Du, Ning Wang, Kai Wang, Shiguo Lian

机构 * Unicom Data Intelligence（中国联通数据智能研究所） ； Data Science & Artificial Intelligence Research Institute（数据科学与人工智能研究院） 纠错

专题命中 安全评测 ：safety(title,abstract)；分类 cs.CL、cs.AI、cs.CY

Comments 21 pages, 13 figures, 4 tables

Unmasking the Canvas: A Dynamic Benchmark for Image Generation Jailbreaking and LLM Content Safety

Variath Madhupal Gautham Nair, Vishal Varma Dantuluri

专题命中 安全评测 ：safety(title,abstract)；分类 cs.CL、cs.AI、cs.CY

Trustworthy artificial intelligence in the energy sector: Landscape analysis and evaluation framework

Sotiris Pelekis, Evangelos Karakolis, George Lampropoulos, Spiros Mouzakitis, Ourania Markaki, Christos Ntanos, Dimitris Askounis

专题命中 安全评测 ：trustworthy(title,abstract)；分类 cs.AI、cs.CY、cs.LG

A Comprehensive Survey and Classification of Evaluation Criteria for Trustworthy Artificial Intelligence

Louise McCormack, Malika Bendechache

专题命中 安全评测 ：trustworthy(title,abstract)；分类 cs.AI、cs.CY、cs.LG

Comments This work has been accepted for publication in AI and Ethics

FormalAlign: Automated Alignment Evaluation for Autoformalization

Jianqiao Lu, Yingjia Wan, Yinya Huang, Jing Xiong, Zhengying Liu, Zhijiang Guo

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 23 pages, 13 tables, 3 figures

Toward General Instruction-Following Alignment for Retrieval-Augmented Generation

Guanting Dong, Xiaoshuai Song, Yutao Zhu, Runqi Qiao, Zhicheng Dou, Ji-Rong Wen

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Working in progress

Revisiting the Superficial Alignment Hypothesis

Mohit Raghavendra, Vaskar Nath, Sean Hendryx

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Improving Concept Alignment in Vision-Language Concept Bottleneck Models

Nithish Muthuchamy Selvaraj, Xiaobao Guo, Adams Wai-Kin Kong, Alex Kot

专题命中 安全评测 ：alignment(title,abstract)；trustworthy(abstract)

AlignBench: Benchmarking Chinese Alignment of Large Language Models

Xiao Liu, Xuanyu Lei, Shengyuan Wang, Yue Huang, Zhuoer Feng, Bosi Wen, Jiale Cheng, Pei Ke, Yifan Xu, Weng Lam Tam, Xiaohan Zhang, Lichao Sun, Xiaotao Gu, Hongning Wang, Jing Zhang, Minlie Huang, Yuxiao Dong, Jie Tang

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to ACL 2024

Emergence in Multi-Agent Systems: A Safety Perspective

Philipp Altmann, Julian Schönberger, Steffen Illium, Maximilian Zorn, Fabian Ritz, Tom Haider, Simon Burton, Thomas Gabor

专题命中 安全评测 ：safety(title,abstract)；alignment(abstract)

Comments 18 pages, 3 figures, accepted for publication at the International Symposium on Leveraging Applications of Formal Methods (ISoLA 2024)

FUTURE-AI: International consensus guideline for trustworthy and deployable artificial intelligence in healthcare

Karim Lekadir, Aasa Feragen, Abdul Joseph Fofanah, Alejandro F Frangi, Alena Buyx, Anais Emelie, Andrea Lara, Antonio R Porras, An-Wen Chan, Arcadi Navarro, Ben Glocker, Benard O Botwe, Bishesh Khanal, Brigit Beger, Carol C Wu, Celia Cintas, Curtis P Langlotz, Daniel Rueckert, Deogratias Mzurikwao, Dimitrios I Fotiadis, Doszhan Zhussupov, Enzo Ferrante, Erik Meijering, Eva Weicken, Fabio A González, Folkert W Asselbergs, Fred Prior, Gabriel P Krestin, Gary Collins, Geletaw S Tegenaw, Georgios Kaissis, Gianluca Misuraca, Gianna Tsakou, Girish Dwivedi, Haridimos Kondylakis, Harsha Jayakody, Henry C Woodruf, Horst Joachim Mayer, Hugo JWL Aerts, Ian Walsh, Ioanna Chouvarda, Irène Buvat, Isabell Tributsch, Islem Rekik, James Duncan, Jayashree Kalpathy-Cramer, Jihad Zahir, Jinah Park, John Mongan, Judy W Gichoya, Julia A Schnabel, Kaisar Kushibar, Katrine Riklund, Kensaku Mori, Kostas Marias, Lameck M Amugongo, Lauren A Fromont, Lena Maier-Hein, Leonor Cerdá Alberich, Leticia Rittner, Lighton Phiri, Linda Marrakchi-Kacem, Lluís Donoso-Bach, Luis Martí-Bonmatí, M Jorge Cardoso, Maciej Bobowicz, Mahsa Shabani, Manolis Tsiknakis, Maria A Zuluaga, Maria Bielikova, Marie-Christine Fritzsche, Marina Camacho, Marius George Linguraru, Markus Wenzel, Marleen De Bruijne, Martin G Tolsgaard, Marzyeh Ghassemi, Md Ashrafuzzaman, Melanie Goisauf, Mohammad Yaqub, Mónica Cano Abadía, Mukhtar M E Mahmoud, Mustafa Elattar, Nicola Rieke, Nikolaos Papanikolaou, Noussair Lazrak, Oliver Díaz, Olivier Salvado, Oriol Pujol, Ousmane Sall, Pamela Guevara, Peter Gordebeke, Philippe Lambin, Pieta Brown, Purang Abolmaesumi, Qi Dou, Qinghua Lu, Richard Osuala, Rose Nakasi, S Kevin Zhou, Sandy Napel, Sara Colantonio, Shadi Albarqouni, Smriti Joshi, Stacy Carter, Stefan Klein, Steffen E Petersen, Susanna Aussó, Suyash Awate, Tammy Riklin Raviv, Tessa Cook, Tinashe E M Mutsvangwa, Wendy A Rogers, Wiro J Niessen, Xènia Puig-Bosch, Yi Zeng, Yunusa G Mohammed, Yves Saint James Aquino, Zohaib Salahuddin, Martijn P A Starmans

专题命中 安全评测 ：trustworthy(title,abstract)；分类 cs.AI、cs.CY、cs.LG

MM-Instruct: Generated Visual Instructions for Large Multimodal Model Alignment

Jihao Liu, Xin Huang, Jinliang Zheng, Boxiao Liu, Jia Wang, Osamu Yoshie, Yu Liu, Hongsheng Li

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Dataset and models are available at https://github.com/jihaonew/MM-Instruct

Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models

Abhishek Kumar, Robert Morabito, Sanzhar Umbet, Jad Kabbara, Ali Emami

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 9 pages (excluding references), accepted to ACL 2024 Main Conference

SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models

Lijun Li, Bowen Dong, Ruohui Wang, Xuhao Hu, Wangmeng Zuo, Dahua Lin, Yu Qiao, Jing Shao

专题命中 安全评测 ：safety(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at ACL 2024 Findings

Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models

Yihong Dong, Xue Jiang, Huanyu Liu, Zhi Jin, Bin Gu, Mengfei Yang, Ge Li

专题命中 安全评测 ：trustworthy(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to ACL

Heterogeneous Value Alignment Evaluation for Large Language Models

Zhaowei Zhang, Ceyao Zhang, Nian Liu, Siyuan Qi, Ziqi Rong, Song-Chun Zhu, Shuguang Cui, Yaodong Yang

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments v3 is the camera-ready version for AAAI-24 Workshop on Public Sector LLMs: Algorithmic and Sociotechnical Design. 7 pages of main content, 1 page of references, 3 pages of appendices, and 7 figures. Our full prompts are released in the repo: https://github.com/zowiezhang/HVAE

JADE: A Linguistics-based Safety Evaluation Platform for Large Language Models

Mi Zhang, Xudong Pan, Min Yang

专题命中 安全评测 ：safety(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments A preprint work. Benchmark link: https://github.com/whitzard-ai/jade-db. Website link: https://whitzard-ai.github.io/jade.html

Unveiling Safety Vulnerabilities of Large Language Models

George Kour, Marcel Zalmanovici, Naama Zwerdling, Esther Goldbraich, Ora Nova Fandina, Ateret Anaby-Tavor, Orna Raz, Eitan Farchi

专题命中 安全评测 ：safety(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments To be published in GEM workshop. Conference on Empirical Methods in Natural Language Processing (EMNLP). 2023

Creating Trustworthy LLMs: Dealing with Hallucinations in Healthcare AI

Muhammad Aurangzeb Ahmad, Ilker Yaramis, Taposh Dutta Roy

专题命中 安全评测 ：trustworthy(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Sociotechnical Safety Evaluation of Generative AI Systems

Laura Weidinger, Maribeth Rauh, Nahema Marchal, Arianna Manzini, Lisa Anne Hendricks, Juan Mateos-Garcia, Stevie Bergman, Jackie Kay, Conor Griffin, Ben Bariach, Iason Gabriel, Verena Rieser, William Isaac

专题命中 安全评测 ：safety(title,abstract)；分类 cs.CL、cs.AI、cs.CY

Comments main paper p.1-29, 5 figures, 2 tables

From Instructions to Intrinsic Human Values -- A Survey of Alignment Goals for Big Models

Jing Yao, Xiaoyuan Yi, Xiting Wang, Jindong Wang, Xing Xie

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.CY

Comments 20 pages, 5 figures

Guideline for Trustworthy Artificial Intelligence -- AI Assessment Catalog

Maximilian Poretschkin, Anna Schmitz, Maram Akila, Linara Adilova, Daniel Becker, Armin B. Cremers, Dirk Hecker, Sebastian Houben, Michael Mock, Julia Rosenzweig, Joachim Sicking, Elena Schulz, Angelika Voss, Stefan Wrobel

专题命中 安全评测 ：trustworthy(title,abstract)；分类 cs.AI、cs.CY、cs.LG

DICES Dataset: Diversity in Conversational AI Evaluation for Safety

Lora Aroyo, Alex S. Taylor, Mark Diaz, Christopher M. Homan, Alicia Parrish, Greg Serapio-Garcia, Vinodkumar Prabhakaran, Ding Wang

专题命中 安全评测 ：safety(title,abstract)；AI safety(abstract)

Trustworthy AI Inference Systems: An Industry Research View

Rosario Cammarota, Matthias Schunter, Anand Rajan, Fabian Boemer, Ágnes Kiss, Amos Treiber, Christian Weinert, Thomas Schneider, Emmanuel Stapf, Ahmad-Reza Sadeghi, Daniel Demmler, Joshua Stock, Huili Chen, Siam Umar Hussain, Sadegh Riazi, Farinaz Koushanfar, Saransh Gupta, Tajan Simunic Rosing, Kamalika Chaudhuri, Hamid Nejatollahi, Nikil Dutt, Mohsen Imani, Kim Laine, Anuj Dubey, Aydin Aysu, Fateme Sadat Hosseini, Chengmo Yang, Eric Wallace, Pamela Norton

专题命中 安全评测 ：trustworthy(title,abstract)；分类 cs.AI、cs.CY、cs.LG

BERTMap: A BERT-based Ontology Alignment System

Yuan He, Jiaoyan Chen, Denvar Antonyrajah, Ian Horrocks

专题命中 安全评测 ：alignment(title,abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Full version (with appendix) of the accepted paper in 36th AAAI Conference on Artificial Intelligence 2022