大模型对齐与安全

Civil Society in the Loop: Feedback-Driven Adaptation of (L)LM-Assisted Classification in an Open-Source Telegram Monitoring Tool

Milena Pustet, Elisabeth Steffen, Helena Mihaljević, Grischa Stanjek, Yannis Illies

机构 * HTW Berlin（柏林应用技术大学） 纠错

专题命中 安全训练 ：alignment(abstract)；分类 cs.CL、cs.AI、cs.CY

7B Fully Open Source Moxin-LLM/VLM -- From Pretraining to GRPO-based Reinforcement Learning Enhancement

Pu Zhao, Xuan Shen, Zhenglun Kong, Yixin Shen, Sung-En Chang, Arash Akbari, Timothy Rupprecht, Lei Lu, Enfu Nan, Changdi Yang, Yumei He, Weiyan Shi, Xingchen Xu, Yu Huang, Wei Jiang, Wei Wang, Yue Chen, Yong He, Yanzhi Wang

机构 * Northeastern University（东北大学） ； Harvard University（哈佛大学） ； Cornell University（康奈尔大学） ； Tulane University（路易斯安那州立大学） ； University of Washington（华盛顿大学） ； Futurewei Technologies（未来科技） ； AIBAO LLC 纠错

专题命中 安全训练 ：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

On the Surprising Efficacy of LLMs for Penetration-Testing

Andreas Happe, Jürgen Cito

专题命中 安全训练 ：alignment(abstract)；safety(abstract)

NSFW-Classifier Guided Prompt Sanitization for Safe Text-to-Image Generation

Yu Xie, Chengjie Zeng, Lingyun Zhang, Yanwei Fu

机构 * Purple Mountain Laboratories（紫金山实验室） ； Fudan University（复旦大学） 纠错

专题命中 安全训练 ：safety(abstract)；jailbreak(abstract)

Learning to Route LLMs with Confidence Tokens

Yu-Neng Chuang, Prathusha Kameswara Sarma, Parikshit Gopalan, John Boccio, Sara Bolouki, Xia Hu, Helen Zhou

机构 * Rice University（里士大学） ； Apple, Inc.（苹果公司） 纠错

专题命中 安全训练 ：trustworthy(abstract)；分类 cs.CL、cs.AI、cs.LG

ThinkGuard: Deliberative Slow Thinking Leads to Cautious Guardrails

Xiaofei Wen, Wenxuan Zhou, Wenjie Jacky Mo, Muhao Chen

机构 * University of California, Davis（加州大学戴维斯分校） ； University of Southern California（南加州大学） 纠错

专题命中 安全训练 ：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ACL 2025

PPO-BR: Dual-Signal Entropy-Reward Adaptation for Trust Region Policy Optimization

Ben Rahman

专题命中 安全训练 ：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments This manuscript builds upon an earlier version posted to TechRxiv. This arXiv version includes an updated comparison with GRPO (Group Relative Policy Optimization)

RTP-LX: Can LLMs Evaluate Toxicity in Multilingual Scenarios?

Adrian de Wynter, Ishaan Watts, Tua Wongsangaroonsri, Minghui Zhang, Noura Farra, Nektar Ege Altıntoprak, Lena Baur, Samantha Claudet, Pavel Gajdusek, Can Gören, Qilong Gu, Anna Kaminska, Tomasz Kaminski, Ruby Kuo, Akiko Kyuba, Jongho Lee, Kartik Mathur, Petter Merok, Ivana Milovanović, Nani Paananen, Vesa-Matti Paananen, Anna Pavlenko, Bruno Pereira Vidal, Luciano Strika, Yueh Tsao, Davide Turcato, Oleksandr Vakhno, Judit Velcsov, Anna Vickers, Stéphanie Visser, Herdyan Widarmanto, Andrey Zaikin, Si-Qing Chen

机构 * Microsoft Research India（微软印度研究院） 纠错

专题命中 安全训练 ：safety(abstract)；分类 cs.CL、cs.CY、cs.LG

Comments AAAI 2025--camera ready + extended abstract

ClimaEmpact: Domain-Aligned Small Language Models and Datasets for Extreme Weather Analytics

Deeksha Varshney, Keane Ong, Rui Mao, Erik Cambria, Gianmarco Mengaldo

机构 * College of Design and Engineering, National University of Singapore（新加坡国立大学设计与工程学院） ； Department of Mathematics, National University of Singapore（新加坡国立大学数学系） ； College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院） 纠错

专题命中 安全训练 ：alignment(abstract)；分类 cs.CL、cs.AI、cs.LG

Knowledge Acquisition on Mass-shooting Events via LLMs for AI-Driven Justice

Benign John Ihugba, Afsana Nasrin, Ling Wu, Lin Li, Lijun Qian, Xishuang Dong

专题命中 安全训练 ：safety(abstract)；分类 cs.CL、cs.AI、cs.CY

A Dual-Space Framework for General Knowledge Distillation of Large Language Models

Xue Zhang, Songming Zhang, Yunlong Liang, Fandong Meng, Yufeng Chen, Jinan Xu, Jie Zhou

专题命中 安全训练 ：alignment(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 19 pages, 9 figures, 11 tables, under review. Code is available at: https://github.com/songmzhang/DSKDv2. arXiv admin note: text overlap with arXiv:2406.17328

Do LLMs Consider Security? An Empirical Study on Responses to Programming Questions

Amirali Sajadi, Binh Le, Anh Nguyen, Kostadin Damevski, Preetha Chatterjee

专题命中 安全训练 ：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to EMSE

Automatic Curriculum Expert Iteration for Reliable LLM Reasoning

Zirui Zhao, Hanze Dong, Amrita Saha, Caiming Xiong, Doyen Sahoo

专题命中 安全训练 ：alignment(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 20 pages

Towards Conversational AI for Disease Management

Anil Palepu, Valentin Liévin, Wei-Hung Weng, Khaled Saab, David Stutz, Yong Cheng, Kavita Kulkarni, S. Sara Mahdavi, Joëlle Barral, Dale R. Webster, Katherine Chou, Avinatan Hassidim, Yossi Matias, James Manyika, Ryutaro Tanno, Vivek Natarajan, Adam Rodman, Tao Tu, Alan Karthikesalingam, Mike Schaekermann

专题命中 安全训练 ：alignment(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 62 pages, 7 figures in main text, 36 figures in appendix

Iterative Nash Policy Optimization: Aligning LLMs with General Preferences via No-Regret Learning

Yuheng Zhang, Dian Yu, Baolin Peng, Linfeng Song, Ye Tian, Mingyue Huo, Nan Jiang, Haitao Mi, Dong Yu

专题命中 安全训练 ：RLHF(abstract)；分类 cs.CL、cs.AI、cs.LG

TalkWithMachines: Enhancing Human-Robot Interaction for Interpretable Industrial Robotics Through Large/Vision Language Models

Ammar N. Abbas, Csaba Beleznai

专题命中 安全训练 ：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments This paper has been accepted for publication in the proceedings of the 2024 Eighth IEEE International Conference on Robotic Computing (IRC)

Large Language Model Unlearning via Embedding-Corrupted Prompts

Chris Yuhao Liu, Yaxuan Wang, Jeffrey Flanigan, Yang Liu

专题命中 安全训练 ：alignment(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments NeurIPS 2024 Poster

Conditional Language Policy: A General Framework for Steerable Multi-Objective Finetuning

Kaiwen Wang, Rahul Kidambi, Ryan Sullivan, Alekh Agarwal, Christoph Dann, Andrea Michi, Marco Gelmi, Yunxuan Li, Raghav Gupta, Avinava Dubey, Alexandre Ramé, Johan Ferret, Geoffrey Cideron, Le Hou, Hongkun Yu, Amr Ahmed, Aranyak Mehta, Léonard Hussenot, Olivier Bachem, Edouard Leurent

专题命中 安全训练 ：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 40 pages. Findings of EMNLP 2024

Harm Amplification in Text-to-Image Models

Susan Hao, Renee Shelby, Yuchi Liu, Hansa Srinivasan, Mukul Bhutani, Burcu Karagol Ayan, Ryan Poplin, Shivani Poddar, Sarah Laszlo

专题命中 安全训练 ：safety(abstract)；分类 cs.AI、cs.CY、cs.LG

Building a Domain-specific Guardrail Model in Production

Mohammad Niknazar, Paul V Haley, Latha Ramanan, Sang T. Truong, Yedendra Shrinivasan, Ayan Kumar Bhowmick, Prasenjit Dey, Ashish Jagmohan, Hema Maheshwari, Shom Ponoth, Robert Smith, Aditya Vempaty, Nick Haber, Sanmi Koyejo, Sharad Sundararajan

专题命中 安全训练 ：safety(abstract)；分类 cs.AI、cs.CY、cs.LG

Regulating AI Adaptation: An Analysis of AI Medical Device Updates

Kevin Wu, Eric Wu, Kit Rodolfa, Daniel E. Ho, James Zou

专题命中 安全训练 ：safety(abstract)；分类 cs.AI、cs.CY、cs.LG

Journal ref CHIL 2024

Foundation Model Transparency Reports

Rishi Bommasani, Kevin Klyman, Shayne Longpre, Betty Xiong, Sayash Kapoor, Nestor Maslej, Arvind Narayanan, Percy Liang

专题命中 安全训练 ：trustworthy(abstract)；分类 cs.AI、cs.CY、cs.LG

Journal ref Published in AIES 2024

MOSSBench: Is Your Multimodal Language Model Oversensitive to Safe Queries?

Xirui Li, Hengguang Zhou, Ruochen Wang, Tianyi Zhou, Minhao Cheng, Cho-Jui Hsieh

专题命中 安全训练 ：safety(abstract)；分类 cs.CL、cs.AI、cs.LG

Aligning Large Language Models by On-Policy Self-Judgment

Sangkyu Lee, Sungdong Kim, Ashkan Yousefpour, Minjoon Seo, Kang Min Yoo, Youngjae Yu

专题命中 安全训练 ：alignment(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published as a main conference paper at ACL 2024

Are aligned neural networks adversarially aligned?

Nicholas Carlini, Milad Nasr, Christopher A. Choquette-Choo, Matthew Jagielski, Irena Gao, Anas Awadalla, Pang Wei Koh, Daphne Ippolito, Katherine Lee, Florian Tramer, Ludwig Schmidt

专题命中 安全训练 ：alignment(abstract)；分类 cs.CL、cs.AI、cs.LG

Stream of Search (SoS): Learning to Search in Language

Kanishk Gandhi, Denise Lee, Gabriel Grand, Muxin Liu, Winson Cheng, Archit Sharma, Noah D. Goodman

专题命中 安全训练 ：alignment(abstract)；分类 cs.CL、cs.AI、cs.LG

DistillSpec: Improving Speculative Decoding via Knowledge Distillation

Yongchao Zhou, Kaifeng Lyu, Ankit Singh Rawat, Aditya Krishna Menon, Afshin Rostamizadeh, Sanjiv Kumar, Jean-François Kagy, Rishabh Agarwal

专题命中 安全训练 ：alignment(abstract)；分类 cs.CL、cs.AI、cs.LG

On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes

Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos, Matthieu Geist, Olivier Bachem

专题命中 安全训练 ：RLHF(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at ICLR 2024. First two authors contributed equally

Data-Driven Traffic Reconstruction and Kernel Methods for Identifying Stop-and-Go Congestion

Edgar Ramirez Sanchez, Shreyaa Raghavan, Cathy Wu

专题命中 安全训练 ：safety(abstract)；分类 cs.AI、cs.CY、cs.LG

Comments Presented at NeurIPS 2023 workshops: Tackling Climate Change with Machine Learning & Computational Sustainability

Effect of roundabout design on the behavior of road users: A case study of roundabouts with application of Unsupervised Machine Learning

Tasnim M. Dwekat, Ayda A. Almsre, Huthaifa I. Ashqar

专题命中 安全训练 ：safety(abstract)；分类 cs.AI、cs.CY、cs.LG