Towards Red Teaming in Multimodal and Multilingual Translation
FAIR, Meta(Meta AI 基础人工智能研究中心)
Christophe Ropers, David Dale, Prangthip Hansanti, Gabriel Mejia Gonzalez, Ivan Evtimov, Corinne Wong, Christophe Touret, Kristina Pereyra, Seohyun Sonia Kim, C… 展开作者
Christophe Ropers, David Dale, Prangthip Hansanti, Gabriel Mejia Gonzalez, Ivan Evtimov, Corinne Wong, Christophe Touret, Kristina Pereyra, Seohyun Sonia Kim, Cristian Canton Ferrer, Pierre Andrews, Marta R. Costa-jussà
备注:
arXiv admin note: substantial text overlap with arXiv:2312.05187
Assessing performance in Natural Language Processing is becoming increasingly complex. One particular challenge is the potential for evaluation datasets to overlap with training data, either directly or indirectly, which can lead to skewed results and overestimation of model performance. As a consequence, human evaluation is gaining increasing interest as a means to assess the performance and reliability of models. One such method is the red teaming approach, which aims to generate edge cases where a model will produce critical errors. While this methodology is becoming standard practice for generative AI, its application to the realm of conditional AI remains largely unexplored. This paper presents the first study on human-based red teaming for Machine Translation (MT), marking a significant step towards understanding and improving the performance of translation models. We delve into both human-based red teaming and a study on automation, reporting lessons learned and providing recommendations for both translation models and red teaming drills. This pioneering work opens up new avenues for research and development in the field of MT.
Red Teaming LLMs as Socio-Technical Practice: From Exploration and Data Creation to Evaluation
Responsible Tech Research IBM Research Yorktown Heights New York USA(负责技术研究 IBM 研究 Yorktown Heights 新 York 美国);IBM Research(IBM 研究);The Pennsylvania State University(宾夕法尼亚州立大学);University of Notre Dame(诺特达美大学)
RedTopic: Toward Topic-Diverse Red Teaming of Large Language Models
Fudan University(复旦大学);City University of Hong Kong(香港城市大学);Deakin University(德克萨斯大学);Hon Hai Research Institute(鸿海研究室);Hong Kong University of Science and Technology(香港理工大学)
Against The Achilles' Heel: A Survey on Red Teaming for Generative Models
LibrAI;Tsinghua University(清华大学);Harbin Institute of Technology(哈尔滨工业大学);Monash University(莫纳什大学);MBZUAI(穆罕默德·本·扎耶德人工智能大学);The University of Melbourne(墨尔本大学)
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Center for AI Safety(人工智能安全中心);Carnegie Mellon University(卡内基梅隆大学);UC Berkeley(加州大学伯克利分校);Microsoft(微软)
Holistic Automated Red Teaming for Large Language Models through Top-Down Test Case Generation and Multi-turn Interaction
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
Summon a Demon and Bind it: A Grounded Theory of LLM Red Teaming
Paul G. Allen School of Computer Science & Engineering(保罗·G·艾伦计算机科学与工程学院);IT University of Copenhagen(哥本哈根信息技术大学);Berkeley Center for Human-Compatible AI, UC Berkeley(加州大学伯克利分校伯克利人类兼容人工智能中心);NVIDIA Corporation(英伟达公司)
Stop Fixating on Prompts: Reasoning Hijacking and Constraint Tightening for Red-Teaming LLM Agents
School of Software, Henan University(河南大学软件学院);Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);University of Chinese Academy of Sciences(中国科学院大学);Peking University(北京大学)
Learning diverse attacks on large language models for robust red-teaming and safety tuning
KAIST(韩国科学技术院);Mila – Québec AI Institute(米拉-魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);McGill University(麦吉尔大学);National University of Singapore(新加坡国立大学);University of Edinburgh(爱丁堡大学);CIFAR(加拿大高级研究所)
University of Toronto(多伦多大学);Vector Institute for AI(人工智能向量研究所);Johns Hopkins Medical Institutions(约翰霍普金斯医学机构);University Health Network(医疗网络);Algoma University(阿尔戈马大学);University of Iowa Hospitals & Clinics(爱荷华大学医院与诊所);University of Edinburgh(爱丁堡大学)
Resource Consumption Red-Teaming for Large Vision-Language Models
China Mobile Research Institute(中国移动研究院);Beijing University of Posts and Telecommunications(北京邮电大学);Nanyang Technological University(南洋理工大学);University of Science and Technology of China(中国科学技术大学);North China Electric Power University(华北电力大学)
Prompt Optimization and Evaluation for LLM Automated Red Teaming
Fuel iX Applied Research(Fuel iX应用研究);North Carolina State University(北卡罗来纳州立大学);University of Minnesota(明尼苏达大学);TELUS Digital(TELUS数字);Learn Prompting
RedAgent: Red Teaming Large Language Models with Context-aware Autonomous Language Agent
The State Key Laboratory of Blockchain and Data Security, Zhejiang University(浙江大学区块链与数据安全国家重点实验室);School of Cyber Science and Technology, Zhejiang University(浙江大学网络科学与技术学院);Palo Alto Networks(派拓网络);University of North Texas(北得克萨斯大学)