Johannes Welbl, Amelia Glaese, Jonathan Uesato, Sumanth Dathathri, John Mellor, Lisa Anne Hendricks, Kirsty Anderson, Pushmeet Kohli, Ben Coppin, Po-Sen Huang
备注：
23 pages, 6 figures, published in Findings of EMNLP 2021
Large language models (LM) generate remarkably fluent text and can be efficiently adapted across NLP tasks. Measuring and guaranteeing the quality of generated text in terms of safety is imperative for deploying LMs in the real world; to this end, prior work often relies on automatic evaluation of LM toxicity. We critically discuss this approach, evaluate several toxicity mitigation strategies with respect to both automatic and human evaluation, and analyze consequences of toxicity mitigation in terms of model bias and LM quality. We demonstrate that while basic intervention strategies can effectively optimize previously established automatic metrics on the RealToxicityPrompts dataset, this comes at the cost of reduced LM coverage for both texts about, and dialects of, marginalized groups. Additionally, we find that human raters often disagree with high automatic toxicity scores after strong toxicity reduction interventions -- highlighting further the nuances involved in careful evaluation of LM toxicity.
Unified Detoxifying and Debiasing in Language Generation via Inference-time Adaptive Optimization
Tsinghua University（清华大学）；Beijing Academy of（北京学院（文本不完整，仅提取现有内容））；Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；National Research Center for Information Science Technology（国家信息科学技术研究中心）；Microsoft Research Asia（微软亚洲研究院）；International Innovation Center of Tsinghua University（清华大学国际创新中心）
Detoxifying Large Language Models via Autoregressive Reward Guided Representation Editing
SKLCCSE, Beihang University（北航智能计算与复杂系统研究院）；National University of Singapore（新加坡国立大学）；Zhongguancun Laboratory, Beijing（中关村实验室）；Institute of Dataspace, Hefei（合肥数据空间研究院）；Nanyang Technological University（南洋理工大学）
Detoxifying Text with MaRCo: Controllable Revision with Experts and Anti-Experts
Paul G. Allen School of Computer Science & Engineering, University of Washington（华盛顿大学保罗·G·艾伦计算机科学与工程学院）；Allen Institute for AI（艾伦人工智能研究所）；Language Technologies Institute, Carnegie Mellon University（卡内基梅隆大学语言技术研究所）
MaRCo 结合可控生成与专家乘积，利用无毒和有毒语言模型似然识别并替换候选词，在微妙毒性去毒任务上自动指标优于基线且人类评估偏好达 2.1 倍。
Foundational Challenges in Assuring Alignment and Safety of Large Language Models
University of Cambridge（剑桥大学）；University of Oxford（牛津大学）；Harvard University（哈佛大学）；University of California, Berkeley（加利福尼亚大学伯克利分校）；University of Sussex（萨塞克斯大学）；Stanford University（斯坦福大学）；Modulo Research（莫杜洛研究所）；New York University（纽约大学）；ETH Zurich（苏黎世联邦理工学院）；UNC Chapel Hill（北卡罗来纳大学教堂山分校）；Massachusetts Institute of Technology（麻省理工学院）；Peking University（北京大学）；LMU Munich（慕尼黑大学）；University of Virginia（弗吉尼亚大学）；Universitat Politècnica de València（瓦伦西亚理工大学）；Center for the Governance of AI（人工智能治理中心）；Newcastle University（纽卡斯尔大学）；Mila - Quebec AI Institute, Université de Montréal（蒙特利尔大学米拉-魁北克人工智能研究所）
State Key Laboratory of AI Safety（人工智能安全国家重点实验室）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Challenges and Applications of Large Language Models
University College London（伦敦大学学院）；UK Health Security Agency（英国卫生安全局）；EleutherAI；University of Cambridge（剑桥大学）；Stability AI；Meta AI Research（Meta人工智能研究院）；InstaDeep
Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Research
School of Computing, The University of Georgia（佐治亚大学计算机学院）；Department of Mathematical and Statistical Sciences, University of Alberta（阿尔伯塔大学数学与统计科学系）；University of California, Los Angeles（加州大学洛杉矶分校）；Department of Radiology, Massachusetts General Hospital and Harvard Medical School（麻省总医院放射科和哈佛医学院）
Large Language Models for Sentiment Analysis to Detect Social Challenges: A Use Case with South African Languages
University of Johannesburg, South Africa（约翰内斯堡大学）；IU International University of Applied Sciences（国际应用科学大学）；Karlsruhe University of Applied Sciences（卡尔斯鲁厄应用科学大学）
ParsiNLU: A Suite of Language Understanding Challenges for Persian
Allen Institute for AI（艾伦人工智能研究所）；Google（谷歌）；George Washington U.（乔治华盛顿大学）；UC Irvine（加州大学欧文分校）；U. of Pittsburgh（匹兹堡大学）；TaskRabbit（TaskRabbit公司）；Arizona State U.（亚利桑那州立大学）；UC Santa Cruz（加州大学圣克鲁兹分校）；U. of Southern California（南加州大学）；IMRSV Data Labs（IMRSV数据实验室）；EPFL（洛桑联邦理工学院）；U. of Illinois - Chicago（伊利诺伊大学芝加哥分校）；U. of Maryland Baltimore County（马里兰大学巴尔的摩县分校）；Rutgers U.（罗格斯大学）；U. of Pennsylvania（宾夕法尼亚大学）；Expedia Inc.（Expedia公司）；U. of Houston（休斯顿大学）；Indiana U. - Bloomington（印第安纳大学伯明顿分校）；Microsoft（微软）
Exploring Large Language Models for Financial Applications: Techniques, Performance, and Challenges with FinMA
Lyman Briggs College, Michigan State University（密歇根州立大学Lyman Briggs学院）；Department of Finance, Michigan State University（密歇根州立大学金融系）；African Institute for Mathematical Sciences, Rwanda（刚果(金)数学科学研究所）
The State of Large Language Models for African Languages: Progress and Challenges
University of Gondar（贡德大学）；AIMS Research and Innovation Centre（AIMS研究与创新中心）；Bahir Dar University（巴赫达尔大学）；Bayero University Kano（卡诺巴耶罗大学）；Addis Ababa University（亚的斯亚贝巴大学）；Imperial College London（伦敦帝国学院）；University of Hamburg（汉堡大学）
Inference Optimizations for Large Language Models: Effects, Challenges, and Practical Considerations
Business Faculty Ansbach University of Applied Sciences（安斯巴赫应用技术大学商学院）；Faculty of Business University of Applied Science Ansbach（安斯巴赫应用技术大学商学院）；Faculty of Business University of Applied Science Heilbronn（海尔布隆应用技术大学商学院）
