In the pursuit of developing Large Language Models (LLMs) that adhere to societal standards, it is imperative to detect the toxicity in the generated text. The majority of existing toxicity metrics rely on encoder models trained on specific toxicity datasets, which are susceptible to out-of-distribution (OOD) problems and depend on the dataset's definition of toxicity. In this paper, we introduce a robust metric grounded on LLMs to flexibly measure toxicity according to the given definition. We first analyze the toxicity factors, followed by an examination of the intrinsic toxic attributes of LLMs to ascertain their suitability as evaluators. Finally, we evaluate the performance of our metric with detailed analysis. Our empirical results demonstrate outstanding performance in measuring toxicity within verified factors, improving on conventional metrics by 12 points in the F1 score. Our findings also indicate that upstream toxicity significantly influences downstream metrics, suggesting that LLMs are unsuitable for toxicity evaluations within unverified factors.
The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs
Beijing University of Posts and Telecommunications（北京邮电大学）；Jinan University（暨南大学）；Beihang University（北京航空航天大学）；China Academy of Information and Communications Technology（中国信息通信研究院）
Lost in Pronunciation: Detecting Chinese Offensive Language Disguised by Phonetic Cloaking Replacement
School of Computer Science, The University of Sydney（悉尼大学计算机科学学院）；Business School, The Hong Kong University of Science and Technology（香港科技大学商学院）；Australian AI Institute, University of Technology Sydney（悉尼科技大学澳大利亚人工智能研究所）；School of Advanced Technology, Xi’an Jiaotong-Liverpool University（西安交通大学利物浦大学先进技术学院）
HypothesisMed: Inference-Time Answer Fusion and Structured Hypothesis-Space Reporting for Biomedical Question Answering
Department of Computer Science Rensselaer Polytechnic Institute（计算机科学系雷士打理工学院）；Department of Biomedical Engineering Rensselaer Polytechnic Institute（生物医学工程系雷士打理工学院）
CASTLE: A Comprehensive Benchmark for Evaluating Student-Tailored Personalized Safety in Large Language Models
East China Normal University（华东师范大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Zhejiang University（浙江大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；The Education University of Hong Kong（香港教育大学）
American Stories: A Large-Scale Structured Text Dataset of Historical U.S. Newspapers
Harvard University（哈佛大学）；Massachusetts Institute of Technology（麻省理工学院）；Princeton University（普林斯顿大学）；National Bureau of Economic Research（美国国家经济研究局）；New York University（纽约大学）；Northwestern University（西北大学）
Bleeding Pathways: Vanishing Discriminability in LLM Hidden States Fuels Jailbreak Attacks
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）；Ant Group（蚂蚁集团）
