守护者与违规者：大语言模型有害内容生成与安全缓解研究综述
Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLM
AI总结：
综述大语言模型在内容创作中带来的风险与挑战，系统回顾相关研究，提出危害与防御统一分类法，分析越狱策略，评估缓解措施，指出评估局限，明确未来研究方向以推动语言技术稳健且符合伦理发展。
摘要：
大语言模型（LLMs）在数字平台上彻底改变了内容创作，在自然语言生成和理解方面提供了前所未有的能力。与此同时，它们也因无意中产生有毒、冒犯性或有偏见的内容而带来风险。本文系统回顾了近期关于无意毒性、对抗性越狱攻击和综合缓解策略的研究。探讨了大语言模型作为危害制造者和安全推动者的双重角色，提出了相关危害与防御的统一分类法，分析了新兴的多模态和大语言模型辅助越狱策略，并评估了缓解措施，包括基于人类反馈的强化学习（RLHF）、提示工程和安全对齐。综述突出了大语言模型安全的演变态势，指出了当前评估方法的局限性，并概述了未来研究方向。
英文摘要：
Large Language Models (LLMs) have revolutionized content creation across digital platforms, offering unprecedented capabilities in natural language generation and understanding. Meanwhile, they pose risks by inadvertently producing toxic, offensive, or biased content. This dual role of LLMs, both as powerful tools for text generation and as potential sources of harmful language, presents a pressing sociotechnical challenge. In this survey, we systematically review recent studies encompassing unintentional toxicity, adversarial jailbreak attacks, and comprehensive mitigation strategies. We explore LLMs' dual role as both generators of harm and enablers of safety through detection, classification, content moderation, and prevention. We propose a unified taxonomy of LLM-related harms and defenses, analyze emerging multimodal and LLM-assisted jailbreak strategies, and assess mitigation efforts, including reinforcement learning with human feedback (RLHF), prompt engineering, and safety alignment. Our review highlights the evolving landscape of LLM safety and identifies limitations in current evaluation methodologies. Ultimately, our review outlines future research directions to guide the development of robust and ethically aligned language technologies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
