通过内容完整性保护实现身份鲁棒的语言模型生成
Identity-Robust Language Model Generation via Content Integrity Preservation
AI总结:
本文提出了一种轻量级、无需训练的框架,通过保留语义关键属性并中和非关键身份信息,实现身份鲁棒的语言模型生成,有效减少身份依赖性偏见。
摘要:
大型语言模型(LLM)的输出往往在用户的社会人口属性上存在差异,导致事实准确性、实用性和安全性上的不一致,即使在客观问题中,当人口信息无关时也是如此。与之前关于刻板印象或代表性偏见的研究不同,本文研究了身份依赖性核心响应质量的退化。我们实证表明,这种退化源于偏见的生成行为,尽管事实知识在不同身份间被稳健地编码。受这种不匹配的启发,我们提出了一种轻量级、无需训练的框架,用于身份鲁棒生成,该框架可选择性地中和非关键身份信息,同时保留语义上重要的属性,从而保持输出内容完整性。在四个基准和18种社会人口身份上的实验显示,与普通提示相比,身份依赖性偏见平均减少了77%,相对于基于提示的防御方法,减少了45%。我们的工作填补了缓解提示中用户身份线索对核心生成质量影响的关键空白。
英文摘要:
Large Language Model (LLM) outputs often vary across user sociodemographic attributes, leading to disparities in factual accuracy, utility, and safety, even for objective questions where demographic information is irrelevant. Unlike prior work on stereotypical or representational bias, this paper studies identity-dependent degradation of core response quality. We show empirically that such degradation arises from biased generation behavior, despite factual knowledge being robustly encoded across identities. Motivated by this mismatch, we propose a lightweight, training-free framework for identity-robust generation that selectively neutralizes non-critical identity information while preserving semantically essential attributes, thus maintaining output content integrity. Experiments across four benchmarks and 18 sociodemographic identities demonstrate an average 77% reduction in identity-dependent bias compared to vanilla prompting and a 45% reduction relative to prompt-based defenses. Our work addresses a critical gap in mitigating the impact of user identity cues in prompts on core generation quality.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。