arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

用于性别包容性改写和反叙事生成的激活引导的LoRA

LoRA for Gender-Inclusive Rewriting and Activation Steering for Counter-Narrative Generation

Akhil Rajeev P, Manoj Balaji J

arXiv 2607.23083首次发表:更新:

发表机构

IHLC(未提及具体中文译名)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

本文针对LT-EDI 2026共享任务,用参数高效的LoRA微调实现性别包容性改写,还用推理时表示工程方法结合约束提示生成反叙事,通过PCA得出主引导方向注入中间表示,取得一定分数,同时分析了引导行为的关键失败模式。

AI 中文摘要

性别包容性语言生成旨在将有偏见的文本转换为包容性的替代文本,同时保留语义和上下文连贯性。本文介绍了用于LT-EDI 2026共享任务的IHLC系统,解决性别包容性改写和反叙事生成问题。对于性别包容性改写,采用参数高效的低秩适应(LoRA)微调,官方得分80.00%。主要贡献是一种计算高效的推理时表示工程方法用于反叙事生成。通过主成分分析(PCA)从对比隐藏状态激活中得出主引导方向,并在推理时注入Gemma-3-4B-it的中间表示,结合约束提示,官方得分78.12%。还进行了引导行为的人工分析,识别出关键失败模式。研究结果凸显了激活引导作为参数更新的轻量级替代方案在可控和社会对齐语言生成中的实际潜力和当前局限性。

英文摘要

Gender-inclusive language generation seeks to transform biased text into inclusive alternatives while preserving semantic meaning and contextual coherence. This paper presents the IHLC system for the LT-EDI 2026 Shared Task, addressing both gender-inclusive rewriting and counter-narrative generation. For gender-inclusive rewriting, we employ parameter-efficient Low-Rank Adaptation (LoRA) fine-tuning, achieving an official score of 80.00%. Our primary contribution is a compute-efficient inference-time representation engineering approach for counter-narrative generation. We derive a principal steering direction from contrastive hidden-state activations using principal component analysis (PCA) and inject it into the intermediate representations of Gemma-3-4B-it during inference, enabling behavioral steering toward inclusive responses without modifying model weights. Combined with constrained prompting, this approach produces polite and contextually appropriate counter-narratives, achieving an official score of 78.12%. We further present a manual analysis of steering behavior, identifying key failure modes including semantic drift, residual bias leakage, layer sensitivity, over-steering, and text degeneration. Our findings highlight both the practical potential and current limitations of activation steering as a lightweight alternative to parameter updates for controllable and socially aligned language generation.

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑