引导大语言模型实现文化本地化生成
Steering LLMs for Culturally Localized Generation
AI总结：
本文通过机制可解释性揭示LLM中的文化表征，提出文化嵌入（CuE）方法，提升文化忠实度并引导生成长尾文化概念，为文化本地化提供可控方法。
摘要：
LLMs在全球范围内部署，但生成的响应偏向于训练数据丰富的文化。现有文化本地化方法如提示或后训练对齐均为黑箱，难以控制且无法判断失败是因知识缺失还是提示不当。本文利用机制可解释性揭示并操控LLM中的文化表征。通过稀疏自编码器，我们识别出可解释的特征，编码文化相关信息，并将其聚合为文化嵌入（CuE）。我们使用CuE分析隐含文化偏见并在未指定提示下生成文化概念。在多个模型上，CuE引导提升文化忠实度，并激发显著更罕见的长尾文化概念。值得注意的是，CuE引导与黑箱本地化方法互补，当应用于提示增强输入时效果更佳。这表明模型可能受益于更好的提示策略，而非必然缺乏长尾知识表示，尽管这因文化而异。我们的结果为LLM中的文化表征提供了诊断洞察，并提供了一种可控方法以引导至期望的文化。
英文摘要：
LLMs are deployed globally, yet produce responses biased towards cultures with abundant training data. Existing cultural localization approaches such as prompting or post-training alignment are black-box, hard to control, and do not reveal whether failures reflect missing knowledge or poor elicitation. In this paper, we address these gaps using mechanistic interpretability to uncover and manipulate cultural representations in LLMs. Leveraging sparse autoencoders, we identify interpretable features that encode culturally salient information and aggregate them into Cultural Embeddings (CuE). We use CuE both to analyze implicit cultural biases under underspecified prompts and to construct white-box steering interventions. Across multiple models, we show that CuE-based steering increases cultural faithfulness and elicits significantly rarer, long-tail cultural concepts than prompting alone. Notably, CuE-based steering is complementary to black-box localization methods, offering gains when applied on top of prompt-augmented inputs. This also suggests that models do benefit from better elicitation strategies, and don't necessarily lack long-tail knowledge representation, though this varies across cultures. Our results provide both diagnostic insight into cultural representations in LLMs and a controllable method to steer towards desired cultures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
