CLaS-Bench: 一种跨语言对齐与引导基准
CLaS-Bench: A Cross-Lingual Alignment and Steering Benchmark
AI总结：
CLaS-Bench是首个多语言引导基准，通过评估32种语言中的语言强制行为，验证了残差基于DiffMean方法在跨语言引导中的有效性。
摘要：
理解并控制大型语言模型（LLMs）的行为在多语言自然语言处理中变得越来越重要。除了提示或微调之外，即在推理过程中操纵内部表示，已成为一种更高效且可解释的技术，用于将模型适应到目标语言。然而，目前还没有专门的基准或评估协议来量化引导技术的有效性。我们引入了CLaS-Bench，一个轻量级的并行问题基准，用于评估LLMs在32种语言中的语言强制行为，使系统性评估多语言引导方法成为可能。我们评估了广泛的一系列引导技术，包括残差流DiffMean干预、探测器导出的方向、语言特定的神经元、PCA/LDA向量、稀疏自编码器和提示基线。引导性能是沿着两个轴测量的：语言控制和语义相关性，结合成一个单一的调和均值引导分数。我们发现，在不同语言中，简单的残差基于DiffMean方法始终优于所有其他方法。此外，层间分析揭示了语言特定的结构主要出现在后期层中，引导方向根据语言家族聚类。CLaS-Bench是第一个标准化的多语言引导基准，使语言表示的严格科学分析和引导作为低成本适应替代方案的实用评估成为可能。
英文摘要：
Understanding and controlling the behavior of large language models (LLMs) is an increasingly important topic in multilingual NLP. Beyond prompting or fine-tuning, , i.e.,~manipulating internal representations during inference, has emerged as a more efficient and interpretable technique for adapting models to a target language. Yet, no dedicated benchmarks or evaluation protocols exist to quantify the effectiveness of steering techniques. We introduce CLaS-Bench, a lightweight parallel-question benchmark for evaluating language-forcing behavior in LLMs across 32 languages, enabling systematic evaluation of multilingual steering methods. We evaluate a broad array of steering techniques, including residual-stream DiffMean interventions, probe-derived directions, language-specific neurons, PCA/LDA vectors, Sparse Autoencoders, and prompting baselines. Steering performance is measured along two axes: language control and semantic relevance, combined into a single harmonic-mean steering score. We find that across languages simple residual-based DiffMean method consistently outperforms all other methods. Moreover, a layer-wise analysis reveals that language-specific structure emerges predominantly in later layers and steering directions cluster based on language family. CLaS-Bench is the first standardized benchmark for multilingual steering, enabling both rigorous scientific analysis of language representations and practical evaluation of steering as a low-cost adaptation alternative.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
