预测引导向量与适配器权重用于少样本作者风格迁移
Predicting Steering Vectors and Adapter Weights for Few-Shot Author-Style Transfer
浏览论文内容
中文总结 AI 辅助
针对少样本作者风格迁移，提出对比激活引导、预测引导向量网络和预测LoRA适配器的超网络三种方法，发现超网络在风格模仿与输出质量间取得最佳平衡，且引导向量可存在多个正交方向。
中文摘要 AI 辅助
将大型语言模型适应于个体作者的写作风格，仅凭少量示例即具有挑战性，而科学写作更加剧了这一难度：正式规范使得表面变化极少，且作者撰写的是各自领域的内容，因此提取的“风格”极易与内容纠缠。我们研究了基于每位作者少量示例摘要的风格条件化摘要生成，并提出了三种方法：（1）对比激活引导，（2）预测引导向量的网络，以及（3）预测LoRA适配器的超网络。我们发现风格模仿与输出质量之间存在一致的权衡：微调能获取大部分可用的风格信号，但会损失流畅性，而超网络在已见和未见作者上均实现了最佳权衡。我们的引导在作者级别运作，将作者的摘要与针对相同内容的风格中性生成进行对比。这固定了主题，无需预定义风格清单，并优于基于清单的引导。此外，我们的分析表明，手动提取与预测的引导向量近乎正交但得分相当，这表明此处的风格条件化可允许多达两个不相关的方向，而非要求某一特定轴。
英文摘要
Adapting large language models to an individual author's style from a few examples is challenging, and scientific writing sharpens the difficulty: formal conventions leave little surface variation, and authors write about their own topics, so extracted ``style'' easily entangles with content. We study style-conditioned abstract generation from a few example abstracts per author and propose three methods: (1) contrastive activation steering, (2) a network that predicts steering vectors, and (3) a hypernetwork that predicts LoRA adapters. We find a consistent trade-off between style imitation and output quality: fine-tuning buys most of the available style signal but forfeits fluency, while the hypernetwork achieves the best trade-off on both seen and unseen authors. Our steering operates at author level, contrasting an author's abstracts against style-neutral generations for the same content. This holds topic fixed, removes the need for a predefined style inventory, and outperforms inventory-based steering. % [EDIT 1a] softened "no single optimal axis" claim Moreover, our analyses demonstrate that manually extracted and predicted steering vectors are near-orthogonal yet score comparably, indicating that style conditioning here can admit at least two unrelated directions rather than requiring one particular axis.
发表机构
- Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
机构由 AI 辅助整理，请以论文原文为准。
