arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2507.04889cs.LG
收藏

在模拟数据上微调优于提示工程以实现智能体语气风格

Fine-tuning on simulated data outperforms prompting for agent tone of voice

Restack

Ingo Marquardt, Philippe Brule

AI总结:

该研究证明在模拟数据上微调小型语言模型比系统提示更有效,能以极少数据高效塑造对话语气风格且不损失内容质量。

摘要:

在面向客户的语音应用中部署语言模型(LMs)需要具备对话流畅性并遵守特定的风格指南。由于指令遵循限制和上下文偏差等问题,使用复杂的系统提示可靠地实现这一目标可能具有挑战性。本研究考察了微调与系统提示在使语言模型对齐特定行为目标方面的有效性,该目标为:以适合语音交互的自然对话语气进行回应。我们使用低秩适应(LoRA)在源自维基百科的合成生成数据集上微调了一个小型开放权重模型(Llama3.2-1B-Instruct)。此外,我们还微调了两个闭源模型(gpt-4o-mini、gpt-4.1-mini)。我们的结果表明,微调优于系统提示,即使仅在100个数据样本上训练,也能实现高比例的对话式回应。语义相似性分析证实,微调并未降低内容质量。有趣的是,使用8位整数量化的微调比使用bfloat16精度更快地收敛到目标风格,这可能是由于隐式正则化效应。我们得出结论,在模拟数据上微调小型开放权重语言模型是一种高效且数据高效的方法,可用于灌输特定的风格行为,为需要细腻回应风格的实际应用提供了一种优于复杂系统提示的替代方案。

英文摘要:

Deploying language models (LMs) in customer-facing speech applications requires conversational fluency and adherence to specific stylistic guidelines. This can be challenging to achieve reliably using complex system prompts due to issues like instruction following limitations and in-context bias. This study investigates the effectiveness of fine-tuning versus system prompting for aligning LMs with a specific behavioral target: responding in a natural, conversational tone suitable for voice interactions. We fine-tuned a small, open-weights model (`Llama3.2-1B-Instruct`) using Low-Rank Adaptation (LoRA) on a synthetically generated dataset derived from Wikipedia. Additionally, we fine-tuned two closed-source models (`gpt-4o-mini`, `gpt-4.1-mini`). Our results demonstrate that fine-tuning outperformed system prompting, achieving a high percentage of conversational responses, even when trained on only 100 data samples. Semantic similarity analysis confirmed that fine-tuning did not degrade content quality. Interestingly, fine-tuning with 8-bit integer quantization converged faster towards the target style than using bfloat16 precision, potentially due to implicit regularization effects. We conclude that fine-tuning small, open-weights LMs on simulated data is a highly effective and data-efficient method for instilling specific stylistic behaviors, offering a preferable alternative to complex system prompting for practical applications requiring nuanced response styles.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑