参数高效调优实现LLM文本输入的规模化个性化：以缩写扩展为例的案例研究
Parameter Efficient Tuning Allows Scalable Personalization of LLMs for Text Entry: A Case Study on Abbreviation Expansion
AI总结：
本文通过案例研究对比微调、提示调优和检索增强生成在缩写扩展中的个性化效果，发现参数高效调优（尤其是提示调优）能高效实现可扩展的个性化，且软提示初始化可提升准确性。
摘要：
缩写扩展是一种通过限制输入量并利用语言模型提供扩展建议来加速交流的策略。本文基于先前的对话对大型语言模型（LLM）的建议进行个性化，以增强预测的相关性，尤其是在用户数据较少（约1000个样本）的情况下。具体而言，我们比较了微调、提示调优和检索增强生成在缩写输入扩展文本建议中的表现。我们的案例研究使用了一个部署的80亿参数LLM，面向一位真实患有肌萎缩侧索硬化症（ALS）的用户，并在电影角色个性化上的实验表明：（1）在某些场景下定制可能是必要的，而提示调优能很好地泛化到这些场景；（2）在领域内数据上进行微调（样本数低至600个）仍显示出一些收益，但（3）检索增强的少样本选择也优于微调。（4）参数高效调优实现了高效且可扩展的个性化。对于提示调优，我们还发现将学习到的“软提示”初始化为用户相关概念标记比随机初始化能获得更高的准确性。
英文摘要：
Abbreviation expansion is a strategy used to speed up communication by limiting the amount of typing and using a language model to suggest expansions. Here we look at personalizing a Large Language Model's (LLM) suggestions based on prior conversations to enhance the relevance of predictions, particularly when the user data is small (~1000 samples). Specifically, we compare fine-tuning, prompt-tuning, and retrieval augmented generation of expanded text suggestions for abbreviated inputs. Our case study with a deployed 8B parameter LLM on a real user living with ALS, and experiments on movie character personalization indicates that (1) customization may be necessary in some scenarios and prompt-tuning generalizes well to those, (2) fine-tuning on in-domain data (with as few as 600 samples) still shows some gains, however (3) retrieval augmented few-shot selection also outperforms fine-tuning. (4) Parameter efficient tuning allows for efficient and scalable personalization. For prompt-tuning, we also find that initializing the learned "soft-prompts" to user relevant concept tokens leads to higher accuracy than random initialization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
