POPI：通过优化自然语言偏好推断实现个性化LLM
POPI: Personalizing LLMs via Optimized Natural Language Preference Inference
AI总结：
POPI提出一种用户级个性化框架，通过自然语言接口分离共享推断模型和生成器，统一优化目标提升个性化质量并减少上下文开销。
摘要：
大型语言模型（LLMs）通常与群体层面的偏好对齐，尽管在个体用户间存在显著差异。我们引入POPI，一种用户级个性化框架，将问题分解为两个通过自然语言接口连接的组件：一个共享推断模型，将异质用户信号压缩为简洁的偏好摘要，以及一个共享生成器，根据此摘要生成个性化响应。两个组件均在统一的偏好优化目标下训练，强化学习处理非可微的推断步骤。此目标分解为生成器近似误差和摘要信息量，揭示了单个损失如何同时驱动准确生成和信息性摘要。由于接口是自然语言，学习到的摘要可一次推断每个用户并重用于不同生成器——包括冻结的、黑盒商业API。在四个个性化基准测试中，POPI通常在提高个性化质量的同时，将上下文开销减少一个数量级。
英文摘要：
Large language models (LLMs) are typically aligned with population-level preferences, despite substantial variation across individual users. We introduce POPI, a user-level personalization framework that separates the problem into two components connected by a natural-language interface: a shared inference model that distills heterogeneous user signals into a concise preference summary, and a shared generator that conditions on this summary to produce personalized responses. Both components are trained under a unified preference-optimization objective, with reinforcement learning handling the non-differentiable inference step. This objective decomposes into generator approximation error and summary informativeness, revealing how a single loss simultaneously drives accurate generation and informative summarization. Because the interface is natural language, learned summaries can be inferred once per user and reused across different generators -- including frozen, black-box commercial APIs. Across four personalization benchmarks, POPI generally improves personalization quality while reducing context overhead by up to an order of magnitude.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
