ALIGN：面向可靠、负责和个性化 LLM 决策的基于提示的属性对齐
ALIGN: Prompt-based Attribute Alignment for Reliable, Responsible, and Personalized LLM-based Decision-Making
AI总结：
提出 ALIGN 开源框架，通过基于提示的细粒度属性对齐实现 LLM 决策者的动态个性化，支持定性与定量分析，并在公共意见调查和医疗分诊领域验证了其有效性。
摘要：
大型语言模型（LLM）正日益被用作决策辅助工具。然而，用户拥有影响其决策的多元价值观和偏好，这要求为 LLM 对齐与个性化开发新方法。现有 LLM 比较工具主要关注基准测试任务，如基于知识的问答。相比之下，我们提出的 ALIGN 系统通过基于提示的对齐来匹配一组细粒度属性，专注于基于 LLM 的决策者的动态个性化。该系统的核心功能包括稳健的配置管理、带推理的结构化输出生成，以及多种带有可替换 LLM 主干的算法实现，从而支持不同类型的分析。我们的用户界面支持对多个 LLM 及其与各种属性的对齐情况进行定性并排比较，后端采用模块化设计以便于算法集成。此外，我们在两个不同领域进行了定量分析，比较了对齐方法：公共意见调查的人口统计对齐，以及医疗分诊决策中的价值对齐。整个 ALIGN 框架是开源的，将推动关于可靠、负责和个性化 LLM 决策者的新研究。
英文摘要：
Large language models (LLMs) are increasingly being used as decision aids. However, users have diverse values and preferences that can affect their decision-making, which requires novel methods for LLM alignment and personalization. Existing LLM comparison tools largely focus on benchmarking tasks, such as knowledge-based question answering. In contrast, our proposed ALIGN system focuses on dynamic personalization of LLM-based decision-makers through prompt-based alignment to a set of fine-grained attributes. Key features of our system include robust configuration management, structured output generation with reasoning, and several algorithm implementations with swappable LLM backbones, enabling different types of analyses. Our user interface enables a qualitative, side-by-side comparison of LLMs and their alignment to various attributes, with a modular backend for easy algorithm integration. Additionally, we perform a quantitative analysis comparing alignment approaches in two different domains: demographic alignment for public opinion surveys and value alignment for medical triage decision-making. The entire ALIGN framework is open source and will enable new research on reliable, responsible, and personalized LLM-based decision-makers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
