arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.06273cs.AIcs.HC
收藏

PREDICT:通过评估从候选轨迹推断出的分解偏好进行偏好推理

PREDICT: Preference Reasoning by Evaluating Decomposed preferences Inferred from Candidate Trajectories

Apple., Inc(苹果公司)

Stephane Aroca-Ouellette, Natalie Mackraz, Barry-John Theobald, Katherine Metcalf

AI总结:

针对大语言模型推断人类偏好过于宽泛的问题,提出含迭代优化、偏好分解、多轨迹验证的PREDICT方法,在两种环境中显著提升偏好推断准确率。

摘要:

适配人类偏好对于打造能提供个性化、有效交互的AI智能体至关重要。近期研究已展现大语言模型(LLMs)从用户交互中推断偏好的潜力,但它们往往生成宽泛笼统的偏好,未能捕捉人类偏好独特且个体化的特性。本文提出PREDICT,一种旨在提升偏好推断精准度与适配性的方法。PREDICT包含三个核心要素:(1)对推断出的偏好进行迭代优化,(2)将偏好分解为各组成部分,(3)跨多条轨迹验证偏好。我们在两种不同环境中评估PREDICT:网格世界环境与全新的文本域环境(PLUME)。PREDICT能更准确地推断细致的人类偏好,相比现有基线在网格世界环境中提升66.2%,在PLUME中提升41.0%。

英文摘要:

Accommodating human preferences is essential for creating AI agents that deliver personalized and effective interactions. Recent work has shown the potential for LLMs to infer preferences from user interactions, but they often produce broad and generic preferences, failing to capture the unique and individualized nature of human preferences. This paper introduces PREDICT, a method designed to enhance the precision and adaptability of inferring preferences. PREDICT incorporates three key elements: (1) iterative refinement of inferred preferences, (2) decomposition of preferences into constituent components, and (3) validation of preferences across multiple trajectories. We evaluate PREDICT on two distinct environments: a gridworld setting and a new text-domain environment (PLUME). PREDICT more accurately infers nuanced human preferences improving over existing baselines by 66.2\% (gridworld environment) and 41.0\% (PLUME).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑