Accommodating human preferences is essential for creating AI agents that deliver personalized and effective interactions. Recent work has shown the potential for LLMs to infer preferences from user interactions, but they often produce broad and generic preferences, failing to capture the unique and individualized nature of human preferences. This paper introduces PREDICT, a method designed to enhance the precision and adaptability of inferring preferences. PREDICT incorporates three key elements: (1) iterative refinement of inferred preferences, (2) decomposition of preferences into constituent components, and (3) validation of preferences across multiple trajectories. We evaluate PREDICT on two distinct environments: a gridworld setting and a new text-domain environment (PLUME). PREDICT more accurately infers nuanced human preferences improving over existing baselines by 66.2\% (gridworld environment) and 41.0\% (PLUME).
Department of Electrical Engineering and Computer Science, Pennsylvania State University, PA, USA(电气工程与计算机科学系,宾夕法尼亚州立大学);College of Information Sciences and Technology, Pennsylvania State University, PA, USA(信息科学与技术学院,宾夕法尼亚州立大学)
Learning What Matters Now: Dynamic Preference Inference under Contextual Shifts
School of Artificial Intelligence, Xidian University(西安电子科技大学人工智能学院);State Key Laboratory of General Artificial Intelligence, BIGAI(通用人工智能国家重点实验室,BIGAI)
Alignment Revisited: Are Large Language Models Consistent in Stated and Revealed Preferences?
Department of Information Systems and Business Analytics(信息系统与商业分析系);University at Albany - State University of New York(阿尔巴尼大学 - 新 york 状态大学);Santa Clara, CA(圣克拉拉,加州);Princeton Junction, NJ(普林斯顿 junction,新泽西)
From Profiling to Synthesis: Benchmarking Implicit Behavioral Alignment in Personalized LLM Agents
National University of Singapore(新加坡国立大学);University of Toronto(多伦多大学);University of Minnesota Twin Cities(明尼苏达大学双城分校);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));University of Oxford(牛津大学)