We present a formal language for specifying qualitative preferences over temporal goals and a preference-based planning method in stochastic systems. Using automata-theoretic modeling, the proposed specification allows us to express preferences over different sets of outcomes, where each outcome describes a set of temporal sequences of subgoals. We define the value of preference satisfaction given a stochastic process over possible outcomes and develop an algorithm for time-constrained probabilistic planning in labeled Markov decision processes where an agent aims to maximally satisfy its preference formula within a pre-defined finite time duration. We present experimental results using a stochastic gridworld example and discuss possible extensions of the proposed preference model.
In this paper, we study planning in stochastic systems, modeled as Markov decision processes (MDPs), with preferences over temporally extended goals. Prior work on temporal planning with preferences assumes that the user preferences form a total order, meaning that every pair of outcomes are comparable with each other. In this work, we consider the case where the preferences over possible outcomes are a partial order rather than a total order. We first introduce a variant of deterministic finite automaton, referred to as a preference DFA, for specifying the user's preferences over temporally e…
Preferences play a key role in determining what goals/constraints to satisfy when not all constraints can be satisfied simultaneously. In this paper, we study how to synthesize preference satisfying plans in stochastic systems, modeled as an MDP, given a (possibly incomplete) combinative preference model over temporally extended goals. We start by introducing new semantics to interpret preferences over infinite plays of the stochastic system. Then, we introduce a new notion of improvement to enable comparison between two prefixes of an infinite play. Based on this, we define two solution conce…
Arizona State University（亚利桑那州立大学）；SUNY Binghamton（纽约州立大学宾汉姆顿分校）
To be responsive to dynamically changing real-world environments, an intelligent agent needs to perform complex sequential decision-making tasks that are often guided by commonsense knowledge. The previous work on this line of research led to the framework called "interleaved commonsense reasoning and probabilistic planning" (icorpp), which used P-log for representing commmonsense knowledge and Markov Decision Processes (MDPs) or Partially Observable MDPs (POMDPs) for planning under uncertainty. A main limitation of icorpp is that its implementation requires non-trivial engineering efforts to …
Preference-Based Planning in Stochastic Environments: From Partially-Ordered Temporal Goals to Most Preferred Policies
Missouri State University（密苏里州立大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Florida（佛罗里达大学）；Oden Institute for Computational Engineering and Sciences（奥登计算工程与科学研究所）
Simplifying Complex Observation Models in Continuous POMDP Planning with Probabilistic Guarantees and Practice
Technion Autonomous Systems Program (TASP), Technion - Israel Institute of Technology（以色列理工学院自主系统项目（TASP），以色列理工学院）；Department of Aerospace Engineering, Technion - Israel Institute of Technology（以色列理工学院航空航天工程系）
iCORPP: Interleaved Commonsense Reasoning and Probabilistic Planning on Robots
The State University of New York at Binghamton（纽约州立大学宾汉姆顿分校）；Sony AI, Sony Corporation of America（索尼美国公司索尼人工智能部门）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）
Behavior2Trip: Towards Personalized Travel Planning via User Behavior Trajectory
Meituan Inc.（美团公司）；School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）；Beijing Institute of Technology（北京理工大学）；University of Edinburgh（爱丁堡大学）
Prior studies have found that explaining robot decisions and actions helps to increase system transparency, improve user understanding, and enable effective human-robot collaboration. In this paper, we present a system for generating personalized explanations of robot path planning via user feedback. We consider a robot navigating in an environment modeled as a Markov decision process (MDP), and develop an algorithm to automatically generate a personalized explanation of an optimal MDP policy, based on the user preference regarding four elements (i.e., objective, locality, specificity, and cor…
Oxford Robotics Institute, University of Oxford（牛津大学牛津机器人研究所）
Previous work on planning as active inference addresses finite horizon problems and solutions valid for online planning. We propose solving the general Stochastic Shortest-Path Markov Decision Process (SSP MDP) as probabilistic inference. Furthermore, we discuss online and offline methods for planning under uncertainty. In an SSP MDP, the horizon is indefinite and unknown a priori. SSP MDPs generalize finite and infinite horizon MDPs and are widely used in the artificial intelligence community. Additionally, we highlight some of the differences between solving an MDP using dynamic programming …
Tufts University（塔夫茨大学）；University of Toronto（多伦多大学）
提升概率推理（Poole，2003）和提升随机规划的符号动态规划（Boutilier et al，2001）是在随机系统中使用抽象的算法努力的同时引入的。多年来，这些想法演变成两个不同的研究路线，每一个都有丰富的文献支持。提升概率推理侧重于在有限域假设下对基于模板的图形模型进行有效的算术运算，而符号动态规划侧重于在丰富的量化逻辑动作模型和开放域推理中支持顺序决策。鉴于它们的共同动机，但不同的重点，这两条研究路线产生了高度互补的创新。在本章中，我们的目标是通过从概率推理的角度概述提升的随机规划来帮助缩小这两个研究领域之间的差距，并显示出与本书其他章节的密切联系。这也使我们能够将广义提升推理定义为一种范式，它统一了这些领域，并阐明了未来研究的开放问题，这些问题可以使提升推理和随机规划受益。
Preference-Guided Planning: An Active Elicitation Approach
University of Texas, Dallas（得克萨斯大学达拉斯分校）；Indiana University, Bloomington（印第安纳大学伯明顿分校）；Washington State University（华盛顿州立大学）；University of Pennsylvania（宾夕法尼亚大学）
Arizona State University（亚利桑那州立大学）；Palo Alto Research Center（帕洛阿尔托研究中心）；Università degli Studi di Brescia（布雷西亚大学）；Free University of Bozen-Bolzano（博尔扎诺自由大学）；IBM India Research Laboratory（IBM印度研究院）
Hewlett Packard（惠普公司）；Sapienza University of Rome（罗马大学）；Pontifical Catholic University of Rio Grande do Sul（南里奥格兰德天主教大学）
Recent approaches to goal recognition have leveraged planning landmarks to achieve high-accuracy with low runtime cost. These approaches, however, lack a probabilistic interpretation. Furthermore, while most probabilistic models to goal recognition assume that the recognizer has access to a prior probability representing, for example, an agent's preferences, virtually no goal recognition approach actually uses the prior in practice, simply assuming a uniform prior. In this paper, we provide a model to both extend landmark-based goal recognition with a probabilistic interpretation and allow the…
Structured Preference Optimization for Vision-Language Long-Horizon Task Planning
Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；The Chinese University of Hong Kong（香港中文大学）；Spatialtemporal AI（时空人工智能）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；The University of Hong Kong（香港大学）；Shanghai Jiaotong University（上海交通大学）
It is well known that reinforcement learning can be cast as inference in an appropriate probabilistic model. However, this commonly involves introducing a distribution over agent trajectories with probabilities proportional to exponentiated rewards. In this work, we formulate reinforcement learning as Bayesian inference without resorting to rewards, and show that rewards are derived from agent's preferences, rather than the other way around. We argue that agent preferences should be specified stochastically rather than deterministically. Reinforcement learning via inference with stochastic pre…
LLMAP: LLM-Assisted Multi-Objective Route Planning with User Preferences
School of Electrical and Computer Engineering, Purdue University（普渡大学电气与计算机工程学院）；Department of Computer Science and Engineering, Yonsei University（延世大学计算机科学与工程系）；Polytechnic Institute, Purdue University（普渡大学理工学院）
