行动胜于言语：马尔可夫决策过程中的速率-奖励权衡
Actions Speak Louder Than Words: Rate-Reward Trade-off in Markov Decision Processes
AI总结：
该研究针对马尔可夫决策过程中智能体通过行动通信的场景，刻画了无限时域下奖励与通信速率的信息论权衡，提出Act2Comm框架实现控制与通信的平衡，实验验证了其兼顾可靠通信与一定控制性能的能力。
摘要：
通信对决策系统的影响已在专用通信信道的假设下得到了广泛研究。我们转而考虑通过行动进行通信的场景：在马尔可夫决策过程（MDP）框架下，消息被嵌入到与环境交互的智能体的行动中。我们将MDP环境概念化为有限状态信道（FSC），其中智能体的行动作为信道输入，而另一个智能体（即接收方）观测到的MDP状态作为信道输出。在此场景中，我们将环境视为一条通信信道，智能体通过其行动在该信道上传输信息，同时还试图最大化自身获得的奖励。我们首先刻画了无限时域下平均奖励与可靠通信速率之间的最优信息论权衡关系。随后，我们提出了一种用于设计联合控制/编码策略的新型框架，命名为Act2Comm，该框架可将消息无缝嵌入到行动中。从通信角度看，Act2Comm是一种针对输入输出约束下不可微FSC的基于学习的信道编码方案。从控制角度看，Act2Comm学习到的MDP策略具备通信能力，但需要付出一定控制性能下降的代价。总体而言，Act2Comm有效平衡了该环境下控制与通信的双重目标。实验结果验证了Act2Comm在维持一定控制性能水平的同时，实现可靠通信的能力。
英文摘要：
The impact of communication on decision-making systems has been extensively studied under the assumption of dedicated communication channels. We instead consider communicating through actions, where the message is embedded into the actions of an agent which interacts with the environment in a Markov decision process (MDP) framework. We conceptualize the MDP environment as a finite-state channel (FSC), where the actions of the agent serve as the channel input, while the states of the MDP observed by another agent (i.e., receiver) serve as the channel output. Here, we treat the environment as a communication channel over which the agent communicates through its actions, while at the same time, trying to maximize its reward. We first characterize the optimal information theoretic trade-off between the average reward and the rate of reliable communication in the infinite-horizon regime. Then, we propose a novel framework to design a joint control/coding policy, termed \textit{Act2Comm}, which seamlessly embeds messages into actions. From a communication perspective, \textit{Act2Comm} functions as a learning-based channel coding scheme for non-differentiable FSCs under input-output constraints. From a control standpoint, \textit{Act2Comm} learns an MDP policy that incorporates communication capabilities, though at the cost of some control performance. Overall, \textit{Act2Comm} effectively balances the dual objectives of control and communication in this environment. Experimental results validate \textit{Act2Comm}'s capability to enable reliable communication while maintaining a certain level of control performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
