用于交互式文本环境的带指数倾斜的语言决策 Transformers
Language Decision Transformers with Exponential Tilt for Interactive Text Environments
AI总结:
提出 Language Decision Transformers (LDTs) 框架,通过引入指数倾斜、新颖目标条件化和未来观测模型扩展决策 Transformers,在 Jericho 等挑战性文本游戏中实现最高分并首次解决离线 RL 问题。
摘要:
基于文本的游戏环境具有挑战性,因为智能体必须处理长文本序列,使用文本执行组合动作并从稀疏奖励中学习。我们通过提出 Language Decision Transformers (LDTs) 来应对这些挑战,这是一个基于 transformer 语言模型和决策 Transformers (DTs) 的框架。我们的 LDTs 通过 3 个组件扩展了 DTs:(1) 指数倾斜,用于引导智能体实现高可获得目标;(2) 新颖的目标条件化方法,其结果优于传统的 return-to-go(所有未来奖励的总和);(3) 未来观测模型,可提升智能体性能。LDTs 是首个在这些具有挑战性的游戏上利用 DTs 解决离线 RL 问题的方法。我们的实验表明,在一些最具挑战性的 Jericho 游戏(如 Enchanter)中,LDTs 在许多不同类型的智能体中获得了最高分。
英文摘要:
Text-based game environments are challenging because agents must deal with long sequences of text, execute compositional actions using text and learn from sparse rewards. We address these challenges by proposing Language Decision Transformers (LDTs), a framework that is based on transformer language models and decision transformers (DTs). Our LDTs extend DTs with 3 components: (1) exponential tilt to guide the agent towards high obtainable goals, (2) novel goal conditioning methods yielding better results than the traditional return-to-go (sum of all future rewards), and (3) a model of future observations that improves agent performance. LDTs are the first to address offline RL with DTs on these challenging games. Our experiments show that LDTs achieve the highest scores among many different types of agents on some of the most challenging Jericho games, such as Enchanter.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。