首页/论文/2002.09043

arXiv 2002.090432020-02-20cs.LGstat.ML

oIRL: Robust Adversarial Inverse Reinforcement Learning with Temporally Extended Actions

McGill University（麦吉尔大学）；Mila（Mila（魁北克人工智能研究所））；MIT（麻省理工学院）；DeepMind

David Venuto, Jhelum Chakravorty, Leonard Boussioux, Junhao Wang, Gavin McCracken, Doina Precup

英文摘要：

Explicit engineering of reward functions for given environments has been a major hindrance to reinforcement learning methods. While Inverse Reinforcement Learning (IRL) is a solution to recover reward functions from demonstrations only, these learned rewards are generally heavily \textit{entangled} with the dynamics of the environment and therefore not portable or \emph{robust} to changing environments. Modern adversarial methods have yielded some success in reducing reward entanglement in the IRL setting. In this work, we leverage one such method, Adversarial Inverse Reinforcement Learning (AIRL), to propose an algorithm that learns hierarchical disentangled rewards with a policy over options. We show that this method has the ability to learn \emph{generalizable} policies and reward functions in complex transfer learning tasks, while yielding results in continuous control benchmarks that are comparable to those of the state-of-the-art methods.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

TCRL: 时序耦合对抗训练用于最坏情况下的鲁棒约束强化学习

TCRL: Temporal-Coupled Adversarial Training for Robust Constrained Reinforcement Learning in Worst-Case Scenarios

Northeastern University（东北大学）；Zhejiang University（浙江大学）；Aalborg University（奥胡斯大学）

TCRL通过引入时序耦合对抗训练框架，解决传统方法在处理时序耦合扰动时的不足，提升约束强化学习在最坏情况下的鲁棒性。

2026-02-13cs.LG

Membership Inference Attacks Against Temporally Correlated Data in Deep Reinforcement Learning

McGill University（麦吉尔大学）；Mila（米拉研究所）；University of Waterloo（滑铁卢大学）；DarwinAI（达尔文人工智能公司）；DeepMind（深度思维）

While significant research advances have been made in the field of deep reinforcement learning, there have been no concrete adversarial attack strategies in literature tailored for studying the vulnerability of deep reinforcement learning algorithms to membership inference attacks. In such attacking systems, the adversary targets the set of collected input data on which the deep reinforcement learning algorithm has been trained. To address this gap, we propose an adversarial attack framework designed for testing the vulnerability of a state-of-the-art deep reinforcement learning algorithm to a…

2022-11-16cs.CRcs.LG

时间统一对抗扰动用于时间序列预测

Temporally Unified Adversarial Perturbations for Time Series Forecasting

本文提出时间统一对抗扰动方法，通过时间一致性约束和梯度累积方法提升时间序列预测模型的对抗攻击效果。

2026-02-12cs.LG

基于策略梯度的少样本学习视频到视频生成对抗网络

Video to Video Generative Adversarial Network for Few-shot Learning Based on Policy Gradient

Northwestern University（西北大学）；Allstate Insurance Company（好事达保险公司）

本文提出 RL-V2V-GAN，结合策略梯度、ConvLSTM 与细粒度时空 GAN，实现无需成对输入的无监督条件视频到视频合成，并在少样本目标域中生成时间连贯视频。

2024-10-28cs.CVcs.LG

使用带约束的生成对抗网络创建时间相关的高分辨率负荷注入曲线

Creating Temporally Correlated High-Resolution Profiles of Load Injection Using Constrained Generative Adversarial Networks

Eversource Energy（Eversource能源公司）；Arizona State University（亚利桑那州立大学）

本文提出一种仅用智能电表慢时间尺度聚合数据训练的带硬约束GAN，通过凸优化强制时间一致性，从15分钟平均功率生成逐分钟负荷曲线，提升配电系统高速状态估计能力。

2024-09-01cs.LGeess.SP

VideoMoCo: Contrastive Video Representation Learning with Temporally Adversarial Examples

Tencent AI Lab（腾讯AI实验室）；Tencent Data Platform（腾讯数据平台）

MoCo is effective for unsupervised image representation learning. In this paper, we propose VideoMoCo for unsupervised video representation learning. Given a video sequence as an input sample, we improve the temporal feature representations of MoCo from two perspectives. First, we introduce a generator to drop out several frames from this sample temporally. The discriminator is then learned to encode similar feature representations regardless of frame removals. By adaptively dropping out different frames during training iterations of adversarial learning, we augment this input sample to train …

2021-03-17cs.CVcs.LGcs.MM

更多相关论文收起更多论文

混合使得马尔可夫上下文在线性赌博机中变得廉价

Mixing Makes Markovian Contexts Cheap for Linear Bandits

University of California, Los Angeles（加州大学洛杉矶分校）；Meta, Superintelligence Lab（Meta超智能实验室）

针对马尔可夫上下文线性赌博机问题，提出一种基于均匀几何遍历性的约简方法，通过构建平稳替代动作集和延迟更新方案，实现了与标准线性赌博机相当的最坏情况遗憾界。

2026-09-09cs.LG

Learning Temporal Coherence via Self-Supervision for GAN-based Video Generation

Technical University of Munich（慕尼黑工业大学）

Our work explores temporal self-supervision for GAN-based video generation tasks. While adversarial training successfully yields generative models for a variety of areas, temporal relationships in the generated data are much less explored. Natural temporal changes are crucial for sequential generation tasks, e.g. video super-resolution and unpaired video translation. For the former, state-of-the-art methods often favor simpler norm losses such as $L^2$ over adversarial training. However, their averaging nature easily leads to temporally smooth results with an undesirable lack of spatial detail…

2020-05-21cs.CVcs.LG

通过奖励进行 scouting：由 VLM-TO-IRL 驱动的电子竞技玩家选拔

Scouting By Reward: VLM-TO-IRL-Driven Player Selection For Esports

Johns Hopkins University（约翰霍普金斯大学）；University of Pennsylvania（宾夕法尼亚大学）；Cornell University（康奈尔大学）

本文提出一种基于逆强化学习的电子竞技玩家选拔框架，通过学习专业选手的奖励函数，利用多模态双分支架构和 GAIL 目标实现玩家风格匹配评估，提升人才发现效率。

2026-04-15cs.LG

Spatiotemporally Constrained Action Space Attacks on Deep Reinforcement Learning Agents

Iowa State University（爱荷华州立大学）；Tandon School of Engineering, New York University（纽约大学坦登工程学院）

Robustness of Deep Reinforcement Learning (DRL) algorithms towards adversarial attacks in real world applications such as those deployed in cyber-physical systems (CPS) are of increasing concern. Numerous studies have investigated the mechanisms of attacks on the RL agent's state space. Nonetheless, attacks on the RL agent's action space (AS) (corresponding to actuators in engineering systems) are equally perverse; such attacks are relatively less studied in the ML literature. In this work, we first frame the problem as an optimization problem of minimizing the cumulative reward of an RL agent…

2019-11-19cs.AIcs.CRcs.LG

Learning Multi-Task Transferable Rewards via Variational Inverse Reinforcement Learning

Seoul National University（首尔国立大学）

Many robotic tasks are composed of a lot of temporally correlated sub-tasks in a highly complex environment. It is important to discover situational intentions and proper actions by deliberating on temporal abstractions to solve problems effectively. To understand the intention separated from changing task dynamics, we extend an empowerment-based regularization technique to situations with multiple tasks based on the framework of a generative adversarial network. Under the multitask environments with unknown dynamics, we focus on learning a reward and policy from the unlabeled expert examples.…

2022-06-19cs.AIcs.LGcs.RO

Adversarial Cross-Domain Action Recognition with Co-Attention

Stanford University（斯坦福大学）

Action recognition has been a widely studied topic with a heavy focus on supervised learning involving sufficient labeled videos. However, the problem of cross-domain action recognition, where training and testing videos are drawn from different underlying distributions, remains largely under-explored. Previous methods directly employ techniques for cross-domain image recognition, which tend to suffer from the severe temporal misalignment problem. This paper proposes a Temporal Co-attention Network (TCoN), which matches the distributions of temporally aligned action features between source and…

2019-12-22cs.CVcs.LGeess.IV

重新思考鲁棒性评估：针对基于学习的四足运动控制器的对抗攻击

Rethinking Robustness Assessment: Adversarial Attacks on Learning-based Quadrupedal Locomotion Controllers

Robotic Systems Lab, ETH Zurich（苏黎世联邦理工学院机器人系统实验室）；Computational Robotics Lab, ETH Zurich（苏黎世联邦理工学院计算机器人实验室）；ETH AI Center（苏黎世联邦理工学院人工智能中心）

本文提出一种利用序列对抗攻击识别四足运动控制器弱点的计算方法，揭示最先进鲁棒控制器在低幅度对抗序列下会失效，并验证该方法能有效增强原始策略鲁棒性及评估黑盒策略安全性。

2024-05-30cs.LGcs.RO

通过鲁棒时间自集成提升脉冲神经网络的鲁棒性-精度权衡

Boosting the Robustness-Accuracy Trade-off of SNNs by Robust Temporal Self-Ensemble

本文提出鲁棒时间自集成（RTE）训练框架，通过增强各时间子网络鲁棒性并降低对抗扰动跨时间迁移，显著改善脉冲神经网络的鲁棒性-精度权衡。

2025-08-15cs.CVcs.LG

CLAWSAT: Towards Both Robust and Accurate Code Models

Michigan State University（密歇根州立大学）；CSAIL, MIT（麻省理工学院计算机科学与人工智能实验室）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）；UC Santa Barbara（加州大学圣巴巴拉分校）

We integrate contrastive learning (CL) with adversarial learning to co-optimize the robustness and accuracy of code models. Different from existing works, we show that code obfuscation, a standard code transformation operation, provides novel means to generate complementary `views' of a code that enable us to achieve both robust and accurate code models. To the best of our knowledge, this is the first systematic study to explore and exploit the robustness and accuracy benefits of (multi-view) code obfuscations in code models. Specifically, we first adopt adversarial codes as robustness-promoti…

2023-03-03cs.LGcs.PLcs.SE