The space of value functions is a fundamental concept in reinforcement learning. Characterizing its geometric properties may provide insights for optimization and representation. Existing works mainly focus on the value space for Markov Decision Processes (MDPs). In this paper, we study the geometry of the robust value space for the more general Robust MDPs (RMDPs) setting, where transition uncertainties are considered. Specifically, since we find it hard to directly adapt prior approaches to RMDPs, we start with revisiting the non-robust case, and introduce a new perspective that enables us to characterize both the non-robust and robust value space in a similar fashion. The key of this perspective is to decompose the value space, in a state-wise manner, into unions of hypersurfaces. Through our analysis, we show that the robust value space is determined by a set of conic hypersurfaces, each of which contains the robust values of all policies that agree on one state. Furthermore, we find that taking only extreme points in the uncertainty set is sufficient to determine the robust value space. Finally, we discuss some other aspects about the robust value space, including its non-convexity and policy agreement on multiple states.
Feature attributions are a popular tool for explaining the behavior of Deep Neural Networks (DNNs), but have recently been shown to be vulnerable to attacks that produce divergent explanations for nearby inputs. This lack of robustness is especially problematic in high-stakes applications where adversarially-manipulated explanations could impair safety and trustworthiness. Building on a geometric understanding of these attacks presented in recent work, we identify Lipschitz continuity conditions on models' gradient that lead to robust gradient-based attributions, and observe that smoothness ma…
Massachusetts Institute of Technology（麻省理工学院）；Harvard University（哈佛大学）；MIT-IBM Watson AI Lab（MIT-IBM沃森人工智能实验室）
Adversarial examples are often cited by neuroscientists and machine learning researchers as an example of how computational models diverge from biological sensory systems. Recent work has proposed adding biologically-inspired components to visual neural networks as a way to improve their adversarial robustness. One surprisingly effective component for reducing adversarial vulnerability is response stochasticity, like that exhibited by biological neurons. Here, using recently developed geometrical techniques from computational neuroscience, we investigate how adversarial perturbations influence…
A Dual Basis Approach for Structured Robust Euclidean Distance Geometry
Department of Statistics and Data Science University of Central Florida（统计与数据科学系 佛罗里达中央大学）；Department of Mathematics Tufts University（数学系 塔夫茨大学）；Department of Statistics and Data Science Department of Computer Science University of Central Florida（统计与数据科学系 计算机科学系 佛罗里达中央大学）
Clinically Aligned Geometry Constraints for Robust IVUS Vessel Boundary Segmentation
AIM For Health Lab, Monash University（莫纳什大学AIM健康实验室）；Department of Data Science and Artificial Intelligence, Faculty of IT, Monash University（莫纳什大学信息技术学院数据科学与人工智能系）；Monash University Victorian Heart Institute（莫纳什大学维多利亚心脏研究所）；School of Computing Technologies, RMIT University（皇家墨尔本理工大学计算技术学院）；National Cerebral and Cardiovascular Center（国立循环器病研究中心）；Department of Cardiology, Chonnam National University Hospital and Medical School（全南大学医院和医学院心脏病学系）
