Developing an agent in reinforcement learning (RL) that is capable of performing complex control tasks directly from high-dimensional observation such as raw pixels is yet a challenge as efforts are made towards improving sample efficiency and generalization. This paper considers a learning framework for Curiosity Contrastive Forward Dynamics Model (CCFDM) in achieving a more sample-efficient RL based directly on raw pixels. CCFDM incorporates a forward dynamics model (FDM) and performs contrastive learning to train its deep convolutional neural network-based image encoder (IE) to extract conducive spatial and temporal information for achieving a more sample efficiency for RL. In addition, during training, CCFDM provides intrinsic rewards, produced based on FDM prediction error, encourages the curiosity of the RL agent to improve exploration. The diverge and less-repetitive observations provide by both our exploration strategy and data augmentation available in contrastive learning improve not only the sample efficiency but also the generalization. Performance of existing model-free RL methods such as Soft Actor-Critic built on top of CCFDM outperforms prior state-of-the-art pixel-based RL methods on the DeepMind Control Suite benchmark.
Guangdong University of Technology（广东工业大学）；The University of Sydney（悉尼大学）
Unpaired image-to-image translation aims to find a mapping between the source domain and the target domain. To alleviate the problem of the lack of supervised labels for the source images, cycle-consistency based methods have been proposed for image structure preservation by assuming a reversible relationship between unpaired images. However, this assumption only uses limited correspondence between image pairs. Recently, contrastive learning (CL) has been used to further investigate the image correspondence in unpaired image translation by using patch-based positive/negative learning. Patch-ba…
The paper proposes a new algorithm called SymBa that aims to achieve more biologically plausible learning than Back-Propagation (BP). The algorithm is based on the Forward-Forward (FF) algorithm, which is a BP-free method for training neural networks. SymBa improves the FF algorithm's convergence behavior by addressing the problem of asymmetric gradients caused by conflicting converging directions for positive and negative samples. The algorithm balances positive and negative losses to enhance performance and convergence speed. Furthermore, it modifies the FF algorithm by adding Intrinsic Clas…
CCLF: A Contrastive-Curiosity-Driven Learning Framework for Sample-Efficient Reinforcement Learning
Alibaba-NTU Singapore Joint Research Institute（阿里巴巴-新加坡南洋理工大学联合研究院）；Nanyang Technological University（南洋理工大学）；Alibaba Group（阿里巴巴集团）；Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly (LILY)（南洋理工大学-不列颠哥伦比亚大学活跃老年生活卓越联合研究中心（LILY））
Neural networks represent data as projections on trained weights in a high dimensional manifold. The trained weights act as a knowledge base consisting of causal class dependencies. Inference built on features that identify these dependencies is termed as feed-forward inference. Such inference mechanisms are justified based on classical cause-to-effect inductive reasoning models. Inductive reasoning based feed-forward inference is widely used due to its mathematical simplicity and operational ease. Nevertheless, feed-forward models do not generalize well to untrained situations. To alleviate t…
California Institute of Technology（加州理工学院）；University of Southern California（南加州大学）；Mississippi State University（密西西比州立大学）
Contrastive learning has recently demonstrated superior performance to supervised learning, despite requiring no training labels. We explore how contrastive learning can be applied to hundreds of thousands of unlabeled Mars terrain images, collected from the Mars rovers Curiosity and Perseverance, and from the Mars Reconnaissance Orbiter. Such methods are appealing since the vast majority of Mars images are unlabeled as manual annotation is labor intensive and requires extensive domain knowledge. Contrastive learning, however, assumes that any given pair of distinct images contain distinct sem…
Albeit having gained significant progress lately, large-scale graph representation learning remains expensive to train and deploy for two main reasons: (i) the repetitive computation of multi-hop message passing and non-linearity in graph neural networks (GNNs); (ii) the computational cost of complex pairwise contrastive learning loss. Two main contributions are made in this paper targeting this twofold challenge: we first propose an adaptive-view graph neural encoder (AVGE) with a limited number of message passing to accelerate the forward pass computation, and then we propose a structure-awa…
We study a new family of inverse problems for recovering representations of corrupted data. We assume access to a pre-trained representation learning network R(x) that operates on clean images, like CLIP. The problem is to recover the representation of an image R(x), if we are only given a corrupted version A(x), for some known forward operator A. We propose a supervised inversion method that uses a contrastive objective to obtain excellent representations for highly corrupted images. Using a linear probe on our robust representations, we achieve a higher accuracy than end-to-end supervised ba…
Beijing University of Posts and Telecommunications（北京邮电大学）；The University of Sydney（悉尼大学）；Zhejiang Lab（之江实验室）
Session-based recommendation (SBR) problem, which focuses on next-item prediction for anonymous users, has received increasingly more attention from researchers. Existing graph-based SBR methods all lack the ability to differentiate between sessions with the same last item, and suffer from severe popularity bias. Inspired by nowadays emerging contrastive learning methods, this paper presents a Simple Contrastive Graph Neural Network for Session-based Recommendation (SimCGNN). In SimCGNN, we first obtain normalized session embeddings on constructed session graphs. We next construct positive and…
Recent advancements in explainable machine learning provide effective and faithful solutions for interpreting model behaviors. However, many explanation methods encounter efficiency issues, which largely limit their deployments in practical scenarios. Real-time explainer (RTX) frameworks have thus been proposed to accelerate the model explanation process by learning a one-feed-forward explainer. Existing RTX frameworks typically build the explainer under the supervised learning paradigm, which requires large amounts of explanation labels as the ground truth. Considering that accurate explanati…
Molecular Graph Contrastive Learning with Line Graph
School of Statistics and Mathematics, Central University of Finance and Economics（中央财经大学统计与数学学院）；School of Journalism, Communication University of China（中国传媒大学新闻学院）
The aim of this paper is to demonstrate the efficacy of using Contrastive Random Walk as a curiosity method to achieve faster convergence to the optimal policy.Contrastive Random Walk defines the transition matrix of a random walk with the help of neural networks. It learns a meaningful state representation with a closed loop. The loss of Contrastive Random Walk serves as an intrinsic reward and is added to the environment reward. Our method works well in non-tabular sparse reward scenarios, in the sense that our method receives the highest reward within the same iterations compared to other m…
Harbin Institute of Technology(Shenzhen)（哈尔滨工业大学（深圳））；The Chinese University of Hong Kong（香港中文大学）；Huawei Technologies co.ltd（华为技术有限公司）；Shenzhen University（深圳大学）
Recently, hyperbolic space has risen as a promising alternative for semi-supervised graph representation learning. Many efforts have been made to design hyperbolic versions of neural network operations. However, the inspiring geometric properties of this unique geometry have not been fully explored yet. The potency of graph models powered by the hyperbolic space is still largely underestimated. Besides, the rich information carried by abundant unlabelled samples is also not well utilized. Inspired by the recently active and emerging self-supervised learning, in this study, we attempt to enhanc…
Politecnico di Torino（都灵理工大学）；PIC4SeR (PoliTo Interdepartmental Centre for Service Robotics)（PIC4SeR（都灵理工大学服务机器人跨部门中心））
The development of precision agriculture has gradually introduced automation in the agricultural process to support and rationalize all the activities related to field management. In particular, service robotics plays a predominant role in this evolution by deploying autonomous agents able to navigate in fields while executing different tasks without the need for human intervention, such as monitoring, spraying and harvesting. In this context, global path planning is the first necessary step for every robotic mission and ensures that the navigation is performed efficiently and with complete fi…
Generalized Out-of-distribution Fault Diagnosis (GOOFD) via Internal Contrastive Learning
Zhejiang University and the University of Illinois Urbana–Champaign Joint Institute（浙江大学与伊利诺伊大学厄巴纳-香槟分校联合研究院）；College of Computer Science and Technology in Zhejiang University（浙江大学计算机科学与技术学院）
Department of Informatics Knowledge Technology Institute（信息学知识技术研究所）
Hierarchical abstraction and curiosity-driven exploration are two common paradigms in current reinforcement learning approaches to break down difficult problems into a sequence of simpler ones and to overcome reward sparsity. However, there is a lack of approaches that combine these paradigms, and it is currently unknown whether curiosity also helps to perform the hierarchical abstraction. As a novelty and scientific contribution, we tackle this issue and develop a method that combines hierarchical reinforcement learning with curiosity. Herein, we extend a contemporary hierarchical actor-criti…
Climate models often require post-processing in order to make accurate estimates of local climate risk. The most common post-processing applied is bias-correction and spatial resolution enhancement. However, the statistical methods typically used for this not only are incapable of capturing multivariate spatial correlation information but are also reliant on rich observational data often not available outside of developed countries, limiting their potential. Here we propose an alternative approach to this challenge based on a combination of image super resolution (SR) and contrastive learning …
Unsupervised learning has made substantial progress over the last few years, especially by means of contrastive self-supervised learning. The dominating dataset for benchmarking self-supervised learning has been ImageNet, for which recent methods are approaching the performance achieved by fully supervised training. The ImageNet dataset is however largely object-centric, and it is not clear yet what potential those methods have on widely different datasets and tasks that are not object-centric, such as in digital pathology. While self-supervised learning has started to be explored within this …
Parameter Estimation using Reinforcement Learning Causal Curiosity: Limits and Challenges
Digital Scholarship at Oxford（牛津大学数字人文中心）；University of Oxford（牛津大学）；Faculty of Engineering and Applied Sciences（工程与应用科学学院）；Cranfield University（克兰菲尔德大学）
SaliencyI2PLoc: saliency-guided image-point cloud localization using contrastive learning
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing（测绘遥感信息工程国家重点实验室）；Wuhan University（武汉大学）；School of Electrical and Electronic Engineering（电气与电子工程学院）；Nanyang Technological University（南洋理工大学）；School of Geography and Environment（地理与环境学院）；Jiangxi Normal University（江西师范大学）
Recently, the cross-modal pre-training task has been a hotspot because of its wide application in various down-streaming researches including retrieval, captioning, question answering and so on. However, exiting methods adopt a one-stream pre-training model to explore the united vision-language representation for conducting cross-modal retrieval, which easily suffer from the calculation explosion. Moreover, although the conventional double-stream structures are quite efficient, they still lack the vital cross-modal interactions, resulting in low performances. Motivated by these challenges, we …
Single-Pass Contrastive Learning Can Work for Both Homophilic and Heterophilic Graph
National University of Singapore（新加坡国立大学）；University of Washington（华盛顿大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）
Illinois Institute of Technology（伊利诺伊理工学院）；Hong Kong University of Science and Technology（香港科技大学）；Texas State University（德克萨斯州立大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））
Neural network binarization accelerates deep models by quantizing their weights and activations into 1-bit. However, there is still a huge performance gap between Binary Neural Networks (BNNs) and their full-precision (FP) counterparts. As the quantization error caused by weights binarization has been reduced in earlier works, the activations binarization becomes the major obstacle for further improvement of the accuracy. BNN characterises a unique and interesting structure, where the binary and latent FP activations exist in the same forward pass (i.e., $\text{Binarize}(\mathbf{a}_F) = \mathb…
Tsinghua University（清华大学）；Microsoft Research Asia（微软亚洲研究院）
Contrastive learning methods for unsupervised visual representation learning have reached remarkable levels of transfer performance. We argue that the power of contrastive learning has yet to be fully unleashed, as current methods are trained only on instance-level pretext tasks, leading to representations that may be sub-optimal for downstream tasks requiring dense pixel predictions. In this paper, we introduce pixel-level pretext tasks for learning dense feature representations. The first task directly applies contrastive learning at the pixel level. We additionally propose a pixel-to-propag…
Stochastic Forward-Forward Learning through Representational Dimensionality Compression
Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai 200433, China（脑启发式智能科学与技术研究院，复旦大学，上海200433，中国）；Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence (Fudan University), Ministry of Education, China（计算神经科学与脑启发式智能重点实验室（复旦大学），教育部，中国）；MOE Frontiers Center for Brain Science, Fudan University, Shanghai 200433, China（教育部前沿脑科学中心（复旦大学），上海200433，中国）；Ji Hua Laboratory, Foshan 528200, China（济华实验室，佛山528200，中国）
CF-JEPA: Mask-free forward prediction with asymmetric encoder utilization for time-series representation learning
Graduate School of Artificial Intelligence Convergence Engineering, Changwon National University（昌原国立大学人工智能融合工程研究生院）；Department of Artificial Intelligence Engineering, Changwon National University（昌原国立大学人工智能工程系）
Stanford University（斯坦福大学）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；NVIDIA（英伟达）；Caltech（加州理工学院）
Manipulating volumetric deformable objects in the real world, like plush toys and pizza dough, bring substantial challenges due to infinite shape variations, non-rigid motions, and partial observability. We introduce ACID, an action-conditional visual dynamics model for volumetric deformable objects based on structured implicit neural representations. ACID integrates two new techniques: implicit representations for action-conditional dynamics and geodesics-based contrastive learning. To represent deformable dynamics from partial RGB-D observations, we learn implicit representations of occupanc…
Breaking Chains with Trees: Model-Parallel Deep Learning with $\mathcal{O}(\log N)$ Time Complexity
Center for Cognitive Interaction Technology, Bielefeld University（比勒费尔德大学认知交互技术中心）；Ruhr Universität Bochum（波鸿鲁尔大学）；Birla Institute of Technology and Science, Pilani, Goa Campus（比拉理工学院皮拉尼校区果阿分校）；Royal Holloway, University of London（伦敦大学皇家霍洛威学院）
Physikalisch Technische Bundesanstalt（德国联邦物理技术研究院）；Fraunhofer Heinrich Hertz Institute（弗劳恩霍夫海因里希赫兹研究所）；University of Oldenburg（奥尔登堡大学）
Clinical 12-lead electrocardiography (ECG) is one of the most widely encountered kinds of biosignals. Despite the increased availability of public ECG datasets, label scarcity remains a central challenge in the field. Self-supervised learning represents a promising way to alleviate this issue. In this work, we put forward the first comprehensive assessment of self-supervised representation learning from clinical 12-lead ECG data. To this end, we adapt state-of-the-art self-supervised methods based on instance discrimination and latent forecasting to the ECG domain. In a first step, we learn co…
A zero-shot RL agent is an agent that can solve any RL task in a given environment, instantly with no additional planning or learning, after an initial reward-free learning phase. This marks a shift from the reward-centric RL paradigm towards "controllable" agents that can follow arbitrary instructions in an environment. Current RL agents can solve families of related tasks at best, or require planning anew for each task. Strategies for approximate zero-shot RL ave been suggested using successor features (SFs) [BBQ+ 18] or forward-backward (FB) representations [TO21], but testing has been limi…
