In this study, we introduce a measure for machine perception, inspired by the concept of Just Noticeable Difference (JND) of human perception. Based on this measure, we suggest an adversarial image generation algorithm, which iteratively distorts an image by an additive noise until the model detects the change in the image by outputting a false label. The noise added to the original image is defined as the gradient of the cost function of the model. A novel cost function is defined to explicitly minimize the amount of perturbation applied to the input image while enforcing the perceptual similarity between the adversarial and input images. For this purpose, the cost function is regularized by the well-known total variation and bounded range terms to meet the natural appearance of the adversarial image. We evaluate the adversarial images generated by our algorithm both qualitatively and quantitatively on CIFAR10, ImageNet, and MS COCO datasets. Our experiments on image classification and object detection tasks show that adversarial images generated by our JND method are both more successful in deceiving the recognition/detection models and less perturbed compared to the images generated by the state-of-the-art methods, namely, FGV, FSGM, and DeepFool methods.
Just Noticeable Difference Modeling for Deep Visual Features
College of Computing and Data Science, Nanyang Technological University, Singapore（新加坡南洋理工大学计算与数据科学学院）；Department of Computer Science and Technology, Harbin Institute of Technology, Harbin, China（哈尔滨工业大学计算机科学与技术学院）；School of Artificial Intelligence, Beijing Normal University, Beijing, China（北京师范大学人工智能学院）；School of Computer Science, Peking University, Beijing, China（北京大学计算机学院）
Zhejiang University（浙江大学）；The University of Technology, Sydney（悉尼科技大学）
Scene classification is a fundamental perception task for environmental understanding in today's robotics. In this paper, we have attempted to exploit the use of popular machine learning technique of deep learning to enhance scene understanding, particularly in robotics applications. As scene images have larger diversity than the iconic object images, it is more challenging for deep learning methods to automatically learn features from scene images with less samples. Inspired by human scene understanding based on object knowledge, we address the problem of scene classification by encouraging d…
Perception-Consistency Multimodal Large Language Models Reasoning via Caption-Regularized Policy Optimization
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（多模态人工智能系统国家重点实验室，自动化研究所，中国科学院）；Pengcheng Laboratory（鹏城实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））
Just Noticeable Difference Modeling for Token Compression in Vision-Language-Action Models
The Hong Kong Polytechnic University（香港理工大学）；Nanyang Technological University（南洋理工大学）；Bytedance Inc.（字节跳动公司）；Shandong University（山东大学）；CNRS（法国国家科学研究中心）；CentraleSupelec（中央理工学院）；Université Paris-Saclay（巴黎萨克雷大学）
Just Noticeable Difference for Large Multimodal Models
Institute of Image Communication and Information Processing, Shanghai Jiao Tong University, Shanghai 200240, China（上海交通大学图像通信与信息处理研究所）；Shanghai AI Laboratory, Shanghai 200232, China（上海人工智能实验室）；School of Communication and Electronic Engineering, East China Normal University, Shanghai 200241, China（华东师范大学通信与电子工程学院）；School of Computer Science and Engineering, Nanyang Technological University, Singapore 639798（南洋理工大学计算机科学与工程学院）
University of British Columbia（英属哥伦比亚大学）；Department of Statistics, University of British Columbia（英属哥伦比亚大学统计系）；Department of Electrical and Computer Engineering, University of British Columbia（英属哥伦比亚大学电气与计算机工程系）；School of Information and Communications Engineering, Xi’an Jiaotong University（西安交通大学信息与通信工程学院）
Recently, generative adversarial networks (GANs) can generate photo-realistic fake facial images which are perceptually indistinguishable from real face photos, promoting research on fake face detection. Though fake face forensics can achieve high detection accuracy, their anti-forensic counterparts are less investigated. Here we explore more \textit{imperceptible} and \textit{transferable} anti-forensics for fake face imagery detection based on adversarial attacks. Since facial and background regions are often smooth, even small perturbation could cause noticeable perceptual impairment in fak…
Learning image representations to capture fine-grained semantics has been a challenging and important task enabling many applications such as image search and clustering. In this paper, we present Graph-Regularized Image Semantic Embedding (Graph-RISE), a large-scale neural graph learning framework that allows us to train embeddings to discriminate an unprecedented O(40M) ultra-fine-grained semantic labels. Graph-RISE outperforms state-of-the-art image embedding algorithms on several evaluation tasks, including image classification and triplet ranking. We provide case studies to demonstrate th…
Contourlet Refinement Gate Framework for Thermal Spectrum Distribution Regularized Infrared Image Super-Resolution
School of Computer Science, Northwestern Polytechnical University（西北工业大学计算机学院）；The National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology（空天地一体化大数据应用技术国家工程实验室）；Graduate School of Information, Production and Systems, Waseda University（早稻田大学信息生产系统研究科）；School of Software, Dalian University of Technology（大连理工大学软件学院）；School of Mechanical Engineering, Dalian University of Technology（大连理工大学机械工程学院）
Based on the notion of just noticeable differences (JND), a stair quality function (SQF) was recently proposed to model human perception on JPEG images. Furthermore, a k-means clustering algorithm was adopted to aggregate JND data collected from multiple subjects to generate a single SQF. In this work, we propose a new method to derive the SQF using the Gaussian Mixture Model (GMM). The newly derived SQF can be interpreted as a way to characterize the mean viewer experience. Furthermore, it has a lower information criterion (BIC) value than the previous one, indicating that it offers a better …
In subjective full-reference image quality assessment, differences between perceptual image qualities of the reference image and its distorted versions are evaluated, often using degradation category ratings (DCR). However, the DCR has been criticized since differences between rating categories on this ordinal scale might not be perceptually equidistant, and observers may have different understandings of the categories. Pair comparisons (PC) of distorted images, followed by Thurstonian reconstruction of scale values, overcome these problems. In addition, PC is more sensitive than DCR, and it c…
Rain removal aims to remove rain streaks from images/videos and reduce the disruptive effects caused by rain. It not only enhances image/video visibility but also allows many computer vision algorithms to function properly. This paper makes the first attempt to conduct a comprehensive study on the robustness of deep learning-based rain removal methods against adversarial attacks. Our study shows that, when the image/video is highly degraded, rain removal methods are more vulnerable to the adversarial attacks as small distortions/perturbations become less noticeable or detectable. In this paper…
