South China University of Technology（华南理工大学）；The University of Adelaide（阿德莱德大学）；Tencent AI Lab（腾讯人工智能实验室）；University of Texas at Arlington（德克萨斯大学阿灵顿分校）
Generative adversarial networks (GANs) aim to generate realistic data from some prior distribution (e.g., Gaussian noises). However, such prior distribution is often independent of real data and thus may lose semantic information (e.g., geometric structure or content in images) of data. In practice, the semantic information might be represented by some latent distribution learned from data, which, however, is hard to be used for sampling in GANs. In this paper, rather than sampling from the pre-defined prior distribution, we propose a Local Coordinate Coding (LCC) based sampling method to improve GANs. We derive a generalization bound for LCC based GANs and prove that a small dimensional input is sufficient to achieve good generalization. Extensive experiments on various real-world datasets demonstrate the effectiveness of the proposed method.
Spiking Neural Networks (SNNs) have shown great potential in solving deep learning problems in an energy-efficient manner. However, they are still limited to simple classification tasks. In this paper, we propose Spiking-GAN, the first spike-based Generative Adversarial Network (GAN). It employs a kind of temporal coding scheme called time-to-first-spike coding. We train it using approximate backpropagation in the temporal domain. We use simple integrate-and-fire (IF) neurons with very high refractory period for our network which ensures a maximum of one spike per neuron. This makes the model …
South China University of Technology（华南理工大学）；Pazhou Laboratory（琶洲实验室）；The University of Adelaide（阿德莱德大学）；Tencent AI Laboratory（腾讯人工智能实验室）；University of Texas at Arlington（德克萨斯大学阿灵顿分校）
Generative adversarial networks (GANs) have shown remarkable success in generating realistic data from some predefined prior distribution (e.g., Gaussian noises). However, such prior distribution is often independent of real data and thus may lose semantic information (e.g., geometric structure or content in images) of data. In practice, the semantic information might be represented by some latent distribution learned from data. However, such latent distribution may incur difficulties in data sampling for GANs. In this paper, rather than sampling from the predefined prior distribution, we prop…
Fluorescence microscopy plays a vital role in understanding the subcellular structures of living cells. However, it requires considerable effort in sample preparation related to chemical fixation, staining, cost, and time. To reduce those factors, we present a virtual fluorescence staining method based on deep neural networks (VirFluoNet) to transform fluorescence images of molecular labels into other molecular fluorescence labels in the same field-of-view. To achieve this goal, we develop and train a conditional generative adversarial network (cGAN) to perform digital fluorescence imaging dem…
CoQui: A Coordinate-Conditioned Quantum Implicit Generative Adversarial Network for End-to-End Image Generation
Shanghai Maritime University（上海海事大学）；Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局）；Singapore Institute of Technology（新加坡理工学院）；Singapore Management University (SMU)（新加坡管理大学）；Tongji University（同济大学）；Tsinghua University（清华大学）
We present TIPSy-GAN, a new approach to improve the accuracy and stability in unsupervised adversarial 2D to 3D human pose estimation. In our work we demonstrate that the human kinematic skeleton should not be assumed as a single spatially codependent structure; in fact, we posit when a full 2D pose is provided during training, there is an inherent bias learned where the 3D coordinate of a keypoint is spatially codependent on the 2D coordinates of all other keypoints. To investigate our hypothesis we follow previous adversarial approaches but train two generators on spatially independent parts…
The end-to-end learning-based video compression has attracted substantial attentions by paving another way to compress video signals as stacked visual features. This paper proposes an efficient end-to-end deep video codec with jointly optimized compression and enhancement modules (JCEVC). First, we propose a dual-path generative adversarial network (DPEG) to reconstruct video details after compression. An $α$-path facilitates the structure information reconstruction with a large receptive field and multi-frame references, while a $β$-path facilitates the reconstruction of local textures. Both …
University of Science and Technology of China（中国科学技术大学）
Versatile Video Coding (H.266/VVC) standard achieves better image quality when keeping the same bits than any other conventional image codec, such as BPG, JPEG, and etc. However, it is still attractive and challenging to improve the image quality with high compression ratio on the basis of traditional coding techniques. In this paper, we design the multi-scale grouped dense network (MSGDN) to further reduce the compression artifacts by combining the multi-scale and grouped dense block, which are integrated as the post-process network of VVC intra coding. Besides, to improve the subjective qual…
RSC-SNN: Exploring the Trade-off Between Adversarial Robustness and Accuracy in Spiking Neural Networks via Randomized Smoothing Coding
Chongqing University（重庆大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Xi’an Jiaotong University（西安交通大学）；Software Security Technology Company Ltd（软件安全技术有限公司）
We observe that despite their hierarchical convolutional nature, the synthesis process of typical generative adversarial networks depends on absolute pixel coordinates in an unhealthy manner. This manifests itself as, e.g., detail appearing to be glued to image coordinates instead of the surfaces of depicted objects. We trace the root cause to careless signal processing that causes aliasing in the generator network. Interpreting all signals in the network as continuous, we derive generally applicable, small architectural changes that guarantee that unwanted information cannot leak into the hie…
Peking University（北京大学）；National Tsinghua University（国立清华大学）；UC San Diego（加州大学圣地亚哥分校）
We propose an adversarial defense method that achieves state-of-the-art performance among attack-agnostic adversarial defense methods while also maintaining robustness to input resolution, scale of adversarial perturbation, and scale of dataset size. Based on convolutional sparse coding, we construct a stratified low-dimensional quasi-natural image space that faithfully approximates the natural image space while also removing adversarial perturbations. We introduce a novel Sparse Transformation Layer (STL) in between the input image and the first layer of the neural network to efficiently proj…
Person re-identification (ReID) aims at matching persons across different views/scenes. In addition to accuracy, the matching efficiency has received more and more attention because of demanding applications using large-scale data. Several binary coding based methods have been proposed for efficient ReID, which either learn projections to map high-dimensional features to compact binary codes, or directly adopt deep neural networks by simply inserting an additional fully-connected layer with tanh-like activations. However, the former approach requires time-consuming hand-crafted feature extract…
SecDiff: Diffusion-Aided Secure Deep Joint Source-Channel Coding Against Adversarial Attacks
College of Computing and Data Science, Nanyang Technological University（计算与数据科学学院，南洋理工大学）；Department of Electrical and Electronic Engineering, University of Hong Kong（电子与电气工程系，香港大学）；School of Electronics, Electrical Engineering and Computer Science, Queen’s University Belfast（电子、电气工程与计算机科学学院，女王学院贝尔法斯特分校）；Department of Electrical and Computer Engineering, Sungkyunkwan University（电气与计算机工程系，成均馆大学）；State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications（网络与交换技术国家重点实验室，北京邮电大学）
Johns Hopkins University（约翰斯·霍普金斯大学）；Technion Israel Institute of Technology（以色列理工学院）
The Convolutional Sparse Coding (CSC) model has recently gained considerable traction in the signal and image processing communities. By providing a global, yet tractable, model that operates on the whole image, the CSC was shown to overcome several limitations of the patch-based sparse model while achieving superior performance in various applications. Contemporary methods for pursuit and learning the CSC dictionary often rely on the Alternating Direction Method of Multipliers (ADMM) in the Fourier domain for the computational convenience of convolutions, while ignoring the local characteriza…
Recent Spiking Neural Networks (SNNs) works focus on an image classification task, therefore various coding techniques have been proposed to convert an image into temporal binary spikes. Among them, rate coding and direct coding are regarded as prospective candidates for building a practical SNN system as they show state-of-the-art performance on large-scale datasets. Despite their usage, there is little attention to comparing these two coding schemes in a fair manner. In this paper, we conduct a comprehensive analysis of the two codings from three perspectives: accuracy, adversarial robustnes…
Sun Yat-Sen University（中山大学）；National University of Singapore（新加坡国立大学）；QiHoo 360 Artificial intelligence institute（奇虎360人工智能研究院）
As the rapid growth of multi-modal data, hashing methods for cross-modal retrieval have received considerable attention. Deep-networks-based cross-modal hashing methods are appealing as they can integrate feature learning and hash coding into end-to-end trainable frameworks. However, it is still challenging to find content similarities between different modalities of data due to the heterogeneity gap. To further address this problem, we propose an adversarial hashing network with attention mechanism to enhance the measurement of content similarities by selectively focusing on informative parts…
Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）；Swarthmore College（斯沃斯莫尔学院）；Villanova University（维拉诺瓦大学）
Although deep learning has shown great success in recent years, researchers have discovered a critical flaw where small, imperceptible changes in the input to the system can drastically change the output classification. These attacks are exploitable in nearly all of the existing deep learning classification frameworks. However, the susceptibility of deep sparse coding models to adversarial examples has not been examined. Here, we show that classifiers based on a deep sparse coding model whose classification accuracy is competitive with a variety of deep neural network models are robust to adve…
Inter-LPCM: Learning-based Inter-Frame Predictive Coding for LiDAR Point Cloud Compression
School of Control Science and Engineering, Shandong University（控制科学与工程学院，山东大学）；Key Laboratory of Machine Intelligence and System Control, Ministry of Education（教育部机器智能与系统控制重点实验室）；School of Computer Science and Technology, Shandong University（计算机科学与技术学院，山东大学）；School of Engineering and Sustainable Development, De Montfort University（工程与可持续发展学院，德蒙福特大学）
Targeted perturbations reveal brain-like local coding axes in robustified, but not standard, ANN-based brain models
Center for Excellence in Computational Cognition, Georgia Tech（计算认知卓越中心，佐治亚理工学院）；School of Mathematics, Georgia Tech（数学学院，佐治亚理工学院）；School of Psychology, Georgia Tech（心理学学院，佐治亚理工学院）
研究用小规模对抗探针检验基于 ANN 的视觉脑模型，发现鲁棒模型具有更稳定、可泛化且类脑的局部编码轴，可作为超越预测准确率的模型评估判据。
Soochow University（苏州大学）；Hefei University of Technology（合肥工业大学）；Adobe Research（奥多比研究院）；Nanjing University of Information Science and Technology（南京信息工程大学）；National University of Singapore（新加坡国立大学）
We investigate the high-dimensional data clustering problem by proposing a novel and unsupervised representation learning model called Robust Flexible Auto-weighted Local-coordinate Concept Factorization (RFA-LCF). RFA-LCF integrates the robust flexible CF, robust sparse local-coordinate coding and the adaptive reconstruction weighting learning into a unified model. The adaptive weighting is driven by including the joint manifold preserving constraints on the recovered clean data, basis concepts and new representation. Specifically, our RFA-LCF uses a L2,1-norm based flexible residue to encode…
Beijing University of Posts and Telecommunications（北京邮电大学）；Peng Cheng Laboratory（鹏城实验室）
As one novel approach to realize end-to-end wireless image semantic transmission, deep learning-based joint source-channel coding (deep JSCC) method is emerging in both deep learning and communication communities. However, current deep JSCC image transmission systems are typically optimized for traditional distortion metrics such as peak signal-to-noise ratio (PSNR) or multi-scale structural similarity (MS-SSIM). But for low transmission rates, due to the imperfect wireless channel, these distortion metrics lose significance as they favor pixel-wise preservation. To account for human visual pe…
School of Computer Science and Technology, Soochow University（苏州大学计算机科学与技术学院）；Department of Computer Science, University of Georgia（佐治亚大学计算机科学系）
Concept Factorization (CF) and its variants may produce inaccurate representation and clustering results due to the sensitivity to noise, hard constraint on the reconstruction error and pre-obtained approximate similarities. To improve the representation ability, a novel unsupervised Robust Flexible Auto-weighted Local-coordinate Concept Factorization (RFA-LCF) framework is proposed for clustering high-dimensional data. Specifically, RFA-LCF integrates the robust flexible CF by clean data space recovery, robust sparse local-coordinate coding and adaptive weighting into a unified model. RFA-LCF…
