Quantization has been applied to multiple domains in Deep Neural Networks (DNNs). We propose Depthwise Quantization (DQ) where $\textit{quantization}$ is applied to a decomposed sub-tensor along the $\textit{feature axis}$ of weak statistical dependence. The feature decomposition leads to an exponential increase in $\textit{representation capacity}$ with a linear increase in memory and parameter cost. In addition, DQ can be directly applied to existing encoder-decoder frameworks without modification of the DNN architecture. We use DQ in the context of Hierarchical Auto-Encoder and train end-to-end on an image feature representation. We provide an analysis on cross-correlation between spatial and channel features and we propose a decomposition of the image feature representation along the channel axis. The improved performance of the depthwise operator is due to the increased representation capacity from implicit feature decoupling. We evaluate DQ on the likelihood estimation task, where it outperforms the previous state-of-the-art on CIFAR-10, ImageNet-32 and ImageNet-64. We progressively train with increasing image size a single hierarchical model that uses 69% less parameters and has a faster convergence than the previous works.
Depthwise separable convolution has shown great efficiency in network design, but requires time-consuming training procedure with full training-set available. This paper first analyzes the mathematical relationship between regular convolutions and depthwise separable convolutions, and proves that the former one could be approximated with the latter one in closed form. We show depthwise separable convolutions are principal components of regular convolutions. And then we propose network decoupling (ND), a training-free method to accelerate convolutional neural networks (CNNs) by transferring pre…
Decoupling Dark Knowledge via Block-wise Logit Distillation for Feature-level Alignment
College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息与电子工程学院）；ZJU-UIUC Institute, Zhejiang University（浙江大学-伊利诺伊大学厄巴纳-香槟分校联合学院）
University of Science and Technology of China（中国科学技术大学）；GuangZhou University（广州大学）
Scene text editing (STE) aims to replace text with the desired one while preserving background and styles of the original text. However, due to the complicated background textures and various text styles, existing methods fall short in generating clear and legible edited text images. In this study, we attribute the poor editing performance to two problems: 1) Implicit decoupling structure. Previous methods of editing the whole image have to learn different translation rules of background and text regions simultaneously. 2) Domain gap. Due to the lack of edited real scene text images, the netwo…
Euler-inspired Decoupling Neural Operator for Efficient Pansharpening
School of Software Technology, Zhejiang University（浙江大学软件技术学院）；School of Computer Science and Technology , Chongqing University of Post and Telecommunications（重庆邮电大学计算机科学与技术学院）
University of Helsinki（赫尔辛基大学）；Helsinki University Hospital（赫尔辛基大学医院）；Institute for Molecular Medicine Finland (FIMM)（芬兰分子医学研究所）
Many current neural networks for medical imaging generalise poorly to data unseen during training. Such behaviour can be caused by networks overfitting easy-to-learn, or statistically dominant, features while disregarding other potentially informative features. For example, indistinguishable differences in the sharpness of the images from two different scanners can degrade the performance of the network significantly. All neural networks intended for clinical practice need to be robust to variation in data caused by differences in imaging equipment, sample preparation and patient populations. …
LocalDyGS: Multi-view Global Dynamic Scene Modeling via Adaptive Local Implicit Feature Decoupling
Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology Shenzhen Graduate School, Peking University（广东省超高清沉浸媒体技术重点实验室 香港大学深圳研究生院）；Pengcheng Lab（鹏城实验室）；University of Birmingham（伯明翰大学）
A key challenge in self-supervised video representation learning is how to effectively capture motion information besides context bias. While most existing works implicitly achieve this with video-specific pretext tasks (e.g., predicting clip orders, time arrows, and paces), we develop a method that explicitly decouples motion supervision from context bias through a carefully designed pretext task. Specifically, we take the keyframes and motion vectors in compressed videos (e.g., in H.264 format) as the supervision sources for context and motion, respectively, which can be efficiently extracte…
MEDN: Motion-Emotion Feature Decoupling Network for Micro-Expression Recognition
School of Computer Science and Technology, Xidian University（西安电子科技大学计算机科学与技术学院）；Xi’an Key Laboratory of Big Data and Intelligent Vision, Xi’an, Shaanxi（西安大数据与智能视觉重点实验室）；Key Laboratory of Collaborative Intelligence Systems, Ministry of Education, Xidian University（教育部协同智能系统重点实验室）；National Engineering Research Center of Educational Big Data and the Faculty of Artificial Intelligence in Education, Central China Normal University（教育大数据国家工程研究中心和华中师范大学人工智能教育学院）
University of California, Berkeley（加州大学伯克利分校）；Princeton University（普林斯顿大学）
Iterative refinement -- start with a random guess, then iteratively improve the guess -- is a useful paradigm for representation learning because it offers a way to break symmetries among equally plausible explanations for the data. This property enables the application of such methods to infer representations of sets of entities, such as objects in physical scenes, structurally resembling clustering algorithms in latent space. However, most prior works differentiate through the unrolled refinement process, which can make optimization challenging. We observe that such methods can be made diffe…
School of Computer and Communication Sciences, EPFL（洛桑联邦理工学院计算机与通信科学学院）
Classic image-restoration algorithms use a variety of priors, either implicitly or explicitly. Their priors are hand-designed and their corresponding weights are heuristically assigned. Hence, deep learning methods often produce superior image restoration quality. Deep networks are, however, capable of inducing strong and hardly predictable hallucinations. Networks implicitly learn to be jointly faithful to the observed data while learning an image prior; and the separation of original data and hallucinated data downstream is then not possible. This limits their wide-spread adoption in image r…
Content-decoupled Contrastive Learning-based Implicit Degradation Modeling for Blind Image Super-Resolution
School of Control and Computer Engineering, North China Electric Power University（华北电力大学控制与计算机工程学院）；School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）
Continuous Splatting meets Retinex: Continuous Gaussian Splatting and Implicit Reflectance Modeling for Low-Light Image Enhancement
College of Mechanical and Vehicle Engineering, Chongqing University（重庆大学机械与运载工程学院）；School of Engineering Science, University of Science and Technology of China（中国科学技术大学工程科学学院）；National Innovation Center of Intelligent and Connected Vehicles（国家智能网联汽车创新中心）；School of Vehicle and Mobility, Tsinghua University（清华大学车辆与运载学院）
Decoupling Feature Representations of Ego and Other Modalities for Incomplete Multi-modal Brain Tumor Segmentation
Huazhong University of Science and Technology（华中科技大学）；Wuhan National Laboratory for Optoelectronics（武汉光电国家研究中心）；College of Life Science and Technology, Huazhong University of Science and Technology（华中科技大学生命科学与技术学院）；Union Hospital, Tongji Medical College, Huazhong University of Science and Technology（华中科技大学同济医学院附属协和医院）
Edit-Your-Motion: Space-Time Diffusion Decoupling Learning for Video Motion Editing
Xidian University（西安电子科技大学）；School of Artificial Intelligence（人工智能学院）；Key Laboratory of Intelligent Perception and Image Understanding of the Ministry of Education of China（智能感知与图像理解教育部重点实验室）；International Research Center of Intelligent Perception and Computation（智能感知与计算国际研究中心）
Morphological Decoupling-Based Skeletal Classification for Clinical Assessment of Malocclusion
The Affiliated Stomatological Hospital of Nanjing Medical University（南京医科大学附属口腔医院）；Nanjing Medical University（南京医科大学）；Jiangsu Province Engineering Research Center of Stomatological Translational Medicine（江苏省口腔转化医学工程研究中心）；The Chinese University of Hong Kong（香港中文大学）
Dehaze-GaussianImage: Zero-Shot Dehazing via Efficient 2D Gaussian Splatting Representation
College of Mechanical and Vehicle Engineering, Chongqing University（重庆大学机械与车辆工程学院）；Department of Electrical and Computer Engineering, Carnegie Mellon University（卡内基梅隆大学电气与计算机工程系）；National Innovation Center of Intelligent and Connected Vehicles（国家智能网联汽车创新中心）；School of Vehicle and Mobility, Tsinghua University（清华大学车辆与运载学院）
Referring image segmentation aims to segment the objects referred by a natural language expression. Previous methods usually focus on designing an implicit and recurrent feature interaction mechanism to fuse the visual-linguistic features to directly generate the final segmentation mask without explicitly modeling the localization information of the referent instances. To tackle these problems, we view this task from another perspective by decoupling it into a "Locate-Then-Segment" (LTS) scheme. Given a language expression, people generally first perform attention to the corresponding target i…
