Large-scale pre-trained Vision-Language Models (VLMs) have demonstrated strong few-shot learning capabilities. However, these methods typically learn holistic representations where an image's domain-invariant structure is implicitly entangled with its domain-specific style. This presents an opportunity to further enhance generalization by disentangling these visual cues. In this paper, we propose Fourier-Attentive Representation Learning (FARL), a novel framework that addresses this by explicitly disentangling visual representations using Fourier analysis. The core of our method is a dual cross-attention mechanism, where learnable representation tokens separately query an image's structural features (from the phase spectrum) and stylistic features (from the amplitude spectrum). This process yields enriched, disentangled tokens that are then injected deep into the VLM encoders to guide adaptation. Our design, which includes an asymmetric injection strategy, forces the model to learn a more robust vision-language alignment. Extensive experiments on 15 datasets demonstrate the effectiveness of our approach.
Korea Advanced Institute of Science and Technology（韩国科学技术院）
In this paper, we propose a novel joint deblurring and multi-frame interpolation (DeMFI) framework, called DeMFI-Net, which accurately converts blurry videos of lower-frame-rate to sharp videos at higher-frame-rate based on flow-guided attentive-correlation-based feature bolstering (FAC-FB) module and recursive boosting (RB), in terms of multi-frame interpolation (MFI). The DeMFI-Net jointly performs deblurring and MFI where its baseline version performs feature-flow-based warping with FAC-FB module to obtain a sharp-interpolated frame as well to deblur two center-input frames. Moreover, its e…
Cutting and pasting image segments feels intuitive: the choice of source templates gives artists flexibility in recombining existing source material. Formally, this process takes an image set as input and outputs a collage of the set elements. Such selection from sets of source templates does not fit easily in classical convolutional neural models requiring inputs of fixed size. Inspired by advances in attention and set-input machine learning, we present a novel architecture that can generate in one forward pass image collages of source templates using set-structured representations. This pape…
The past few years have witnessed great progress in the domain of face recognition thanks to advances in deep learning. However, cross pose face recognition remains a significant challenge. It is difficult for many deep learning algorithms to narrow the performance gap caused by pose variations; the main reasons for this relate to the intra-class discrepancy between face images in different poses and the pose imbalances of training datasets. Learning pose-robust features by traversing to the feature space of frontal faces provides an effective and cheap way to alleviate this problem. In this p…
Indian Institute of Technology Gandhinagar（甘地讷格尔印度理工学院）
Competitive diving is a well recognized aquatic sport in which a person dives from a platform or a springboard into the water. Based on the acrobatics performed during the dive, diving is classified into a finite set of action classes which are standardized by FINA. In this work, we propose an attention guided LSTM-based neural network architecture for the task of diving classification. The network takes the frames of a diving video as input and determines its class. We evaluate the performance of the proposed model on a recently introduced competitive diving dataset, Diving48. It contains ove…
Zhengzhou University（郑州大学）；University of Technology Sydney（悉尼科技大学）
Single image deraining (SID) is an important and challenging topic in emerging vision applications, and most of emerged deraining methods are supervised relying on the ground truth (i.e., paired images) in recent years. However, in practice it is rather common to have no un-paired images in real deraining task, in such cases how to remove the rain streaks in an unsupervised way will be a very challenging task due to lack of constraints between images and hence suffering from low-quality recovery results. In this paper, we explore the unsupervised SID task using unpaired data and propose a nove…
Vehicle re-identification (re-ID) focuses on matching images of the same vehicle across different cameras. It is fundamentally challenging because differences between vehicles are sometimes subtle. While several studies incorporate spatial-attention mechanisms to help vehicle re-ID, they often require expensive keypoint labels or suffer from noisy attention mask if not trained with expensive labels. In this work, we propose a dedicated Semantics-guided Part Attention Network (SPAN) to robustly predict part attention masks for different views of vehicles given only image-level semantic labels d…
Centre for Visual Information Technology (CVIT) International Institute of Information Technology, Hyderabad（海得拉巴国际信息技术学院视觉信息技术中心（CVIT））
Precise boundary annotations of image regions can be crucial for downstream applications which rely on region-class semantics. Some document collections contain densely laid out, highly irregular and overlapping multi-class region instances with large range in aspect ratio. Fully automatic boundary estimation approaches tend to be data intensive, cannot handle variable-sized images and produce sub-optimal results for aforementioned images. To address these issues, we propose BoundaryNet, a novel resizing-free approach for high-precision semi-automatic layout annotation. The variable-sized user…
Tsinghua University（清华大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Shanghai Jiao Tong University（上海交通大学）；The Chinese University of Hong Kong（香港中文大学）；Shenzhen University（深圳大学）；Peng Cheng Laboratory（鹏城实验室）；ETH Zürich（苏黎世联邦理工学院）
Alibaba Group（阿里巴巴集团）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
Adapting semantic segmentation models to new domains is an important but challenging problem. Recently enlightening progress has been made, but the performance of existing methods are unsatisfactory on real datasets where the new target domain comprises of heterogeneous sub-domains (e.g., diverse weather characteristics). We point out that carefully reasoning about the multiple modalities in the target domain can improve the robustness of adaptation models. To this end, we propose a condition-guided adaptation framework that is empowered by a special attentive progressive adversarial training …
Illinois Institute of Technology（伊利诺伊理工学院）；Texas State University（德克萨斯州立大学）；Shandong University（山东大学）
With the remarkable success of deep learning recently, efficient network compression algorithms are urgently demanded for releasing the potential computational power of edge devices, such as smartphones or tablets. However, optimal network pruning is a non-trivial task which mathematically is an NP-hard problem. Previous researchers explain training a pruned network as buying a lottery ticket. In this paper, we investigate the Magnitude-Based Pruning (MBP) scheme and analyze it from a novel perspective through Fourier analysis on the deep learning model to guide model designation. Besides expl…
FB-Diff: Fourier Basis-guided Diffusion for Temporal Interpolation of 4D Medical Imaging
Computer Aided Medical Procedures, Technical University of Munich（技术大学慕尼黑计算机辅助医学程序）；Institute of Medical Robotics, Shanghai Jiao Tong University（上海交通大学医学机器人研究所）；Munich Center for Machine Learning, Munich（慕尼黑机器学习中心）；School of Computing, Macquarie University（麦考瑞大学计算机学院）
Case Western Reserve University（凯斯西储大学）；Clarkson University（克拉克森大学）
We propose in this paper a novel two-stage Projection Correction Modeling (PCM) framework for image reconstruction from (non-uniform) Fourier measurements. PCM consists of a projection stage (P-stage) motivated by the multi-scale Galerkin method and a correction stage (C-stage) with an edge guided regularity fusing together the advantages of total variation (TV) and total fractional variation (TFV). The P-stage allows for continuous modeling of the underlying image of interest. The given measurements are projected onto a space in which the image is well represented. We then enhance the reconst…
LarvSeg: Exploring Image Classification Data For Large Vocabulary Semantic Segmentation via Category-wise Attentive Classifier
School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）；Center of Data Science, Peking University（北京大学数据科学中心）；Microsoft Research Asia（微软亚洲研究院）
LISA: Language-guided Interference-aware Spatial-Frequency Attention for Driver Gaze Estimation
School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications（北京邮电大学计算机科学学院（国家级试点软件工程学院））；Key Laboratory of Trustworthy Distributed Computing and Service (BUPT), Ministry of Education（教育部可信分布式计算与服务重点实验室（BUPT））；School of Electrical Engineering, Guangxi University（广西大学电气工程学院）；Zhejiang University（浙江大学）
Tsinghua University（清华大学）；University of Connecticut（康涅狄格大学）
Fourier ptychography (FP) is a newly developed computational imaging approach that achieves both high resolution and wide field of view by stitching a series of low-resolution images captured under angle-varied illumination. So far, many supervised data-driven models have been applied to solve inverse imaging problems. These models need massive amounts of data to train, and are limited by the dataset characteristics. In FP problems, generic datasets are always scarce, and the optical aberration varies greatly under different acquisition conditions. To address these dilemmas, we model the forwa…
Deep Attentive Features for Prostate Segmentation in 3D Transrectal Ultrasound
Shenzhen University（深圳大学）；The Chinese University of Hong Kong（香港中文大学）；The Hong Kong Polytechnic University（香港理工大学）；Sun Yat-Sen University（中山大学）；the First Affiliated Hospital（第一附属医院）
Queensland University of Technology (QUT)（昆士兰科技大学）
Person re-identification (re-ID) concerns the matching of subject images across different camera views in a multi camera surveillance system. One of the major challenges in person re-ID is pose variations across the camera network, which significantly affects the appearance of a person. Existing development data lack adequate pose variations to carry out effective training of person re-ID systems. To solve this issue, in this paper we propose an end-to-end pose-driven attention-guided generative adversarial network, to generate multiple poses of a person. We propose to attentively learn and tr…
Hefei Institute of Physical Science, Chinese Academy of Sciences（中国科学院合肥物质科学研究院）；University of Science and Technology of China（中国科学技术大学）；SenseBrain Technology (SenseTime Research USA)（商汤美国研究院思脑科技）；Nanyang Technological University（南洋理工大学）；Xi’an Jiaotong University（西安交通大学）；Pazhou Laboratory (Huangpu)（琶洲实验室（黄埔））
Existing convolutional neural networks widely adopt spatial down-/up-sampling for multi-scale modeling. However, spatial up-sampling operators (\emph{e.g.}, interpolation, transposed convolution, and un-pooling) heavily depend on local pixel attention, incapably exploring the global dependency. In contrast, the Fourier domain obeys the nature of global modeling according to the spectral convolution theorem. Unlike the spatial domain that performs up-sampling with the property of local similarity, up-sampling in the Fourier domain is more challenging as it does not follow such a local property.…
Few-shot object detection (FSOD) localizes and classifies objects in an image given only a few data samples. Recent trends in FSOD research show the adoption of metric and meta-learning techniques, which are prone to catastrophic forgetting and class confusion. To overcome these pitfalls in metric learning based FSOD techniques, we introduce Attention Guided Cosine Margin (AGCM) that facilitates the creation of tighter and well separated class-specific feature clusters in the classification head of the object detector. Our novel Attentive Proposal Fusion (APF) module minimizes catastrophic for…
In this work, we propose a new approach for efficient edge-preserving image deconvolution. Our algorithm is based on a novel type of explicit image filter - guided filter. The guided filter can be used as an edge-preserving smoothing operator like the popular bilateral filter, but has better behaviors near edges. We propose an efficient iterative algorithm with the decouple of deblurring and denoising steps in the restoration process. In deblurring step, we proposed two cost function which could be computed with fast Fourier transform efficiently. The solution of the first one is used as the g…
King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）
This work introduced a novel GAN architecture for unsupervised image translation on the task of face style transform. A spectral attention-based mechanism is embedded into the design along with spatial attention on the image contents. We proved that neural network has the potential of learning complex transformations such as Fourier transform, within considerable computational cost. The model is trained and tested in comparison to the baseline model, which only uses spatial attention. The performance improvement of our approach is significant especially when the source and target domain includ…
UC San Diego（加州大学圣迭戈分校）；The University of Hong Kong（香港大学）；Tencent AI Lab（腾讯人工智能实验室）
Vision Transformers (ViTs) take all the image patches as tokens and construct multi-head self-attention (MHSA) among them. Complete leverage of these image tokens brings redundant computations since not all the tokens are attentive in MHSA. Examples include that tokens containing semantically meaningless or distractive image backgrounds do not positively contribute to the ViT predictions. In this work, we propose to reorganize image tokens during the feed-forward process of ViT models, which is integrated into ViT during training. For each forward inference, we identify the attentive image tok…
Persistent Homology-Guided Frequency Filtering for Image Compression
North Carolina School of Science and Mathematics（北卡罗来纳科学与数学学校）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
CFSPMNet: Cross-subject Fourier-guided Spatial-Patch Mamba Network for EEG Motor Imagery Decoding in Stroke Patients
School of Artificial Intelligence, Chongqing University of Technology（重庆理工大学人工智能学院）；Chongqing Key Laboratory of Embodied Intelligence Perception and Autonomous Learning for Humanoid Robots（重庆市人形机器人感知与自主学习重点实验室）；Key Laboratory of Advanced Equipment Intelligence of the Chongqing Education Commission（重庆市教育委员会先进设备智能重点实验室）；School of Smart Health, Chongqing Polytechnic University of Electronic Technology（重庆理工大学电子工程学院智能健康学院）；Department of Language Science and Technology, The Hong Kong Polytechnic University（香港理工大学语言科学与技术系）；School of Pharmacy and Bioengineering, Chongqing University of Technology（重庆理工大学药学院与生物工程学院）
The University of Hong Kong（香港大学）；Tencent AI Lab（腾讯人工智能实验室）；University of Oxford（牛津大学）
Studies on self-supervised visual representation learning (SSL) improve encoder backbones to discriminate training samples without labels. While CNN encoders via SSL achieve comparable recognition performance to those via supervised learning, their network attention is under-explored for further improvement. Motivated by the transformers that explore visual attention effectively in recognition scenarios, we propose a CNN Attention REvitalization (CARE) framework to train attentive CNN encoders guided by transformers in SSL. The proposed CARE framework consists of a CNN stream (C-stream) and a …
West Virginia University（西弗吉尼亚大学）；University at Buffalo, State University of New York（纽约州立大学布法罗分校）；The Hong Kong University of Science and Technology（香港科技大学）
Global Positioning Systems (GPS) have played a crucial role in various navigation applications. Nevertheless, localizing the perfect destination within the last few meters remains an important but unresolved problem. Limited by the GPS positioning accuracy, navigation systems always show users a vicinity of a destination, but not its exact location. Street view images (SVI) in maps as an immersive media technology have served as an aid to provide the physical environment for human last-meters wayfinding. However, due to the large diversity of geographic context and acquisition conditions, the …
Shandong University（山东大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；City University of Hong Kong（香港城市大学）；Shandong Jianzhu University（山东建筑大学）
University of Kentucky（肯塔基大学）；Case Western Reserve University（凯斯西储大学）
In compressive sensing, it is challenging to reconstruct image of high quality from very few noisy linear projections. Existing methods mostly work well on piecewise constant images but not so well on piecewise smooth images such as natural images, medical images that contain a lot of details. We propose a two-stage method called GeoCS to recover images with rich geometric information from very limited amount of noisy measurements. The method adopts the shearlet transform that is mathematically proven to be optimal in sparsely representing images containing anisotropic features such as edges, …
