Human-Object Interaction (HOI), as an important problem in computer vision, requires locating the human-object pair and identifying the interactive relationships between them. The HOI instance has a greater span in spatial, scale, and task than the individual object instance, making its detection more susceptible to noisy backgrounds. To alleviate the disturbance of noisy backgrounds on HOI detection, it is necessary to consider the input image information to generate fine-grained anchors which are then leveraged to guide the detection of HOI instances. However, it is challenging for the following reasons. i) how to extract pivotal features from the images with complex background information is still an open question. ii) how to semantically align the extracted features and query embeddings is also a difficult issue. In this paper, a novel end-to-end transformer-based framework (FGAHOI) is proposed to alleviate the above problems. FGAHOI comprises three dedicated components namely, multi-scale sampling (MSS), hierarchical spatial-aware merging (HSAM) and task-aware merging mechanism (TAM). MSS extracts features of humans, objects and interaction areas from noisy backgrounds for HOI instances of various scales. HSAM and TAM semantically align and merge the extracted features and query embeddings in the hierarchical spatial and task perspectives in turn. In the meanwhile, a novel training strategy Stage-wise Training Strategy is designed to reduce the training pressure caused by overly complex tasks done by FGAHOI. In addition, we propose two ways to measure the difficulty of HOI detection and a novel dataset, i.e., HOI-SDC for the two challenges (Uneven Distributed Area in Human-Object Pairs and Long Distance Visual Modeling of Human-Object Pairs) of HOI instances detection.
Existing fine-grained visual categorization methods often suffer from three challenges: lack of training data, large number of fine-grained categories, and high intraclass vs. low inter-class variance. In this work we propose a generic iterative framework for fine-grained categorization and dataset bootstrapping that handles these three challenges. Using deep metric learning with humans in the loop, we learn a low dimensional feature embedding with anchor points on manifolds for each category. These anchor points capture intra-class variances and remain discriminative between classes. In each …
We present a model for temporally precise action spotting in videos, which uses a dense set of detection anchors, predicting a detection confidence and corresponding fine-grained temporal displacement for each anchor. We experiment with two trunk architectures, both of which are able to incorporate large temporal contexts while preserving the smaller-scale features required for precise localization: a one-dimensional version of a u-net, and a Transformer encoder (TE). We also suggest best practices for training models of this kind, by applying Sharpness-Aware Minimization (SAM) and mixup data …
Pose variation and subtle differences in appearance are key challenges to fine-grained classification. While deep networks have markedly improved general recognition, many approaches to fine-grained recognition rely on anchoring networks to parts for better accuracy. Identifying parts to find correspondence discounts pose variation so that features can be tuned to appearance. To this end previous methods have examined how to find parts and extract pose-normalized features. These methods have generally separated fine-grained recognition into stages which first localize parts using hand-engineer…
Peking University（北京大学）；China North Vehicle Research Institute（中国北方车辆研究所）
Road detection or traversability analysis has been a key technique for a mobile robot to traverse complex off-road scenes. The problem has been mainly formulated in early works as a binary classification one, e.g. associating pixels with road or non-road labels. Whereas understanding scenes with fine-grained labels are needed for off-road robots, as scenes are very diverse, and the various mechanical performance of off-road robots may lead to different definitions of safe regions to traverse. How to define and annotate fine-grained labels to achieve meaningful scene understanding for a robot t…
Attention-based learning for fine-grained image recognition remains a challenging task, where most of the existing methods treat each object part in isolation, while neglecting the correlations among them. In addition, the multi-stage or multi-scale mechanisms involved make the existing methods less efficient and hard to be trained end-to-end. In this paper, we propose a novel attention-based convolutional neural network (CNN) which regulates multiple object parts among different input images. Our method first learns multiple attention region features of each input image through the one-squeez…
Learning with noisy labels (LNL) aims to ensure model generalization given a label-corrupted training set. In this work, we investigate a rarely studied scenario of LNL on fine-grained datasets (LNL-FG), which is more practical and challenging as large inter-class ambiguities among fine-grained classes cause more noisy labels. We empirically show that existing methods that work well for LNL fail to achieve satisfying performance for LNL-FG, arising the practical need of effective solutions for LNL-FG. To this end, we propose a novel framework called stochastic noise-tolerated supervised contra…
ACT-Net: Anchor-context Action Detection in Surgery Videos
University of Birmingham（伯明翰大学）；Southern University of Science and Technology（南方科技大学）；Research Institute of Trustworthy Autonomous Systems（可信自主系统研究院）；National University of Singapore（新加坡国立大学）；Third medical center of Chinese PLAGH（中国人民解放军总医院第三医学中心）；Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局高性能计算研究所）
Existing metric learning losses can be categorized into two classes: pair-based and proxy-based losses. The former class can leverage fine-grained semantic relations between data points, but slows convergence in general due to its high training complexity. In contrast, the latter class enables fast and reliable convergence, but cannot consider the rich data-to-data relations. This paper presents a new proxy-based loss that takes advantages of both pair- and proxy-based methods and overcomes their limitations. Thanks to the use of proxies, our loss boosts the speed of convergence and is robust …
Huazhong University of Science and Technology（华中科技大学）；Huawei Tech.（华为技术有限公司）；Baidu Inc.（百度公司）；Microsoft Dynamics 365 AI Research（微软Dynamics 365人工智能研究院）；Zhejiang Gongshang University（浙江工商大学）；Dalian University of Technology（大连理工大学）
Temporal language localization in videos aims to ground one video segment in an untrimmed video based on a given sentence query. To tackle this task, designing an effective model to extract ground-ing information from both visual and textual modalities is crucial. However, most previous attempts in this field only focus on unidirectional interactions from video to query, which emphasizes which words to listen and attends to sentence information via vanilla soft attention, but clues from query-by-video interactions implying where to look are not taken into consideration. In this paper, we propo…
ShanghaiTech University（上海科技大学）；University of Toronto（多伦多大学）；Shanghai Engineering Research Center of Intelligent Vision and Imaging（上海智能视觉与成像工程技术研究中心）
Recent years have witnessed the tremendous progress of 3D GANs for generating view-consistent radiance fields with photo-realism. Yet, high-quality generation of human radiance fields remains challenging, partially due to the limited human-related priors adopted in existing methods. We present HumanGen, a novel 3D human generation scheme with detailed geometry and $\text{360}^{\circ}$ realistic free-view rendering. It explicitly marries the 3D human generation with various priors from the 2D generator and 3D reconstructor of humans through the design of "anchor image". We introduce a hybrid fe…
South China University of Technology（华南理工大学）；Peng Cheng Laboratory（鹏城实验室）
Fine-grained visual classification can be addressed by deep representation learning under supervision of manually pre-defined targets (e.g., one-hot or the Hadamard codes). Such target coding schemes are less flexible to model inter-class correlation and are sensitive to sparse and imbalanced data distribution as well. In light of this, this paper introduces a novel target coding scheme -- dynamic target relation graphs (DTRG), which, as an auxiliary feature regularization, is a self-generated structural output to be mapped from input images. Specifically, online computation of class-level fea…
Sketches have been used to conceptualise and depict visual objects from pre-historic times. Sketch research has flourished in the past decade, particularly with the proliferation of touchscreen devices. Much of the utilisation of sketch has been anchored around the fact that it can be used to delineate visual concepts universally irrespective of age, race, language, or demography. The fine-grained interactive nature of sketches facilitates the application of sketches to various visual understanding tasks, like image retrieval, image-generation or editing, segmentation, 3D-shape modelling etc. …
National University of Singapore（新加坡国立大学）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）
State-of-the-art CNN based recognition models are often computationally prohibitive to deploy on low-end devices. A promising high level approach tackling this limitation is knowledge distillation, which let small student model mimic cumbersome teacher model's output to get improved generalization. However, related methods mainly focus on simple task of classification while do not consider complex tasks like object detection. We show applying the vanilla knowledge distillation to detection model gets minor gain. To address the challenge of distilling knowledge in detection model, we propose a …
BeautyGRPO: Aesthetic Alignment for Face Retouching via Dynamic Path Guidance and Fine-Grained Preference Modeling
School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, China（中山大学计算机科学与技术学院，深圳校区）；vivo BlueImage Lab, vivo Mobile Communication Co., Ltd., China（vivo蓝影实验室，vivo移动通信有限公司）
DRAFE: Domain-Robust Asymmetric Fusion of Heterogeneous Detection Transformers for Cross-City Fine-Grained Traffic Object Detection
Iowa State University（爱荷华州立大学）；University of Arkansas（阿肯色大学）；Parsons Corporation（帕森斯公司）；Jacobs（雅各布斯公司）；University of South Florida（南佛罗里达大学）；Neel-Schaffer, Inc.（尼尔-沙弗公司）
针对跨城市细粒度交通目标检测中细粒度识别与域泛化难兼顾的问题，提出 DRAFE 模型，通过两阶段训练及多模块融合策略，在 AI City Challenge 2026 Track 6 任务中取得 0.4022 mAP 排名第六的成绩，性能优于初步集成模型。
Taming a Retrieval Framework to Read Images in Humanlike Manner for Augmenting Generation of MLLMs
Emory University（埃默里大学）；University of Electronic Science and Technology of China（电子科技大学）；University of Illinois Chicago（伊利诺伊大学香槟分校）；The Hong Kong Polytechnic University（香港理工大学）
Skeleton-Guided Sparse Anchors for Rotated Instance Segmentation in Cell Microscopy
Aier Institute of Digital Ophthalmology and Visual Science（爱尔数字眼科学与视觉科学研究所）；Changsha Aier Eye Hospital（长沙爱尔眼科医院）；Shanghai Jiao Tong University（上海交通大学）；Nanyang Technological University（南洋理工大学）；Obstetrics and Gynecology Hospital, Institute of Reproduction and Development, Fudan University（复旦大学附属妇产科医院 生殖与发育研究院）
FedeCouple: Fine-Grained Balancing of Global-Generalization and Local-Adaptability in Federated Learning
Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology (Shandong Academy of Sciences)（计算能力网络与信息安全部分省计算机科学中心，青岛科技大学（山东省科学院））；Shandong Provincial Key Laboratory of Industrial Network and Information System Security, Shandong Fundamental Research Center for Computer Science（山东省工业网络与信息系统安全省重点实验室，山东省计算机科学基础研究中心）；School of Computer Science and Technology, Shandong University（山东大学计算机科学与技术学院）；School of Computer Science and Cyber Engineering, Guangzhou University（广州大学计算机科学与网络工程学院）；State Key Lab. of Industrial Control Technology, Zhejiang University（浙江省工业控制技术重点实验室，浙江大学）
Contextrast++: Robust Multi-Scale Contextual Contrastive Learning for Semantic Segmentation
Information & Electronics Research Institute, KAIST（韩国科学技术院信息与电子研究院）；Zoox Inc.（Zoox公司）；KAIST (Korea Advanced Institute of Science and Technology)（韩国科学技术院）；Hanwha Aerospace（韩华宇航）；School of Electrical Engineering, KAIST（韩国科学技术院电气工程学院）
