Boosting Single-domain Generalized Object Detection via Vision-Language Knowledge Interaction
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences（先进交叉学科研究院，中国科学院大学）；Institute of Microelectronics of the Chinese Academy of Sciences（中国科学院微电子研究所）
Single-Domain Generalized Object Detection~(S-DGOD) aims to train an object detector on a single source domain while generalizing well to diverse unseen target domains, making it suitable for multimedia applications that involve various domain shifts, such as intelligent video surveillance and VR/AR technologies. With the success of large-scale Vision-Language Models, recent S-DGOD approaches exploit pre-trained vision-language knowledge to guide invariant feature learning across visual domains. However, the utilized knowledge remains at a coarse-grained level~(e.g., the textual description of adverse weather paired with the image) and serves as an implicit regularization for guidance, struggling to learn accurate region- and object-level features in varying domains. In this work, we propose a new cross-modal feature learning method, which can capture generalized and discriminative regional features for S-DGOD tasks. The core of our method is the mechanism of Cross-modal and Region-aware Feature Interaction, which simultaneously learns both inter-modal and intra-modal regional invariance through dynamic interactions between fine-grained textual and visual features. Moreover, we design a simple but effective strategy called Cross-domain Proposal Refining and Mixing, which aligns the position of region proposals across multiple domains and diversifies them, enhancing the localization ability of detectors in unseen scenarios. Our method achieves new state-of-the-art results on S-DGOD benchmark datasets, with improvements of +8.8\%~mPC on Cityscapes-C and +7.9\%~mPC on DWD over baselines, demonstrating its efficacy.
Australian National University（澳大利亚国立大学）；Beihang University（北京航空航天大学）；NICTA（国家信息与通信技术研究院）
Real-time object detection is one of the core problems in computer vision. The cascade boosting framework proposed by Viola and Jones has become the standard for this problem. In this framework, the learning goal for each node is asymmetric, which is required to achieve a high detection rate and a moderate false positive rate. We develop new boosting algorithms to address this asymmetric learning problem. We show that our methods explicitly optimize asymmetric loss objectives in a totally corrective fashion. The methods are totally corrective in the sense that the coefficients of all selected …
Boosting Gaze Object Prediction via Pixel-level Supervision from Vision Foundation Model
Xi’an University of Architecture and Technology（西安建筑科技大学）；School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）
NICTA（澳大利亚国家信息与通信技术研究院）；Australian National University（澳大利亚国立大学）；Beihang University（北京航空航天大学）
Object detection is one of the key tasks in computer vision. The cascade framework of Viola and Jones has become the de facto standard. A classifier in each node of the cascade is required to achieve extremely high detection rates, instead of low overall classification error. Although there are a few reported methods addressing this requirement in the context of object detection, there is no a principled feature selection method that explicitly takes into account this asymmetric node learning objective. We provide such a boosting algorithm in this work. It is inspired by the linear asymmetric …
Temple University（天普大学）；University of Pittsburgh（匹兹堡大学）；Toyota Research Institute（丰田研究院）；TuSimple（图森未来）
Pavement crack detection is a critical task for insuring road safety. Manual crack detection is extremely time-consuming. Therefore, an automatic road crack detection method is required to boost this progress. However, it remains a challenging task due to the intensity inhomogeneity of cracks and complexity of the background, e.g., the low contrast with surrounding pavements and possible shadows with similar intensity. Inspired by recent advances of deep learning in computer vision, we propose a novel network architecture, named Feature Pyramid and Hierarchical Boosting Network (FPHBN), for pa…
Convolution Neural Networks (CNN) have been extremely successful in solving intensive computer vision tasks. The convolutional filters used in CNNs have played a major role in this success, by extracting useful features from the inputs. Recently researchers have tried to boost the performance of CNNs by re-calibrating the feature maps produced by these filters, e.g., Squeeze-and-Excitation Networks (SENets). These approaches have achieved better performance by Exciting up the important channels or feature maps while diminishing the rest. However, in the process, architectural complexity has in…
Convolutional Neural Networks (CNNs) have dominated computer vision for years, due to its ability in capturing locality and translation invariance. Recently, many vision transformer architectures have been proposed and they show promising performance. A key component in vision transformers is the fully-connected self-attention which is more powerful than CNNs in modelling long range dependencies. However, since the current dense self-attention uses all image patches (tokens) to compute attention matrix, it may neglect locality of images patches and involve noisy tokens (e.g., clutter backgroun…
Hyper Adversarial Tuning for Boosting Adversarial Robustness of Pretrained Large Vision Models
Zhejiang University（浙江大学）；Shanghai Maritime University（上海海事大学）；Zhejiang Gongshang University（浙江工商大学）；Nanjing University of Information Science and Technology（南京信息工程大学）
Boosting Vision-Language Models for Histopathology Classification: Predict all at once
Université Catholique de Louvain (UCLouvain)（天主教鲁汶大学（UCLouvain））；Université de Mons (UMons)（蒙斯大学（UMons））；École de Technologie Supérieure (ÉTS)（高等技术学院（ÉTS））；Centre de Recherche du Centre Hospitalier de l’Université de Montréal (CRCHUM)（蒙特利尔大学中心医院研究中心（CRCHUM））
Knowledge Boosting: Rethinking Medical Contrastive Vision-Language Pre-Training
Southeast University（东南大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Case Western Reserve University（凯斯西储大学）；Centre de Recherche en Information Biomédicale Sino-Français（中法生物医学信息研究中心）
FullLoRA: Efficiently Boosting the Robustness of Pretrained Vision Transformers
Key Laboratory of AI Safety of CAS, Institute of Computing Technology, Chinese Academy of Sciences (CAS), Beijing 100190, China, and the University of Chinese Academy of Sciences, Beijing 100049, China（中国科学院计算技术研究所人工智能安全重点实验室，中国科学院（CAS）及中国科学院大学，北京）
Set-level Guidance Attack: Boosting Adversarial Transferability of Vision-Language Pre-training Models
Southern University of Science and Technology（南方科技大学）；The University of Hong Kong（香港大学）；Monash University（莫纳什大学）；Temple University（天普大学）；Peng Cheng Laboratory（鹏城实验室）
Boosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learning
School of Information Science and Technology, University of Science and Technology of China（中国科学技术大学信息科学技术学院）；Intelligent System Department, Zhongxing Telecom Equipment(ZTE)（中兴通讯股份有限公司智能系统部）；Northeastern University（东北大学）
Dealicious Inc.（Dealicious公司）；Institute of Advanced Research on Artificial Intelligence (IARAI)（人工智能高等研究院）；Athena RC（雅典娜研究中心）
Vision transformers have achieved remarkable progress in vision tasks such as image classification and detection. However, in instance-level image retrieval, transformers have not yet shown good performance compared to convolutional networks. We propose a number of improvements that make transformers outperform the state of the art for the first time. (1) We show that a hybrid architecture is more effective than plain transformers, by a large margin. (2) We introduce two branches collecting global (classification token) and local (patch tokens) information, from which we form a global image re…
DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）；Huazhong University of Science and Technology（华中科技大学）
Recently, vision-language joint representation learning has proven to be highly effective in various scenarios. In this paper, we specifically adapt vision-language joint learning for scene text detection, a task that intrinsically involves cross-modal interaction between the two modalities: vision and language, since text is the written form of language. Concretely, we propose to learn contextualized, joint representations through vision-language pre-training, for the sake of enhancing the performance of scene text detectors. Towards this end, we devise a pre-training architecture with an ima…
DM-KG: A Novel Method for Boosting Spatial Cognition of Vision-Language Models in Street View Imagery
Institute of Surveying and Mapping, Information Engineering University（信息工程大学测绘学院）；Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences（中国科学院地理科学与资源研究所）
Boosting Generative Adversarial Transferability with Self-supervised Vision Transformer Features
School of Cyberspace Science and Technology, Beijing Institute of Technology（网络安全科学与技术学院，北京理工大学）；School of Computer Science and Technology, Beijing Institute of Technology（计算机科学与技术学院，北京理工大学）
Boosting 3D Neuron Segmentation with 2D Vision Transformer Pre-trained on Natural Images
School of Computer Science, The University of Sydney（悉尼大学计算机科学学院）；Data61, CSIRO（澳大利亚联邦科学与工业研究组织Data61）；SEU-ALLEN Joint Center, Institute for Brain and Intelligence, Southeast University（东南大学SEU-ALLEN联合中心脑与智能研究院）
Boosting Vision Semantic Density with Anatomy Normality Modeling for Medical Vision-language Pre-training
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）；The First Affiliated Hospital of College of Medicine, Zhejiang University（浙江大学医学院附属第一医院）；Hupan Lab（虎盘实验室）
Due to the limits of bandwidth and storage space, digital images are usually down-scaled and compressed when transmitted over networks, resulting in loss of details and jarring artifacts that can lower the performance of high-level visual tasks. In this paper, we aim to generate an artifact-free high-resolution image from a low-resolution one compressed with an arbitrary quality factor by exploring joint compression artifacts reduction (CAR) and super-resolution (SR) tasks. First, we propose a context-aware joint CAR and SR neural network (CAJNN) that integrates both local and non-local featur…
TinyAlign: Boosting Lightweight Vision-Language Models by Mitigating Modal Alignment Bottlenecks
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing（北京未来区块链与隐私计算先进创新中心）；Beihang University（北京航空航天大学）；Hangzhou International Innovation Institute（杭州国际创新研究院）；Xreal；Renmin University（中国人民大学）；Peking University（北京大学）；Beijing Academy of Blockchain and Edge Computing (BABEC)（北京区块链与边缘计算研究院）
IRPO: Boosting Image Restoration via Post-training GRPO
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Tsinghua University（清华大学）；Technical University of Munich（慕尼黑技术大学）；Zhejiang University（浙江大学）；Shanghai Jiao Tong University（上海交通大学）；Fudan University（复旦大学）；National University of Singapore（新加坡国立大学）
Automatically describing an image with a natural language has been an emerging challenge in both fields of computer vision and natural language processing. In this paper, we present Long Short-Term Memory with Attributes (LSTM-A) - a novel architecture that integrates attributes into the successful Convolutional Neural Networks (CNNs) plus Recurrent Neural Networks (RNNs) image captioning framework, by training them in an end-to-end manner. To incorporate attributes, we construct variants of architectures by feeding image representations and attributes into RNNs in different ways to explore th…
Boosting Quantitive and Spatial Awareness for Zero-Shot Object Counting
Northwestern Polytechnical University（西北工业大学）；Institute of Artificial Intelligence (TeleAI), China Telecom（中国电信人工智能研究院）；University of Science and Technology of China（中国科学技术大学）；Fudan University（复旦大学）
Boosting Medical Vision-Language Pretraining via Momentum Self-Distillation under Limited Computing Resources
Faculty of Information Technology, University of Science, Ho Chi Minh City, Vietnam（信息科技学院，科学大学，胡志明市，越南）；Vietnam National University, Ho Chi Minh City, Vietnam（越南国家大学，胡志明市，越南）
