The basic principles in designing convolutional neural network (CNN) structures for predicting objects on different levels, e.g., image-level, region-level, and pixel-level are diverging. Generally, network structures designed specifically for image classification are directly used as default backbone structure for other tasks including detection and segmentation, but there is seldom backbone structure designed under the consideration of unifying the advantages of networks designed for pixel-level or region-level predicting tasks, which may require very deep features with high resolution. Towards this goal, we design a fish-like network, called FishNet. In FishNet, the information of all resolutions is preserved and refined for the final task. Besides, we observe that existing works still cannot \emph{directly} propagate the gradient information from deep layers to shallow layers. Our design can better handle this problem. Extensive experiments have been conducted to demonstrate the remarkable performance of the FishNet. In particular, on ImageNet-1k, the accuracy of FishNet is able to surpass the performance of DenseNet and ResNet with fewer parameters. FishNet was applied as one of the modules in the winning entry of the COCO Detection 2018 challenge. The code is available at https://github.com/kevin-ssy/FishNet.
Nanjing University（南京大学）；The University of Hong Kong（香港大学）；Nanjing University of Science and Technology（南京理工大学）；IIAI（智能信息处理与应用研究院）；SenseTime Research（商汤科技研究院）
Although using convolutional neural networks (CNNs) as backbones achieves great successes in computer vision, this work investigates a simple backbone network useful for many dense prediction tasks without convolutions. Unlike the recently-proposed Transformer model (e.g., ViT) that is specially designed for image classification, we propose Pyramid Vision Transformer~(PVT), which overcomes the difficulties of porting Transformer to various dense prediction tasks. PVT has several merits compared to prior arts. (1) Different from ViT that typically has low-resolution outputs and high computation…
3D object detection using LiDAR data is an indispensable component for autonomous driving systems. Yet, only a few LiDAR-based 3D object detection methods leverage segmentation information to further guide the detection process. In this paper, we propose a novel multi-task framework that jointly performs 3D object detection and panoptic segmentation. In our method, the 3D object detection backbone in Bird's-Eye-View (BEV) plane is augmented by the injection of Range-View (RV) feature maps from the 3D panoptic segmentation backbone. This enables the detection backbone to leverage multi-view inf…
Sichuan University（四川大学）；Chengdu PU Chip Science and Technology Company, Ltd.（成都普芯科技有限公司）
In this paper, we focus on devising a versatile framework for dense pixelwise prediction whose goal is to assign a discrete or continuous label to each pixel for an image. It is well-known that the reduced feature resolution due to repeated subsampling operations poses a serious challenge to Fully Convolutional Network (FCN) based models. In contrast to the commonly-used strategies, such as dilated convolution and encoder-decoder structure, we introduce the Flattening Module to produce high-resolution predictions without either removing any subsampling operations or building a complicated deco…
The University of Hong Kong（香港大学）；SenseTime Research（商汤科技研究院）；Shanghai AI Laboratory（上海人工智能实验室）
This paper presents a simple MLP-like architecture, CycleMLP, which is a versatile backbone for visual recognition and dense predictions. As compared to modern MLP architectures, e.g., MLP-Mixer, ResMLP, and gMLP, whose architectures are correlated to image size and thus are infeasible in object detection and segmentation, CycleMLP has two advantages compared to modern approaches. (1) It can cope with various image sizes. (2) It achieves linear computational complexity to image size by using local windows. In contrast, previous MLPs have $O(N^2)$ computations due to fully spatial connections. …
Pre-training Everywhere: Parameter-Efficient Fine-Tuning for Medical Image Analysis via Target Parameter Pre-training
School of Computer Science and Engineering, Northwestern Polytechnical University（西北工业大学计算机科学与工程学院）；Shandong Artificial Intelligence Institute, Qilu University of Technology（齐鲁工业大学山东人工智能研究院）；School of Computer Science, The University of Sydney（悉尼大学计算机科学学院）
Santa Clara University（圣克拉拉大学）；ETH Zürich（苏黎世联邦理工学院）；Westlake University（西湖大学）；Northeastern University（东北大学）
The emerging technology of snapshot compressive imaging (SCI) enables capturing high dimensional (HD) data in an efficient way. It is generally implemented by two components: an optical encoder that compresses HD signals into a 2D measurement and an algorithm decoder that retrieves the HD data upon the hardware-encoded measurement. Over a broad range of SCI applications, hyperspectral imaging (HSI) and video compressive sensing have received significant research attention in recent years. Among existing SCI reconstruction algorithms, deep learning-based methods stand out as their promising per…
Frozen-DETR: Enhancing DETR with Image Understanding from Frozen Foundation Models
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Tongyi Lab, Alibaba Group（阿里巴巴集团通义实验室）；Peng Cheng Laboratory（鹏城实验室）；Pazhou Laboratory (Huangpu)（琶洲实验室（黄埔））
UniGP: Taming Diffusion Transformer for Prior-Preserved Unified Generation and Perception
The Hong Kong University of Science and Technology（香港科学与技术大学）；DAMO Academy, Alibaba Group（阿里云达摩院）；Hupan Lab（虎扑实验室）；Zhejiang University（浙江大学）；Tsinghua University（清华大学）；The Chinese University of Hong Kong（香港中文大学）；Zeekr Automobile R&D Co., Ltd.（小鹏汽车研发有限公司）
HybridMIM: A Hybrid Masked Image Modeling Framework for 3D Medical Image Segmentation
Medical College of Tianjin University（天津大学医学院）；The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The University of Hong Kong（香港大学）；Haihe Hospital, Tianjin University（天津大学海河医院）；College of Computing and Intelligence, Tianjin University（天津大学计算与智能学院）
Revisiting Feature Prediction for Learning Visual Representations from Video
FAIR at Meta（Meta FAIR实验室）；Inria（法国国家信息与自动化研究所）；École normale supérieure, CNRS, PSL Research University（巴黎高等师范学院、法国国家科学研究中心、PSL大学）；Univ. Gustave Eiffel, CNRS, LIGM（古斯塔夫·埃菲尔大学、法国国家科学研究中心、LIGM实验室）；Courant Institute, New York University（纽约大学柯朗数学科学研究所）；Center for Data Science, New York University（纽约大学数据科学中心）
GuidedMorph: Two-Stage Deformable Registration for Breast MRI
Department of Electrical and Computer Engineering, Duke University（电子工程系，杜克大学）；Department of Biostatistics & Bioinformatics, Duke University（生物统计学与生物信息学系，杜克大学）；Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida（健康结果与生物医学信息学系，医学院，佛罗里达大学）
GeoChat: Grounded Large Vision-Language Model for Remote Sensing
Mohamed bin Zayed University of AI（穆罕默德·本·扎耶德人工智能大学）；Birla Institute of Technology & Science, Hyderabad（海得拉巴贝拉理工学院）；Australian National University（澳大利亚国立大学）；Linköping University（林雪平大学）
