Learnable Graph Matching: Incorporating Graph Partitioning with Deep Feature Learning for Multiple Object Tracking
Institute of Automation Chinese Academy of Sciences (CASIA)（中国科学院自动化研究所）；TuSimple（图森未来）；School of Artificial Intelligence University of Chinese Academy of Sciences (UCAS)（中国科学院大学人工智能学院）；Centre for Artificial Intelligence and Robotics HKISI_CAS（香港中科院人工智能与机器人中心）
Data association across frames is at the core of Multiple Object Tracking (MOT) task. This problem is usually solved by a traditional graph-based optimization or directly learned via deep learning. Despite their popularity, we find some points worth studying in current paradigm: 1) Existing methods mostly ignore the context information among tracklets and intra-frame detections, which makes the tracker hard to survive in challenging cases like severe occlusion. 2) The end-to-end association methods solely rely on the data fitting power of deep neural networks, while they hardly utilize the advantage of optimization-based assignment methods. 3) The graph-based optimization methods mostly utilize a separate neural network to extract features, which brings the inconsistency between training and inference. Therefore, in this paper we propose a novel learnable graph matching method to address these issues. Briefly speaking, we model the relationships between tracklets and the intra-frame detections as a general undirected graph. Then the association problem turns into a general graph matching between tracklet graph and detection graph. Furthermore, to make the optimization end-to-end differentiable, we relax the original graph matching into continuous quadratic programming and then incorporate the training of it into a deep graph network with the help of the implicit function theorem. Lastly, our method GMTracker, achieves state-of-the-art performance on several standard MOT datasets. Our code will be available at https://github.com/jiaweihe1996/GMTracker .
Peking University（北京大学）；Peng Cheng Laboratory（鹏城实验室）；Centre de Recherche Cerveau et Cognition (CERCO), UMR5549 CNRS - Univ. Toulouse 3（大脑与认知研究中心（CERCO），CNRS UMR5549 - 图卢兹第三大学）；Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）
Spiking Neural Networks (SNNs) have attracted enormous research interest due to temporal information processing capability, low power consumption, and high biological plausibility. However, the formulation of efficient and high-performance learning algorithms for SNNs is still challenging. Most existing learning methods learn weights only, and require manual tuning of the membrane-related parameters that determine the dynamics of a single spiking neuron. These parameters are typically chosen to be the same for all neurons, which limits the diversity of neurons and thus the expressiveness of th…
Enhancing Frequency for Single Image Super-Resolution with Learnable Separable Kernels
Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, East China University of Science；Technology, Shanghai, 200237, China. Department of Computer Science；Engineering, East China University of Science
Video scene detection is the task of dividing videos into temporal semantic chapters. This is an important preliminary step before attempting to analyze heterogeneous video content. Recently, Optimal Sequential Grouping (OSG) was proposed as a powerful unsupervised solution to solve a formulation of the video scene detection problem. In this work, we extend the capabilities of OSG to the learning regime. By giving the capability to both learn from examples and leverage a robust optimization formulation, we can boost performance and enhance the versatility of the technology. We present a compre…
The University of Adelaide（阿德莱德大学）；Shenzhen University（深圳大学）
Deep networks excel in learning patterns from large amounts of data. On the other hand, many geometric vision tasks are specified as optimization problems. To seamlessly combine deep learning and geometric vision, it is vital to perform learning and geometric optimization end-to-end. Towards this aim, we present BPnP, a novel network module that backpropagates gradients through a Perspective-n-Points (PnP) solver to guide parameter updates of a neural network. Based on implicit differentiation, we show that the gradients of a "self-contained" PnP solver can be derived accurately and efficientl…
Task-driven Image Fusion with Learnable Fusion Loss
Xi’an Jiaotong University（西安交通大学）；ETH Zürich（苏黎世联邦理工学院）；City University of Hong Kong（香港城市大学）；Tsinghua University（清华大学）；Northwestern Polytechnical University（西北工业大学）
Multi-Branch Non-Homogeneous Image Dehazing via Concentration Partitioning and Image Fusion
School of Software and Internet of Things Engineering, Zhejiang Gongshang University（软件与物联网工程学院，浙江工商大学）；School of Computer Science and Technology, Tiangong University（计算机科学与技术学院，天工大学）
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）；New York University（纽约大学）
Estimation of differential geometric quantities in discrete 3D data representations is one of the crucial steps in the geometry processing pipeline. Specifically, estimating normals and sharp feature lines from raw point cloud helps improve meshing quality and allows us to use more precise surface reconstruction techniques. When designing a learnable approach to such problems, the main difficulty is selecting neighborhoods in a point cloud and incorporating geometric relations between the points. In this study, we present a geometric attention mechanism that can provide such properties in a le…
Fine-grained visual recognition aims to capture discriminative characteristics amongst visually similar categories. The state-of-the-art research work has significantly improved the fine-grained recognition performance by deep metric learning using triplet network. However, the impact of intra-category variance on the performance of recognition and robust feature representation has not been well studied. In this paper, we propose to leverage intra-class variance in metric learning of triplet network to improve the performance of fine-grained recognition. Through partitioning training images wi…
University of Massachusetts Dartmouth（马萨诸塞大学达特茅斯分校）；Fred Hutchinson Cancer Research Center（弗雷德·哈钦森癌症研究中心）；Worcester Polytechnic Institute（伍斯特理工学院）；University of California Berkeley（加州大学伯克利分校）；Tsinghua University（清华大学）
Tissue microarray (TMA) images have been used increasingly often in cancer studies and the validation of biomarkers. TACOMA---a cutting-edge automatic scoring algorithm for TMA images---is comparable to pathologists in terms of accuracy and repeatability. Here we consider how this algorithm may be further improved. Inspired by the recent success of deep learning, we propose to incorporate representations learnable through computation. We explore representations of a group nature through unsupervised learning, e.g., hierarchical clustering and recursive space partition. Information carried by c…
PE-CSNet: An equivariant network architecture with learnable patch-based sparse representation
The Hong Kong Polytechnic University（香港理工大学）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；University of Chinese Academy of Sciences（中国科学院大学）
iPac: Incorporating Intra-image Patch Context into Graph Neural Networks for Medical Image Classification
Nuffield Department of Surgical Sciences, University of Oxford, UK（牛津大学外科科学系）；School of Computing and Digital Technology, Birmingham City University, UK（伯明翰城市大学计算与数字技术学院）；Department of Computer Science, University of Exeter, UK（埃克塞特大学计算机科学系）
