We present the first learning-based visual odometry (VO) model, which generalizes to multiple datasets and real-world scenarios and outperforms geometry-based methods in challenging scenes. We achieve this by leveraging the SLAM dataset TartanAir, which provides a large amount of diverse synthetic data in challenging environments. Furthermore, to make our VO model generalize across datasets, we propose an up-to-scale loss function and incorporate the camera intrinsic parameters into the model. Experiments show that a single model, TartanVO, trained only on synthetic data, without any finetuning, can be generalized to real-world datasets such as KITTI and EuRoC, demonstrating significant advantages over the geometry-based methods on challenging trajectories. Our code is available at https://github.com/castacks/tartanvo.
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Recent studies show that models trained on synthetic datasets are able to achieve better generalizable person re-identification (GPReID) performance than that trained on public real-world datasets. On the other hand, due to the limitations of real-world person ReID datasets, it would also be important and interesting to use large-scale synthetic datasets as test sets to benchmark person ReID algorithms. Yet this raises a critical question: is synthetic dataset reliable for benchmarking generalizable person re-identification? In the literature there is no evidence showing this. To address this,…
Synthetic image generation has opened up new opportunities but has also created threats in regard to privacy, authenticity, and security. Detecting fake images is of paramount importance to prevent illegal activities, and previous research has shown that generative models leave unique patterns in their synthetic images that can be exploited to detect them. However, the fundamental problem of generalization remains, as even state-of-the-art detectors encounter difficulty when facing generators never seen during training. To assess the generalizability and robustness of synthetic image detectors…
Generalizable Synthetic Image Detection via Language-guided Contrastive Learning
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；State Key Laboratory of Internet of Things for Smart City（智慧城市物联网国家重点实验室）；Department of Computer and Information Science, Faculty of Science and Technology, University of Macau（澳门大学计算机与信息科学系）；Faculty of Applied Sciences, Macao Polytechnic University（澳门 polytechnic 大学应用科学系）
Peking University（北京大学）；National University of Defense Technology（国防科技大学）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）
In this work, we tackle 6-DoF grasp detection for transparent and specular objects, which is an important yet challenging problem in vision-based robotic systems, due to the failure of depth cameras in sensing their geometry. We, for the first time, propose a multiview RGB-based 6-DoF grasp detection network, GraspNeRF, that leverages the generalizable neural radiance field (NeRF) to achieve material-agnostic object grasping in clutter. Compared to the existing NeRF-based 3-DoF grasp detection methods that rely on densely captured input images and time-consuming per-scene optimization, our sys…
Forgery-aware Adaptive Transformer for Generalizable Synthetic Image Detection
Institute of Information Science, Beijing Jiaotong University（北京交通大学信息科学研究所）；Beijing Key Laboratory of Advanced Information Science and Network Technology（北京先进信息科学与网络技术重点实验室）；Baidu VIS（百度视觉技术部）
Existing person re-identification models often have low generalizability, which is mostly due to limited availability of large-scale labeled data in training. However, labeling large-scale training data is very expensive and time-consuming, while large-scale synthetic dataset shows promising value in learning generalizable person re-identification models. Therefore, in this paper a novel and practical person re-identification task is proposed,i.e. how to use labeled synthetic dataset and unlabeled real-world dataset to train a universal model. In this way, human annotations are no longer requi…
Learning to Curate What You Generate for Generalizable Few-Shot Class-Incremental Learning
University of Science and Technology Beijing（北京科技大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Singapore Management University（新加坡管理大学）
Although previous CNN based face anti-spoofing methods have achieved promising performance under intra-dataset testing, they suffer from poor generalization under cross-dataset testing. The main reason is that they learn the network with only binary supervision, which may learn arbitrary cues overfitting on the training dataset. To make the learned feature explainable and more generalizable, some researchers introduce facial depth and reflection map as the auxiliary supervision. However, many other generalizable cues are unexplored for face anti-spoofing, which limits their performance under c…
Dual Data Alignment Makes AI-Generated Image Detector Easier Generalizable
Tencent YouTu Lab（腾讯优图实验室）；East China University of Science and Technology（东华大学）；Peking University（北京大学）；Renmin University of China（中国人民大学）；Shenzhen University（深圳大学）；Hong Kong University of Science and Technology（香港科技大学）
University of Science and Technology of China（中国科学技术大学）；Microsoft Research Asia（微软亚洲研究院）
Human can infer the 3D geometry of a scene from a sketch instead of a realistic image, which indicates that the spatial structure plays a fundamental role in understanding the depth of scenes. We are the first to explore the learning of a depth-specific structural representation, which captures the essential feature for depth estimation and ignores irrelevant style information. Our S2R-DepthNet (Synthetic to Real DepthNet) can be well generalized to unseen real-world data directly even though it is only trained on synthetic data. S2R-DepthNet consists of: a) a Structure Extraction (STE) module…
Inception Institute of Artificial Intelligence (IIAI)（Inception人工智能研究院（IIAI））；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Recently, large-scale synthetic datasets are shown to be very useful for generalizable person re-identification. However, synthesized persons in existing datasets are mostly cartoon-like and in random dress collocation, which limits their performance. To address this, in this work, an automatic approach is proposed to directly clone the whole outfits from real-world person images to virtual 3D characters, such that any virtual person thus created will appear very similar to its real-world counterpart. Specifically, based on UV texture mapping, two cloning methods are designed, namely registere…
StereoCarla: A High-Fidelity Driving Dataset for Generalizable Stereo
School of Computer Science, Wuhan University（武汉大学计算机学院）；IAIR, Xi’an Jiaotong University（西安交通大学IAIR）；Waytous；CASIA（中国科学院自动化所）；University of Bologna（博洛尼亚大学）；University of California, Berkeley（加州大学伯克利分校）；AIR, Tsinghua University（清华大学AIR）
FetchBot: Learning Generalizable Object Fetching in Cluttered Scenes via Zero-Shot Sim2Real
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；CFCS, School of Computer Science, Peking University（计算机科学系，北京大学）；Galbot；Beijing Academy of Artificial Intelligence（北京人工智能研究院）
REArtGS++: Generalizable Articulation Reconstruction with Temporal Geometry Constraint via Planar Gaussian Splatting
Hefei Institutes of Physical Science, Chinese Academy of Sciences（合肥物理科学研究院，中国科学院）；University of Science and Technology of China（中国科学技术大学）；Hefei University of Technology（合肥工业大学）；RoboticsX, Tencent（腾讯机器人院）；Shanghai Jiao Tong University（上海交通大学）
Simon Fraser University（西蒙弗雷泽大学）；Hinge Health（Hinge Health公司）
Modern deep learning training procedures rely on model regularization techniques such as data augmentation methods, which generate training samples that increase the diversity of data and richness of label information. A popular recent method, mixup, uses convex combinations of pairs of original samples to generate new samples. However, as we show in our experiments, mixup can produce undesirable synthetic samples, where the data is sampled off the manifold and can contain incorrect labels. We propose $ζ$-mixup, a generalization of mixup with provably and demonstrably desirable properties that…
The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutions
Department of Computer Science and Engineering, University of Nevada, Reno（计算机科学与工程系，内华达大学雷诺分校）；Centre for Frontier AI Research (CFAR), and Institute of High Performance Computing (IHPC), A*STAR, Singapore（前沿人工智能研究中心（CFAR）和高性能计算研究所（IHPC），A*STAR，新加坡）
Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies（广东省新型安全智能技术重点实验室）；Harbin Institute of Technology（哈尔滨工业大学）
Typical methods for blind image super-resolution (SR) focus on dealing with unknown degradations by directly estimating them or learning the degradation representations in a latent space. A potential limitation of these methods is that they assume the unknown degradations can be simulated by the integration of various handcrafted degradations (e.g., bicubic downsampling), which is not necessarily true. The real-world degradations can be beyond the simulation scope by the handcrafted degradations, which are referred to as novel degradations. In this work, we propose to learn a latent representa…
Training real-world neural network models to achieve high performance and generalizability typically requires a substantial amount of labeled data, spanning a broad range of variation. This data-labeling process can be both labor and cost intensive. To achieve desirable predictive performance, a trained model is typically applied into a domain where the data distribution is similar to the training dataset. However, for many agricultural machine learning problems, training datasets are collected at a specific location, during a specific period in time of the growing season. Since agricultural s…
Seoul National University（首尔大学）；Google（谷歌）；UC Merced（加州大学默塞德分校）；Yonsei University（延世大学）
Most image super-resolution (SR) methods are developed on synthetic low-resolution (LR) and high-resolution (HR) image pairs that are constructed by a predetermined operation, e.g., bicubic downsampling. As existing methods typically learn an inverse mapping of the specific function, they produce blurry results when applied to real-world images whose exact formulation is different and unknown. Therefore, several methods attempt to synthesize much more diverse LR samples or learn a realistic downsampling model. However, due to restrictive assumptions on the downsampling process, they are still …
Post-hoc analysis is a popular category in eXplainable artificial intelligence (XAI) study. In particular, methods that generate heatmaps have been used to explain the deep neural network (DNN), a black-box model. Heatmaps can be appealing due to the intuitive and visual ways to understand them but assessing their qualities might not be straightforward. Different ways to assess heatmaps' quality have their own merits and shortcomings. This paper introduces a synthetic dataset that can be generated adhoc along with the ground-truth heatmaps for more objective quantitative assessment. Each sampl…
Learning Continuous Source Responses For Generalizable AI-Generated Image Detection
Ant Group（蚂蚁集团）；Huazhong University of Science and Technology（华中科技大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Jilin University（吉林大学）
EndoMatcher: Generalizable Endoscopic Image Matcher via Multi-Domain Pre-training for Robot-Assisted Surgery
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（多模态人工智能系统国家重点实验室，自动化研究所，中国科学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（人工智能学院，中国科学院大学）；Department of Pulmonary and Critical Care Medicine, The First Affiliated Hospital of Sun Yat-sen University（中山大学附属第一医院呼吸与危重症医学科）；Hong Kong Institute of Science & Innovation（香港创新科技署）；Centre of AI and Robotics, Hong Kong Institute of Science & Innovation（人工智能与机器人中心，香港创新科技署）
MIRROR: Manifold Ideal Reference ReconstructOR for Generalizable AI-Generated Image Detection
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Advanced Interdisciplinary Sciences, UCAS（北京大学交叉信息学院）；Huazhong University of Science and Technology（华中科技大学）；Tencent YouTu Lab（腾讯YouTu实验室）；Southwest University（西南大学）；Peking University（北京大学）；The University of Sydney（悉尼大学）；Shenzhen University（深圳大学）；Tsinghua University（清华大学）
