Sim2real transfer for robot policy learning suffers due to mismatch between simulation and reality. Existing methods typically address each gap in isolation through separate adaptation modules, which are composed or layered when both gaps coexist. Yet the basis for attempting sim2real in the first place is that there is shared structure between a task in simulation and reality, where equivalent actions from equivalent configurations produce equivalent long term outcomes regardless of domain specific differences in rendering or physics. In this paper, we study whether we can identify and exploit this shared structure from raw observations to train a policy that enables zero shot transfer. We introduce BIFROST, which learns a shared history encoder on paired cross-domain data via cross-domain bisimulation objective: observation-action sequences leading to equivalent long-term behavior are mapped to nearby latent states, regardless of domain. Policies trained on these latent states in simulation transfer zero-shot to reality. We provide empirical evidence on sim2sim visual navigation and sim2real contact rich manipulation task and visual servoing task that BIFROST achieves effective transfer where domain adaptation and co-training baselines fail under both visual and dynamics domain gaps.
Few-shot learning aims to fast adapt a deep model from a few examples. While pre-training and meta-training can create deep models powerful for few-shot generalization, we find that pre-training and meta-training focuses respectively on cross-domain transferability and cross-task transferability, which restricts their data efficiency in the entangled settings of domain shift and task shift. We thus propose the Omni-Training framework to seamlessly bridge pre-training and meta-training for data-efficient few-shot learning. Our first contribution is a tri-flow Omni-Net architecture. Besides the …
Bridging Data Gaps in Healthcare: A Scoping Review of Transfer Learning in Biomedical Data Analysis
Duke-NUS Medical School（杜克-新加坡国立大学医学院）；Duke University（杜克大学）；Singapore General Hospital（新加坡中央医院）；Columbia University Mailman School of Public Health（哥伦比亚大学梅尔曼公共卫生学院）；National University of Singapore（新加坡国立大学）
University of Sydney（悉尼大学）；The University of Hong Kong（香港大学）；Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局）
Deep learning based image enhancement models have largely improved the readability of fundus images in order to decrease the uncertainty of clinical observations and the risk of misdiagnosis. However, due to the difficulty of acquiring paired real fundus images at different qualities, most existing methods have to adopt synthetic image pairs as training data. The domain shift between the synthetic and the real images inevitably hinders the generalization of such models on clinical data. In this work, we propose an end-to-end optimized teacher-student framework to simultaneously conduct image e…
Bridging the Synthetic-to-Real Gap for Few-Shot Cryo-ET Classification
Columbia University（哥伦比亚大学）；Carnegie Mellon University（卡内基梅隆大学）；National Institute of Technology Karnataka（卡纳塔克邦国立理工学院）；University of Minnesota – Twin Cities（明尼苏达大学双城分校）
TRAM: Bridging Trust Regions and Sharpness Aware Minimization
University of Edinburgh（爱丁堡大学）；Kempner Institute, Harvard University（哈佛大学肯普纳研究所）；Allen Institute for AI（艾伦人工智能研究所）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
EnvBridge: Bridging Diverse Environments with Cross-Environment Knowledge Transfer for Embodied AI
Panasonic Connect Co., Ltd.（松下连接有限公司）；National University of Singapore（新加坡国立大学）；Panasonic R&D Center Singapore（松下新加坡研发中心）；Panasonic R&D Center Germany（松下德国研发中心）
University of Freiburg（弗莱堡大学）；Toyota Research Institute（丰田研究院）
The majority of learning-based semantic segmentation methods are optimized for daytime scenarios and favorable lighting conditions. Real-world driving scenarios, however, entail adverse environmental conditions such as nighttime illumination or glare which remain a challenge for existing approaches. In this work, we propose a multimodal semantic segmentation model that can be applied during daytime and nighttime. To this end, besides RGB images, we leverage thermal images, making our network significantly more robust. We avoid the expensive annotation of nighttime images by leveraging an exist…
