Deep neural networks obtained by standard training have been constantly plagued by adversarial examples. Although adversarial training demonstrates its capability to defend against adversarial examples, unfortunately, it leads to an inevitable drop in the natural generalization. To address the issue, we decouple the natural generalization and the robust generalization from joint training and formulate different training strategies for each one. Specifically, instead of minimizing a global loss on the expectation over these two generalization errors, we propose a bi-expert framework called \emph{Generalist} where we simultaneously train base learners with task-aware strategies so that they can specialize in their own fields. The parameters of base learners are collected and combined to form a global learner at intervals during the training process. The global learner is then distributed to the base learners as initialized parameters for continued training. Theoretically, we prove that the risks of Generalist will get lower once the base learners are well trained. Extensive experiments verify the applicability of Generalist to achieve high accuracy on natural examples while maintaining considerable robustness to adversarial ones. Code is available at https://github.com/PKU-ML/Generalist.
Hidden-Shot: Towards One-Shot Task Generalization for Low-Level Vision Generalist Models
Pratt School of Engineering, Duke University(杜克大学普拉特工程学院);Department of Engineering Science, University of Oxford(牛津大学工程科学系);Khoury College of Computer Science, Northeastern University(东北大学库里计算机科学学院)
Value-order Decomposition for Generalist Anomaly Detection
Dalian University of Technology(大连理工大学);Xi’an Chang’an Vanke City Primary School(西安长安万科城小学);Key Laboratory of Social Computing and Cognitive Intelligence (Dalian University of Technology), Ministry of Education(社会计算与认知智能教育部重点实验室(大连理工大学))
Towards Generalist Intelligence in Dentistry: Vision Foundation Models for Oral and Maxillofacial Radiology
Shanghai Jiao Tong University(上海交通大学);Shanghai Ninth People’s Hospital, Shanghai Jiao Tong University School of Medicine, Department of Oral Craniomaxillofacial(上海第九人民医院,上海交通大学医学院,口腔颅面医学部);Shanghai Jiao Tong University, School of Computer Science(上海交通大学,计算机科学学院);University of Electronic Science and Technology of China, School of Mechanical and Electrical Engineering(电子科技大学,机械与电子工程学院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Shanghai Jiao Tong University, College of Stomatology(上海交通大学,口腔医学院);National Center for Stomatology(国家口腔医学中心);National Clinical Medical Research Center for Oral Diseases(国家口腔疾病临床医学研究中心);Shanghai Key Laboratory of Stomatology(上海口腔医学重点实验室);Shanghai Research Institute of Stomatology(上海口腔研究所);Chinese Academy of Medical Science, Research Unit of Oral and Maxillofacial Regenerative Medicine(中国医学科学院,口腔颌面再生医学研究单元)
ReFineVLA: Multimodal Reasoning-Aware Generalist Robotic Policies via Teacher-Guided Fine-Tuning
VinRobotics;University of Texas at Arlington(德克萨斯大学阿灵顿分校);Max Planck Research School for Intelligent Systems (IMPRS-IS)(智能系统Max Planck研究学校);Intelligent Autonomous Systems Lab(智能自主系统实验室);TU Darmstadt(德累斯顿技术大学);Automation & Control Institute(自动化与控制研究所);TU Wien(维也纳技术大学);Austrian Institute of Technology (AIT)(奥地利技术研究所)
Decoupling Wavelet Sub-bands for Single Source Domain Generalization in Fundus Image Segmentation
Indian Statistical Institute(印度统计研究所);University of Oxford(牛津大学);Radcliffe Department of Medicine, University of Oxford(牛津大学拉德克利夫医学系);Department of Engineering Science, University of Oxford(牛津大学工程科学系)
Metis-SPECS: Decoupling Multimodal Learning via Self-distilled Preference-based Cold Start
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);MAIS, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所MAIS);Meituan(美团)
OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agent
University of Science and Technology of China(中国科学技术大学);Shanghai AI Laboratory(上海人工智能实验室);National University of Singapore(新加坡国立大学);The Hong Kong University of Science and Technology(香港科学与技术大学);The University of Hong Kong(香港大学);CUHK MMLab(香港中文大学MMLab);Xi’an Jiaotong University(西安交通大学);Nanjing University(南京大学);Harbin Institute of Technology(哈尔滨工业大学)
Parameter Decoupling Strategy for Semi-supervised 3D Left Atrium Segmentation
School of Biological Science and Medical Engineering, Beihang University(北京航空航天大学生物科学与医学工程学院);School of Computer Science and Technology, Xidian University(西安电子科技大学计算机科学与技术学院);Hefei Innovation Research Institute, Beihang University(北京航空航天大学合肥创新研究院);Beijing Advanced Innovation Centre for Biomedical Engineering, Beihang University(北京航空航天大学生物医学工程高精尖创新中心);Beijing Advanced Innovation Centre for Big Data-Based Precision Medicine, Beihang University(北京航空航天大学大数据精准医疗高精尖创新中心)