Compact Task-Aligned Imitation Learning for Laboratory Automation
Spatial Robotics Research Center, Fujitsu Limited.(富士通株式会社空间机器人研究中心);Faculty of Science and Engineering, Waseda University(早稻田大学理工学部);National Institute of Advanced Industrial Science and Technology(国家工业科学与技术研究院)
Kanata Suzuki, Hanon Nakamura, Kana Miyamoto, Tetsuya Ogata
Robotic laboratory automation has traditionally relied on carefully engineered motion pipelines and task-specific hardware interfaces, resulting in high design cost and limited flexibility. While recent imitation learning techniques can generate general robot behaviors, their large model sizes often require high-performance computational resources, limiting applicability in practical laboratory environments. In this study, we propose a compact imitation learning framework for laboratory automation using small foundation models. The proposed method, TVF-DiT, aligns a self-supervised vision foundation model with a vision-language model through a compact adapter, and integrates them with a Diffusion Transformer-based action expert. The entire model consists of fewer than 500M parameters, enabling inference on low-VRAM GPUs. Experiments on three real-world laboratory tasks - test tube cleaning, test tube arrangement, and powder transfer - demonstrate an average success rate of 86.6%, significantly outperforming alternative lightweight baselines. Furthermore, detailed task prompts improve vision-language alignment and task performance. These results indicate that small foundation models, when properly aligned and integrated with diffusion-based policy learning, can effectively support practical laboratory automation with limited computational resources.
Surgical Task Automation Using Actor-Critic Frameworks and Self-Supervised Imitation Learning
University of Edinburgh(爱丁堡大学);Institute for Imaging, Data and Communications(成像、数据与通信研究所);School of Engineering(工程学院);TECNALIA(泰克纳利亚);University of Strathclyde(思克莱德大学);Centre for Precision Manufacturing(精密制造中心);Department of Biomedical Engineering(生物医学工程系)
Contact Skill Imitation Learning for Robot-Independent Assembly Programming
FZI Research Center for Information Technology(FZI信息科技研究中心);IAR Institute for Anthropomatics and Robotics, HIS Humanoids and Intelligence Systems Lab, KIT Karlsruhe Institute of Technology(卡尔斯鲁厄理工学院(KIT)IAR人形机器人与机器人学研究所、HIS人形与智能系统实验室)
Learning Rhythmic Trajectories with Geometric Constraints for Laser-Based Skincare Procedures
The Hong Kong Polytechnic University(香港理工大学);Politecnico di Torino(都灵理工大学);Università degli Studi di Genova(热那亚大学);Istituto Italiano di Tecnologia(意大利技术研究院);Massachusetts Institute of Technology(麻省理工学院);Università di Genova(热那亚大学)
PRIMAL: Pathfinding via Reinforcement and Imitation Multi-Agent Learning
Carnegie Mellon University(卡内基梅隆大学);Commonwealth Scientific and Industrial Research Organisation (CSIRO)(联邦科学与工业研究组织(CSIRO));University of Southern California(南加州大学)
Dual-Arm Hierarchical Planning for Laboratory Automation: Vibratory Sieve Shaker Operations
College of Intelligence Science and Technology, and the National Key Laboratory of Equipment State Sensing and Smart Support, National University of Defense Technology(智能科学与技术学院,装备状态感知与智能支持国家重点实验室,国防科技大学);College of Aerospace Science and Engineering, National University of Defense Technology(航空航天科学与工程学院,国防科技大学)
Perspective on Utilizing Foundation Models for Laboratory Automation in Materials Research
The University of Tokyo(东京大学);RIKEN Center for Biosystems Dynamics Research(理化学研究所生物系统动力学研究中心);Intellectual Moonshine;OMRON SINIC X Corporation(欧姆龙 SINIC X 公司);NexaScience;RIKEN Advanced General Intelligence for Science Program(理化学研究所科学高级通用智能项目);Institute of Science Tokyo(东京科学大学)
Towards Robotic Laboratory Automation Plug & Play: Survey and Concept Proposal on Teaching-free Robot Integration with the LAPP Digital Twin
Takeda Manufacturing Austria AG(武田奥地利制造公司);Doctoral School of Applied Informatics and Applied Mathematics, Óbuda University(欧布达大学应用信息学与应用数学博士学院);Óbuda University(欧布达大学);Baxalta Innovations GmbH, a Takeda company(百特创新有限公司(武田旗下公司));Alba Regia Technical Faculty, Óbuda University(欧布达大学阿尔巴雷吉亚技术学院);Antal Bejczy Center for Intelligent Robotics, Óbuda University(欧布达大学安塔尔·贝伊奇智能机器人中心)
MATTERIX: toward a digital twin for robotics-assisted chemistry laboratory automation
University of Toronto(多伦多大学);Acceleration Consortium(加速联盟);Vector Institute(向量研究所);University of Liverpool(利物浦大学);University of Salento(萨勒诺大学);NVIDIA Canadian Institute for Advanced Research(NVIDIA加拿大高级研究机构);Georgia Institute of Technology(佐治亚理工学院)
From Prompts to Protocols: An AI Agent for Laboratory Automation
Department of Computer Science, University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校计算机科学系);Department of Chemistry, University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校化学系)
Accelerating Discovery in Natural Science Laboratories with AI and Robotics: Perspectives and Challenges from the 2024 IEEE ICRA Workshop, Yokohama, Japan
University of Liverpool(利物浦大学);SiLA Consortium(SiLA联盟);Acceleration Consortium(加速联盟);University of Toronto(多伦多大学);Technical University of Munich(慕尼黑工业大学);University of Tokyo(东京大学);University of British Columbia(不列颠哥伦比亚大学);Roche Diagnostics GmbH(罗氏诊断有限公司);Unchained Labs(Unchained实验室);University of Rostock(罗斯托克大学);Argonne National Laboratory(阿贡国家实验室);Takeda(武田药品工业);Óbuda University(欧布达大学);Institute of Science Tokyo(东京科学大学);Florida State University(佛罗里达州立大学)
BioProVLA-Agent: An Affordable, Protocol-Driven, Vision-Enhanced VLA-Enabled Embodied Multi-Agent System with Closed-Loop-Capable Reasoning for Biological Laboratory Manipulation
Key Laboratory of Smart Manufacturing in Energy Chemical Process Ministry of Education, East China University of Science and Technology, Shanghai, CN(能源化工过程智能制造教育部重点实验室,东华大学,上海,中国);Department of Computer Science and Engineering, East China University of Science and Technology, Shanghai, CN(东华大学计算机科学与工程系,上海,中国);Department of Laboratory Medicine, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, CN(复旦大学附属瑞金医院检验医学科,上海,中国);School of Information Science and Technology, Shihezi University, Shihezi, CN(石河子大学信息科学与技术学院,石河子,中国)