The biological skin enables animals to sense various stimuli. Extensive efforts have been made recently to develop smart skin-like sensors to extend the capabilities of biological skins; however, simultaneous sensing of several types of stimuli in a large area remains challenging because this requires large-scale sensor integration with numerous wire connections. We propose a simple, highly sensitive, and multimodal sensing approach, which does not require integrating multiple sensors. The proposed approach is based on an optical interference technique, which can encode the information of various stimuli as a spatial pattern. In contrast to the existing approach, the proposed approach, combined with a deep neural network, enables us to freely select the sensing mode according to our purpose. As a key example, we demonstrate simultaneous sensing mode of three different physical quantities, contact force, contact location, and temperature, using a single soft material without requiring complex integration. Another unique property of the proposed approach is spatially continuous sensing with ultrahigh resolution of few tens of micrometers, which enables identifying the shape of the object in contact. Furthermore, we present a haptic soft device for a human-machine interface. The proposed approach encourages the development of high-performance optical skins.
Flexible Sampling for Long-tailed Skin Lesion Classification
Monash-Airdoc Research(莫纳什-艾尔多克研究);Monash University(莫纳什大学);Monash Medical AI Group(莫纳什医学人工智能小组);Faculty of Information Technology(信息科技学院);Harbin Engineering University(哈尔滨工程大学)
A Multimodal Vision Foundation Model for Clinical Dermatology
Monash University(莫纳什大学);The University of Queensland(昆士兰大学);Medical University of Vienna(维也纳医科大学);Alfred Hospital(阿尔弗雷德医院);Princess Alexandra Hospital(亚历山德拉公主医院);NVIDIA AI Technology Center(英伟达人工智能技术中心);University of Florence(佛罗伦萨大学);Melanoma Institute Australia(澳大利亚黑色素瘤研究所);Royal Prince Alfred Hospital(阿尔弗雷德王子医院);Hospital General Universitario de Alicante(阿利坎特大学综合医院)
Trustworthy and Fair SkinGPT-R1 for Democratizing Dermatological Reasoning across Diverse Ethnicities
School of Data Science, The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen)(数据科学学院,香港中文大学(深圳));Faculty of Information Technology, Monash University(信息技术学院,莫纳什大学);Department of Dermatology, Tianjin Institute of Integrative Dermatology, Tianjin Academy of Traditional Chinese Medicine Affiliated Hospital(皮肤科,天津整合皮肤科研究所,天津中医研究院附属医院);Department of Dermatology, The First Affiliated Hospital, Shantou University Medical College(皮肤科,汕头大学医学院第一附属医院);Department of Dermatology, Beijing AnZhen Hospital, Capital Medical University(皮肤科,北京安贞医院,首都医科大学);School of Computing and Data Science, The University of Hong Kong(计算与数据科学学院,香港大学);Institute of Automation, Chinese Academy of Sciences(自动化研究所,中国科学院);Department of Dermatology, Beijing Aerospace General Hospital(皮肤科,北京航天总医院)
Great Bay University(大湾区大学);Nanyang Technological University(南洋理工大学);University of Oulu(奥卢大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Shenzhen University(深圳大学)
Dynamic Multimodal Evaluation with Flexible Complexity by Vision-Language Bootstrapping
School of Artificial Intelligence, Shanghai Jiao Tong University(上海交通大学人工智能学院);Shanghai AI Laboratory(上海人工智能实验室);Tongji University(同济大学);The University of Hong Kong(香港大学)
Flexible Multimodal Neuroimaging Fusion for Alzheimer's Disease Progression Prediction
Department of Computer Science and Engineering(计算机科学与工程系);Department of Biomedical Informatics(生物医学信息学系);Department of Neurology(神经病学系);Translational Data Analytics Institute(转化数据分析研究所)
Adaptive Look, Flexible Thought: Learning How and How Long to Think in Multimodal Reasoning
Sun Yat-sen University(中山大学);Shenzhen Loop Area Institute(深圳河套学院);Beijing Zhongguancun Academy(北京中关村研究院);Huawei Technologies Co., Ltd(华为技术有限公司);Harbin Institute of Technology(哈尔滨工业大学)
OmniGen: Unified Multimodal Sensor Generation for Autonomous Driving
Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区);Westlake University(西湖大学);Li Auto Inc.(Li汽车公司);The University of Toronto(多伦多大学);University of Macau(澳门大学)
Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performance
Shanghai AI Laboratory(上海人工智能实验室);Fudan University(复旦大学);Tsinghua University(清华大学);SenseTime Research(商汤科技研究院);Nanjing University(南京大学);The Chinese University of Hong Kong(香港中文大学);Shanghai Jiao Tong University(上海交通大学)
Multi-FRuGaL: Multimodal Flexible Redundancy-aware Decomposed Gated Learning for Cancer Diagnosis and Prognosis
Division of Computational Pathology, Department of Pathology and Laboratory Medicine, Indiana University School of Medicine(计算病理学部,病理学与实验室医学部,印第安纳大学医学院);IU Melvin and Bren Simon Comprehensive Cancer Center(印第安纳大学Melvin和Bren Simon综合癌症中心);Departments of Biostatistics and Health Data Science(生物统计学与健康数据科学部);Radiology and Imaging Sciences(放射学与影像科学部);Neurological Surgery(神经外科);Indiana University School of Medicine(印第安纳大学医学院);Department of Computer Science, Luddy School of Informatics, Computing, and Engineering(计算机科学部,Luddy信息、计算与工程学院)
Towards Scalable SOAP Note Generation: A Weakly Supervised Multimodal Framework
Department of Computer Science, University of Maryland, Baltimore County(计算机科学系,马里兰大学巴尔的摩县分校);Department of Dermatology, Johns Hopkins University School of Medicine(皮肤科系,约翰霍普金斯大学医学院)
Simultaneous Tactile-Visual Perception for Learning Multimodal Robot Manipulation
Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院);School of Psychological and Cognitive Sciences, Peking University(北京大学心理与认知科学学院);Beijing Key Lab of Behavior and Mental Health, Peking University(北京大学行为与心理健康北京市重点实验室);Beijing Institute for General Artificial Intelligence(北京一般人工智能研究院);State Key Lab for General Artificial Intelligence(一般人工智能国家重点实验室);Embodied Intelligence Lab, PKU-Wuhan Institute for Artificial Intelligence(具身智能实验室,北京大学武汉人工智能研究院);Department of Computer Science and Technology, University of Cambridge(剑桥大学计算机科学与技术系)