Accurately diagnosing and predicting vehicle malfunctions is crucial for maintenance and safety in the automotive industry. While modern diagnostic systems primarily rely on sequences of vehicular Diagnostic Trouble Codes (DTCs) registered in On-Board Diagnostic (OBD) systems, they often overlook valuable contextual information such as raw sensory data (e.g., temperature, humidity, and pressure). This contextual data, crucial for domain experts to classify vehicle failures, introduces unique challenges due to its complexity and the noisy nature of real-world data. This paper presents BiCarFormer: the first multimodal approach to multi-label sequence classification of error codes into error patterns that integrates DTC sequences and environmental conditions. BiCarFormer is a bidirectional Transformer model tailored for vehicle event sequences, employing embedding fusions and a co-attention mechanism to capture the relationships between diagnostic codes and environmental data. Experimental results on a challenging real-world automotive dataset with 22,137 error codes and 360 error patterns demonstrate that our approach significantly improves classification performance compared to models that rely solely on DTC sequences and traditional sequence models. This work highlights the importance of incorporating contextual environmental information for more accurate and robust vehicle diagnostics, hence reducing maintenance costs and enhancing automation processes in the automotive industry.
MM-NeuroOnco: A Multimodal Benchmark and Instruction Dataset for MRI-Based Brain Tumor Diagnosis
Guangdong Institute of Intelligence Science and Technology(广东智能科学与技术研究院);Center for Brain-Inspired Computing Research (CBICR), Department of Precision Instrument, Tsinghua University(脑启发计算研究中心(CBICR)、精密仪器系,清华大学)
Augmented Intelligence for Multimodal Virtual Biopsy in Breast Cancer Using Generative Artificial Intelligence
Department of Radiology, Fondazione Policlinico Campus Bio-Medico(放射学系,政策临床医学院);Department of Radiology, Università Campus Bio-Medico di Roma(放射学系,罗马生物医学大学);Department of Diagnostics and Intervention, Radiation Physics, Biomedical Engineering, Umeå University(诊断与介入系,辐射物理,生物医学工程,乌梅拉大学)
Beyond Traditional Diagnostics: Transforming Patient-Side Information into Predictive Insights with Knowledge Graphs and Prototypes
Joint SDU-NTU Centre for Artificial Intelligence Research \& School of Software Shandong University Jinan, China;College of Computing;Data Science Nanyang Technological University Singapore
Evidence-Based Intelligent Diagnostic and Therapeutic Visualization System with Large Language Models: Multi-Turn Interaction and Multimodal Treatment Plan Generation
Harbin Institute of Technology, Weihai(哈尔滨工业大学(威海));Harbin Institute of Technology (Weihai) Qingdao Research Institute(哈尔滨工业大学(威海)青岛研究院);Shandong Key Laboratory of Digital Service Computing Technology and Systems(山东省数字服务计算技术与系统重点实验室);Weihai Municipal Hospital(威海市人民医院);Shanghai Taizhu Technology Co., Ltd(上海泰山技术有限公司);Tianjin Zhifu Qihuang Medical Technology Co., Ltd(天津中孚启黄医疗技术有限公司)
LingLan: An Advancing Traditional Chinese Medicine Diagnosis LLM with Multimodal Data
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Guangdong Provincial Laboratory of Traditional Chinese Medicine Hengqin(广东省中医药横琴实验室)
Evaluating Multi-Turn Multimodal Diagnostic Reasoning on Challenging Real-World Clinical Cases
Center for Biomedical Data Science, Duke-NUS Medical School(生物医学数据科学中心,杜克 - 新加坡国立大学医学院);Duke-NUS AI + Medical Sciences Initiative, Duke-NUS Medical School(杜克 - 新加坡国立大学人工智能与医学科学计划,杜克 - 新加坡国立大学医学院);Department of Population Health Sciences, Weill Cornell Medicine(人口健康科学系,威尔康奈尔医学院);System Engineering, College of Engineering, Cornell University(系统工程,康奈尔大学工程学院);Graduate School of Frontier Sciences, The University of Tokyo(前沿科学研究生院,东京大学);RIKEN Center for Advanced Intelligence Project(理化学研究所先进智能项目中心);Department of Biostatistics and Bioinformatics, Duke University(生物统计学与生物信息学系,杜克大学);Perelman School of Medicine, University of Pennsylvania(佩雷尔曼医学院,宾夕法尼亚大学);School of Clinical Medicine, University of Cambridge(临床医学学院,剑桥大学);Hospital of the University of Pennsylvania(宾夕法尼亚大学医院);Children’s Hospital of Philadelphia (CHOP)(费城儿童医院);Department of Anatomic Pathology and Laboratory Medicine, Hospital of the University of Pennsylvania(解剖病理学与检验医学系,宾夕法尼亚大学医院);Department of Pathology and Laboratory Medicine, University of California, San Francisco(病理学与检验医学系,加州大学旧金山分校);Language Technologies Institute, Carnegie Mellon University(语言技术研究所,卡内基梅隆大学)
RetiBridge: Bridging Quantitative Retinal Biomarkers and Qualitative Diagnosis with a Knowledge-Guided Multimodal Large Language Model
University of Liverpool(利物浦大学);Institute of Life Course & Medical Sciences(生命课程与医学科学研究院);Department of Eye and Vision Sciences(眼科与视觉科学系);Computer Science Department(计算机科学系);Cardiovascular & Metabolic Medicine(心血管与代谢医学);Liverpool Centre for Cardiovascular Science(利物浦心血管科学中心)
Adaptive Diagnostic Reasoning Framework for Pathology with Multimodal Large Language Models
Computer Science Department, University of California, Los Angeles, CA, USA(加州大学洛杉矶分校计算机科学系);Mechanical and Aerospace Engineering Department, University of California, Los Angeles, CA, USA(加州大学洛杉矶分校机械与航空航天工程系);Department of Pathology, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan(台湾国防医学院三军总医院病理部);Department of Pathology, National Taiwan University Hospital, Taipei, Taiwan(台湾国立台湾大学医院病理部);Department of Pathology, David Geffen School of Medicine, University of California, Los Angeles, CA, USA(加州大学洛杉矶分校大卫·Geffen医学院病理部);Bioengineering Department, University of California, Los Angeles, CA, USA(加州大学洛杉矶分校生物工程系);Institute for Quantitative and Computational Biosciences, University of California, CA, USA(加州大学定量与计算生物科学研究所)
ConceptM$^3$oE: Concept-Guided Multimodal Mixture of Experts for Interpretable Computational Pathology
University of Texas at Austin(德克萨斯大学奥斯汀分校);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);Dell Children’s Medical Center(德尔儿童医疗中心);The University of Texas MD Anderson Cancer Center(德克萨斯大学MD安德森癌症中心);University of Nevada, Reno(内华达大学里诺分校)
A Survey of Multimodal Ophthalmic Diagnostics: From Task-Specific Approaches to Foundational Models
College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China(深圳大学计算机科学与软件工程学院);Shenzhen Key Laboratory of Visual Object Detection and Recognition, Harbin Institute of Technology, Shenzhen, 518055, China(视觉对象检测与识别深圳重点实验室);Laboratory for Artificial Intelligence in Design, Hong Kong(人工智能设计实验室);School of Artificial Intelligence, Shenzhen University, Shenzhen, China(深圳大学人工智能学院)
Evaluating Multimodal LLMs for Inpatient Diagnosis: Real-World Performance, Safety, and Cost Across Ten Frontier Models
School of Computer Science and Applied Mathematics, University of the Witwatersrand(沃斯兰大学计算机科学与应用数学学院);Wits MIND Institute, University of the Witwatersrand(沃斯兰大学Wits MIND研究所);Grai Labs, Cape Town, South Africa(南非开普敦Grai实验室);Faculty of Health Sciences, University of the Witwatersrand(沃斯兰大学健康科学学院);South African Medical Research Council Vaccines and Infectious Diseases Analytics Research Unit, Faculty of Health Sciences, University of the Witwatersrand(南非医学研究委员会疫苗与传染病分析研究单位,沃斯兰大学健康科学学院)
AI in Oncology: Transforming Cancer Detection through Machine Learning and Deep Learning Applications
School of Basic Medical Sciences, Zhengzhou University(郑州大学基础医学院);School of Electrical and Information Engineering, Zhengzhou University(郑州大学电气与信息工程学院);Academy of Medical Sciences, Zhengzhou University(郑州大学医学科学院);China-US (Henan) Hormel Cancer Institute(中美(河南)荷美尔肿瘤研究院);The Affiliated Cancer Hospital of Zhengzhou University, Henan Cancer Hospital(郑州大学附属肿瘤医院 河南省肿瘤医院);State Key Laboratory of Esophageal Cancer Prevention and Treatment(食管癌防治国家重点实验室);The Collaborative Innovation Center of Henan Province for Cancer Chemoprevention(河南省癌症化预防协同创新中心)
PathFound: An Agentic Multimodal Model Activating Evidence-seeking Pathological Diagnosis
Qing Yuan Research Institute, Shanghai Jiao Tong University(上海交通大学庆元研究院);Shanghai Innovation Institute(上海创新研究院);Department of Pathology, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China(中国科学技术大学生命科学与医学学院病理科);Intelligent Pathology Institute, Division of Life Sciences and Medicine(生命科学与医学学院智能病理研究所);Department of Pathology, Fudan University Shanghai Cancer Center(复旦大学上海癌症中心病理科);Department of Oncology, Shanghai Medical College, Fudan University(复旦大学上海医学院肿瘤科);Institute of Pathology, Fudan University(复旦大学病理研究所);Department of Pathology, The First Affiliated Hospital with Nanjing Medical University(南京医科大学第一附属医院病理科)
Transforming Surgical Interventions with Embodied Intelligence for Ultrasound Robotics
Centre for Artificial Intelligence and Robotics (CAIR), HKISI-CAS(香港中文大学(深圳)人工智能与机器人中心(HKISI-CAS));Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
Multimodal Integration of Longitudinal Noninvasive Diagnostics for Survival Prediction in Immunotherapy Using Deep Learning
Radboud University(拉德堡德大学);The Netherlands Cancer Institute(荷兰癌症研究所);Maastricht University(马斯特里赫特大学);Oncode Institute(Oncode研究所);Leiden University Medical Center(莱顿大学医学中心);Centre Hospitalier Universitaire Vaudois(沃州大学医院);University of Southern Denmark(南丹麦大学)
Thinking Like a Botanist: Challenging Multimodal Language Models with Intent-Driven Chain-of-Inquiry
Department of Robotics and Mechatronics Engineering, University of Dhaka(达卡大学机器人与机电工程系);Department of Computer Science and Engineering, University of Dhaka(达卡大学计算机科学与工程系);Department of Agronomy, Gazipur Agricultural University(加兹ipur农业大学作物学系);Department of Botany, University of Dhaka(达卡大学植物学系)
Multimodal Deep Learning for Phyllodes Tumor Classification from Ultrasound and Clinical Data
Department of Electrical and Computer Engineering, University of Central Florida(电子与计算机工程系,中央佛罗里达大学);Massachusetts General Hospital, Department of Surgery, Section of Breast Surgery(麻省总医院,外科部,乳腺外科);Department of Medicine, University of Central Florida College of Medicine(医学系,中央佛罗里达大学医学院)