Multi-modal Machine Learning for Vehicle Rating Predictions Using Image, Text, and Parametric Data
Massachusetts Institute of Technology(麻省理工学院)
Hanqi Su, Binyang Song, Faez Ahmed
备注:
The paper submitted to IDETC/CIE2023, the International Design Engineering Technical Conferences & Computers and Information in Engineering Conference, has been accepted
Accurate vehicle rating prediction can facilitate designing and configuring good vehicles. This prediction allows vehicle designers and manufacturers to optimize and improve their designs in a timely manner, enhance their product performance, and effectively attract consumers. However, most of the existing data-driven methods rely on data from a single mode, e.g., text, image, or parametric data, which results in a limited and incomplete exploration of the available information. These methods lack comprehensive analyses and exploration of data from multiple modes, which probably leads to inaccurate conclusions and hinders progress in this field. To overcome this limitation, we propose a multi-modal learning model for more comprehensive and accurate vehicle rating predictions. Specifically, the model simultaneously learns features from the parametric specifications, text descriptions, and images of vehicles to predict five vehicle rating scores, including the total score, critics score, performance score, safety score, and interior score. We compare the multi-modal learning model to the corresponding unimodal models and find that the multi-modal model's explanatory power is 4% - 12% higher than that of the unimodal models. On this basis, we conduct sensitivity analyses using SHAP to interpret our model and provide design and optimization directions to designers and manufacturers. Our study underscores the importance of the data-driven multi-modal learning approach for vehicle design, evaluation, and optimization. We have made the code publicly available at http://decode.mit.edu/projects/vehicleratings/.
Interpretability by design using computer vision for behavioral sensing in child and adolescent psychiatry
Technical University of Denmark(丹麦技术大学);Copenhagen University Hospital – Mental Health Services CPH(哥本哈根大学医院——哥本哈根精神卫生服务中心);University of Copenhagen(哥本哈根大学);Hospital Glostrup - University Hospital(格洛斯楚普大学医院)
Uncertainty-Aware Multi-Parametric Magnetic Resonance Image Information Fusion for 3D Object Segmentation
Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院保罗·C·劳特伯生物医学成像研究中心);University of Chinese Academy of Sciences(中国科学院大学);Peng Cheng Laboratory(鹏城实验室);Guangdong Provincial Key Laboratory of Artificial Intelligence in Medical Image Analysis and Application(广东省医学图像分析与应用人工智能重点实验室)
Emotion-Driven Personalized Recommendation for AI-Generated Content Using Multi-Modal Sentiment and Intent Analysis
University of Electronic Science and Technology of China(电子科技大学);Columbia Business School, Columbia University(哥伦比亚大学哥伦比亚商学院);New York University(纽约大学)