arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2305.15218cs.LGcs.AIcs.CV
收藏

使用图像、文本和参数化数据进行车辆评级预测的多模态机器学习

Multi-modal Machine Learning for Vehicle Rating Predictions Using Image, Text, and Parametric Data

Massachusetts Institute of Technology(麻省理工学院)

Hanqi Su, Binyang Song, Faez Ahmed

AI总结:

提出一种融合图像、文本和参数化数据的多模态学习模型用于车辆评级预测,相比单模态模型解释力提升4%-12%,并利用SHAP提供设计优化方向。

摘要:

准确的车辆评级预测有助于设计和配置优质车辆。该预测使车辆设计师和制造商能够及时优化和改进其设计,提升产品性能,并有效吸引消费者。然而,现有的大多数数据驱动方法依赖于单一模态的数据,例如文本、图像或参数化数据,这导致对可用信息的探索有限且不完整。这些方法缺乏对多模态数据的综合分析和探索,可能导致不准确的结论并阻碍该领域的进展。为了克服这一局限性,我们提出了一种多模态学习模型,用于更全面、更准确的车辆评级预测。具体而言,该模型同时从车辆的参数化规格、文本描述和图像中学习特征,以预测五项车辆评级得分,包括总分、专家评分、性能得分、安全得分和内饰得分。我们将该多模态学习模型与相应的单模态模型进行比较,发现多模态模型的解释力比单模态模型高4%至12%。在此基础上,我们使用SHAP进行敏感性分析以解释我们的模型,并为设计师和制造商提供设计和优化方向。我们的研究强调了数据驱动的多模态学习方法在车辆设计、评估和优化中的重要性。我们已在 http://decode.mit.edu/projects/vehicleratings/ 上公开了代码。

英文摘要:

Accurate vehicle rating prediction can facilitate designing and configuring good vehicles. This prediction allows vehicle designers and manufacturers to optimize and improve their designs in a timely manner, enhance their product performance, and effectively attract consumers. However, most of the existing data-driven methods rely on data from a single mode, e.g., text, image, or parametric data, which results in a limited and incomplete exploration of the available information. These methods lack comprehensive analyses and exploration of data from multiple modes, which probably leads to inaccurate conclusions and hinders progress in this field. To overcome this limitation, we propose a multi-modal learning model for more comprehensive and accurate vehicle rating predictions. Specifically, the model simultaneously learns features from the parametric specifications, text descriptions, and images of vehicles to predict five vehicle rating scores, including the total score, critics score, performance score, safety score, and interior score. We compare the multi-modal learning model to the corresponding unimodal models and find that the multi-modal model's explanatory power is 4% - 12% higher than that of the unimodal models. On this basis, we conduct sensitivity analyses using SHAP to interpret our model and provide design and optimization directions to designers and manufacturers. Our study underscores the importance of the data-driven multi-modal learning approach for vehicle design, evaluation, and optimization. We have made the code publicly available at http://decode.mit.edu/projects/vehicleratings/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑