UMDFood：视觉-语言模型助力食品成分汇编
UMDFood: Vision-language models boost food composition compilation
AI总结：
针对现有食品成分汇编效率低、数据滞后及早期机器学习方法信息利用不全的问题，本研究提出视觉-语言模型UMDFood-VL，构建多模态数据库UMDFood-90k，实现高精度食品成分估算，为相关领域应用提供新思路。
摘要：
营养信息在精准营养和食品行业中至关重要。当前的食品成分汇编范式依赖于费力且依赖经验的方法，难以跟上动态变化的消费市场，导致营养数据滞后且不完整。此外，早期的机器学习方法要么忽略食品配料表中的信息，要么忽视食品图像的特征。为此，我们提出了一种新型视觉-语言模型UMDFood-VL，利用包装正面标签和产品图像准确估算食品成分概况。为支撑模型训练，我们构建了UMDFood-90k——这是迄今为止最全面的多模态食品数据库，包含89533个样本，每个样本都标注有基于图像和文本的配料描述以及11项营养注释。UMDFood-VL在脂肪含量估算上的macro-AUCROC（宏平均受试者工作特征曲线下面积）最高可达0.921，显著高于现有基线方法，满足食品成分汇编的实际需求。同时，在所选产品中，高达82.2%的产品其化学分析结果与模型估算结果之间的误差小于10%。这一性能为推广到其他食品和营养相关数据汇编提供了思路，也推动了基于生成式AI的技术在其他需要个性化的食品应用中的发展。
英文摘要：
Nutrition information is crucial in precision nutrition and the food industry. The current food composition compilation paradigm relies on laborious and experience-dependent methods. However, these methods struggle to keep up with the dynamic consumer market, resulting in delayed and incomplete nutrition data. In addition, earlier machine learning methods overlook the information in food ingredient statements or ignore the features of food images. To this end, we propose a novel vision-language model, UMDFood-VL, using front-of-package labeling and product images to accurately estimate food composition profiles. In order to empower model training, we established UMDFood-90k, the most comprehensive multimodal food database to date, containing 89,533 samples, each labeled with image and text-based ingredient descriptions and 11 nutrient annotations. UMDFood-VL achieves the macro-AUCROC up to 0.921 for fat content estimation, which is significantly higher than existing baseline methods and satisfies the practical requirements of food composition compilation. Meanwhile, up to 82.2% of selected products' estimated error between chemical analysis results and model estimation results are less than 10%. This performance sheds light on generalization towards other food and nutrition-related data compilation and catalyzation for the evolution of generative AI-based technology in other food applications that require personalization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
