Although recipe data are very easy to come by nowadays, it is really hard to find a complete recipe dataset - with a list of ingredients, nutrient values per ingredient, and per recipe, allergens, etc. Recipe datasets are usually collected from social media websites where users post and publish recipes. Usually written with little to no structure, using both standardized and non-standardized units of measurement. We collect six different recipe datasets, publicly available, in different formats, and some including data in different languages. Bringing all of these datasets to the needed format for applying a machine learning (ML) pipeline for nutrient prediction [1], [2], includes data normalization using dictionary-based named entity recognition (NER), rule-based NER, as well as conversions using external domain-specific resources. From the list of ingredients, domain-specific embeddings are created using the same embedding space for all recipes - one ingredient dataset is generated. The result from this normalization process is two corpora - one with predefined ingredient embeddings and one with predefined recipe embeddings. On all six recipe datasets, the ML pipeline is evaluated. The results from this use case also confirm that the embeddings merged using the domain heuristic yield better results than the baselines.
Warsaw University of Technology（华沙理工大学）；Adam Mickiewicz University（亚当·密茨凯维奇大学）；Poznan University of Technology（波兹南理工大学）
Food Computing is currently a fast-growing field of research. Natural language processing (NLP) is also increasingly essential in this field, especially for recognising food entities. However, there are still only a few well-defined tasks that serve as benchmarks for solutions in this area. We introduce a new dataset -- called \textit{TASTEset} -- to bridge this gap. In this dataset, Named Entity Recognition (NER) models are expected to find or infer various types of entities helpful in processing recipes, e.g.~food products, quantities and their units, names of cooking processes, physical qua…
Korea University（高丽大学）；Sony AI（索尼人工智能）；Sejong University（世宗大学）
We propose a computational approach for recipe ideation, a downstream task that helps users select and gather ingredients for creating dishes. To perform this task, we developed RecipeMind, a food affinity score prediction model that quantifies the suitability of adding an ingredient to set of other ingredients. We constructed a large-scale dataset containing ingredient co-occurrence based scores to train and evaluate RecipeMind on food affinity score prediction. Deployed in recipe ideation, RecipeMind helps the user expand an initial set of ingredients by suggesting additional ingredients. Ex…
National University of Singapore（新加坡国立大学）；Fudan University（复旦大学）；City University of Hong Kong（香港城市大学）
Understanding food recipe requires anticipating the implicit causal effects of cooking actions, such that the recipe can be converted into a graph describing the temporal workflow of the recipe. This is a non-trivial task that involves common-sense reasoning. However, existing efforts rely on hand-crafted features to extract the workflow graph from recipes due to the lack of large-scale labeled datasets. Moreover, they fail to utilize the cooking images, which constitute an important part of food recipes. In this paper, we build MM-ReS, the first large-scale dataset for cooking workflow constr…
RoDE: Linear Rectified Mixture of Diverse Experts for Food Large Multi-Modal Models
School of Computer Science, Fudan University（复旦大学计算机科学技术学院）；School of Computing and Information Systems, Singapore Management University（新加坡管理大学计算与信息系统学院）
Zhejiang University（浙江大学）；Singapore Management University（新加坡管理大学）；Alibaba-Zhejiang University Joint Institute of Frontier Technologies（阿里巴巴-浙江大学前沿技术联合研究院）
Nowadays, driven by the increasing concern on diet and health, food computing has attracted enormous attention from both industry and research community. One of the most popular research topics in this domain is Food Retrieval, due to its profound influence on health-oriented applications. In this paper, we focus on the task of cross-modal retrieval between food images and cooking recipes. We present Modality-Consistent Embedding Network (MCEN) that learns modality-invariant representations by projecting images and texts to the same embedding space. To capture the latent alignments between mod…
Athens University of Economics and Business（雅典经济与商业大学）
Food is essential to human survival. So much so that we have developed different recipes to suit our taste needs. In this work, we propose a novel way of creating new, fine-dining recipes from scratch using Transformers, specifically auto-regressive language models. Given a small dataset of food recipes, we try to train models to identify cooking techniques, propose novel recipes, and test the power of fine-tuning with minimal data.
Large Language Models for Ingredient Substitution in Food Recipes using Supervised Fine-tuning and Direct Preference Optimization
University of Moratuwa（莫拉图瓦大学）；Massey University（梅西大学）；CSIR-Central Scientific Instruments Organisation（CSIR中央科学仪器组织）；Academy of Scientific and Innovative Research (AcSIR)（科学与创新研究院）
Validating FKG.in: Soundness Assessment in LLM-Augmented Indian Food Knowledge
Ashoka University（阿肖克大学）；Mphasis AI and Applied Tech Lab（美斐西斯人工智能与应用技术实验室）；Koita Centre for Digital Health, Ashoka University（阿肖克大学科伊塔数字健康中心）；Institute for Future Health, UC Irvine（加州大学欧文分校未来健康研究所）
University of Notre Dame（圣母大学）；Brandeis University（布兰迪斯大学）
Learning effective recipe representations is essential in food studies. Unlike what has been developed for image-based recipe retrieval or learning structural text embeddings, the combined effect of multi-modal information (i.e., recipe images, text, and relation data) receives less attention. In this paper, we formalize the problem of multi-modal recipe representation learning to integrate the visual, textual, and relational information into recipe embeddings. In particular, we first present Large-RG, a new recipe graph data with over half a million nodes, making it the largest recipe graph t…
FoodSky: A Food-oriented Large Language Model that Passes the Chef and Dietetic Examination
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Institute of Intelligent Computing Technology, Chinese Academy of Sciences（中国科学院智能计算技术研究所）；Zhongke Shenjian (Suzhou) Technology Co., Ltd（中科神箭（苏州）科技有限公司）
Deep Learning Based Named Entity Recognition Models for Recipes
Infosys Centre of Artificial Intelligence, IIIT-Delhi（IIIT-德里 Infosys 人工智能中心）；Department of Computational Biology, IIIT-Delhi（IIIT-德里 计算生物学系）；Department of Computer Science, IIIT-Delhi（IIIT-德里 计算机科学系）
AI Institute, University of South Carolina（南卡罗来纳大学人工智能研究所）
AI systems have been widely adopted across various domains in the real world. However, in high-value, sensitive, or safety-critical applications such as self-management for personalized health or food recommendation with a specific purpose (e.g., allergy-aware recipe recommendations), their adoption is unlikely. Firstly, the AI system needs to follow guidelines or well-defined processes set by experts; the data alone will not be adequate. For example, to diagnose the severity of depression, mental healthcare providers use Patient Health Questionnaire (PHQ-9). So if an AI system were to be used…
Department of Computational Biology（计算生物学系）；Infosys Center for Artificial Intelligence（Infosys人工智能中心）；Center of Excellence in Healthcare（医疗卓越中心）；Department of Mathematics（数学系）；Department of Computer Science（计算机科学系）；Indraprastha Institute of Information Technology Delhi (IIIT-Delhi)（德里印度教徒普拉斯塔信息技术学院（IIIT-Delhi））；Foodoscope Technologies Private Limited（Foodoscope技术私人有限公司）
