Procedural materials, represented as functional node graphs, are ubiquitous in computer graphics for photorealistic material appearance design. They allow users to perform intuitive and precise editing to achieve desired visual appearances. However, creating a procedural material given an input image requires professional knowledge and significant effort. In this work, we leverage the ability to convert procedural materials into standard Python programs and fine-tune a large pre-trained vision-language model (VLM) to generate such programs from input images. To enable effective fine-tuning, we also contribute an open-source procedural material dataset and propose to perform program-level augmentation by prompting another pre-trained large language model (LLM). Through extensive evaluation, we show that our method outperforms previous methods on both synthetic and real-world examples.
A Vision Language Model for Generating Procedural Plant Architecture Representations from Simulated Images
Biological and Agricultural Engineering(生物与农业工程系);Department of Plant Sciences(植物科学系);Viticulture and Enology(葡萄栽培与酿酒系);Wageningen University & Research(瓦赫宁根大学与研究学院)
Vision-Based Mistake Analysis in Procedural Activities: A Review of Advances and Challenges
Institute of Computer Science, Foundation for Research and Technology – Hellas(计算机科学研究所,希腊基础研究与技术机构);Computer Science Department, University of Crete(克里特大学计算机科学系)
Can You Learn to See Without Images? Procedural Warm-Up for Vision Transformers
Australian Institute for Machine Learning (AIML), University of Adelaide, Australia(澳大利亚机器学习研究所(AIML),阿德莱德大学,澳大利亚);École Polytechnique Fédérale de Lausanne (EPFL), Switzerland(瑞士联邦理工学院(EPFL));Idiap Research Institute, Switzerland(日内瓦研究所)
MemVLN: Episodic and Procedural Memory for Vision-and-Language Navigation
The Chinese University of Hong Kong(香港中文大学);LIGHTSPEED(光速);The University of Hong Kong(香港大学);The Hong Kong University of Science and Technology(香港科技大学)
CAS-IQA: Teaching Vision-Language Models for Synthetic Angiography Quality Assessment
State Key Laboratory of Multimodal Artificial Intelligence Systems(多模态人工智能系统国家重点实验室);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);University of Science and Technology Beijing(北京科技大学);Hunan University of Science and Technology(湖南科技大学);Shanxi University of Finance and Economics(山西财经大学);Beijing Friendship Hospital, Capital Medical University(首都医科大学北京友谊医院)
Procedural Core: A Compact Recurrent Initialization for Vision Transformers
Australian Institute for Machine Learning (AIML), Adelaide University(阿德莱德大学澳大利亚机器学习研究所);Bielefeld University(比勒费尔德大学);Metacognition AI;Idiap Research Institute(Idiap 研究所)
CoPESD: A Multi-Level Surgical Motion Dataset for Training Large Vision-Language Models to Co-Pilot Endoscopic Submucosal Dissection
The Chinese University of Hong Kong(香港中文大学);Shanghai AI Laboratory(上海人工智能实验室);The Qilu Hospital of Shandong University(山东大学齐鲁医院);National University o(国立大学)
Vrije University of Amsterdam University of Amsterdam Centrum Wiskunde \& Informatic;University of Amsterdam National Institute of Informatics Centrum Wiskunde \& Informatic;Centrum Wiskunde \& Informatic Leiden University University College London;Vrije University of Amsterdam;Centrum Wiskunde \& Informatica Technische Universiteit Delft Guangzhou University;Vrije University of Amsterdam Centrum Wiskunde \& Informatic
Evaluation of Medical Vision Language Models HuluMed and MedGemma, and general purpose chatbots Gemma 3, ChatGPT Plus, and Claude Pro on real previously unseen wound images
Department of Computer Science, New Jersey Institute of Technology(新泽西理工学院计算机科学系);Vascular and Endovascular Surgery, Robert Wood Johnson Hospital(罗伯特·伍德·约翰逊医院血管外科);Department of Data Science, New Jersey Institute of Technology(新泽西理工学院数据科学系)
本研究评估了六种视觉语言模型在慢性伤口分析任务上的表现,发现通用模型 ChatGPT 和 Claude 显著优于医学专用模型,表明广泛的多模态推理能力比领域知识更重要。
Text-conditioned Segmentation for Tomato Phenotyping via Procedural Synthetic Data
Computer, Electrical and Mathematical Sciences and Engineering Division, KAUST(计算机、电气与数学科学与工程系,沙特阿卜杜拉国王科技大学);Department of Computer Science, University of Calgary(卡尔加里大学计算机科学系);Department of Computer Science, Kiel University(基尔大学计算机科学系);Department of Artificial Intelligence, Adam Mickiewicz University(亚当·密茨凯维奇大学人工智能系);Department of Electronics and Telecommunications, Polytechnic University of Turin(都灵理工大学电子与电信系)