Patent figure classification facilitates faceted search in patent retrieval systems, enabling efficient prior art search. Existing approaches have explored patent figure classification for only a single aspect and for aspects with a limited number of concepts. In recent years, large vision-language models (LVLMs) have shown tremendous performance across numerous computer vision downstream tasks, however, they remain unexplored for patent figure classification. Our work explores the efficacy of LVLMs in patent figure visual question answering (VQA) and classification, focusing on zero-shot and few-shot learning scenarios. For this purpose, we introduce new datasets, PatFigVQA and PatFigCLS, for fine-tuning and evaluation regarding multiple aspects of patent figures~(i.e., type, projection, patent class, and objects). For a computational-effective handling of a large number of classes using LVLM, we propose a novel tournament-style classification strategy that leverages a series of multiple-choice questions. Experimental results and comparisons of multiple classification approaches based on LVLMs and Convolutional Neural Networks (CNNs) in few-shot settings show the feasibility of the proposed approaches.
From Panel to Pixel: Zoom-In Vision-Language Pretraining from Biomedical Scientific Literature
University of Strasbourg, CNRS, INSERM, ICube, UMR7357, Strasbourg, France(斯特拉斯堡大学、法国国家科学研究中心、法国国家医学研究院、ICube、UMR7357、斯特拉斯堡,法国);Technical University of Munich(慕尼黑技术大学);Stanford University(斯坦福大学);DSAI, The Hong Kong Polytechnic University(香港理工大学DSAI实验室);University of British Columbia(不列颠哥伦比亚大学);Munich Center for Machine Learning(慕尼黑机器学习中心);IHU Strasbourg, Strasbourg(斯特拉斯堡IHU医院,斯特拉斯堡);Vector Institute for AI(人工智能向量研究所);Yale University(耶鲁大学)
VisionQ: VLM-as-a-Judge Taxonomy, Dataset and Benchmark for Qualitative Analysis in Computer Vision
University of Information Technology, VNU-HCM(越南国立胡志明市大学信息技术大学);University College Cork(科克大学);Hanoi University of Science and Technology(河内科技大学);Trinity College Dublin(都柏林圣三一大学)
How Do VLMs Behave When Blind or Misled? Behavioral Evaluation of VLMs on Scientific Figures
University of Aberdeen(阿伯丁大学);International Institute of Information Technology Hyderabad(海德拉巴国际信息技术学院);University of Technology Nuremberg(纽伦堡工业大学);University of Southern California(南加利福尼亚大学)
PlotPick: AI-powered batch extraction of numerical data from scientific figures
Copenhagen Research Centre for Biological and Precision Psychiatry(哥本哈根生物与精准精神病学研究中心);Mental Health Centre Copenhagen(哥本哈根心理健康中心);Copenhagen University Hospital(哥本哈根大学医院)
Enhancing Agentic Autonomous Scientific Discovery with Vision-Language Model Capabilities
Department of Physics, University of Cambridge, Cambridge, United Kingdom(剑桥大学物理系);Kavli Institute for Cosmology, University of Cambridge, Cambridge, United Kingdom(剑桥大学卡弗利天文研究所);Department of Physics and Astronomy, Haverford College, 370 Lancaster Avenue, Haverford, PA 19041, USA(哈弗福德学院物理与天文学系);Division of Physics, Mathematics and Astronomy, California Institute of Technology, Pasadena, CA 91125, USA(加州理工学院物理、数学与天文学系);Institute of Astronomy, University of Cambridge, Cambridge, United Kingdom(剑桥大学天文研究所)
De-rendering, Reasoning, and Repairing Charts with Vision-Language Models
Universidad Torcuato Di Tella(托克托迪特拉大学);Consejo Nacional de Investigaciones Científicas y Técnicas(国家科学与技术研究理事会);Facultad de Ciencias Exactas y Naturales Universidad de Buenos Aires(布宜诺斯艾利斯大学科学与自然学院)
LLaVA-Ultra: Large Chinese Language and Vision Assistant for Ultrasound
ZJU-UIUC Institute, Zhejiang University(浙江大学ZJU-UIUC学院);University of Washington(华盛顿大学);Zhejiang University School of Medicine Sir Run Run Shaw Hospital(浙江大学医学院附属邵逸夫医院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
PPU-Bench:Real World Benchmark for Personalized Partial Unlearning in Vision Language Models
Harbin Institute of Technology(哈尔滨工业大学);Pengcheng Laboratory(鹏城实验室);The Hong Kong Polytechnic University(香港理工大学);Sichuan University(四川大学);Zhejiang Normal University(浙江师范大学)
Can Vision-Language Models Understand and Interpret Dynamic Gestures from Pedestrians? Pilot Datasets and Exploration Towards Instructive Nonverbal Commands for Cooperative Autonomous Vehicles
University of California Merced(加州大学默塞德分校);Aalborg University(奥尔堡大学)
Beyond Text and Tables: Vision-Language Model Integration in ComProScanner for Extracting Materials Data from Scientific Figures with High Accuracy
Energy, Materials and Environment Research Centre, London South Bank University, London SE1 0AA, UK(能源、材料与环境研究中心,伦敦南银行大学);School of Engineering and Design, London South Bank University, London SE1 0AA, UK(工程与设计学院,伦敦南银行大学);Bioscience and Bioengineering Research Centre, London South Bank University, London SE1 0AA, UK(生物科学与生物工程研究中心,伦敦南银行大学);Department of Physics, Kings College London, London WC2R 2LS, UK(物理系,伦敦国王学院)
M3DocDep: Multi-modal, Multi-page, Multi-document Dependency Chunking with Large Vision-Language Models
Human-inspired AI Research, Korea University(韩国大学人智AI研究所);Computer Science and Engineering, Konkuk University(konkuk大学计算机科学与工程系);Department of Computer Science and Engineering, Korea University(韩国大学计算机科学与工程系)
Developing an AI Assistant for Knowledge Management and Workforce Training in State DOTs
Department of Computer Science, University of Houston(计算机科学系,休斯顿大学);Department of Civil and Environmental Engineering, University of Houston(土木与环境工程系,休斯顿大学);Ph.D Department of Civil and Environmental Engineering, University of Houston(土木与环境工程系,休斯顿大学);Engineering Data Science, University of Houston(工程数据科学,休斯顿大学)