Vision-language models (VLMs) are highly effective but often underperform on specialized tasks; for example, Llava-1.5 struggles with chart and diagram understanding due to scarce task-specific training data. Existing training data, sourced from general-purpose datasets, fails to capture the nuanced details needed for these tasks. We introduce MM-Gen, a scalable method that generates task-specific, high-quality synthetic text for candidate images by leveraging stronger models. MM-Gen employs a three-stage targeted process: partitioning data into subgroups, generating targeted text based on task descriptions, and filtering out redundant and outlier data. Fine-tuning VLMs with data generated by MM-Gen leads to significant performance gains, including 29% on spatial reasoning and 15% on diagram understanding for Llava-1.5 (7B). Compared to human-curated caption data, MM-Gen achieves up to 1.6x better improvements for the original models, proving its effectiveness in enhancing task-specific VLM performance and bridging the gap between general-purpose datasets and specialized requirements. Code available at https://github.com/sjoshi804/MM-Gen.
SCOPE-MRI: Bankart Lesion Detection as a Case Study in Data Curation and Deep Learning for Challenging Diagnoses
Pritzker School of Medicine, University of Chicago(芝加哥大学普ritzker医学学院);University of Chicago(芝加哥大学);Department of Computer Science, University of Chicago(芝加哥大学计算机科学系);Department of Orthopaedic Surgery & Rehabilitation Medicine, UChicago Medicine(芝加哥医学中心骨科与康复医学系)
Virginia Tech(弗吉尼亚理工大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Wisconsin-Madison(威斯康星大学麦迪逊分校);University of California, Berkeley(加州大学伯克利分校)
Fair and accurate age prediction using distribution aware data curation and augmentation
Nanyang Technological University(南洋理工大学);Singapore Institute of Manufacturing and Technology, A*STAR(新加坡科技研究局制造技术研究院);Singapore Institute of High Performance Computing, A*STAR(新加坡科技研究局高性能计算研究院);Ben-Gurion University of the Negev(内盖夫本-古里安大学)
Systematic Reward Gap Optimization for Mitigating VLM Hallucinations
School of Software, Beihang University(北京航空航天大学软件学院);Shanghai Innovation Institute(上海创新研究院);Shanghai AI Laboratory(上海人工智能实验室);Tsinghua University(清华大学)
From Data Deluge to Data Curation: A Filtering-WoRA Paradigm for Efficient Text-based Person Search
Beijing Institute of Technology(北京理工大学);National University of Singapore(新加坡国立大学);University of Macau(澳门大学);Institute for Clarity in Documentation(文档清晰研究所);Inria Paris-Rocquencourt(法国国家信息与自动化研究所巴黎-罗康分中心);Rajiv Gandhi University(拉吉夫·甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕尔默研究实验室)
JAVEDIT: Joint Audio-Visual Instruction-Guided Video Editing with Agentic Data Curation
Zhejiang University(浙江大学);Tencent Youtu Lab(腾讯优图实验室);Nanjing University(南京大学);University of Auckland(奥克兰大学);Fudan University(复旦大学);National University of Singapore(新加坡国立大学)