Vision-Language Models (VLMs) have demonstrated impressive capabilities in code generation across various domains. However, their ability to replicate complex, multi-panel visualizations from real-world data remains largely unassessed. To address this gap, we introduce \textbf{\texttt{RealChart2Code}}, a new large-scale benchmark with over 2,800 instances grounded in authentic datasets and featuring tasks with clear analytical intent. Crucially, it is the first benchmark to systematically evaluate chart generation from large-scale raw data and assess iterative code refinement in a multi-turn conversational setting. Our comprehensive evaluation of 14 leading VLMs on \texttt{RealChart2Code} reveals significant performance degradation compared to simpler benchmarks, highlighting their struggles with complex plot structures and authentic data. Our analysis uncovers a substantial performance gap between proprietary and open-weight models and confirms that even state-of-the-art VLMs often fail to accurately replicate intricate, multi-panel charts. These findings provide valuable insights into the current limitations of VLMs and guide future research directions. We release the benchmark and code at \url{https://github.com/Speakn0w/RealChart2Code}.
ChartEdit: How Far Are MLLMs From Automating Chart Analysis? Evaluating MLLMs' Capability via Chart Editing
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Harbin Institute of Technology(哈尔滨工业大学);Tianjin University(天津大学);Tsinghua University(清华大学)
Do LVLMs Understand Charts? Analyzing and Correcting Factual Errors in Chart Captioning
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Columbia University(哥伦比亚大学);Salesforce AI Research(Salesforce AI研究院);DAMO Academy, Alibaba Group(阿里巴巴达摩院)
ChartReasoner: Code-Driven Modality Bridging for Long-Chain Reasoning in Chart Question Answering
MAIS, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所MAIS研究所);Shenyang Institute of Computing Technology, Chinese Academy of Sciences(中国科学院沈阳计算技术研究所);Beijing Wenge Technology Co., Ltd.(北京文耕科技有限公司);University of Chinese Academy of Sciences(中国科学院大学)
ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answering
York University(约克大学);Dialpad Inc.(Dialpad公司);RBC(加拿大皇家银行);MILA - Quebec AI Institute(米拉-魁北克人工智能研究所);Qatar Computing Research Institute (QCRI)(卡塔尔计算研究所);Nanyang Technological University(南洋理工大学);Salesforce Research(Salesforce研究院)
Distill Visual Chart Reasoning Ability from LLMs to MLLMs
School of Computer Science, Fudan University(复旦大学计算机学院);Weixin Group, Tencent(腾讯Weixin部门);Shanghai Innovation Institute(上海创新研究院);Shanghai Key Lab of Intelligent Information Processing(上海智能信息处理重点实验室)
ChartThinker: A Contextual Chain-of-Thought Approach to Optimized Chart Summarization
Sun Yat-sen University(中山大学);Alibaba Group(阿里巴巴集团);Guangdong Provincial Key Laboratory of Fire Science and Intelligent Emergency Technology(广东省消防科学与智能应急技术重点实验室);Pazhou Lab(琶洲实验室)
ChartAttack: Testing the Vulnerability of LLMs to Malicious Prompting in Chart Generation
INSAIT, Sofia University "St. Kliment Ohridski"(INSAIT索菲亚大学"圣克莱门特·欧赫里迪斯基");Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science, TU Darmstadt and National Research Center for Applied Cybersecurity ATHENE(无处不在知识处理实验室(UKP实验室)、计算机科学系、图腾达姆斯塔特大学和应用网络安全国家研究中心ATHENE);Arizona State University(亚利桑那州立大学)
Advancing Sentiment Analysis in Tamil-English Code-Mixed Texts: Challenges and Transformer-Based Solutions
Instituto Politécnico Nacional (IPN)(墨西哥国立理工学院);Centro de Investigación en Computación (CIC)(计算机研究中心);Escuela Superior de Ingeniería Mecánica y Eléctrica (ESIME)(机械与电气工程高等学院)
ChartSync: A Benchmark for Visuo-Logical Cascading Chart Editing
Jianghan University(江汉大学);HiThink Research(海思睿研);University of Science and Technology of China(中国科学技术大学);Southeast University(东南大学);Zhejiang University(浙江大学)
A chart review process aided by natural language processing and multi-wave adaptive sampling to expedite validation of code-based algorithms for large database studies
Brigham and Women’s Hospital(布列根和妇女医院);Harvard Medical School(哈佛医学院);Center for Drug Evaluation and Research, Food and Drug Administration(食品药品监督管理局药品评估与研究中心);Harvard Pilgrim Health Care Institute(哈佛朝圣者医疗保健研究所)
Large Language Models are Qualified Benchmark Builders: Rebuilding Pre-Training Datasets for Advancing Code Intelligence Tasks
College of Computer, National University of Defense Technology(计算机学院,国防科技大学);Sun Yat-sen University(中山大学);College of Banking and Financial Studies(金融学院)
ClimateViz: A Benchmark for Statistical Reasoning and Fact Verification on Scientific Charts
University of Oxford(牛津大学);Qilu University of Technology(Shandong Academy of Sciences)(齐鲁工业大学(山东科学院));Hong Kong University of Science and Technology(香港科学大学);Hong Kong University of Science and Technology (Guangzhou)(香港科学大学(广州))
The Vault: A Comprehensive Multilingual Dataset for Advancing Code Understanding and Generation
FPT Software AI Center(FPT软件人工智能中心);Hanoi University of Science and Technology(河内科技大学);McGill University(麦吉尔大学);Mila - Quebec AI Institute(米拉-魁北克人工智能研究所);Fulbright University(富布赖特大学)
本文提出多语言代码-文本数据集 The Vault,包含 4300 万高质量样本,通过规则与深度学习方法提取构建,实验表明在其上微调的代码大模型在代码生成、搜索和摘要任务上优于 CodeSearchNet 等数据集训练结果。