Flowcharts are typically presented as images, driving the trend of using vision-language models (VLMs) for end-to-end flowchart understanding. However, two key challenges arise: (i) Limited controllability--users have minimal influence over the downstream task, as they can only modify input images, while the training of VLMs is often out of reach for most researchers. (ii) Lack of explainability--it is difficult to trace VLM errors to specific causes, such as failures in visual encoding or reasoning. We propose TextFlow, addressing aforementioned issues with two stages: (i) Vision Textualizer--which generates textual representations from flowchart images; and (ii) Textual Reasoner--which performs question-answering based on the text representations. TextFlow offers three key advantages: (i) users can select the type of text representations (e.g., Graphviz, Mermaid, PlantUML), or further convert them into executable graph object to call tools, enhancing performance and controllability; (ii) it improves explainability by helping to attribute errors more clearly to visual or textual processing components; and (iii) it promotes the modularization of the solution, such as allowing advanced LLMs to be used in the Reasoner stage when VLMs underperform in end-to-end fashion. Experiments on the FlowVQA and FlowLearn benchmarks demonstrate TextFlow's state-of-the-art performance as well as its robustness. All code is publicly available.
Prism: A Framework for Decoupling and Assessing the Capabilities of VLMs
Tongji University(同济大学);Nanjing University of Posts and Telecommunications(南京邮电大学);University of Science and Technology of China(中国科学技术大学);Shanghai AI Laboratory(上海人工智能实验室);The Chinese University of Hong Kong(香港中文大学);Nanjing University(南京大学)
Image-to-Text Logic Jailbreak: Your Imagination can Help You Do Anything
Faculty of Computer Science, University of Exeter(埃克塞特大学计算机学院);College of Electronic and Information Engineering, Nanjing University of Aeronautics and Astronautics(南京航空航天大学电子信息工程学院)
ELITE: Enhanced Language-Image Toxicity Evaluation for Safety
Yonsei University(延世大学);Kyung Hee University(庆熙大学);Korea Institute of Science(韩国科学研究院);Seoul National University(首尔国立大学);Sookmyung Women's University(_sookmyung女子大学)
Do Vision-Language Models Truly Perform Vision Reasoning? A Rigorous Study of the Modality Gap
College of Computing and Data Science, Nanyang Technological University, Singapore(南洋理工大学计算机与数据科学学院,新加坡);Alibaba-NTU Global e-Sustainability CorpLab (ANGEL)(阿里巴巴-国立大学全球可持续发展公司实验室(ANGEL));Tongyi Lab, Alibaba Group, China(阿里云实验室,阿里巴巴集团,中国)
Advancing Complex Wide-Area Scene Understanding with Hierarchical Coresets Selection
Institute of Software Chinese Academy of Sciences(中国科学院软件研究所);University of the Chinese Academy of Sciences(中国科学院大学);Beijing University of Technology(北京理工大学)
Beyond Shortcuts: Mitigating Visual Illusions in Frozen VLMs via Qualitative Reasoning
Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(人工智能研究院,合肥国家科学中心);Anhui Polytechnic University(安徽理工大学);Hefei University of Technology(合肥工业大学);Anhui University(安徽大学);IGS, Imperial College London(帝国理工学院伦敦分校)
Integrating Video and Text: A Balanced Approach to Multimodal Summary Generation and Evaluation
IRIT, University of Toulouse, France(IRIT,图卢兹大学,法国);Institute for Infocomm Research (I 2 R), A*STAR, Singapore(信息通信研究所(I2R),A*STAR,新加坡);CNRS, IRIT, France(CNRS,IRIT,法国)
Manager: Aggregating Insights from Unimodal Experts in Two-Tower VLMs and MLLMs
Research Center for Social Computing and Information Retrieval, Harbin Institute of Technology(社会计算与信息检索研究中心,哈尔滨工业大学);School of Computer Science and Engineering, Central South University(计算机科学与工程学院,中南大学);School of Computing, National University of Singapore(计算学院,新加坡国立大学)
Similarity-as-Evidence: Calibrating Overconfident VLMs for Interpretable and Label-Efficient Medical Active Learning
School of Electronic Technology and Engineering, Xiamen University, Xiamen, China(厦门大学电子技术与工程学院,厦门,中国);School of Informatics, Xiamen University, Xiamen, China(厦门大学信息学院,厦门,中国);School of Engineering, Case Western Reserve University, Cleveland, USA(凯斯西储大学工程学院,克利夫兰,美国)
Revisiting CroPA: A Reproducibility Study and Enhancements for Cross-Prompt Adversarial Transferability in Vision-Language Models
Mehta Family School of Data Science and Artificial Intelligence(梅hta家族数据科学与人工智能学院);Indian Institute of Technology, Roorkee(印度理工学院罗奥克学院);Department of Civil Engineering(土木工程系);Department of Electronics and Communication(电子与通信系)