In this paper, we propose an end-to-end structured multimodal attention (SMA) neural network to mainly solve the first two issues above. SMA first uses a structural graph representation to encode the object-object, object-text and text-text relationships appearing in the image, and then designs a multimodal graph attention network to reason over it. Finally, the outputs from the above modules are processed by a global-local attentional answering module to produce an answer splicing together tokens from both OCR and general vocabulary iteratively by following M4C. Our proposed model outperforms the SoTA models on TextVQA dataset and two tasks of ST-VQA dataset among all models except pre-training based TAP. Demonstrating strong reasoning ability, it also won first place in TextVQA Challenge 2020. We extensively test different OCR methods on several reasoning models and investigate the impact of gradually increased OCR performance on TextVQA benchmark. With better OCR results, different models share dramatic improvement over the VQA accuracy, but our model benefits most blessed by strong textual-visual reasoning ability. To grant our method an upper bound and make a fair testing base available for further works, we also provide human-annotated ground-truth OCR annotations for the TextVQA dataset, which were not given in the original release. The code and ground-truth OCR annotations for the TextVQA dataset are available at https://github.com/ChenyuGAO-CS/SMA
Spatially Aware Multimodal Transformers for TextVQA
Georgia Institute of Technology（佐治亚理工学院）；Facebook AI Research (FAIR)（Facebook人工智能研究院（FAIR））；Google（谷歌）；University of Illinois, Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Allen Institute for AI（艾伦人工智能研究所）
EE-MLLM: A Data-Efficient and Compute-Efficient Multimodal Large Language Model
University of Science and Technology of China（中国科学技术大学）；WeChat, Tencent Inc.（腾讯微信）；Fudan University（复旦大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）
AeroRAG: Structured Multimodal Retrieval-Augmented LLM for Fine-Grained Aerial Visual Reasoning
Research Center for Space Computing System, Zhejiang Lab, Hangzhou（杭州浙大实验室空间计算系统研究中心）；Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou（中国科学院大学杭州高等研究院）；School of Cyber Science and Engineering, Zhengzhou University（郑州大学计算机科学与工程学院）；School of Computer Science and Technology, Xi’an Jiaotong University（西安交通大学计算机科学与技术学院）
Revisiting Change VQA in Remote Sensing with Structured and Native Multimodal Qwen Models
Computer Engineering Department, College of Computer and Information Sciences, King Saud University（计算机工程系，计算机与信息科学学院，沙特国王大学）；Applied Computer Science Department, College of Applied Computer Science, King Saud University（应用计算机科学系，应用计算机科学学院，沙特国王大学）
Enhancing Instruction-Following Capability of Visual-Language Models by Reducing Image Redundancy
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Kuaishou Technology（快手科技）；CAIR, HKISI, Chinese Academy of Sciences（中国科学院香港创新研究院人工智能与机器人创新中心）
Can Atomic Step Decomposition Enhance the Self-structured Reasoning of Multimodal Large Models?
Sun Yat-sen University（中山大学）；Hong Kong University of Science and Technology（香港科学与技术大学）；Shanghai Jiaotong University（上海交通大学）；University of Hong Kong（香港大学）；Huawei Noah’s Ark Lab（华为诺亚实验室）
AtomThink: Multimodal Slow Thinking with Atomic Step Reasoning
Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；ETH Zurich（苏黎世联邦理工学院）；Hong Kong University of Science and Technology（香港科技大学）；University of Hong Kong（香港大学）；Noah’s Ark Lab（诺亚实验室）；Yinwang Intelligent Technology Co., Ltd.（亿纬智能科技有限公司）
Stitch and Tell: A Structured Multimodal Data Augmentation Method for Spatial Understanding
School of Computer Science, Beijing Institute of Technology（北京理工大学计算机科学学院）；School of Software and Microelectronics, Peking University（北京大学软件与微电子学院）；Xiaohongshu Inc（小红书公司）
Stitch and Tell通过结构化空间监督提升视觉-语言模型的空间理解能力，有效缓解空间幻觉并提高相关任务性能。
Towards Efficient VLMs: Information-Theoretic Driven Compression via Adaptive Structural Pruning
School of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）；Zhongtai Securities Institute for Financial Studies, Shandong University（山东大学中泰证券金融研究学院）
MMRad-22K: A Structured Multimodal Evidence Dataset for Chest X-ray Report Generation
MoE Key Lab of Artificial Intelligence, AI Institute, School of Computer Science, Shanghai Jiao Tong University（人工智能MOE实验室、人工智能研究院、计算机科学学院、上海交通大学）；School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China (USTC)（生物医学工程学院、生命科学与医学系、中国科学技术大学）；Center for Medical Imaging, Robotics, Analytic Computing & Learning (MIRACLE), Suzhou Institute for Advanced Research, USTC（医学影像、机器人、分析计算与学习中心（MIRACLE）、苏州市先进研究院、中国科学技术大学）；Department of Radiology, The First Affiliated Hospital, Zhejiang University School of Medicine（放射科、浙江大学医学院第一附属医院）
MeniOmni: A Structured Multimodal Benchmark for Holistic Meniscus Injury Assessment
1School of Electronics, Electrical Engineering；Computer Science, Queen's University Belfast, Belfast, UK；2Radiology Department, Affiliated Nantong Clinical College of Nantong University, Nantong First People's Hospital, School of Clinical Medicine, Nantong University, Nantong, Jiangsu, China；3School of Artificial Intelligence；Computer Science, Nantong University, Nantong, China；4Faculty of Data Science, City University of Macau, Macau, China；5Department of Chemistry, University of Oxford, Oxford, UK；6Orthopedics Department, Nantong First People's Hospital, Southeast University, Nantong, Jiangsu, China
InterPruner: Interactive Structured Pruning via Taylor-Implicit Criterion and Language-Prior Modulator for Multimodal Object Detection
Beijing University of Technology（北京工业大学）；Southwest University（西南大学）；China University of Geosciences Wuhan（中国地质大学（武汉））；Tsinghua University（清华大学）；Beijing Institute of Technology（北京理工大学）；Yunnan Normal University（云南师范大学）；Beijing Forestry University（北京林业大学）；Ghent University（根特大学）
Quantum-Inspired Spectral Geometry for Neural Operator Equivalence and Structured Pruning
School of Computer Jiangsu University of Science and Technology（江苏科技大学计算机学院）；School of Computer and Software Nanjing University of Information Science and Technology（南京信息工程大学计算机与软件学院）；Department of Electrical and Computer Engineering University of Nevada Las Vegas（内华达大学拉斯维加斯分校电气与计算机工程系）
CUS-GS: A Compact Unified Structured Gaussian Splatting Framework for Multimodal Scene Representation
School of Computer Science, Hangzhou Dianzi University（杭州电子科技大学计算机科学学院）；CAD & CG, Zhejiang University（浙江大学计算机辅助设计与图形学研究所）；School of Computer Science, University of Bristol（布里斯托大学计算机科学学院）
