How far can we go with textual representations for understanding pictures? In image understanding, it is essential to use concise but detailed image representations. Deep visual features extracted by vision models, such as Faster R-CNN, are prevailing used in multiple tasks, and especially in visual question answering (VQA). However, conventional deep visual features may struggle to convey all the details in an image as we humans do. Meanwhile, with recent language models' progress, descriptive text may be an alternative to this problem. This paper delves into the effectiveness of textual representations for image understanding in the specific context of VQA. We propose to take description-question pairs as input, instead of deep visual features, and fed them into a language-only Transformer model, simplifying the process and the computational cost. We also experiment with data augmentation techniques to increase the diversity in the training set and avoid learning statistical bias. Extensive evaluations have shown that textual representations require only about a hundred words to compete with deep visual features on both VQA 2.0 and VQA-CP v2.
A Single 2D Pose with Context is Worth Hundreds for 3D Human Pose Estimation
Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）；Center for Research in Computer Vision, University of Central Florida（中佛罗里达大学计算机视觉研究中心）；Key Laboratory of Machine Perception, Peking University, Shenzhen Graduate School（北京大学深圳研究生院机器感知重点实验室）
VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks
OpenGVLab（开放通用视觉实验室）；Shanghai AI Laboratory（上海人工智能实验室）；Tsinghua University（清华大学）；SenseTime Research（商汤科技研究院）；Nanjing University（南京大学）；The University of Hong Kong（香港大学）；Beijing Institute of Technology（北京理工大学）；The Hong Kong University of Science and Technology（香港科技大学）；The Chinese University of Hong Kong（香港中文大学）
VisionLLM v2 提出端到端通用多模态大模型，通过 super link 机制连接 MLLM 与任务专用解码器，并在数百个视觉语言任务上联合训练，以共享参数实现目标定位、姿态估计、图像生成编辑等多样化任务，性能可比肩专用模型。
What Large Language Models Bring to Text-rich VQA?
SenseTime Research（商汤科技研究院）；Nanjing University of Science and Technology（南京理工大学）；Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；ByteDance Inc.（字节跳动有限公司）
Revisiting CroPA: A Reproducibility Study and Enhancements for Cross-Prompt Adversarial Transferability in Vision-Language Models
Mehta Family School of Data Science and Artificial Intelligence（梅hta家族数据科学与人工智能学院）；Indian Institute of Technology, Roorkee（印度理工学院罗奥克学院）；Department of Civil Engineering（土木工程系）；Department of Electronics and Communication（电子与通信系）
Core Tokensets for Data-efficient Sequential Training of Transformers
Hessian Center for AI (hessian.AI)（黑森人工智能中心）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；Centre for Cognitive Science, TU Darmstadt（达姆施塔特工业大学认知科学中心）；Department of Computer Science, TU Darmstadt（达姆施塔特工业大学计算机科学系）
WaveFuse: A Unified Deep Framework for Image Fusion with Discrete Wavelet Transform
School of Basic Medical Science, Fudan University（复旦大学基础医学院）；Shanghai Key Laboratory of Medical Image Computing and Computer Assited Intervention（上海市医学图像计算与计算机辅助干预重点实验室）
