Multimodal Large Language Models (MLLMs) have demonstrated remarkable success in visual understanding, yet their performance degrades significantly under real-world visual corruptions. While existing robustness enhancement approaches exist, they are limited: black-box feature alignment lacks interpretability, and white-box text-based reasoning cannot restore lost pixel-level details. This work investigates a fundamental research question: Can MLLMs recover corrupted visual content by themselves? To address this, we propose Robust-U1, a novel framework that equips MLLMs with explicit visual self-recovery capability for robust understanding. The approach comprises three core stages: supervised fine-tuning for initial reconstruction, reinforcement learning with dual rewards (pixel-level SSIM and semantic-level CLIP similarity) for aligning high visual quality, and multimodal reasoning that jointly considers both the corrupted input and the recovered image. Extensive experiments demonstrate that Robust-U1 achieves state-of-the-art robustness on the real-world corruption benchmark and maintains superior performance under adversarial corruptions on general VQA benchmarks. Analysis confirms that high-quality visual recovery directly enhances reasoning performance, establishing self-recovery as a critical mechanism for robust visual understanding. The source code is available at https://github.com/jqtangust/Robust-U1.
Corrupted but Not Broken: Understanding and Mitigating the Negative Impacts of Corrupted Data in Visual Instruction Tuning
Southern University of Science and Technology（南方科技大学）；The Hong Kong University of Science and Technology（香港科技大学）；Huawei Noah’s Ark Lab（华为诺亚实验室）；Hong Kong Baptist University（香港 Baptist 大学）
TriViewBench: Controlled Complexity Scaling for Multi-View Structural Reasoning in MLLMs
School of Intelligence Science and Technology, Nanjing University（南京大学智能科学与技术学院）；National Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）
Decoupling and Regulating Modality Heads for Fake News Detection under Missing Modality
Soochow University（苏州大学）；Hong Kong University of Science and Technology（香港科技大学）；Computer Science and Engineering, The Chinese University of Hong Kong（香港中文大学计算机科学与工程系）；The Hong Kong University of Science and Technology（香港科技大学）；FiT, Tencent（腾讯FiT）；Alibaba Group（阿里巴巴集团）；School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；School of Education, Zhejiang Normal University（浙江师范大学教育学院）
Jury Duty: Calibration and Orientation Failures in MLLM-as-a-Judge Under Cultural Ambiguity
Salesforce AI Research（Salesforce AI 研究院）；University of Cambridge（剑桥大学）；University of Colorado Boulder（科罗拉多大学博尔德分校）；Carnegie Mellon University（卡内基梅隆大学）；UIUC（伊利诺伊大学厄巴纳-香槟分校）
Can MLLMs Decode the Creative Leap? Introducing C4 for Cross-Concept Understanding
School of Computer Science and Engineering, Northeastern University（东北大学计算机科学与工程学院）；School of Computing and Information Systems, Singapore Management University（新加坡管理大学计算与信息系统学院）；School of Computer Science and Engineering, Shandong University of Science and Technology（山东科技大学计算机科学与工程学院）
On the Limits of Representation-Based Test-Time Adaptation for Multimodal LLMs
IDEAS Research Institute（IDEAS研究所）；Warsaw University of Technology（华沙理工大学）；Akces NCBR（NCBR Akces）；NASK National Research Institute（NASK国家研究所）；Computer Vision Center, Barcelona（巴塞罗那计算机视觉中心）
From What to Which: Decoding Modifier Grounding in Frozen MLLMs
AI for Good (AIGO)（人工智能促进美好（AIGO））；Istituto Italiano di Tecnologia（意大利技术研究院）；University of Siena（锡耶纳大学）；University of Genoa（热那亚大学）；University of Verona（维罗纳大学）
Towards Blind Bitstream-corrupted Video Recovery via a Visual Foundation Model-driven Framework
School of EEE, Nanyang Technological University（南洋理工大学电子工程系）；School of EIC, Huazhong University of Science and Technology（华中科技大学电子信息学院）；Dept. of EEE, The Hong Kong Polytechnic University（香港理工大学电子工程系）
Massachusetts Institute of Technology（麻省理工学院）；Rice University（莱斯大学）
Let $t_1,\ldots,t_n \in \mathbb{R}^d$ and consider the location recovery problem: given a subset of pairwise direction observations $\{(t_i - t_j) / \|t_i - t_j\|_2\}_{i<j \in [n] \times [n]}$, where a constant fraction of these observations are arbitrarily corrupted, find $\{t_i\}_{i=1}^n$ up to a global translation and scale. We propose a novel algorithm for the location recovery problem, which consists of a simple convex program over $dn$ real variables. We prove that this program recovers a set of $n$ i.i.d. Gaussian locations exactly and with high probability if the observations are given…
This paper is concerned with the problem of recovering a structured signal from a relatively small number of corrupted random measurements. Sharp phase transitions have been numerically observed in practice when different convex programming procedures are used to solve this problem. This paper is devoted to presenting theoretical explanations for these phenomenons by employing some basic tools from Gaussian process theory. Specifically, we identify the precise locations of the phase transitions for both constrained and penalized recovery procedures. Our theoretical results show that these phas…
Investigating Structural Pruning and Recovery Techniques for Compressing Multimodal Large Language Models: An Empirical Study
Technical University of Munich, Germany（慕尼黑技术大学，德国）；Helmholtz Munich, Munich Center for Machine Learning, Germany（海德堡慕尼黑，慕尼黑机器学习中心，德国）；University of Tübingen, Tübingen AI Center, Germany（图宾根大学，图宾根人工智能中心，德国）；University of Trento, Italy（特伦托大学，意大利）；Beijing University of Posts and Telecommunications, China（北京邮电大学，中国）
Massachusetts Institute of Technology（麻省理工学院）；Rice University（莱斯大学）
Let $t_1,\ldots,t_{n_l} \in \mathbb{R}^d$ and $p_1,\ldots,p_{n_s} \in \mathbb{R}^d$ and consider the bipartite location recovery problem: given a subset of pairwise direction observations $\{(t_i - p_j) / \|t_i - p_j\|_2\}_{i,j \in [n_l] \times [n_s]}$, where a constant fraction of these observations are arbitrarily corrupted, find $\{t_i\}_{i \in [n_ll]}$ and $\{p_j\}_{j \in [n_s]}$ up to a global translation and scale. We study the recently introduced ShapeFit algorithm as a method for solving this bipartite location recovery problem. In this case, ShapeFit consists of a simple convex progra…
Reference-Efficient Multimodal Bitstream-Corrupted Video Recovery via Depth-Conditioned Adaptation
Nanyang Technological University（南洋理工大学）；Huazhong University of Science and Technology（华中科技大学）；Harbin Engineering University（哈尔滨工程大学）；Institute for Infocomm Research, Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局信息通信研究院）；National Tsing Hua University（国立清华大学）
National University of Singapore（新加坡国立大学）；Cornell University（康奈尔大学）；Didi Research（滴滴研究院）；Peking University（北京大学）；Shanghai Jiaotong University（上海交通大学）；AI Institute（360人工智能研究院）
This paper studies the Tensor Robust Principal Component (TRPCA) problem which extends the known Robust PCA (Candes et al. 2011) to the tensor case. Our model is based on a new tensor Singular Value Decomposition (t-SVD) (Kilmer and Martin 2011) and its induced tensor tubal rank and tensor nuclear norm. Consider that we have a 3-way tensor ${\mathcal{X}}\in\mathbb{R}^{n_1\times n_2\times n_3}$ such that ${\mathcal{X}}={\mathcal{L}}_0+{\mathcal{E}}_0$, where ${\mathcal{L}}_0$ has low tubal rank and ${\mathcal{E}}_0$ is sparse. Is that possible to recover both components? In this work, we prove …
School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）；CSE Department, University of Texas at Arlington（德克萨斯大学阿灵顿分校计算机科学与工程系）
In many real-world applications, image data often come with noises, corruptions or large errors. One approach to deal with noise image data is to use data recovery techniques which aim to recover the true uncorrupted signals from the observed noise images. In this paper, we first introduce a novel corruption recovery transformation (CRT) model which aims to recover multiple (or a collection of) corrupted images using a single affine transformation. Then, we show that the introduced CRT can be efficiently constructed through learning from training data. Once CRT is learned, we can recover the t…
Huazhong University of Science & Technology（华中科技大学）；Microsoft Research（微软研究院）
We introduce Corrupted Image Modeling (CIM) for self-supervised visual pre-training. CIM uses an auxiliary generator with a small trainable BEiT to corrupt the input image instead of using artificial [MASK] tokens, where some patches are randomly selected and replaced with plausible alternatives sampled from the BEiT output distribution. Given this corrupted image, an enhancer network learns to either recover all the original image pixels, or predict whether each visual token is replaced by a generator sample or not. The generator and the enhancer are simultaneously trained and synergistically…
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Peking University（北京大学）；Nanjing University of Information Science and Technology（南京信息工程大学）
In this paper, we study the problem of matrix recovery, which aims to restore a target matrix of authentic samples from grossly corrupted observations. Most of the existing methods, such as the well-known Robust Principal Component Analysis (RPCA), assume that the target matrix we wish to recover is low-rank. However, the underlying data structure is often non-linear in practice, therefore the low-rankness assumption could be violated. To tackle this issue, we propose a novel method for matrix recovery in this paper, which could well handle the case where the target matrix is low-rank in an im…
We study a data model in which the data matrix D can be expressed as D = L + S + C, where L is a low rank matrix, S an element-wise sparse matrix and C a matrix whose non-zero columns are outlying data points. To date, robust PCA algorithms have solely considered models with either S or C, but not both. As such, existing algorithms cannot account for simultaneous element-wise and column-wise corruptions. In this paper, a new robust PCA algorithm that is robust to simultaneous types of corruption is proposed. Our approach hinges on the sparse approximation of a sparsely corrupted column so that…
