Multimodal models typically combine a powerful large language model (LLM) with a vision encoder and are then trained on multimodal data via instruction tuning. While this process adapts LLMs to multimodal settings, it remains unclear whether this adaptation compromises their original language reasoning capabilities. In this work, we explore the effects of multimodal instruction tuning on language reasoning performance. We focus on LLaVA, a leading multimodal framework that integrates LLMs such as Vicuna or Mistral with the CLIP vision encoder. We compare the performance of the original LLMs with their multimodal-adapted counterparts across eight language reasoning tasks. Our experiments yield several key insights. First, the impact of multimodal learning varies between Vicuna and Mistral: we observe a degradation in language reasoning for Mistral but improvements for Vicuna across most tasks. Second, while multimodal instruction learning consistently degrades performance on mathematical reasoning tasks (e.g., GSM8K), it enhances performance on commonsense reasoning tasks (e.g., CommonsenseQA). Finally, we demonstrate that a training-free model merging technique can effectively mitigate the language reasoning degradation observed in multimodal-adapted Mistral and even improve performance on visual tasks.
EVOREFUSE: Evolutionary Prompt Optimization for Evaluation and Mitigation of LLM Over-Refusal to Pseudo-Malicious Instructions
Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University（航天信息安全部门、教育部、武汉大学计算机科学与工程学院）；Ant Group（蚂蚁集团）；Ant International（蚂蚁国际）；School of Computing Technologies, Royal Melbourne Institute of Technology（皇家墨尔本理工学院计算机技术学院）
Zero-Shot Vision-and-Language Navigation with Collision Mitigation in Continuous Environment
Seoul National University（首尔大学）；AI Institute of Seoul National University (AIIS)（首尔大学人工智能研究院）；Dept. of Computer Science and Engineering, Seoul National University（首尔大学计算机科学与工程系）
Understanding Textual Capability Degradation in Speech LLMs via Parameter Importance Analysis
National Engineering Research Center of Speech and Language Information Processing（语音与语言信息处理国家级工程研究中心）；University of Science and Technology of China（中国科学技术大学）
SwitchCIT: Switching for Continual Instruction Tuning
Coordinated Science Laboratory（协调科学实验室）；Department of Electrical and Computer Engineering（电气与计算机工程系）；Department of Mathematics（数学系）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Know Thy Enemy: Securing LLMs Against Prompt Injection via Diverse Data Synthesis and Instruction-Level Chain-of-Thought Learning
State Key Laboratory of Complex System Modeling and Simulation Technology（复杂系统建模与仿真技术国家重点实验室）；Science and Technology on Integrated Information System Laboratory, Institute of Software Chinese Academy of Sciences（中国科学院软件研究所综合信息系统技术国家级重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Nanyang Technological University（南洋理工大学）；Beijing Forestry University（北京林业大学）
Spectral Characterization and Mitigation of Sequential Knowledge Editing Collapse
Shandong University（山东大学）；School of Computer Science, Beijing University of Posts and Telecommunications（北京邮电大学计算机学院）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）
Visual-Instructed Degradation Diffusion for All-in-One Image Restoration
State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), CASIA（多模态人工智能系统国家重点实验室（MAIS）、中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Ant Group（蚂蚁集团）；PeopleAI Inc.（PeopleAI公司）
Attention Hijackers: Detect and Disentangle Attention Hijacking in LVLMs for Hallucination Mitigation
Center for Future Media, University of Electronic Science and Technology of China（电子科技大学未来媒体中心）；Southwestern University of Finance and Economics（西南财经大学）；Tongji University（同济大学）
Revealing and Enhancing Core Visual Regions: Harnessing Internal Attention Dynamics for Hallucination Mitigation in LVLMs
School of Electronic Engineering, Xidian University（西安电子科技大学电子工程学院）；School of Computer Science and Technology, Xidian University（西安电子科技大学计算机科学与技术学院）；College of Computer and Information, Hohai University（河海大学计算机与信息学院）；Institute for Infocomm Research, A*STAR（A*STAR信息与通信研究所）
School of Artificial Intelligence, UCAS（人工智能学院，UCAS）；State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA（多模态人工智能系统国家重点实验室，CASIA）；Centre for Artificial Intelligence and Robotics, HKISI-CAS（人工智能与机器人中心，HKISI-CAS）；Huawei Inc（华为公司）
Reflective Instruction Tuning: Mitigating Hallucinations in Large Vision-Language Models
Southern University of Science and Technology（南方科技大学）；The University of Hong Kong（香港大学）；Shenzhen Polytechnic University（深圳职业技术大学）；ZTE Corporation（中兴通讯股份有限公司）；Peng Cheng Laboratory（鹏城实验室）
Although many long-range imaging systems are designed to support extended vision applications, a natural obstacle to their operation is degradation due to atmospheric turbulence. Atmospheric turbulence causes significant degradation to image quality by introducing blur and geometric distortion. In recent years, various deep learning-based single image atmospheric turbulence mitigation methods, including CNN-based and GAN inversion-based, have been proposed in the literature which attempt to remove the distortion in the image. However, some of these methods are difficult to train and often fail…
ControlFusion: A Controllable Image Fusion Framework with Language-Vision Degradation Prompts
Electronic Information School, Wuhan University（武汉大学电子信息学院）；School of Computer Science and Engineering, Macau University of Science and Technology（澳门科学技术大学计算机科学与工程学院）；Faculty of Computing, Harbin Institute of Technology（哈尔滨工业大学计算机学院）
Multimodal Continual Instruction Tuning with Dynamic Gradient Guidance
Harbin Institute of Technology（哈尔滨工业大学）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东人工智能与数字经济发展实验室）；Chongqing Research Institute of HIT（重庆哈工大研究院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）
FreqEdit: Preserving High-Frequency Features for Robust Multi-Turn Image Editing
School of Artificial Intelligence, Sun Yat-sen University（中山大学人工智能学院）；Lingnan University（岭大大学）；Video Rebirth；The Hong Kong Polytechnic University（香港理工大学）
