Semantic Router: On the Feasibility of Hijacking MLLMs via a Single Adversarial Perturbation
The Chinese University of Hong Kong, Shenzhen, China(香港中文大学(深圳));School of Data Science, School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen, China(数据科学学院、人工智能学院、香港中文大学(深圳))
Multimodal Large Language Models (MLLMs) are increasingly deployed in stateless systems, such as autonomous driving and robotics. This paper investigates a novel threat: Semantic-Aware Hijacking. We explore the feasibility of hijacking multiple stateless decisions simultaneously using a single universal perturbation. We introduce the Semantic-Aware Universal Perturbation (SAUP), which acts as a semantic router, "actively" perceiving input semantics and routing them to distinct, attacker-defined targets. To achieve this, we conduct theoretical and empirical analysis on the geometric properties in the latent space. Guided by these insights, we propose the Semantic-Oriented (SORT) optimization strategy and annotate a new dataset with fine-grained semantics to evaluate performance. Extensive experiments on three representative MLLMs demonstrate the fundamental feasibility of this attack, achieving a 66% attack success rate over five targets using a single frame against Qwen.
Is Our Benchmark Enough? An Analysis of Continual Learning for MLLMs
School of Computer Science and Statistics, Trinity College Dublin(都柏林圣三一学院计算机科学与统计学院);Department of Mathematics and Computer Science, Eindhoven University of Technology(埃因霍温理工大学数学与计算机科学系)
CL-MoE: Enhancing Multimodal Large Language Model with Dual Momentum Mixture-of-Experts for Continual Visual Question Answering
School of Computer Science and Technology, East China Normal University(华东师范大学计算机科学与技术学院);Shanghai Open University(上海开放大学);ZhuQingTing Data Technology (Zhejiang) Co., Ltd(朱清庭数据科技(浙江)有限公司)
Pro-Router: Token-Aware Progressive Model Routing with Adaptive Edge-Cloud Collaboration for Efficient Multimodal LLM Inference
Anyscale(安尼斯科尔公司);Mississippi State University(密西西比州立大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Institute of AI for Industries, Chinese Academy of Sciences(中国科学院产业人工智能研究院)
Open Eyes, Then Reason: Fine-grained Visual Mathematical Understanding in MLLMs
Australian Institute for Machine Learning, University of Adelaide(阿德莱德大学澳大利亚机器学习研究所);Georgia Institute of Technology(佐治亚理工学院);Nanjing University of Science and Technology(南京理工大学);University of Oxford(牛津大学);The Ohio State University(俄亥俄州立大学)
Enhancing Multimodal Continual Instruction Tuning with BranchLoRA
Mohamed bin Zayed University of Artificial Intelligence(Mohamed bin Zayed大学人工智能学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Kyoto University(京都大学);Tencent AI Lab(腾讯AI实验室);Tomorrow Advancing Life(明天生命科技)
PASs-MoE: Mitigating Misaligned Co-drift among Router and Experts via Pathway Activation Subspaces for Continual Learning
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);National University of Singapore(新加坡国立大学);Southeast University, Nanjing, China(南京东南大学);Wuhan AI Research, Wuhan, China(武汉人工智能研究院)
$γ-$MoD: Exploring Mixture-of-Depth Adaptation for Multimodal Large Language Models
Technical University Of Denmark(丹麦技术大学);OpenGVLab(开放视觉实验室);Shanghai AI Laboratory(上海人工智能实验室);Xiamen University(厦门大学);National University of Singapore(新加坡国立大学);MBZUAI(穆罕默德·本·扎耶德人工智能大学)
VisionWeave: Weaving Elastic Visual Representations as a Native Capability of MLLMs
University of Science and Technology of China(中国科学技术大学);Alibaba Token Hub, Alibaba Group(阿里巴巴集团,阿里巴巴Token Hub);The Chinese University of Hong Kong(香港中文大学);Zhejiang University(浙江大学)
Synergistic Perception-Reasoning Governance: Grounding Medical MLLMs with Verifiable Anatomical Evidence
Huazhong University of Science and Technology(华中科技大学);Imperial Global Singapore, Imperial College London(帝国理工学院新加坡全球中心);Nanyang Technological University(南洋理工大学);National University of Singapore(新加坡国立大学);Anhui University(安徽大学)
Memory-Augmented Multimodal Large Language Models for Small Object Understanding in Streaming Aerial Videos
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));University of Freiburg(弗莱堡大学);Hangzhou City University(杭州城市大学);XGRIDS(XGRIDS公司);Fudan University(复旦大学);Hong Kong Baptist University(香港浸会大学)
SpaR3D-MoE: Adaptive 3D Spatial Reasoning from Sparse Views Meets Geometry-Inductive Mixture-of-Experts
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所多模态人工智能系统国家重点实验室);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);YUKUN Intelligent World(宇琨智能世界)
Progressive Multimodal Alignment for Continual Instruction Tuning
Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences(中国科学院脑科学与智能技术卓越创新中心);Kyoto University(京都大学);Migu Culture Technology Co.,Ltd.(咪咕文化科技有限公司);State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology(脑认知与类脑智能技术国家重点实验室)
College of Computer Science and Technology, National University of Defense Technology(国防科技大学计算机科学与技术学院);School of Computer Science, Beijing University of Posts and Telecommunications(北京邮电大学计算机学院)
ViCO: A Training Strategy towards Semantic Aware Dynamic High-Resolution
Shanghai Jiao Tong University(上海交通大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Fudan University(复旦大学);Nanjing University(南京大学);Donghua University(东华大学);The Chinese University of Hong Kong(香港中文大学)