Learning to Generate via Understanding: Understanding-Driven Intrinsic Rewarding for Unified Multimodal Models
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Peking University(北京大学);University of Chinese Academy of Sciences(中国科学院大学);Sun Yat-sen University(中山大学);Baidu Inc.(百度公司)
Recently, unified multimodal models (UMMs) have made remarkable progress in integrating visual understanding and generation, demonstrating strong potential for complex text-to-image (T2I) tasks. Despite their theoretical promise, a persistent capability gap exists: UMMs typically exhibit superior visual understanding but comparatively weaker generative capabilities. This discrepancy arises largely from the intrinsic decoupling between the understanding and generation processes. While a UMM can accurately interpret fine-grained visual details, it often struggles to produce semantically coherent images from complex textual prompts. To address this challenge, we explore UMMs' internal understanding capability to enhance generation quality. We propose a token-level intrinsic text-image alignment reward mechanism, GvU, enabling the UMM to act simultaneously as teacher and student: it evaluates its own outputs using the understanding branch to guide the generations accordingly. Building upon this, we design a self-supervised reinforcement learning framework, allowing UMMs to iteratively improve their generation quality through understanding-based intrinsic reward signals--without reliance on external supervision. Experimental results show that our method substantially boosts UMMs' generation, which in turn strengthens their fine-grained visual understanding, narrowing the capability gap between UMMs' visual understanding and generation.
Spatial Preference Rewarding for MLLMs Spatial Understanding
S-Lab, Nanyang Technological University(南洋理工大学S实验室);Shanghai AI Laboratory(上海人工智能实验室);Sensetime Research(商汤科技研究院);Zhejiang University of Technology(浙江工业大学);UCAS-Terminus AI Lab,University of Chinese Academy of Sciences(中国科学院大学Terminus AI实验室)
INDIGO: Intrinsic Multimodality for Domain Generalization
Adobe Media and Data Science Research Lab(奥多比媒体与数据科学研究实验室);INRIA(法国国家信息与自动化研究所);Universite Grenoble Alpes(格勒诺布尔阿尔卑斯大学);Indian Institute of Technology, Hyderabad(印度理工学院海得拉巴分校)
Reliability-Prioritized Fine-Grained Generation in Multimodal Large
Beijing Key Laboratory of Intelligent Information Technology, School of Computer Science & Technology, Beijing Institute of Technology(北京智能信息科技重点实验室,计算机科学与技术学院,北京理工大学);Guangdong Laboratory of Machine Perception and Intelligent Computing, Shenzhen MSU-BIT University(广东机器感知与智能计算实验室,深圳MSU-BIT大学);Department of Electrical and Computer System Engineering, Monash University(电子与计算机系统工程系,莫纳什大学)
Forbid Your Attention: Fooling Multimodal Large Language Models by Selectively Removing Intrinsic Focus in Spectral Domain
Institute for Math & AI, Wuhan University(武汉大学数学与人工智能学院);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院);School of Computer Science and Technology, Huazhong University of Science and Technology(华中科技大学计算机科学与技术学院);Department of Computer Science and Engineering, The Hong Kong University of Science and Technology(香港科技大学计算机科学与工程系);Cyberspace Institute of Advanced Technology, Guangzhou University(广州大学先进技术网络空间学院);College of Computer and Information Engineering, Zhejiang Gongshang University(浙江工商大学计算机与信息工程学院)
Modeling Pedestrian Intrinsic Uncertainty for Multimodal Stochastic Trajectory Prediction via Energy Plan Denoising
School of Computing, Macquarie University(麦考瑞大学计算学院);Data 61, CSIRO(联邦科学与工业研究组织Data 61研究院);School of Computer Science and Engineering, University of New South Wales(新南威尔士大学计算机科学与工程学院)
Faithful-MR1: Faithful Multimodal Reasoning via Anchoring and Reinforcing Visual Attention
AMAP, Alibaba Group(阿里云实验室,阿里巴巴集团);University of Chinese Academy of Sciences(中国科学院大学);Tsinghua University(清华大学);Nanyang Technological University(南洋理工大学)
Towards Unified Multimodal Editing with Enhanced Knowledge Collaboration
Zhejiang University(浙江大学);National University of Singapore(新加坡国立大学);Hefei University of Technology(合肥工业大学);Nanyang Technological University(南洋理工大学);Singapore Management University(新加坡管理大学)
Multimodal Inverse Attention Network with Intrinsic Discriminant Feature Exploitation for Fake News Detection
School of Information Science and Engineering, Shandong Normal University(山东师范大学信息科学与工程学院);Faculty of Engineering & Information Technology, University of Technology Sydney(悉尼科技大学工程与信息技术学院)
E-MRL: Cross-view Aligned Evidence-driven Multimodal Reinforcement Learning for Reliable 3D Tumor Analysis
Zhejiang University(浙江大学);DAMO Academy, Alibaba Group(阿里巴巴集团达摩院);Hupan Lab(华平实验室);Huazhong University of Science and Technology(华中科技大学);East China Normal University(华东师范大学);Shanghai Jiao Tong University(上海交通大学)
Seeing Clearly without Training: Mitigating Hallucinations in Multimodal LLMs for Remote Sensing
School of Computer Science, Wuhan University, China(武汉大学计算机学院);Zhongguancun Academy, China(中关村学院);School of Computer Science, Chongqing University, China(重庆大学计算机学院)
Bifocal: Training-Free Mixed Granularity Visual Token Pruning for Multimodal LLMs
Eastern Institute of Technology(东方理工大学);Shanghai Jiao Tong University(上海交通大学);Uppsala University(乌普萨拉大学);University of Science and Technology of China(中国科学技术大学)
PRISM: Self-Pruning Intrinsic Selection Method for Training-Free Multimodal Data Selection
LMU Munich(慕尼黑大学);Munich Research Center, Huawei Technologies(慕尼黑研究中心,华为技术);METEOR;School of Computer Science, Wuhan University(武汉大学计算机学院);Munich Center for Machine Learning(慕尼黑机器学习中心)
Period-LLM: Extending the Periodic Capability of Multimodal Large Language Model
The Hong Kong University of Science & Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science & Technology(香港科技大学);Zhejiang University(浙江大学);Lappeenranta-Lahti University of Technology(拉佩兰塔-拉赫蒂技术大学)
Octopus: Agentic Multimodal Reasoning with Six-Capability Orchestration
Sun Yat-sen University(中山大学);Shanghai Jiao Tong University(上海交通大学);Zhejiang University(浙江大学);Tsinghua University(清华大学);Peking University(北京大学);The Hong Kong University of Science and Technology(香港科技大学);University of Chinese Academy of Sciences(中国科学院大学)
Uncovering Intrinsic Capabilities: A Paradigm for Data Curation in Vision-Language Models
Harbin Institute of Technology, Shenzhen, China(哈尔滨工业大学(深圳));Hong Kong Baptist University, China(香港 Baptist大学);City University of Hong Kong, China(香港城市大学)
Multimodal Doctor-in-the-Loop: A Clinically-Guided Explainable Framework for Predicting Pathological Response in Non-Small Cell Lung Cancer
Research Unit of Computer Systems and Bioinformatics, Department of Engineering(计算机系统与生物信息学研究单位,工程系);Research Unit of Radiation Oncology, Department of Medicine and Surgery(放射肿瘤学研究单位,医学与外科系);Research Unit of Anatomical Pathology, Department of Medicine and Surgery(病理学研究单位,医学与外科系)
Weakly supervised multimodal segmentation of acoustic borehole images with depth-aware cross-attention
Federal University of Bahia, Institute of Geosciences, Department of Geophysics(巴伊亚联邦大学,地质科学学院,地球物理学系);Grupo de Estudos e Aplicação de Inteligência Artificial em Geofísica (GAIA)(地质物理中人工智能研究与应用小组)