Change detection (CD) is a fundamental task for monitoring and analyzing land cover dynamics. While recent high performance models and high quality datasets have significantly advanced the field, a critical limitation persists. Current models typically acquire limited knowledge from single-type annotated data and cannot concurrently leverage diverse binary change detection (BCD) and semantic change detection (SCD) datasets. This constraint leads to poor generalization and limited versatility. The recent advancements in Multimodal Large Language Models (MLLMs) introduce new possibilities for a unified CD framework. We leverage the language priors and unification capabilities of MLLMs to develop UniChange, the first MLLM-based unified change detection model. UniChange integrates generative language abilities with specialized CD functionalities. Our model successfully unifies both BCD and SCD tasks through the introduction of three special tokens: [T1], [T2], and [CHANGE]. Furthermore, UniChange utilizes text prompts to guide the identification of change categories, eliminating the reliance on predefined classification heads. This design allows UniChange to effectively acquire knowledge from multi-source datasets, even when their class definitions conflict. Experiments on four public benchmarks (WHU-CD, S2Looking, LEVIR-CD+, and SECOND) demonstrate SOTA performance, achieving IoU scores of 90.41, 53.04, 78.87, and 57.62, respectively, surpassing all previous methods. The code is available at https://github.com/Erxucomeon/UniChange.
UniVLR: Unifying Text and Vision in Visual Latent Reasoning for Multimodal LLMs
University of Science and Technology of China(中国科学技术大学);Tsinghua University(清华大学);National University of Singapore(新加坡国立大学);Zhongguancun Academy(中关村学院)
Unifying Language Agent Algorithms with Graph-based Orchestration Engine for Reproducible Agent Research
Om AI Research(Om AI研究机构);Binjiang Institute of Zhejiang University(浙江大学滨江学院);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院)
From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Models
School of Computer Science, Sichuan University(四川大学计算机学院);School of Electronic and Computer Engineering, Peking University Shenzhen Graduate School(北京大学深圳研究生院电子与计算机工程学院);Institute of Artificial Intelligence (TeleAI), China Telecom and Northwestern Polytechnical University(中国电信与西北工业大学人工智能研究院(TeleAI))
CLIP-GS: Unifying Vision-Language Representation with 3D Gaussian Splatting
Institute of Information Science, Beijing Jiaotong University(北京交通大学信息科学学院);National University of Singapore(新加坡国立大学);Meituan(美团);Georgia Institute of Technology(佐治亚理工学院);Picsart AI Research (PAIR)(Picsart AI研究(PAIR))
FlowInOne:Unifying Multimodal Generation as Image-in, Image-out Flow Matching
University of Electronic Science and Technology of China(电子科技大学);Central South University(中南大学);National University of Singapore(新加坡国立大学);University of Science and Technology of China(中国科学技术大学);Microsoft(微软)
UINO-FSS: Unifying Representation Learning and Few-shot Segmentation via Hierarchical Distillation and Mamba-HyperCorrelation
School of Artificial Intelligence and the National Engineering Laboratory of Big Data System Computing Technology, Shenzhen University(人工智能学院和大数据系统计算技术国家工程实验室,深圳大学);Guangdong Provincial Key Laboratory of Intelligent Information Processing, China(广东省智能信息处理重点实验室,中国);School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University(生物医学工程学院,深圳大学医学院,深圳大学);Department of Computer Science, University of Nottingham Ningbo China(计算机科学系,宁波大学中国)
MEDBind: Unifying Language and Multimodal Medical Data Embeddings
Peter Munk Cardiac Centre(彼得·芒克心脏中心);University Health Network (UHN)(大学网络健康中心);University of Toronto (U of T)(多伦多大学);Ted Rogers Centre for Heart Research(泰德·罗杰斯心脏研究中心);Vector Institute(向量研究所);Toronto General Hospital Research Institute(多伦多总医院研究所);Toronto General Hospital(多伦多总医院);Department of Medical Imaging(医学影像系)
Mixture-of-Thought-Tokens: Unifying Perception and Reasoning for Free-form Multimodal Grounding
Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University(西安交通大学人工智能与机器人研究所);Xingchen AGI Lab, China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd(星辰通用人工智能实验室,中国电信人工智能技术(北京)有限公司);University of Science and Technology Beijing(北京科技大学);Beijing University of Posts and Telecommunications(北京邮电大学);Shanghai Jiao Tong University(上海交通大学)
UniMedVL: Unifying Medical Multimodal Understanding and Generation through Observation-Knowledge-Analysis
Shanghai Artificial Intelligence Laboratory;Shanghai Innovation Institute;Shanghai Jiao Tong University;Shanghai Institute of Optics;Fudan University;University of Cambridge;Monash University;DAMO Academy, Alibaba Group;Imperial College London;The University of Hong Kong;The Hong Kong University of Science;Hupan Lab;The Chinese University of Hong Kong
One Framework to Rule Them All: Unifying Multimodal Tasks with LLM Neural-Tuning
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);College of Information Science and Engineering, Ritsumeikan University(立命馆大学信息科学与工程学院)
TIGeR: Unifying Text-to-Image Generation and Retrieval with Large Multimodal Models
National University of Singapore(新加坡国立大学);Nanyang Technological University(南洋理工大学);University of Science and Technology of China(中国科学技术大学);Hong Kong Polytechnic University(香港理工大学);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳))
DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformers
Centre for Artificial Intelligence and Robotics, HKISI, CAS(中国科学院香港中文大学信息工程中心人工智能与机器人中心);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
UltraDub: Towards Authentic Dubbing by Unifying Visually-Steered Flow Learning and Trajectory Guidance
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Hangzhou Dianzi University(杭州电子科技大学);ByteDance(字节跳动);University of Science and Technology of China(中国科学技术大学)
LAVENDER: Unifying Video-Language Understanding as Masked Language Modeling
Microsoft(微软)
近年来,统一视觉语言框架有了很大的发展,其中大多数采用编码器-解码器架构,将图像-文本任务统一为序列到序列生成。然而,现有的视频语言(VidL)模型仍然需要在模型架构和每个任务的训练目标的任务特定的设计。在这项工作中,我们探索了一个统一的VidL框架LAVENDER,其中Masked Language Modeling(MLM)被用作所有预训练和下游任务的公共接口。这样的统一导致简化的模型架构,其中在多模态编码器之上仅需要轻量级MLM头,而不是具有更多参数的解码器。令人惊讶的是,实验结果表明,这个统一的框架实现了竞争力的性能在14个VidL基准,涵盖视频问答,文本到视频检索和视频描述。广泛的分析进一步证明了LAVENDER优于现有VidL方法的优点:(i)在多任务微调时仅使用一组参数值支持所有下游任务;(ii)对各种下游任务的Few-Shot泛化;以及(iii)对视频问答任务启用zero-shot评估。代码可在https://github.com/microsoft/LAVENDER上获得。
Med-UniC: Unifying Cross-Lingual Medical Vision-Language Pre-Training by Diminishing Bias
The Ohio State University(俄亥俄州立大学);Imperial College London(伦敦帝国学院);The Hong Kong University of Science and Technology(香港科技大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Peking University(北京大学)
InstructSeq: Unifying Vision Tasks with Instruction-conditioned Multi-modal Sequence Generation
The Chinese University of Hong Kong(香港中文大学);Fudan University(复旦大学);Sensetime Research Institute(商汤研究院);Tsinghua University(清华大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
Actor as Its Own Critic: Unifying Region Understanding and Localization via CycleGRPO
National University of Singapore(新加坡国立大学);University of Chinese Academy of Sciences(中国科学院大学);Nanyang Technological University(南洋理工大学);Wuhan University(武汉大学)
Vision-Centric Activation and Coordination for Multimodal Large Language Models
MoE Key Lab of Artificial Intelligence, Shanghai Jiao Tong University(人工智能MoE实验室,上海交通大学);Ant Group(蚂蚁集团);Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo(宁波数字孪生研究所,东部技术研究所,宁波)
UniRS: Unifying Multi-temporal Remote Sensing Tasks through Vision Language Models
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications(北京邮电大学);Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院空天信息创新研究院);School of Geographic Sciences, Hunan Normal University(湖南师范大学);City University of Hong Kong(香港城市大学)