A Novel Adaptive Fine-Tuning Algorithm for Multimodal Models: Self-Optimizing Classification and Selection of High-Quality Datasets in Remote Sensing
Hangzhou Institute of Technology of Xidian University（西安电子科技大学杭州研究院）；Xidian University（西安电子科技大学）；Key Laboratory of Coal Resources Exploration and Comprehensive Utilization, Ministry of Natural Resources（自然资源部煤炭资源勘查与综合利用重点实验室）；Shaanxi Water Development Group Co., Ltd（陕西水务发展集团有限公司）
Yi Ren, Tianyi Zhang, Zhixiong Han, Weibin Li, Zhiyang Wang, Wenbo Ji, Chenhao Qin, Chenbin Liang, Licheng Jiao
We propose an adaptive fine-tuning algorithm for multimodal large models. The core steps of this algorithm involve two stages of truncation. First, the vast amount of data is projected into a semantic vector space, and the MiniBatchKMeans algorithm is used for automated clustering. This classification ensures that the data within each cluster exhibit high semantic similarity. Next, we process the data in each cluster, calculating the translational difference between the original and perturbed data in the multimodal large model's vector space. This difference serves as a generalization metric for the data. Based on this metric, we select the data with high generalization potential for training. We applied this algorithm to train the InternLM-XComposer2-VL-7B model on two 3090 GPUs using one-third of the GeoChat multimodal remote sensing dataset. The results demonstrate that our algorithm outperforms the state-of-the-art baselines. various baselines. The model trained on our optimally chosen one-third dataset, based on experimental validation, exhibited only 1% reduction in performance across various remote sensing metrics compared to the model trained on the full dataset. This approach significantly preserved general-purpose capabilities while reducing training time by 68.2%. Furthermore, the model achieved scores of 89.86 and 77.19 on the UCMerced and AID evaluation datasets, respectively, surpassing the GeoChat dataset by 5.43 and 5.16 points. It only showed a 0.91-point average decrease on the LRBEN evaluation dataset.
MGIMM: Multi-Granularity Instruction Multimodal Model for Attribute-Guided Remote Sensing Image Detailed Description
National Engineering Research Center for Multimedia Software（国家多媒体软件工程技术研究中心）；School of Computer Science, Wuhan University（武汉大学计算机学院）；Hubei Luojia Laboratory（湖北珞珈实验室）
PhysDrive: A Multimodal Remote Physiological Measurement Dataset for In-vehicle Driver Monitoring
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Tsinghua University（清华大学）；Sichuan Agricultural University（四川农业大学）
Balanced Diffusion-Guided Fusion for Multimodal Remote Sensing Classification
Department of Information Engineering and Computer Science, University of Trento（特伦托大学信息工程与计算机科学系）；College of Electrical and Information Engineering, Hunan University（湖南大学电气与信息工程学院）；Key Laboratory of Collaborative Intelligent Systems of Ministry of Education, Xidian University（西安电子科技大学教育部协同智能系统重点实验室）；School of Electronic Engineering, Xidian University（西安电子科技大学电子工程学院）；Academy of Artificial Intelligence, Inner Mongolia Normal University（内蒙古师范大学人工智能学院）
FLORO: A Multimodal Geospatial Foundation Model for Ecological Remote Sensing Across Sensors and Scales
Biological and Environmental Science and Engineering Division, King Abdullah University of Science and Technology（国王阿卜杜勒·阿齐兹科技大学生物与环境科学与工程 division）；Computer, Electrical and Mathematical Science and Engineering Division, King Abdullah University of Science and Technology（国王阿卜杜勒·阿齐兹科技大学计算机、电气与数学科学与工程 division）
Multimodal Remote Sensing Benchmark Datasets for Land Cover Classification with A Shared and Specific Feature Learning Model
Remote Sensing Technology Institute, German Aerospace Center（德国航空航天中心遥感技术研究所）；Technical University of Munich（慕尼黑工业大学）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；INRIA（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔理工学院）
SAMChat: Introducing Chain of Thought Reasoning and GRPO to a Multimodal Small Language Model for Small Scale Remote Sensing
Center for the Image Analysis (OGAM) and Department of Electrical and Electronics Engineering of Middle East Technical University (METU)（图像分析中心（OGAM）和中东部技术大学（METU）电子与电气工程系）
FUSAR-KLIP: Towards Multimodal Foundation Models for Remote Sensing
Key Laboratory for Information Science of Electromagnetic Waves (MoE), Fudan University（电磁波信息科学重点实验室（MoE），复旦大学）；Institute of Zhejiang Laboratory（浙江实验室研究院）；College of Electronic Science and Technology, NUDT（电子科学与技术学院，南大学）
Multimodal-Multiresolution Foundation Model for Lunar Remote Sensing
IBM Research（IBM研究院）；The University of Alabama in Huntsville（阿拉巴马大学亨茨维尔分校）；University Space Research Association (USRA)（大学空间研究协会）；NASA Goddard Space Flight Center（美国国家航空航天局戈达德太空飞行中心）；SETI Institute（搜寻地外文明研究所）；NASA Ames Research Center（美国国家航空航天局艾姆斯研究中心）；University of Maryland, Baltimore County (UMBC)（马里兰大学巴尔的摩县分校）；Howard University（霍华德大学）
GeoPixel: Pixel Grounding Large Multimodal Model in Remote Sensing
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Australian National University（澳大利亚国立大学）；The University of Western Australia（西澳大利亚大学）
MMLSv2: A Multimodal Dataset for Martian Landslide Detection in Remote Sensing Imagery
Fairfield University（费尔菲尔德大学）；Michigan Technological University（密歇根技术大学）；Computer Vision Center（计算机视觉中心）；Universitat Autònoma de Barcelona（巴塞罗那自治大学）；ESPOL Polytechnic University（ESPOL多科大学）；University of Kerala（喀拉拉大学）；University of Mississippi（密西西比大学）
GeoHeight-Bench: Towards Height-Aware Multimodal Reasoning in Remote Sensing
KTH Royal Institute of Technology（瑞典皇家理工学院）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Technical University of Munich（慕尼黑工业大学）
LHRS-Bot-Nova: Improved Multimodal Large Language Model for Remote Sensing Vision-Language Interpretation
Jiangsu Provincial Key Laboratory of Geographic Information Science and Technology（江苏省地理信息科学与技术重点实验室）；Key Laboratory for Land Satellite Remote Sensing Applications of Ministry of Natural Resources（自然资源部陆地卫星遥感应用重点实验室）；School of Geography and Ocean Science, Nanjing University（南京大学地理与海洋科学学院）；Department of Aerospace and Geodesy, Data Science in Earth Observation, Technical University of Munich（慕尼黑工业大学航空测量与地理系地球观测数据科学研究所）
Cross-City Matters: A Multimodal Remote Sensing Benchmark Dataset for Cross-City Semantic Segmentation using High-Resolution Domain Adaptation Networks
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Technical University of Munich（慕尼黑工业大学）；Southeast University（东南大学）；Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔国立理工学院）；GIPSA-Lab（格勒诺布尔图像、信号、自动化与控制实验室）
Revisiting Change VQA in Remote Sensing with Structured and Native Multimodal Qwen Models
Computer Engineering Department, College of Computer and Information Sciences, King Saud University（计算机工程系，计算机与信息科学学院，沙特国王大学）；Applied Computer Science Department, College of Applied Computer Science, King Saud University（应用计算机科学系，应用计算机科学学院，沙特国王大学）
LithoBench: Benchmarking Large Multimodal Models for Remote-Sensing Lithology Interpretation
School of Computer Science, China University of Geosciences（中国地质大学（北京）计算机学院）；PRADA Lab, King Abdullah University of Science and Technology（科廷大学PRADA实验室）；Department of Computer Science, University of Exeter（埃克塞特大学计算机科学系）
Multimodal Large Language Models for Remote Sensing Image Understanding: Domain-Specific or General-Purpose?
School of Artificial Intelligence and Robotics, Hunan University（湖南大学人工智能与机器人学院）；Intelligent Game and Decision Lab (IGDL)（智能游戏与决策实验室）；Yuelushan Center for Industrial Innovation（岳麓山工业创新中心）
Uni-RS: A Spatially Faithful Unified Understanding and Generation Model for Remote Sensing
Institute of Remote Sensing and Geographic Information System, School of Earth and Space Sciences, Peking University（遥感与地理信息系统研究所，地球与空间科学学院，北京大学）；Department of Civil and Environmental Engineering, The Hong Kong University of Science and Technology（土木与环境工程系，香港科学与技术大学）
Multimodal Feature Fusion Network with Text Difference Enhancement for Remote Sensing Change Detection
College of Electronics and Information Engineering, Wuyi University（威怡大学电子与信息工程学院）；School of Electronic and Information Engineering and the Key Laboratory of Big Data and Intelligent Robot, Ministry of Education, South China University of Technology（电子与信息工程学院和大数据与智能机器人重点实验室，华南理工大学）；State Key Laboratory of Lunar and Planetary Sciences, Macau University of Science and Technology（澳门大学地球和行星科学国家重点实验室）；College of Engineering and Computer Science, California State University, Northridge（工程与计算机科学学院，加州大学北岭分校）；Department of Geography, The University of Hong Kong（地理系，香港大学）；Faculty of Computer Science and Engineering, S（计算机科学与工程学院，S）
TinyRS-R1: Compact Multimodal Language Model for Remote Sensing
Center for the Image Analysis (OGAM) and Department of Electrical and Electronics Engineering of Middle East Technical University (METU)（图像分析中心（OGAM）和中欧技术大学（METU）电子与电气工程系）
A Unified Framework with Multimodal Fine-tuning for Remote Sensing Semantic Segmentation
School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）科学与工程学院）；School of Information Science and Engineering, Wuhan University of Science and Technology（武汉科技大学信息科学与工程学院）；Department of Geography, The University of Hong Kong（香港大学地理系）
Seeing Clearly without Training: Mitigating Hallucinations in Multimodal LLMs for Remote Sensing
School of Computer Science, Wuhan University, China（武汉大学计算机学院）；Zhongguancun Academy, China（中关村学院）；School of Computer Science, Chongqing University, China（重庆大学计算机学院）
LMFNet: An Efficient Multimodal Fusion Approach for Semantic Segmentation in High-Resolution Remote Sensing
State Key Laboratory of information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University（武汉大学测绘遥感信息工程国家重点实验室）；Electronic Information School, Wuhan University（武汉大学电子信息学院）
