Efficient Prompt Tuning of Large Vision-Language Model for Fine-Grained Ship Classification
College of Computer Science and Technology, National University of Defense Technology(国防科技大学计算机科学与技术学院);College of Physics and Information Engineering, Fuzhou University(福州大学物理与信息工程学院);School of Computer Science, Wuhan University(武汉大学计算机学院);College of Computer, National University of Defense Technology(国防科技大学计算机学院)
Long Lan, Fengxiang Wang, Xiangtao Zheng, Zengmao Wang, Xinwang Liu
Fine-grained ship classification in remote sensing (RS-FGSC) poses a significant challenge due to the high similarity between classes and the limited availability of labeled data, limiting the effectiveness of traditional supervised classification methods. Recent advancements in large pre-trained Vision-Language Models (VLMs) have demonstrated impressive capabilities in few-shot or zero-shot learning, particularly in understanding image content. This study delves into harnessing the potential of VLMs to enhance classification accuracy for unseen ship categories, which holds considerable significance in scenarios with restricted data due to cost or privacy constraints. Directly fine-tuning VLMs for RS-FGSC often encounters the challenge of overfitting the seen classes, resulting in suboptimal generalization to unseen classes, which highlights the difficulty in differentiating complex backgrounds and capturing distinct ship features. To address these issues, we introduce a novel prompt tuning technique that employs a hierarchical, multi-granularity prompt design. Our approach integrates remote sensing ship priors through bias terms, learned from a small trainable network. This strategy enhances the model's generalization capabilities while improving its ability to discern intricate backgrounds and learn discriminative ship features. Furthermore, we contribute to the field by introducing a comprehensive dataset, FGSCM-52, significantly expanding existing datasets with more extensive data and detailed annotations for less common ship classes. Extensive experimental evaluations demonstrate the superiority of our proposed method over current state-of-the-art techniques. The source code will be made publicly available.
ShipTraj-R1: Reinforcing Ship Trajectory Prediction in Large Language Models via Group Relative Policy Optimization
School of Artificial Intelligence, Optics, and Electronics (iOPEN)(人工智能、光学与电子学院);Northwestern Polytechnical University(西北工业大学);Department of Logistics and Maritime Studies(物流与海洋研究系);The Hong Kong Polytechnic University(香港理工大学);State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing(测绘遥感信息工程国家重点实验室);Wuhan University(武汉大学)
Prompting Video-Language Foundation Models with Domain-specific Fine-grained Heuristics for Video Question Answering
School of Information Science and Technology, Hangzhou Normal University(杭州师范大学信息科学与技术学院);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);School of Computer Science and Technology, University of Chinese Academy of Sciences(中国科学院大学计算机科学与技术学院);School of Computer Science and Technology, Hangzhou Dianzi University(杭州电子科技大学计算机科学与技术学院)
Structured-Condensed Prompt Tuning in Vision-Language Models for Fine-grained Image Recognition
School of Information Science and Technology, Northwest University(西北大学信息科学与技术学院);Laboratory of Intelligent Information Processing, Institute of Computing Technology(中国科学院计算技术研究所智能信息处理重点实验室)
CLIP-AUTT: Test-Time Personalization with Action Unit Prompting for Fine-Grained Video Emotion Recognition
LIVIA, ILLS, Dept. of Systems Engineering, ETS Montreal(LIVIA, ILLS, 系统工程学院, 蒙特利尔高等技术学院);LIVIA, Dept. of Software and IT Engineering, ETS Montreal(LIVIA, 软件与IT工程学院, 蒙特利尔高等技术学院);Dept. of Computer Science, École Polytechnique(计算机科学系, 巴黎综合理工学院)
Vision Language Model for Interpretable and Fine-grained Detection of Safety Compliance in Diverse Workplaces
School of Mechanical, Aerospace, and Manufacturing Engineering(机械、航空航天与制造工程学院);University of Connecticut(康涅狄格大学);Department of Computer Science(计算机科学系);University of California Irvine(加州大学欧文分校)
Unlabeled Data Improves Fine-Grained Image Zero-shot Classification with Multimodal LLMs
Computer Science Department, University of California, Los Angeles(加州大学洛杉矶分校计算机科学系);Mechanical and Aerospace Engineering Department, University of California, Los Angeles(加州大学洛杉矶分校机械与航空航天工程系)
Towards Fine-Grained Robustness: Attention-Guided Test-Time Prompt Tuning for Vision-Language Models
School of Computer Science and Engineering(计算机科学与工程学院);Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications(新一代人工智能技术及其交叉应用重点实验室);Southeast University(东南大学);School of Intelligence Science and Engineering(智能科学与工程学院)
Improving Zero-Shot Generalization for CLIP with Synthesized Prompts
University of Science and Technology of China(中国科学技术大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Beijing Wenge Group(北京文格集团);Nanjing University(南京大学)
Causality-guided Prompt Learning for Vision-language Models via Visual Granulation
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(多模态人工智能系统国家重点实验室,自动化研究所,中国科学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(人工智能学院,中国科学院大学)
SISP: A Benchmark Dataset for Fine-grained Ship Instance Segmentation in Panchromatic Satellite Images
State Key Laboratory of Space-Ground Integrated Information Technology, CAST(天地一体化信息技术国家重点实验室);College of Computer Science and Technology, Harbin Engineering University(哈尔滨工程大学计算机科学与技术学院);College of Electrical and Mechanical Engineering, Dalian Minzu University(大连民族大学机电工程学院);School of Geography and Ocean Science, Nanjing University(南京大学地理与海洋科学学院)
A Dual Large Language Models Architecture with Herald Guided Prompts for Parallel Fine Grained Traffic Signal Control
School of Artificial Intelligence, Beijing University of Posts(人工智能学院,北京邮电大学);Beijing Big Data Center(北京大数据中心);Eastern Institute of Technology(东部技术研究所)
EarthGPT-X: A Spatial MLLM for Multi-level Multi-Source Remote Sensing Imagery Understanding with Visual Prompting
School of Interdisciplinary Science, Beijing Institute of Technology(交叉科学学院,北京理工大学);College of Computing and Data Science, Nanyang Technological University(计算与数据科学学院,南洋理工大学);National Key Laboratory of Science and Technology on Space-Born Intelligent Information Processing, Beijing Institute of Technology(空间智能信息处理国家重点实验室,北京理工大学);School of Optics and Photonics, Beijing Institute of Technology(光学与 photonics 学院,北京理工大学);State Key Laboratory of Explosion Science and Safety Protection, Beijing(爆炸科学与安全防护国家重点实验室,北京)
CoF: Coarse to Fine-Grained Image Understanding for Multi-modal Large Language Models
School of Automation, Northwestern Polytechnical University(西北工业大学自动化学院);School of Cybersecurity, Northwestern Polytechnical University(西北工业大学网络空间安全学院);Alibaba Group(阿里巴巴集团);Southeast University(东南大学);Advanced Ocean Institute of Southeast University(东南大学先进海洋研究院);Zhejiang University of Technology(浙江工业大学);Binjiang Institute of Artificial Intelligence(滨江人工智能研究院)
CILP-FGDI: Exploiting Vision-Language Model for Generalizable Person Re-Identification
School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院);Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education(教育部新一代人工智能技术及其交叉应用重点实验室(东南大学))
Funny or Persuasive, but Not Both: Evaluating Fine-Grained Multi-Concept Control in LLMs
University of Cambridge(剑桥大学);CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全中心);Apta(Apta公司);Royal Holloway, University of London(伦敦大学皇家霍洛威学院)
Controllable Image Captioning with Prompt-Conditioned Scene Rewards
POSTECH(浦项科技大学);Graduate School of Artificial Intelligence, POSTECH(浦项科技大学人工智能研究生院);Department of Computer Science and Engineering, POSTECH(浦项科技大学计算机科学与工程系)
Prompt-CAM: Making Vision Transformers Interpretable for Fine-Grained Analysis
The Ohio State University(俄亥俄州立大学);University of Tsukuba(筑波大学);Virginia Tech(弗吉尼亚理工大学);Princeton University(普林斯顿大学);Rensselaer Polytechnic Institute(伦斯勒理工学院)