Exploring Sparse Visual Prompt for Domain Adaptive Dense Prediction
Peking University(北京大学);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));University of Science and Technology of China(中国科学技术大学);Beihang University(北京航空航天大学)
The visual prompts have provided an efficient manner in addressing visual cross-domain problems. In previous works, Visual Domain Prompt (VDP) first introduces domain prompts to tackle the classification Test-Time Adaptation (TTA) problem by warping image-level prompts on the input and fine-tuning prompts for each target domain. However, since the image-level prompts mask out continuous spatial details in the prompt-allocated region, it will suffer from inaccurate contextual information and limited domain knowledge extraction, particularly when dealing with dense prediction TTA problems. To overcome these challenges, we propose a novel Sparse Visual Domain Prompts (SVDP) approach, which holds minimal trainable parameters (e.g., 0.1\%) in the image-level prompt and reserves more spatial information of the input. To better apply SVDP in extracting domain-specific knowledge, we introduce the Domain Prompt Placement (DPP) method to adaptively allocates trainable parameters of SVDP on the pixels with large distribution shifts. Furthermore, recognizing that each target domain sample exhibits a unique domain shift, we design Domain Prompt Updating (DPU) strategy to optimize prompt parameters differently for each sample, facilitating efficient adaptation to the target domain. Extensive experiments were conducted on widely-used TTA and continual TTA benchmarks, and our proposed method achieves state-of-the-art performance in both semantic segmentation and depth estimation tasks.
SP3D: Boosting Sparsely-Supervised 3D Object Detection via Accurate Cross-Modal Semantic Prompts
Fujian Key Laboratory of Sensing and Computing for Smart Cities, Xiamen University(厦门大学福建省智慧城市感知与计算重点实验室);Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University(厦门大学多媒体可信感知与高效计算教育部重点实验室)
Prompt Learning for Oriented Power Transmission Tower Detection in High-Resolution SAR Images
Key Laboratory of Digital Earth Science, Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院空天信息创新研究院数字地球科学重点实验室);International Research Center of Big Data for Sustainable Development Goals(可持续发展目标大数据国际研究中心);College of Resources and Environment, University of Chinese Academy of Sciences(中国科学院大学资源与环境学院)
Exploring Interpretability for Visual Prompt Tuning with Cross-layer Concepts
School of Computer Science and Technology, Tongji University(同济大学计算机科学与技术学院);Microsoft Research Asia(微软亚洲研究院);School of Telecommunication and Engineering, Xidian University(西安电子科技大学电信与工程学院)
Beyond Low-Rank: Low-Rank Sparse Prompting via Spiking Neural Network and Prompt Factorization
Information Materials and Intelligent Sensing Laboratory of Anhui Province(安徽省信息材料与智能感知实验室);Anhui Provincial Key Laboratory of Multimodal Cognitive Computation(安徽省多模态认知计算重点实验室);School of Computer Science and Technology, Anhui University(安徽大学计算机科学与技术学院)
Exploring Conditional Multi-Modal Prompts for Zero-shot HOI Detection
Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);State Key Laboratory of General Artificial Intelligence, Peking University(北京大学通用人工智能国家重点实验室)
GRASP: Guided Region-Aware Sparse Prompting for Adapting MLLMs to Remote Sensing
School of Computing, Kyung Hee University(京畿大学计算机学院);School of Information and Software Engineering, University of Electronic Science and Technology of China(电子科技大学信息与软件工程学院);School of Computer Science and Engineering, University of Electronic Science and Technology of China(电子科技大学计算机科学与工程学院);College of Computer Science and Information Engineering, Harbin Normal University(哈尔滨师范大学计算机科学与信息工程学院);College of Professional and Continuing Education, The Hong Kong Polytechnic University(香港理工大学专业及继续教育学院);School of Computer Science and Technology, Tongji University(同济大学计算机科学与技术学院)
Exploring Multimodal Prompts For Unsupervised Continuous Anomaly Detection
Key Laboratory of Computing Power Network(计算能力网络重点实验室);Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences)(信息安全,教育部,山东计算机科学中心(济南国家超算中心),齐鲁工业大学(山东科学院));Shandong Provincial Key Laboratory of Computing Power Internet(山东省计算能力互联网重点实验室);Service Computing, Shandong Fundamental Research Center for Computer Science(服务计算,山东省计算机基础研究中心);Faculty of Data Science, City University of Macau(数据科学学院,澳门城市大学)
Sparse Anatomical Prompt Semi-Supervised Learning with Masked Image Modeling for CBCT Tooth Segmentation
Institute of Innovative Research, Tokyo Institute of Technology(东京工业大学创新研究院);Stomatological Hospital of Chongqing Medical University(重庆医科大学附属口腔医院);School of Mechanical Engineering, Zhejiang University(浙江大学机械工程学院)
DA2Diff: Exploring Degradation-aware Adaptive Diffusion Priors for All-in-One Weather Restoration
School of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics(南京航空航天大学计算机科学与技术学院);Collaborative Innovation Center of Novel Software Technology and Industrialization(新型软件技术与产业化协同创新中心);College of Computer Science and Technology, Anhui University of Technology(安徽工业大学计算机科学与技术学院);Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院)
Prompt Guiding Multi-Scale Adaptive Sparse Representation-driven Network for Low-Dose CT MAR
School of Information Science and Engineering, Yanshan University(信息科学与工程学院,燕山大学);School of Electronics and Information Technology, Sun Yat-sen University(电子与信息学院,中山大学);School of Computer Science and Technology, Beijing Institute of Technology(计算机科学与技术学院,北京理工大学);Institute of High Performance Computing, Agency for Science, Technology and Research(高性能计算研究所,科技研究局);Institute of Lauterbur Research Center for Biomedical Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(生物医学成像研究所以及深圳先进技术研究所,中国科学院)
Exploring Typographic Visual Prompts Injection Threats in Cross-Modality Generation Models
The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));University of Oxford(牛津大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院);The Hong Kong University of Science and Technology(香港科学与技术大学);Beijing University of Technology(北京工业大学);Tsinghua University(清华大学);City University of Hong Kong(香港城市大学)
DiTCtrl: Exploring Attention Control in Multi-Modal Diffusion Transformer for Tuning-Free Multi-Prompt Longer Video Generation
MMLab, CUHK(香港中文大学多媒体实验室);GVC Lab, Great Bay University(大湾区大学GVC实验室);ARC Lab, Tencent PCG(腾讯平台与内容事业群ARC实验室);Tencent AI Lab(腾讯AI实验室);SHIAE, CUHK(香港中文大学SHIAE实验室)
UCDR-Adapter: Exploring Adaptation of Pre-Trained Vision-Language Models for Universal Cross-Domain Retrieval
Zhejiang University(浙江大学);Carnegie Mellon University(卡内基梅隆大学);University of Washington(华盛顿大学);Dalian University of Technology(大连理工大学);DAMO Academy, Alibaba Group(阿里巴巴达摩院);Microsoft Research(微软研究院)
SegCompass: Exploring Interpretable Alignment with Sparse Autoencoders for Enhanced Reasoning Segmentation
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(深圳先进技术研究院,中国科学院);Peng Cheng Laboratory(鹏城实验室);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Meituan, Beijing(美团,北京);University of Chinese Academy of Sciences(中国科学院大学);College of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院)