尽管统一图像分割（IS）近期取得了进展，开发统一的视频分割（VS）模型仍然是一个挑战。这主要是因为通用的类别指定 VS 任务需要检测所有对象并在连续帧中跟踪它们，而提示引导的 VS 任务则要求在整个视频中利用视觉或文本提示重新识别目标，这使得用同一架构处理不同任务变得困难。我们尝试解决这些问题，并提出了一种新颖的统一 VS 架构，即 UniVS，通过将提示作为查询。UniVS 将来自先前帧的目标提示特征进行平均，作为其初始查询以显式解码掩码，并在掩码解码器中引入了一个目标级提示交叉注意力层，以整合记忆池中的提示特征。通过将先前帧中实体的预测掩码作为它们的视觉提示，UniVS 将不同的 VS 任务转换为提示引导的目标分割，消除了启发式的帧间匹配过程。我们的框架不仅统一了不同的 VS 任务，还自然地实现了通用训练和测试，确保在不同场景下具有稳健的性能。UniVS 在 10 个具有挑战性的 VS 基准上展示了性能与通用性之间值得称道的平衡，涵盖了视频实例、语义、全景、对象和指代分割任务。代码可在 \url{https://github.com/MinghanLi/UniVS} 找到。
英文摘要：
Despite the recent advances in unified image segmentation (IS), developing a unified video segmentation (VS) model remains a challenge. This is mainly because generic category-specified VS tasks need to detect all objects and track them across consecutive frames, while prompt-guided VS tasks require re-identifying the target with visual/text prompts throughout the entire video, making it hard to handle the different tasks with the same architecture. We make an attempt to address these issues and present a novel unified VS architecture, namely UniVS, by using prompts as queries. UniVS averages the prompt features of the target from previous frames as its initial query to explicitly decode masks, and introduces a target-wise prompt cross-attention layer in the mask decoder to integrate prompt features in the memory pool. By taking the predicted masks of entities from previous frames as their visual prompts, UniVS converts different VS tasks into prompt-guided target segmentation, eliminating the heuristic inter-frame matching process. Our framework not only unifies the different VS tasks but also naturally achieves universal training and testing, ensuring robust performance across different scenarios. UniVS shows a commendable balance between performance and universality on 10 challenging VS benchmarks, covering video instance, semantic, panoptic, object, and referring segmentation tasks. Code can be found at \url{https://github.com/MinghanLi/UniVS}.
Boosting Low-Data Instance Segmentation by Unsupervised Pre-training with Saliency Prompt
Northwestern Polytechnical University（西北工业大学）；Zhejiang Lab（之江实验室）；Inception Institute of Artificial Intelligence（起源人工智能研究院）；University of Chinese Academy of Sciences（中国科学院大学）；NetEase（网易）；Huazhong University of Science and Technology（华中科技大学）
Unsupervised Pre-training with Language-Vision Prompts for Low-Data Instance Segmentation
Northwestern Polytechnical University（西北工业大学）；Inception Institute of Artificial Intelligence（Inception人工智能研究院）；Hefei University of Technology（合肥工业大学）；Baidu Inc.（百度公司）
Multiple Queries with Multiple Keys: A Precise Prompt Matching Paradigm for Prompt-based Continual Learning
National Key Laboratory for Novel Software Technology, Nanjing University（新型软件技术国家重点实验室，南京大学）；School of Electronic Science and Engineering, Nanjing University（电子科学与工程学院，南京大学）
Efficient Universal Goal Hijacking with Semantics-guided Prompt Organization
Nanyang Technological University, Singapore（南洋理工大学）；CFAR and IHPC, Agency for Science, Technology and Research (A*STAR), Singapore（CFAR和IHPC，新加坡科技研究局）；New York University, USA（纽约大学）；East China Normal University, China（华东师范大学）
Prompt to Restore, Restore to Prompt: Cyclic Prompting for Universal Adverse Weather Removal
School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机科学与信息工程学院）；School of Information and Communication Engineering, University of Electronic Science and Technology of China（电子科技大学信息与通信工程学院）；School of Computer Science and Technology, Beijing Jiaotong University（北京交通大学计算机科学与技术学院）
PromptHSI: Universal Hyperspectral Image Restoration with Vision-Language Modulated Frequency Adaptation
Institute of Data Science, National Cheng Kung University（成功大学数据科学研究所）；Graduate Institute of Communication Engineering, National Taiwan University（台湾大学通讯工程学系）；NVIDIA（英伟达公司）
UPCMR: A Universal Prompt-guided Model for Random Sampling Cardiac MRI Reconstruction
Leiden University Medical Center（莱顿大学医学中心）；Cardiologs；Philips（飞利浦）；Eindhoven University of Technology（埃因霍温理工大学）；Philips Innovative Technologies（飞利浦创新技术）
