TSP-Transformer：面向整体场景理解的任务特定提示增强Transformer
TSP-Transformer: Task-Specific Prompts Boosted Transformer for Holistic Scene Understanding
AI总结：
针对整体场景理解多子任务需兼顾共性与特性表征的问题，提出TSP-Transformer，通过引入任务特定提示实现通用与任务特定信息的融合学习，在NYUD-v2和PASCAL-Context上达到SOTA性能。
摘要：
整体场景理解包含语义分割、表面法线估计、物体边界检测、深度估计等任务。该问题的核心在于有效学习表征，因为每个子任务不仅依赖相关属性，还依赖各自独特的属性。受视觉提示调优的启发，我们提出了任务特定提示Transformer（简称TSP-Transformer）用于整体场景理解。该模型的特点是前期采用普通Transformer，横向阶段采用任务特定提示Transformer编码器，并在其中加入任务特定提示。通过这种方式，Transformer层从共享部分学习通用信息，同时被赋予任务特定能力。首先，任务特定提示可有效作为每个任务的诱导先验。此外，任务特定提示可被视为开关，以促进不同任务的任务特定表征学习。在NYUD-v2和PASCAL-Context上开展的大量实验表明，我们的方法达到了当前最优性能，验证了该方法在整体场景理解任务中的有效性。我们还在以下链接提供了代码：https://github.com/tb2-sy/TSP-Transformer。
英文摘要：
Holistic scene understanding includes semantic segmentation, surface normal estimation, object boundary detection, depth estimation, etc. The key aspect of this problem is to learn representation effectively, as each subtask builds upon not only correlated but also distinct attributes. Inspired by visual-prompt tuning, we propose a Task-Specific Prompts Transformer, dubbed TSP-Transformer, for holistic scene understanding. It features a vanilla transformer in the early stage and tasks-specific prompts transformer encoder in the lateral stage, where tasks-specific prompts are augmented. By doing so, the transformer layer learns the generic information from the shared parts and is endowed with task-specific capacity. First, the tasks-specific prompts serve as induced priors for each task effectively. Moreover, the task-specific prompts can be seen as switches to favor task-specific representation learning for different tasks. Extensive experiments on NYUD-v2 and PASCAL-Context show that our method achieves state-of-the-art performance, validating the effectiveness of our method for holistic scene understanding. We also provide our code in the following link https://github.com/tb2-sy/TSP-Transformer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
