Hulk：面向人类中心任务的通用知识翻译器
Hulk: A Universal Knowledge Translator for Human-Centric Tasks
AI总结：
Hulk是首个无需任务特定微调的多模态人类中心通用模型，通过统一离散与连续表示，将多种任务转化为模态翻译，在12个基准上11项达到最优。
摘要：
人类中心感知任务，例如行人检测、基于骨架的动作识别和姿态估计，在工业中有着广泛的应用，如元宇宙和体育分析。近期，开发能够惠及广泛人类中心感知任务的人类中心基础模型的趋势日益增长。尽管许多人类中心基础模型已取得成功，但它们未探索人类中心的3D和视觉-语言任务，并且需要针对特定任务的微调。这些限制制约了它们在下游任务和场景中的更广泛应用。为解决这些问题，我们提出了Hulk，首个多模态人类中心通用模型，能够处理2D视觉、3D视觉、基于骨架和视觉-语言任务，无需针对特定任务的微调。实现这一目标的关键在于将各种任务特定的头部压缩为两个通用头部，一个用于离散表示（如语言），另一个用于连续表示（如位置坐标）。两个头部的输出可进一步堆叠成四种不同的输入和输出模态。这种统一表示使Hulk能够将多样化的人类中心任务视为模态翻译，整合广泛任务中的知识。在覆盖8项人类中心任务的12个基准上对Hulk进行全面评估，证明了我们提出的方法的优越性，在11个基准上达到了最先进的性能。代码将在https://github.com/OpenGVLab/Hulk上提供。
英文摘要：
Human-centric perception tasks, e.g., pedestrian detection, skeleton-based action recognition, and pose estimation, have wide industrial applications, such as metaverse and sports analysis. There is a recent surge to develop human-centric foundation models that can benefit a broad range of human-centric perception tasks. While many human-centric foundation models have achieved success, they did not explore 3D and vision-language tasks for human-centric and required task-specific finetuning. These limitations restrict their application to more downstream tasks and situations. To tackle these problems, we present Hulk, the first multimodal human-centric generalist model, capable of addressing 2D vision, 3D vision, skeleton-based, and vision-language tasks without task-specific finetuning. The key to achieving this is condensing various task-specific heads into two general heads, one for discrete representations, \emph{e.g.,} languages, and the other for continuous representations, \emph{e.g.,} location coordinates. The outputs of two heads can be further stacked into four distinct input and output modalities. This uniform representation enables Hulk to treat diverse human-centric tasks as modality translation, integrating knowledge across a wide range of tasks. Comprehensive evaluations of Hulk on 12 benchmarks covering 8 human-centric tasks demonstrate the superiority of our proposed method, achieving state-of-the-art performance in 11 benchmarks. The code will be available on https://github.com/OpenGVLab/Hulk.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
