arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.08556cs.CVcs.AIcs.LGcs.MM

以人为中心的基础模型:感知、生成与智能体建模

Human-Centric Foundation Models: Perception, Generation and Agentic Modeling

The Chinese University of Hong Kong(香港中文大学);Zhejiang University(浙江大学);Tsinghua University(清华大学);Hong Kong University of Science and Technology(香港科技大学)

Shixiang Tang, Yizhou Wang, Lu Chen, Yuan Wang, Sida Peng, Dan Xu, Wanli Ouyang

AI总结:

本综述提出分类法,将以人为中心的基础模型(HcFMs)分为感知、AIGC、统一感知生成、智能体建模四类,回顾技术、讨论挑战与方向,为数字人类建模提供路线图。

摘要:

人类理解与生成对于数字人类和类人实体建模至关重要。近期,受大语言模型和视觉模型等通用模型成功的启发,以人为中心的基础模型(HcFMs)应运而生,将多样的以人为中心的任务统一到单一框架中,超越了传统的特定任务方法。本综述通过提出一种分类法,对HcFMs进行全面概述,将现有方法分为四类:(1)以人为中心的感知基础模型,捕捉细粒度特征以实现多模态2D和3D理解;(2)以人为中心的AIGC基础模型,生成高保真、多样化的人类相关内容;(3)统一感知与生成模型,整合这些能力以增强人类理解与合成;(4)以人为中心的智能体基础模型,超越感知与生成,学习类人智能和交互行为以完成类人实体任务。我们回顾了最先进的技术,讨论了新兴挑战和未来研究方向。本综述旨在为研究人员和从业者提供路线图,助力构建更稳健、多功能和智能的数字人类与实体建模。

英文摘要:

Human understanding and generation are critical for modeling digital humans and humanoid embodiments. Recently, Human-centric Foundation Models (HcFMs) inspired by the success of generalist models, such as large language and vision models, have emerged to unify diverse human-centric tasks into a single framework, surpassing traditional task-specific approaches. In this survey, we present a comprehensive overview of HcFMs by proposing a taxonomy that categorizes current approaches into four groups: (1) Human-centric Perception Foundation Models that capture fine-grained features for multi-modal 2D and 3D understanding. (2) Human-centric AIGC Foundation Models that generate high-fidelity, diverse human-related content. (3) Unified Perception and Generation Models that integrate these capabilities to enhance both human understanding and synthesis. (4) Human-centric Agentic Foundation Models that extend beyond perception and generation to learn human-like intelligence and interactive behaviors for humanoid embodied tasks. We review state-of-the-art techniques, discuss emerging challenges and future research directions. This survey aims to serve as a roadmap for researchers and practitioners working towards more robust, versatile, and intelligent digital human and embodiments modeling.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑