While Human-Object Interaction(HOI) Detection has achieved tremendous advances in recent, it still remains challenging due to complex interactions with multiple humans and objects occurring in images, which would inevitably lead to ambiguities. Most existing methods either generate all human-object pair candidates and infer their relationships by cropped local features successively in a two-stage manner, or directly predict interaction points in a one-stage procedure. However, the lack of spatial configurations or reasoning steps of two- or one- stage methods respectively limits their performance in such complex scenes. To avoid this ambiguity, we propose a novel actor-centric framework. The main ideas are that when inferring interactions: 1) the non-local features of the entire image guided by actor position are obtained to model the relationship between the actor and context, and then 2) we use an object branch to generate pixel-wise interaction area prediction, where the interaction area denotes the object central area. Moreover, we also use an actor branch to get interaction prediction of the actor and propose a novel composition strategy based on center-point indexing to generate the final HOI prediction. Thanks to the usage of the non-local features and the partly-coupled property of the human-objects composition strategy, our proposed framework can detect HOI more accurately especially for complex images. Extensive experimental results show that our method achieves the state-of-the-art on the challenging V-COCO and HICO-DET benchmarks and is more robust especially in multiple persons and/or objects scenes.
OccludeNet: A Causal Journey into Mixed-View Actor-Centric Video Action Recognition under Occlusions
Sanya Science and Education Innovation Park, Wuhan University of Technology, Sanya 572025, China(武汉理工大学三亚科技创新园);Hubei Key Laboratory of Transportation Internet of Things, Wuhan University of Technology, Wuhan 430070, China(武汉理工大学交通物联网重点实验室);State Key Laboratory for Multimedia Info Processing, Peking University, Beijing 100871, China(北京大学信息处理国家重点实验室);Hubei Key Laboratory of Big Data Intelligent Analysis, Hubei University, Wuhan 430062, China(湖北大学大数据智能分析重点实验室);National Engineering Research Center for Multimedia Software, Wuhan University, Wuhan 430072, China(武汉大学多媒体软件工程研究中心);Department of Electrical Engineering, National Tsing Hua University, Hsinchu 30013, Taiwan(清华大学电子工程系)
针对视频动作识别中遮挡数据不足的问题,本文构建涵盖真实与合成场景的大规模 OccludeNet 数据集,并提出基于结构因果模型、后门调整和反事实推理的 CAR 方法以增强遮挡鲁棒性。
UniHCP: A Unified Model for Human-Centric Perceptions
The University of Sydney(悉尼大学);Zhejiang University(浙江大学);Shanghai AI Laboratory(上海人工智能实验室);SenseTime Research(商汤科技研究院);Qing Yuan Research Institute, Shanghai Jiao Tong University(上海交通大学清源研究院)
Exploring object-centric and scene-centric CNN features and their complementarity for human rights violations recognition in images
School of Computer Science and Electronic Engineering, University of Essex, UK(埃塞克斯大学计算机科学与电子工程学院,英国);School of Computer Science, University of Birmingham, UK(伯明翰大学计算机科学学院,英国)
AEI: Actors-Environment Interaction with Adaptive Attention for Temporal Action Proposals Generation
University of Arkansas(阿肯色大学);University of Maryland(马里兰大学);Carnegie Mellon University(卡内基梅隆大学);University of Science VNU-HCM(胡志明市国家大学自然科学大学);Vietnam National University(越南国家大学)
HumanBench: Towards General Human-centric Perception with Projector Assisted Pretraining
The University of Sydney(悉尼大学);SenseTime Research(商汤科技研究院);Zhejiang University(浙江大学);Shanghai AI Laboratory(上海人工智能实验室);Qing Yuan Research Institute, Shanghai Jiao Tong University(上海交通大学清源研究院)
以人为中心的感知包括各种视觉任务,这些任务具有广泛的工业应用,包括监控,自动驾驶和虚拟世界。希望有一个通用的预训练模型,用于以人为中心的多功能下游任务。本文从基准测试和预训练方法两个方面,沿着这条道路进行探索。具体而言,我们提出了一个基于现有数据集的\textbf{HumanBench},以全面评估不同预训练方法在19个数据集上的泛化能力,这些数据集来自6个不同的下游任务,包括人ReID,姿势估计,人体解析,行人属性识别,行人检测和人群计数。为了同时学习人体中的粗粒度和细粒度知识,我们进一步提出了一种\textbf{P}学习器\textbf{A} ssis \textbf{T} ed \textbf{H}原型预训练方法(\textbf{PATH})来学习不同粒度级别的知识.在HumanBench上的综合评估表明,我们的PATH在17个下游数据集上取得了新的最先进的结果,在另外2个数据集上取得了同等的结果。该代码将在\href{https://github.com/OpenGVLab/HumanBench}{https://github.com/OpenGVLab/HumanBench}上公开。
HUNTER: Unsupervised Human-centric 3D Detection via Transferring Knowledge from Synthetic Instances to Real Scenes
ShanghaiTech University(上海科技大学);University of Chinese Academy of Sciences(中国科学院大学);Innovation Academy for Microsatellites, Chinese Academy of Sciences(中国科学院微小卫星创新研究院);The University of Hong Kong(香港大学);The Chinese University of Hong Kong(香港中文大学)
Human-centric Indoor Scene Synthesis Using Stochastic Grammar
UCLA Center for Vision, Cognition, Learning and Autonomy(加州大学洛杉矶分校视觉、认知、学习与自主中心);UPenn Computer Graphics Group(宾夕法尼亚大学计算机图形学组)
我们提出了一种以人为中心的方法来采样和合成3D房间布局及其2D图像,以获得大规模的2D/3D图像数据与完美的每像素地面实况。提出了一种属性空间与或图(S-AOG)来表示室内场景。S-AOG是一个概率文法模型,其中的终端节点是对象实体。人类上下文作为上下文关系在终端节点上用马尔可夫随机场(MRF)编码。我们从室内场景数据集学习分布,并使用Monte Carlo Markov Chain对新布局进行采样。实验表明,我们的方法可以基于三个标准鲁棒地对各种各样的现实房间布局进行采样:(i)与最先进的房间布置方法相比的视觉现实主义,(ii)关于地面实况的示能图的准确性,以及(ii)由人类受试者评估的合成房间的功能性和自然性。该代码可在https://github.com/SiyuanQi/human-centric-scene-synthesis上获得。
CLIP-Guided Adaptable Self-Supervised Learning for Human-Centric Visual Tasks
Key Laboratory of Intelligent Information Processing, Institute of Computing Technology (ICT), Chinese Academy of Sciences (CAS), and University of Chinese Academy of Sciences(智能信息处理重点实验室,计算技术研究所(ICT),中国科学院(CAS)及中国科学院大学)
Human-Centric Intelligence in the Era of Foundation Models: A Survey
The Hong Kong Polytechnic University(香港理工大学);Peking University(北京大学);University of Southern Queensland(南昆士兰大学);Tongji University(同济大学);RIKEN Center for Advanced Intelligence Project(理化学研究所先进智能项目中心);Beijing Institute of Technology(北京理工大学);The University of Sydney(悉尼大学);University of Trento(特伦托大学);Alibaba Group(阿里巴巴集团);Nanyang Technological University(南洋理工大学);Hong Kong University of Science and Technology(香港科技大学)
HumanPCR: Probing MLLM Capabilities in Diverse Human-Centric Scenes
Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS, China(中国科学院智能信息处理重点实验室(中国科学院)、计算技术研究所、中国)