Despite the fact that many 3D human activity benchmarks being proposed, most existing action datasets focus on the action recognition tasks for the segmented videos. There is a lack of standard large-scale benchmarks, especially for current popular data-hungry deep learning based methods. In this paper, we introduce a new large scale benchmark (PKU-MMD) for continuous multi-modality 3D human action understanding and cover a wide range of complex human activities with well annotated information. PKU-MMD contains 1076 long video sequences in 51 action categories, performed by 66 subjects in three camera views. It contains almost 20,000 action instances and 5.4 million frames in total. Our dataset also provides multi-modality data sources, including RGB, depth, Infrared Radiation and Skeleton. With different modalities, we conduct extensive experiments on our dataset in terms of two scenarios and evaluate different methods by various metrics, including a new proposed evaluation protocol 2D-AP. We believe this large-scale dataset will benefit future researches on action detection for the community.
CMD: Self-supervised 3D Action Representation Learning with Cross-modal Mutual Distillation
University of Science and Technology of China(中国科学技术大学);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥综合性国家科学中心人工智能研究院)
USDRL: Unified Skeleton-Based Dense Representation Learning with Multi-Grained Feature Decorrelation
Southeast University(东南大学);Northwestern Polytechnical University(西北工业大学);Tsinghua University(清华大学);Nanjing University of Science and Technology(南京理工大学)
Multimodal Skeleton-Based Action Representation Learning via Decomposition and Composition
School of Computer Science and Engineering, Southeast University, Nanjing 210096, China(东南大学计算机科学与工程学院,南京210096,中国);Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China(新一代人工智能技术及其交叉应用关键实验室(东南大学),中华人民共和国教育部,中国);School of Cyber Science and Engineering, Southeast University, Nanjing 210096, China(东南大学网络安全科学与工程学院,南京210096,中国);Engineering Research Center of Blockchain Application, Supervision And Management (Southeast University), Ministry of Education, China(区块链应用、监督与管理工程研究中心(东南大学),中华人民共和国教育部,中国);Purple Mountain Laboratories, Nanjing 210000, China(紫金山实验室,南京210000,中国)
CmIVTP: Cross-modal Interaction-based Vessel Trajectory Prediction for Maritime Intelligence
Department of Logistics and Maritime Studies, the Hong Kong Polytechnic University(物流及海运研究系,香港理工大学);Research Centre for ESG Advancement (RCESGA), the Hong Kong Polytechnic University(ESG进步研究中心(RCESGA),香港理工大学);School of Navigation, Wuhan University of Technology(航海学院,武汉理工大学)
MMDrive: Interactive Scene Understanding Beyond Vision with Multi-representational Fusion
organization= College of Computer Science;Technology, Jilin University , city= Changchun , country= China;organization= Georgia Institute of Technology , city= Atlanta , country= USA;organization= Qingdao Institute of Software, College of Computer Science;Technology, China University of Petroleum (East China) , city= Qingdao , country= China;organization= Institute for Math \& AI, Wuhan University , city= Wuhan , country= China;organization= University of Michigan, Ann Arbor , country= USA;organization= Thrust of Artificial Intelligence, Hong Kong University of Science;organization= School of Artificial Intelligence;Computer Science, Nantong University , city= Nantong , country= China
Vision-Language Meets the Skeleton: Progressively Distillation with Cross-Modal Knowledge for 3D Action Representation Learning
School of Information and Communication Engineering, University of Electronic Science and Technology of China(电子科技大学信息与通信工程学院);School of Automation Engineering, University of Electronic Science and Technology of China(电子科技大学自动化工程学院);The Hong Kong Polytechnic University(香港理工大学);Hong Kong Polytechnic University Shenzhen Research Institute(香港理工大学深圳研究院);No.1 Orthopedic Hospital of Chengdu(成都市第一骨科医院)
Unified Multi-modal Unsupervised Representation Learning for Skeleton-based Action Understanding
Zhejiang Gongshang University(浙江工商大学);Peking University(北京大学);Zhejiang Key Lab of E-Commerce(浙江省电子商务重点实验室);Huazhong University of Science and Technology(华中科技大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Science and Technology of China(中国科学技术大学);Hefei University of Technology(合肥工业大学)
MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMs
Beijing University of Posts and Telecommunications(北京邮电大学);University of California, Santa Barbara(加州大学圣巴巴拉分校);Center for Research on Intelligent Perception and Computing, NLPR, CASIA(中国科学院自动化研究所模式识别国家重点实验室智能感知与计算研究中心)
Pose-guided Generative Adversarial Net for Novel View Action Synthesis
CRCV, University of Central Florida(中佛罗里达大学计算机视觉研究中心);National University of Singapore(新加坡国立大学);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院)
Affinity Contrastive Learning for Skeleton-based Human Activity Understanding
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);School of Electronic, Electrical and Communication Engineering, Univeristy of Chinese Academy of Sciences(中国科学院大学电子电气与通信工程学院);School of Artificial Intelligence, Beijing University of Posts and Telecommunications(北京邮电大学人工智能学院);Moonshot AI
Masked Motion Predictors are Strong 3D Action Representation Learners
University of Science and Technology of China(中国科学技术大学);Hefei Comprehensive National Science Center(合肥综合性国家科学中心);The University of Sydney(悉尼大学);Merchants Union Consumer Finance Company Limited(招商局消费金融有限公司)
School of Computer Science and Engineering, Southeast University, Nanjing 210096, China(东南大学计算机科学与工程学院);Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China(新一代人工智能技术及交叉应用关键实验室(东南大学));School of Cyber Science and Engineering, Southeast University, Nanjing 210096, China(东南大学网络科学与工程学院);Engineering Research Center of Blockchain Application, Supervision And Management (Southeast University), Ministry of Education, China(区块链应用、监督与管理工程研究中心(东南大学));Purple Mountain Laboratories, Nanjing 210000, China(紫金山实验室)