EventMemAgent: Hierarchical Event-Centric Memory for Online Video Understanding with Adaptive Tool Use
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University（北京未来区块链与隐私计算先进创新中心，人工智能学院，北京航空航天大学）；Hangzhou International Innovation Institute, Beihang University（杭州国际创新研究院，北京航空航天大学）；Paradigm Inc.（4Paradigm公司）
Siwei Wen, Zhangcheng Wang, Xingjian Zhang, Lei Huang, Wenjun Wu
Online video understanding requires models to perform continuous perception and long-range reasoning within potentially infinite visual streams. Its fundamental challenge lies in the conflict between the unbounded nature of streaming media input and the limited context window of Multimodal Large Language Models (MLLMs). Current methods primarily rely on passive processing, which often face a trade-off between maintaining long-range context and capturing the fine-grained details necessary for complex tasks. To address this, we introduce EventMemAgent, an active online video agent framework based on a hierarchical memory module. Our framework employs a dual-layer strategy for online videos: short-term memory detects event boundaries and utilizes event-granular reservoir sampling to process streaming video frames within a fixed-length buffer dynamically; long-term memory structuredly archives past observations on an event-by-event basis. Furthermore, we integrate a multi-granular perception toolkit for active, iterative evidence capture and employ Agentic Reinforcement Learning (Agentic RL) to end-to-end internalize reasoning and tool-use strategies into the agent's intrinsic capabilities. Experiments show that EventMemAgent achieves competitive results on online video benchmarks. The code will be released here: https://github.com/lingcco/EventMemAgent.
GHR-VQA: Graph-guided Hierarchical Relational Reasoning for Video Question Answering
School of ECE National Technical University of Athens（国家技术大学雅典电子工程学院）；Robotics Institute Athena Research Center（雅典研究中心机器人研究所）；University of Luxembourg（卢森堡大学）
Northeastern University（东北大学）；Universitat Oberta de Catalunya（加泰罗尼亚开放大学）
This paper presents Hierarchical Network Dissection, a general pipeline to interpret the internal representation of face-centric inference models. Using a probabilistic formulation, our pipeline pairs units of the model with concepts in our "Face Dictionary", a collection of facial concepts with corresponding sample images. Our pipeline is inspired by Network Dissection, a popular interpretability model for object-centric and scene-centric models. However, our formulation allows to deal with two important challenges of face-centric models that Network Dissection cannot address: (1) spacial ove…
Hand-Centric Motion Refinement for 3D Hand-Object Interaction via Hierarchical Spatial-Temporal Modeling
Zhejiang University（浙江大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Shandong Artificial Intelligence Institute, Qilu University of Technology (Shandong Academy of Sciences)（齐鲁工业大学（山东省科学院）山东省人工智能研究院）
Beihang University（北航）；City University of Macau（澳门城市大学）；Peking University（北京大学）；Mohamed bin Zayed University of Artificial Intelligence（马尔代夫 bin Zayed 人工智能大学）；Beijing Normal University（北京师范大学）
WaterVideoQA: ASV-Centric Perception and Rule-Compliant Reasoning via Multi-Modal Agents
Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou)（香港理工大学（广州）人工智能研究所）；Hubei Key Laboratory of Inland Shipping Technology (Wuhan University of Technology)（湖北内河航运技术重点实验室（武汉理工大学））；School of Navigation, Wuhan University of Technology（武汉理工大学航海学院）；School of Advanced Technology, Xi’an Jiaotong-Liverpool University（西安交通大学利物浦大学先进科技学院）；School of Artificial Intelligence, Nanjing University（南京大学人工智能学院）；School of Information Engineering, Yancheng Institute of Technology（盐城职业技术学院信息工程学院）；School of Engineering, Stanford University（斯坦福大学工程学院）；Centre for AI and Data Science Innovation and the School of Science and Engineering, James Cook University（詹姆斯库克大学人工智能与数据科学创新中心及科学与工程学院）
A Hyperbolic Perspective on Hierarchical Structure in Object-Centric Scene Representations
Aalborg University & Pioneer Centre for AI, Denmark（奥尔堡大学 & 人工智能先锋中心，丹麦）；Universitat de Barcelona & Computer Vision Center, Spain（巴塞罗那大学 & 计算机视觉中心，西班牙）；Milestone Systems, Denmark（Milestone Systems，丹麦）；University of Guelph & Vector Institute, Canada（圭尔夫大学 & 向量研究所，加拿大）
EchoFoley: Event-Centric Hierarchical Control for Video Grounded Creative Sound Generation
ByteDance Intelligent Creation（字节跳动智能创作）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of California, Merced（加州大学默塞德分校）；University of California, Los Angeles（加州大学洛杉矶分校）
RoboMatrix: A Skill-centric Hierarchical Framework for Scalable Robot Task Planning and Execution in Open-World
Waseda University（早稻田大学）；Beijing Institute of Technology（北京理工大学）；The Chinese University of Hong Kong（香港中文大学）；MEGVII Technology（旷视科技）；Chinese Academy of Sciences（中国科学院）
EventFormer: A Node-graph Hierarchical Attention Transformer for Action-centric Video Event Prediction
Beihang University（北京航空航天大学）；University of Science and Technology Beijing（北京科技大学）；School of Computer Science and Engineering（计算机科学与工程学院）；State Key Laboratory of Virtual Reality Technology and Systems（虚拟现实技术与系统国家重点实验室）
Nanyang Technological University（南洋理工大学）；SenseTime Research（商汤科技研究院）；Shanghai AI Laboratory（上海人工智能实验室）
Human-centric perception plays a vital role in vision and graphics. But their data annotations are prohibitively expensive. Therefore, it is desirable to have a versatile pre-train model that serves as a foundation for data-efficient downstream tasks transfer. To this end, we propose the Human-Centric Multi-Modal Contrastive Learning framework HCMoCo that leverages the multi-modal nature of human data (e.g. RGB, depth, 2D keypoints) for effective representation learning. The objective comes with two main challenges: dense pre-train for multi-modality data, efficient usage of sparse human prior…
We present PANDA, the first gigaPixel-level humAN-centric viDeo dAtaset, for large-scale, long-term, and multi-object visual analysis. The videos in PANDA were captured by a gigapixel camera and cover real-world scenes with both wide field-of-view (~1 square kilometer area) and high-resolution details (~gigapixel-level/frame). The scenes may contain 4k head counts with over 100x scale variation. PANDA provides enriched and hierarchical ground-truth annotations, including 15,974.6k bounding boxes, 111.8k fine-grained attribute labels, 12.7k trajectories, 2.2k groups and 2.9k interactions. We be…
The temporal segmentation of events is an essential task and a precursor for the automatic recognition of human actions in the video. Several attempts have been made to capture frame-level salient aspects through attention but they lack the capacity to effectively map the temporal relationships in between the frames as they only capture a limited span of temporal dependencies. To this end we propose a complete end-to-end supervised learning approach that can better learn relationships between actions over time, thus improving the overall segmentation performance. The proposed hierarchical recu…
HCNQA: Enhancing 3D VQA with Hierarchical Concentration Narrowing Supervision
Southern University of Science and Technology, Shenzhen, China（南方科技大学，深圳，中国）；College of William and Mary, Williamsburg, VA 23185, USA（威廉玛丽学院，威廉斯堡，VA 23185，美国）；Peng Cheng Laboratory, Shenzhen, China（鹏城实验室，深圳，中国）
Human-Centric Perception for Child Sexual Abuse Imagery
Universidade Federal de Minas Gerais (UFMG)（米纳斯吉拉斯联邦大学）；Federal Police of Brazil（巴西联邦警察局）；Universidade Estadual de Campinas (UNICAMP)（坎皮纳斯州立大学）；University of Sheffield（谢菲尔德大学）
GESTO: Human-Centric Spatio-Temporal Memory for Reasoning in Dynamic Scenes
KTH Royal Institute of Technology（瑞典皇家理工学院）；University of Stuttgart（斯图加特大学）；Max Planck Research School for Intelligent Systems（马克斯·普朗克智能系统研究所）；Google Research（谷歌研究院）；TU Munich（慕尼黑工业大学）
RobotEQ-Video: A Video-Centric Benchmark for Social Proactive Intelligence with World-State Taxonomy
State Key Laboratory of Autonomous Intelligent Unmanned Systems, Tongji University（同济大学自主智能无人系统全国重点实验室）；Tsinghua University（清华大学）；The Chinese University of Hong Kong（香港中文大学）；Shenzhen MSU-BIT University（深圳北理莫斯科大学）；University of Science and Technology of China（中国科学技术大学）
Focus-to-Perceive Representation Learning: A Cognition-Inspired Hierarchical Framework for Endoscopic Video Analysis
Key Laboratory of Intelligent Computing and Information Processing of Ministry of Education, Xiangtan University（湘潭大学教育部智能计算与信息处理重点实验室）；School of Computer Science, Xiangtan University（湘潭大学计算机科学学院）；Key Laboratory for Artificial Intelligence and International Communication, Hunan Normal University（湖南师范大学人工智能与国际传播重点实验室）
