Multi-modal knowledge graphs (MMKGs), which ground various non-symbolic data (e.g., images and videos) into symbols, have attracted attention as resources enabling knowledge processing and machine learning across modalities. However, the construction of MMKGs for videos consisting of multiple events, such as daily activities, is still in the early stages. In this paper, we construct an MMKG based on synchronized multi-view simulated videos of daily activities. Besides representing the content of daily life videos as event-centric knowledge, our MMKG also includes frame-by-frame fine-grained changes, such as bounding boxes within video frames. In addition, we provide support tools for querying our MMKG. As an application example, we demonstrate that our MMKG facilitates benchmarking vision-language models by providing the necessary vision-language datasets for a tailored task.
Multimodal video and IMU kinematic dataset on daily life activities using affordable devices (VIDIMU)
University of Valladolid（巴利亚多利德大学）；University of Applied Sciences and Arts Western Switzerland (HES-SO) Valais-Wallis（瑞士西部应用科学与艺术大学 (HES-SO) 瓦莱-沃利斯分校）；University of Geneva（日内瓦大学）
Energy Expenditure Estimation Through Daily Activity Recognition Using a Smart-phone
University of Paris-Sud（巴黎南大学）；CNRS-LIMSI（法国国家科学研究中心-计算机科学与机械与信息学实验室）
本文提出了一个3步系统，估计一个人的实时能量消耗的非侵入性的方式。首先，使用用户的智能手机的传感器，我们建立一个决策树模型来识别他的身体活动（\textit{running}，\textit{standing}，.）。然后，我们使用检测到的身体活动、时间和用户的速度来推断他的日常活动（\textit{watching TV}、\textit{going to the bathroom}、.）通过使用强化学习环境，部分可观察马尔可夫决策过程框架。一旦日常活动被识别，我们就使用身体活动纲要将这些信息转化为能量消耗。通过在90%的成功检测到8种身体活动，我们在识别17种不同的日常活动时达到了80%的总体准确率。该结果导致我们估计用户的能量消耗，平均误差为预期估计的26%。
Simulating Human-like Daily Activities with Desire-driven Autonomy
Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）；The University of Hong Kong（香港大学）；School of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）；Academy for Advanced Interdisciplinary Studies, Peking University（北京大学先进跨学科研究学院）；State Key Laboratory of General Artificial Intelligence, BIGAI（通用人工智能国家重点实验室，BIGAI）；Center on Frontiers of Computing Studies, School of Computer Science, Nat’l Eng. Research Center of Visual Technology, Peking University（计算前沿研究中心，计算机科学学院，国家工程视觉技术研究中心，北京大学）
On-Demand Myoelectric Control Using Wake Gestures to Eliminate False Activations During Activities of Daily Living
Department of Electrical Engineering, University of New Brunswick（新不伦瑞克大学电气工程学院）；HCI and SPECTRAL Labs, University of New Brunswick（新不伦瑞克大学HCI与SPECTRAL实验室）
Paving the Way Towards Kinematic Assessment Using Monocular Video: A Preclinical Benchmark of State-of-the-Art Deep-Learning-Based 3D Human Pose Estimators Against Inertial Sensors in Daily Living Activities
An Efficient Data Imputation Technique for Human Activity Recognition
Instituto de Telecomunicações（电信研究所）；Universidade da Beira Interior（贝拉内华大学）；Polytechnic Institute of Viseu（维塞乌理工学院）；Department of Computer Engineering（计算机工程系）；University of Engineering and Technology (UET)（工程与技术大学）；Faculty of Computer Science and Engineering（计算机科学与工程学院）；University Ss Cyril and Methodius（圣西里尔与美多德大学）
InterAct: A Large-Scale Dataset of Dynamic, Expressive and Interactive Activities between Two People in Daily Scenarios
The University of Hong Kong（香港大学）；Centre for Transformative Garment Production（转型服装生产中心）；Great Bay University（大湾大学）；Dongguan Key Laboratory for Intelligence and Information Technology（东莞智能与信息科技重点实验室）；Shandong University（山东大学）；National Institute of Informatics（国家信息研究所）
