EPFL-Smart-Kitchen-30：含3D运动学密集标注的烹饪数据集，用于挑战视频与语言模型

Cooking in the kitchen: Recognizing and Segmenting Human Activities in Videos

As research on action recognition matures, the focus is shifting away from categorizing basic task-oriented actions using hand-segmented video datasets to understanding complex goal-oriented daily human activities in real-world settings. Temporally structured models would seem obvious to tackle this set of problems, but so far, cases where these models have outperformed simpler unstructured bag-of-word types of models are scarce. With the increasing availability of large human activity datasets, combined with the development of novel feature coding techniques that yield more compact representa…

Rethinking Cooking State Recognition with Vision Transformers

To ensure proper knowledge representation of the kitchen environment, it is vital for kitchen robots to recognize the states of the food items that are being cooked. Although the domain of object detection and recognition has been extensively studied, the task of object state classification has remained relatively unexplored. The high intra-class similarity of ingredients during different states of cooking makes the task even more challenging. Researchers have proposed adopting Deep Learning based strategies in recent times, however, they are yet to achieve high performance. In this study, we …

ViewBridge：用于极端视角变化下的活动视角不变性的课程知识蒸馏

ViewBridge: Curriculum Knowledge Distillation for Activity View-Invariance Under Extreme Viewpoint Changes

本文提出ViewBridge框架，通过课程学习和知识蒸馏方法，在严重视角遮挡下学习丰富的视频表示，提升了极端视角变化下的活动识别性能。

将大语言模型嵌入流程控制：一种用于自适应且可信的自动化烹饪的智能体框架

Embedding Large Language Models into Flow Controls: An Agentic Framework for Adaptive and Trustworthy Automated Cooking

针对自动化烹饪机器人的个性化与可信性问题，提出智能体框架，通过多智能体分解需求、分阶段生成执行代码，实验验证其在真实场景中可靠且实用。

Temporal Saliency Adaptation in Egocentric Videos

This work adapts a deep neural model for image saliency prediction to the temporal domain of egocentric video. We compute the saliency map for each video frame, firstly with an off-the-shelf model trained from static images, secondly by adding a a convolutional or conv-LSTM layers trained with a dataset for video saliency prediction. We study each configuration on EgoMon, a new dataset made of seven egocentric videos recorded by three subjects in both free-viewing and task-driven set ups. Our results indicate that the temporal adaptation is beneficial when the viewer is not moving and observin…

WristCompass: 运动耦合作为可学习的视觉概念用于自我相机朝向估计

WristCompass: Kinematic Coupling as a Learnable Visual Concept for Ego-Camera Orientation

提出WristCompass，利用手腕与相机朝向之间的运动耦合作为视觉概念，通过紧凑的4D特征和GRU时序建模，从操作视频中恢复自我相机朝向，零样本迁移至厨房视频并达到与1B参数场景模型相近的性能。