Many everyday tasks ranging from fixing appliances, cooking recipes to car maintenance require expert knowledge, especially when tasks are complex and multi-step. Despite growing interest in AI agents, there is a scarcity of dialogue-video datasets grounded for real world task assistance. In this paper, we propose a simple yet effective approach that transforms single-person instructional videos into task-guidance two-person dialogues, aligned with fine grained steps and video-clips. Our fully automatic approach, powered by large language models, offers an efficient alternative to the substantial cost and effort required for human-assisted data collection. Using this technique, we build HowToDIV, a large-scale dataset containing 507 conversations, 6636 question-answer pairs and 24 hours of videoclips across diverse tasks in cooking, mechanics, and planting. Each session includes multi-turn conversation where an expert teaches a novice user how to perform a task step by step, while observing user's surrounding through a camera and microphone equipped wearable device. We establish the baseline benchmark performance on HowToDIV dataset through Gemma-3 model for future research on this new task of dialogues for procedural-task assistance.
Ego-EXTRA: video-language Egocentric Dataset for EXpert-TRAinee assistance
Department of Mathematics and Computer Science - University of Catania（数学与计算机科学系 - 卡塔尼亚大学）；Next Vision s.r.l. - Spinoff of the University of Catania（Next Vision公司 - 卡塔尼亚大学衍生机构）；Meta Reality Labs Research（Meta现实实验室）
Exo2EgoDVC: Dense Video Captioning of Egocentric Procedural Activities Using Web Instructional Videos
The University of Tokyo（东京大学）；OMRON SINIC X Corp.（欧姆龙SINIC X公司）；National Institute of Advanced Industrial Science and Technology (AIST)（日本产业技术综合研究所）；LY Corporation（LY公司）
APM-Bench: Benchmarking Cross-session Persistent Memory for Egocentric Streaming Video Assistants
Shanghai Jiao Tong University（上海交通大学）；Eastern Institute of Technology, Ningbo（宁波东方理工大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Zhongguancun Academy（中关村学院）；Dalian University of Technology（大连理工大学）；Beihang University（北京航空航天大学）；Hong Kong Polytechnic University（香港理工大学）
EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real World
OpenGVLab, Shanghai AI Laboratory（上海人工智能实验室OpenGVLab）；The University of Tokyo（东京大学）；Nanjing University（南京大学）；Fudan University（复旦大学）；Zhejiang University（浙江大学）；USTC（中国科学技术大学）；SIAT, CAS（中国科学院深圳先进技术研究院）
Honda Research Institute（本田研究所）；School of Informatics, Computing and Engineering, Indiana University（印第安纳大学信息学、计算与工程学院）；Robotics Institute, University of Michigan（密歇根大学机器人研究所）
EgoBlind: Towards Egocentric Visual Assistance for the Blind
National University of Singapore（新加坡国立大学）；Communication University of China（中国传媒大学）；University of Science and Technology of China（中国科学技术大学）；Hefei University of Technoloy（合肥工业大学）
