Long video understanding has emerged as an increasingly important yet challenging task in computer vision. Agent-based approaches are gaining popularity for processing long videos, as they can handle extended sequences and integrate various tools to capture fine-grained information. However, existing methods still face several challenges: (1) they often rely solely on the reasoning ability of large language models (LLMs) without dedicated mechanisms to enhance reasoning in long video scenarios; and (2) they remain vulnerable to errors or noise from external tools. To address these issues, we propose a specialized chain-of-thought (CoT) process tailored for long video analysis. Our proposed CoT with plan-adjust mode enables the LLM to incrementally plan and adapt its information-gathering strategy. We further incorporate heuristic uncertainty estimation of both the LLM and external tools to guide the CoT process. This allows the LLM to assess the reliability of newly collected information, refine its collection strategy, and make more robust decisions when synthesizing final answers. Empirical experiments show that our uncertainty-aware CoT effectively mitigates noise from external tools, leading to more reliable outputs. We implement our approach in a system called VideoAgent2, which also includes additional modules such as general context acquisition and specialized tool design. Evaluation on three dedicated long video benchmarks (and their subsets) demonstrates that VideoAgent2 outperforms the previous state-of-the-art agent-based method, VideoAgent, by an average of 13.1% and achieves leading performance among all zero-shot approaches
Enhancing Video-LLM Reasoning via Agent-of-Thoughts Distillation
School of Artificial Intelligence, Shanghai Jiao Tong University(上海交通大学人工智能学院);Coop. Medianet Innovation Center, Shanghai Jiao Tong University(上海交通大学合作媒体创新中心)
CoT-Segmenter: Enhancing OOD Detection in Dense Road Scenes via Chain-of-Thought Reasoning
Department of Artificial Intelligence, Chung-Ang University(Chung-Ang大学人工智能系);Visual Intelligence Lab., ETRI(ETRI视觉智能实验室);University of Science and Technology (UST)(科技大学(UST));School of Electronics Engineering, Kyungpook National University(Kyungpook国立大学电子工程学院)
ViRC: Enhancing Visual Interleaved Mathematical CoT with Reason Chunking
Jilin University(吉林大学);Ant Digital Technologies, Ant Group(蚂蚁集团蚂蚁数字科技);The Chinese University of Hong Kong(香港中文大学);Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, China(中国教育部知识驱动的人机智能工程研究中心)
UniRect-CoT: Enhancing Generation in Unified Multimodal Models via Reflective Rectification with Inherent Understanding
School of Software Technology, Zhejiang University(浙江大学软件技术学院);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);College of Computer Science(计算机科学学院)
FairCoT: Enhancing Fairness in Text-to-Image Generation via Chain of Thought Reasoning with Multimodal Large Language Models
School of Electronic Engineering and Computer Science, Queen Mary University of London(伦敦女王学院电子工程与计算机科学学院);Department of Knowledge Technologies, Jožef Stefan Institute(Jožef Stefan研究所知识技术系)
Enhancing the Reliability of Segment Anything Model for Auto-Prompting Medical Image Segmentation with Uncertainty Rectification
Artificial Intelligence Innovation and Incubation Institute, Fudan University(复旦大学人工智能创新与孵化研究院);Shanghai Academy of Artificial Intelligence for Science(上海科学人工智能研究院)
Existence Is Chaos: Enhancing 3D Human Motion Prediction with Uncertainty Consideration
Zhejiang University(浙江大学);Ningbo Innovation Center, Zhejiang University(浙江大学宁波创新中心);National Centre for Computer Animation, Bournemouth University(伯恩茅斯大学国家计算机动画中心)
PointLLM-R: Enhancing 3D Point Cloud Reasoning via Chain-of-Thought
Visual Computing Research Center (VCC), College of Computer Science(视觉计算研究中心(VCC),计算机科学学院);Software Engineering (CSSE) Shenzhen University China(软件工程(CSSE)深圳大学中国);VCC, CSSE Shenzhen University China(VCC,CSSE 深圳大学中国);Shenzhen University(深圳大学)
Enhancing Monte Carlo Dropout Performance for Uncertainty Quantification
Institute for Intelligent Systems Research and Innovation (IISRI)(智能系统研究与创新研究所);Deakin University(德金大学);Concordia Institute for Information Systems Engineering(Concordia信息系统工程研究所);Concordia University(Concordia大学);The Graduate School of Biomedical Engineering(生物医学工程研究生院);UNSW Sydney(新南威尔士大学悉尼分校)