Chain-of-Procedure: Hierarchical Visual-Language Reasoning for Procedural QA
NLP;CT Lab, Department of Computer and Information Science, University of Macau(计算机与信息科学系,澳门大学CT实验室);Department of Electrical and Computer Engineering, University of Macau(电气与计算机工程系,澳门大学);Centre for Cognitive and Brain Sciences, University of Macau(认知与脑科学中心,澳门大学)
Guanhua Chen, Yutong Yao, Shenghe Sun, Ci-Jun Gao, Shudong Liu, Lidia S. Chao, Feng Wan, Derek F. Wong
Recent advances in vision-language models (VLMs) have achieved impressive results on standard image-text tasks, yet their potential for visual procedure question answering (VP-QA) remains largely unexplored. VP-QA presents unique challenges where users query next-step actions by uploading images for intermediate states of complex procedures. To systematically evaluate VLMs on this practical task, we propose ProcedureVQA, a novel multimodal benchmark specifically designed for visual procedural reasoning. Through comprehensive analysis, we identify two critical limitations in current VLMs: inadequate cross-modal retrieval of structured procedures given visual states, and misalignment between image sequence granularity and textual step decomposition. To address these issues, we present Chain-of-Procedure (CoP), a hierarchical reasoning framework that first retrieves relevant instructions using visual cues, then performs step refinement through semantic decomposition, and finally generates the next step. Experiments across six VLMs demonstrate CoP's effectiveness, achieving up to 13% absolute improvement over standard baselines.
TAMA: Tool-Augmented Multimodal Agent for Procedural Activity Understanding
Language Technologies Institute, Carnegie Mellon University(卡内基梅隆大学语言技术研究所);National Institute of Advanced Industrial Science and Technology (AIST)(国家先进工业科学与技术研究院)
AutoPRM: Automating Procedural Supervision for Multi-Step Reasoning via Controllable Question Decomposition
University of Chicago(芝加哥大学);Peking University(北京大学);University of California, Berkeley(加州大学伯克利分校);Carnegie Mellon University(卡内基梅隆大学);UNC-Chapel Hill(北卡罗来纳大学教堂山分校)
Beyond Factual Knowledge: Benchmarking and Learning Step-Level Procedural Rule Reasoning in Large Language Models
University of Chinese Academy of Sciences(中国科学院大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院)
BioProBench: A Corpus and Benchmark for Biological Protocol Reasoning in Autonomous Science
School of Electronic and Computer Engineering, Peking University(北京大学电子与计算机工程学院);School of Chemical Biology&Biotechnology, Peking University(北京大学化学生物学与生物技术学院)
MoReBench: Evaluating Procedural and Pluralistic Moral Reasoning in Language Models, More than Outcomes
University of Washington(华盛顿大学);New York University(纽约大学);Scale AI;Harvard University(哈佛大学);University of Michigan(密歇根大学);UNC Chapel Hill(北卡罗来纳大学教堂山分校);Center for AI Safety(人工智能安全中心);Stanford University(斯坦福大学);MIT(麻省理工学院);University of Oxford(牛津大学)
Cognitive Loop of Thought: Reversible Hierarchical Markov Chain for Efficient Mathematical Reasoning
School of Computer Science and Technology, East China Normal University(1 计算机科学与技术学院,华东师范大学);School of Electronic and Electrical Engineering, Shanghai University of Engineering Science(2 电子电气工程学院,上海工程技术大学)
Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models
School of Computer Science, Fudan University(复旦大学计算机科学技术学院);National University of Singapore(新加坡国立大学);Shanghai AI Laboratory(上海人工智能实验室);Midea AI Research Center(美的AI研究中心)
Meta-Cognitive Analysis: Evaluating Declarative and Procedural Knowledge in Datasets and Large Language Models
University of Chinese Academy of Sciences(中国科学院大学);Chinese Information Processing Laboratory(中文信息处理实验室);State Key Laboratory of Computer Science(计算机科学国家重点实验室);Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所)
ReasoningLens: Hierarchical Visualization and Diagnostic Auditing for Large Reasoning Models
Chinese Information Processing Laboratory(中国信息处理实验室);Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);University of Chinese Academy of Sciences(中国科学院大学)
THOR: A Theta-Gamma Hierarchical Oscillatory Reasoning Framework for Multi-hop QA
Suzhou Institute for Advanced Research, University of Science and Technology of China(中国科学技术大学苏州高等研究院);School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China(中国科学技术大学生命科学与医学部生物医学工程学院)
PlaSma: Making Small Language Models Better Procedural Knowledge Models for (Counterfactual) Planning
Allen Institute for Artificial Intelligence(艾伦人工智能研究所);University of Washington(华盛顿大学);University of Southern California(南加州大学);Tohoku University(东北大学)