Temporal logical understanding, a core facet of human cognition, plays a pivotal role in capturing complex sequential events and their temporal relationships within videos. This capability is particularly crucial in tasks like Video Question Answering (VideoQA), where the goal is to process visual data over time together with textual data to provide coherent answers. However, current VideoQA benchmarks devote little focus to evaluating this critical skill due to the challenge of annotating temporal logic. Despite the advancement of vision-language models, assessing their temporal logical reasoning powers remains a challenge, primarily due to the lack QA pairs that demand formal, complex temporal reasoning. To bridge this gap, we introduce the TimeLogic QA (TLQA) framework to automatically generate the QA pairs, specifically designed to evaluate the temporal logical understanding. To this end, TLQA leverages temporal annotations from existing video datasets together with temporal operators derived from logic theory to construct questions that test understanding of event sequences and their temporal relationships. TLQA framework is generic and scalable, capable of leveraging both, existing video action datasets with temporal action segmentation annotations, or video datasets with temporal scene graph annotations, to automatically generate temporal logical questions. We leverage 4 datasets, STAR, Breakfast, AGQA, and CrossTask, and generate two VideoQA dataset variants - small (TLQA-S) and large (TLQA-L) - containing 2k and 10k QA pairs for each category, resulting in 32k and 160k total pairs per dataset. We undertake a comprehensive evaluation of leading-edge VideoQA models, employing the TLQA to benchmark their temporal logical understanding capabilities. We assess the VideoQA model's temporal reasoning performance on 16 categories of temporal logic with varying temporal complexity.
Temporal Evidence Routing with Structured Visual Evidence for TimeLogicQA
Southeast University（东南大学）；National University of Singapore（新加坡国立大学）；Independent Researcher（独立研究员）；Opus AI Research（Opus AI研究院）；University of Science and Technology of China（中国科学技术大学）
MAVIS: Multi-Agent Video Retrieval via Structured Video Understanding
School of Computing and Information Technology, Great Bay University（大湾区大学计算机与信息技术学院）；College of Computer Science, Nankai University（南开大学计算机学院）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Graduate School of Information Science and Technology, The University of Tokyo（东京大学信息科学与技术研究生院）
StreamingCoT: A Dataset for Temporal Dynamics and Multimodal Chain-of-Thought Reasoning in Streaming VideoQA
Henan Institute of Advanced Technology, Zhengzhou University（河南高级技术研究所，郑州大学）；Institute of Automation, CAS（自动化研究所，中国科学院）；UCAS（中国科学院大学）；Peng Cheng Laboratory（鹏城实验室）
Understanding Temporal Logic Consistency in Video-Language Models through Cross-Modal Attention Discriminability
School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China（北京理工大学计算机科学与技术学院，北京，中国）；Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications, Beijing Institute of Technology, Beijing, China（高性能语言信息处理与云计算应用北京工程研究中心，北京理工大学，北京，中国）
Think, Then Verify: A Hypothesis-Verification Multi-Agent Framework for Long Video Understanding
College of Computer Science, Zhejiang University of Technology（浙江工业大学计算机科学学院）；Zhejiang Key Laboratory of Visual Information Intelligent Processing（浙江视觉信息智能处理重点实验室）；UC Berkeley（伯克利大学）；College of Computer Science and Technology, East China Normal University（华东师范大学计算机科学与技术学院）
Zero-shot Compositional Action Recognition with Neural Logic Constraints
College of Computer Science and Technology（计算机科学与技术学院）；Zhejiang University（浙江大学）；AI Chip Center for Emerging Smart Systems（新兴智能系统人工智能芯片中心）；The Hong Kong University of Science and Technology（香港科学与技术大学）
Decoupling Semantics and Logic: A Training-Free Coarse-to-Fine Pipeline for Video Retrieval-Augmented Generation
School of Computer Science & Tech, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）；School of AI and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）
UpstreamQA: A Modular Framework for Explicit Reasoning on Video Question Answering Tasks
Lincoln North Star High School（林肯北星高中）；The Charter School of Wilmington（威尔明顿 charter 学校）；Greenwich High School（格林威治高中）；Santa Susana High School（圣塔苏娜高中）；UC Berkeley（伯克利大学）
ChainReaction: Causal Chain-Guided Reasoning for Modular and Explainable Causal-Why Video Question Answering
Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore（新加坡高性能计算研究所，科技研究局）；Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore（科技研究局前沿人工智能研究中心，新加坡）；College of Computing and Data Science, Nanyang Technological University, Singapore（新加坡南洋理工大学计算与数据科学学院）
Taking an Emotional Look at Video Paragraph Captioning
Tongji University（同济大学）；Hong Kong Polytechnic University（香港理工大学）
将视觉数据转换为自然语言对于机器理解世界并与人类交互至关重要。在这项工作中，一个全面的研究进行视频段落字幕，目标是生成一个给定的视频段落级的描述。然而，目前的研究主要集中在检测客观事实，忽略了需要建立句子之间的逻辑关联，并发现更准确的情感相关的视频内容。这样的问题影响了预测字幕的流畅和丰富的表达，这远远低于人类的语言标准。为了解决这个问题，我们建议构建一个大规模的情感和逻辑驱动的多语言数据集。该数据集被命名为EMVPC（Emotional Video Paragraph Captioning），包含日常生活中广泛使用的53种情绪，与这些情绪相对应的376个常见场景，10，291个高质量视频和20，582个精心制作的中英文段落字幕。在这个新的数据集中还提供了相关的情感类别、场景标签、情感词标签和逻辑词标签。拟议的EMVPC数据集旨在提供全面的视频段落字幕，具有丰富的情感、连贯的逻辑和精致的表达，这也可以有益于视觉语言领域的其他任务。此外，进行了全面的研究，通过实验对现有的基准视频段落字幕数据集和建议EMVPC。从不同的视觉字幕任务的最先进的方案进行了比较，在15个流行的指标，并总结了详细的客观和主观的结果。最后，本文对视频段落字幕技术存在的问题和未来的发展方向进行了展望。这一独特的视角有望推动视频段落字幕研究的进一步发展。
I2VControl: Disentangled and Unified Video Motion Synthesis Control
Intelligent Creation Team, ByteDance（字节跳动智能创作团队）；University of Science and Technology of China (USTC)（中国科学技术大学）；Institute of Automation, Chinese Academy of Sciences (CASIA)（中国科学院自动化研究所）
