This paper introduces MiniGPT4-Video, a multimodal Large Language Model (LLM) designed specifically for video understanding. The model is capable of processing both temporal visual and textual data, making it adept at understanding the complexities of videos. Building upon the success of MiniGPT-v2, which excelled in translating visual features into the LLM space for single images and achieved impressive results on various image-text benchmarks, this paper extends the model's capabilities to process a sequence of frames, enabling it to comprehend videos. MiniGPT4-video does not only consider visual content but also incorporates textual conversations, allowing the model to effectively answer queries involving both visual and text components. The proposed model outperforms existing state-of-the-art methods, registering gains of 4.22%, 1.13%, 20.82%, and 13.1% on the MSVD, MSRVTT, TGIF, and TVQA benchmarks respectively. Our models and code have been made publicly available here https://vision-cair.github.io/MiniGPT4-video/
VTI-CoT: Visual-Textual Interleaved Chain of Thought for Video Reasoning
Beijing University of Posts and Telecommunications（北京邮电大学）；University of Hong Kong（香港大学）；Beijing Shanwei Zhixing Technology Co., Ltd.（北京尚维智行科技有限公司）；Tsinghua University（清华大学）；Beihang University（北航）
ViTCoT: Video-Text Interleaved Chain-of-Thought for Boosting Video Understanding in Large Language Models
School of Computer Science and Engineering, Central South University（计算机科学与工程学院，中南大学）；Research Center for SCIR, Harbin Institute of Technology（SCIR研究中心，哈尔滨工业大学）；NExT Research Center, National University of Singapore（NExT研究中心，新加坡国立大学）
Beyond Textual CoT: Interleaved Text-Image Chains with Deep Confidence Reasoning for Image Editing
Shanghai Jiaotong University（上海交通大学）；Huawei Noah’s Ark Lab（华为诺亚实验室）；East China Normal University（华东师范大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究院，中国科学院）
Think in Strokes, Not Pixels: Process-Driven Image Generation via Interleaved Reasoning
Meta Superintelligence Labs（Meta超级智能实验室）；University of California, San Diego（加州大学圣迭戈分校）；Worcester Polytechnic Institute（伍斯特理工学院）；Northwestern University（西北大学）
TRACE: Temporal Grounding Video LLM via Causal Event Modeling
School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）理工学院）；Tencent PCG（腾讯平台与内容事业群）；Shenzhen Institute of Artificial Intelligence and Robotics for Society (AIRS)（深圳市人工智能与机器人研究院）；Guangdong Provincial Key Laboratory of Future Networks of Intelligence（广东省未来智能网络重点实验室）
Tsinghua University（清华大学）；Meituan Inc（美团公司）；Central South University（中南大学）；Peking University（北京大学）；University of Chinese Academy of Sciences（中国科学院大学）；Nanyang Technological University（南洋理工大学）
CoMM: A Coherent Interleaved Image-Text Dataset for Multimodal Understanding and Generation
The Hong Kong University of Science and Technology（香港科技大学）；Wuhan University（武汉大学）；AI Chip Center for Emerging Smart Systems（新兴智能系统人工智能芯片中心）；Kuaishou Technology（快手科技）
Bridging Modal Isolation in Interleaved Thinking: Supervising Modality Transitions via Stepwise Reinforcement
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Shanghai Jiaotong University（上海交通大学）；Zhejiang University（浙江大学）；University of Chinese Academy of Sciences（中国科学院大学）
ViRC: Enhancing Visual Interleaved Mathematical CoT with Reason Chunking
Jilin University（吉林大学）；Ant Digital Technologies, Ant Group（蚂蚁集团蚂蚁数字科技）；The Chinese University of Hong Kong（香港中文大学）；Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, China（中国教育部知识驱动的人机智能工程研究中心）
Contextual AD Narration with Interleaved Multimodal Sequence
State Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；ESAT, KU Leuven（鲁汶大学ESAT学院）；Ant Group（蚂蚁集团）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
Position Rebinding Cache Reuse: Replay-Free Visual Revisiting for Interleaved Multimodal Reasoning
Sun Yat-sen University（中山大学）；Shenzhen Loop Area Institute（深圳河套学院）；Harbin Institute of Technology (HIT)（哈尔滨工业大学）；Fudan University（复旦大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；The Chinese University of Hong Kong（香港中文大学）
Interleaved Latent Visual Reasoning with Selective Perceptual Modeling
China University of Geosciences, Wuhan（中国地质大学（武汉））；Shanghai Innovation Institute（上海创新研究院）；University of Southern California（南加州大学）；Fudan University（复旦大学）；Zhejiang University（浙江大学）；Shanghai Jiao Tong University（上海交通大学）
Interleaved Scene Graphs for Interleaved Text-and-Image Generation Assessment
University of Washington（华盛顿大学）；Huazhong University of Science and Technology（华中科技大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Notre Dame（圣母大学）
ARMOR: Empowering Multimodal Understanding Model with Interleaved Multimodal Generation Capability
Nankai University（南开大学）；Shanghai Innovation Institute（上海创新研究院）；University of Science and Technology of China（中国科学技术大学）；Wuhan University（武汉大学）；Shanghai AI Laboratory（上海人工智能实验室）
College of Computer Science, Sichuan University（四川大学计算机科学学院）；Southwest China Institute of Electronic Technology（西南中国电子技术研究所）；National Key Laboratory of Fundamental Algorithms（国家基础算法重点实验室）；Models for Engineering Simulation, Sichuan University（工程模拟模型研究所，四川大学）
Advancing Prompt Learning through an External Layer
Shanghai Jiaotong University（上海交通大学）；University of Science and Technology of China（中国科学技术大学）；Zhejiang University（浙江大学）；Defense Innovation Institute（国防科技创新研究院）
UniVLR: Unifying Text and Vision in Visual Latent Reasoning for Multimodal LLMs
University of Science and Technology of China（中国科学技术大学）；Tsinghua University（清华大学）；National University of Singapore（新加坡国立大学）；Zhongguancun Academy（中关村学院）
School of Computer Science and Technology, Soochow University（苏州大学计算机科学与技术学院）；Department of Computer Science, The Hong Kong Polytechnic University（香港理工大学计算机科学系）
Reasoning in the Dark: Interleaved Vision-Text Reasoning in Latent Space
The Hong Kong Polytechnic University（香港理工大学）；Singapore Management University（新加坡管理学院）；Shandong University（山东大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））
