This paper introduces QCaption, a novel video captioning and Q&A pipeline that enhances video analytics by fusing three models: key frame extraction, a Large Multimodal Model (LMM) for image-text analysis, and a Large Language Model (LLM) for text analysis. This approach enables integrated analysis of text, images, and video, achieving performance improvements over existing video captioning and Q&A models; all while remaining fully self-contained, adept for on-premises deployment. Experimental results using QCaption demonstrated up to 44.2% and 48.9% improvements in video captioning and Q&A tasks, respectively. Ablation studies were also performed to assess the role of LLM on the fusion on the results. Moreover, the paper proposes and evaluates additional video captioning approaches, benchmarking them against QCaption and existing methodologies. QCaption demonstrate the potential of adopting a model fusion approach in advancing video analytics.
University of Science and Technology of China（中国科学技术大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；CAS Center for Excellence in Brain Science and Intelligence Technology（中国科学院脑科学与智能技术卓越创新中心）
FLEUR: An Explainable Reference-Free Evaluation Metric for Image Captioning Using a Large Multimodal Model
Seoul National University（首尔大学）；Interdisciplinary Program in Artificial Intelligence, Seoul National University（首尔大学人工智能跨学科项目）；Department of Mathematical Sciences, Seoul National University（首尔大学数学科学系）
EAMA : Entity-Aware Multimodal Alignment Based Approach for News Image Captioning
Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机研究所）；School of Electronics Engineering and Computer Science, Peking University（北京大学电子工程与计算机科学学院）
CCRC: A Change-Aware Captioning and Reasoning Chain for Image Change Captioning and Segmentation
College of Computer Science and Electronic Engineering, Hunan University, China（湖南大学计算机科学与电子工程学院，中国）；Guangxi Minzu University, China（广西民族大学，中国）；National University of Defense Technology, China（国防科学技术大学，中国）
Detecting and Understanding Hateful Contents in Memes Through Captioning and Visual Question-Answering
The University of Sydney, School of Computer Science（悉尼大学计算机科学学院）；University of Technology Sydney, School of Computer Science（新南威尔士大学技术学院）；University of Technology Sydney, TD School（新南威尔士大学TD学院）；Australian Catholic University, Peter Faber Business School（澳大利亚天主教大学彼得·法伯商学院）
ABMAMBA: Multimodal Large Language Model with Aligned Hierarchical Bidirectional Scan for Efficient Video Captioning
Keio University（庆应义塾大学）；National Institute of Informatics（国立信息学研究所）；National Institute of Informatics Research and Development Center for Large Language Models（国立信息学研究所大型语言模型研发中心）
Player-Centric Multimodal Prompt Generation for Large Language Model Based Identity-Aware Basketball Video Captioning
Beijing University of Technology（北京理工大学）；Shanghai Jiao Tong University（上海交通大学）；Chinese Academy of Sciences（中国科学院）；The Hong Kong Polytechnic University（香港理工大学）
BTCChat: Advancing Remote Sensing Bi-temporal Change Captioning with Multimodal Large Language Model
State Key Laboratory of Networking and Switching Technology（网络与交换技术国家重点实验室）；Beijing University of Posts and Telecommunications（北京邮电大学）；Aerospace Information Research Institute（航天信息研究所）；Chinese Academy of Sciences（中国科学院）；School of Geographical Sciences（地理科学学院）；Hunan Normal University（湖南师范大学）
Center for Research on Intelligent Perception and Computing（智能感知与计算研究中心）；National Laboratory of Pattern Recognition（模式识别国家重点实验室）；Center for Excellence in Brain Science and Intelligence Technology（脑科学与智能技术卓越创新中心）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Modelling Visual Semantics via Image Captioning to extract Enhanced Multi-Level Cross-Modal Semantic Incongruity Representation with Attention for Multimodal Sarcasm Detection
