VideoQA-SC: Adaptive Semantic Communication for Video Question Answering
State Key Laboratory of Advanced Rail Autonomous Operation（先进轨道交通自主运行全国重点实验室）；School of Electronic and Information Engineering, Beijing Jiaotong University（北京交通大学电子信息工程学院）；Frontiers Science Center for Smart High-speed Railway System（智能高速铁路系统前沿科学中心）；School of Information Engineering, Zhengzhou University（郑州大学信息工程学院）
Jiangyuan Guo, Wei Chen, Yuxuan Sun, Jialong Xu, Bo Ai
Although semantic communication (SC) has shown its potential in efficiently transmitting multimodal data such as texts, speeches and images, SC for videos has focused primarily on pixel-level reconstruction. However, these SC systems may be suboptimal for downstream intelligent tasks. Moreover, SC systems without pixel-level video reconstruction present advantages by achieving higher bandwidth efficiency and real-time performance of various intelligent tasks. The difficulty in such system design lies in the extraction of task-related compact semantic representations and their accurate delivery over noisy channels. In this paper, we propose an end-to-end SC system, named VideoQA-SC for video question answering (VideoQA) tasks. Our goal is to accomplish VideoQA tasks directly based on video semantics over noisy or fading wireless channels, bypassing the need for video reconstruction at the receiver. To this end, we develop a spatiotemporal semantic encoder for effective video semantic extraction, and a learning-based bandwidth-adaptive deep joint source-channel coding (DJSCC) scheme for efficient and robust video semantic transmission. Experiments demonstrate that VideoQA-SC outperforms traditional and advanced DJSCC-based SC systems that rely on video reconstruction at the receiver under a wide range of channel conditions and bandwidth constraints. In particular, when the signal-to-noise ratio is low, VideoQA-SC can improve the answer accuracy by 5.17% while saving almost 99.5\% of the bandwidth at the same time, compared with the advanced DJSCC-based SC system. Our results show the great potential of SC system design for video applications.
Answering from Sure to Uncertain: Uncertainty-Aware Curriculum Learning for Video Question Answering
School of Computing and Information Systems, The University of Melbourne（墨尔本大学计算与信息系统学院）；Computer Science & Software Engineering, University of Western Australia（西澳大学计算机科学与软件工程系）；School of Computing and Communications, Lancaster University（兰卡斯特大学计算与通讯学院）；Department of Data Science & AI, Monash University（莫纳什大学数据科学与人工智能系）
Prompting Video-Language Foundation Models with Domain-specific Fine-grained Heuristics for Video Question Answering
School of Information Science and Technology, Hangzhou Normal University（杭州师范大学信息科学与技术学院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）；School of Computer Science and Technology, Hangzhou Dianzi University（杭州电子科技大学计算机科学与技术学院）
ReasVQA: Advancing VideoQA with Imperfect Reasoning Process
Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机研究所）；National Key Laboratory of General Artificial Intelligence, Peking University（北京大学通用人工智能国家重点实验室）
Multilevel Hierarchical Network with Multiscale Sampling for Video Question Answering
University of Chinese Academy of Sciences（中国科学院大学）；Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences（中国科学院重庆绿色智能技术研究院）；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳人工智能与机器人研究院）；University College London（伦敦大学学院）
UpstreamQA: A Modular Framework for Explicit Reasoning on Video Question Answering Tasks
Lincoln North Star High School（林肯北星高中）；The Charter School of Wilmington（威尔明顿 charter 学校）；Greenwich High School（格林威治高中）；Santa Susana High School（圣塔苏娜高中）；UC Berkeley（伯克利大学）
StreamingCoT: A Dataset for Temporal Dynamics and Multimodal Chain-of-Thought Reasoning in Streaming VideoQA
Henan Institute of Advanced Technology, Zhengzhou University（河南高级技术研究所，郑州大学）；Institute of Automation, CAS（自动化研究所，中国科学院）；UCAS（中国科学院大学）；Peng Cheng Laboratory（鹏城实验室）
End-to-End Video Question Answering with Frame Scoring Mechanisms and Adaptive Sampling
Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机研究所）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；Center for Data Science, Peking University（北京大学数据科学中心）
MarioQA: Answering Questions by Watching Gameplay Videos
POSTECH（浦项科技大学）
我们提出了一个框架来分析模型的各个方面的视频问答（VideoQA）使用可定制的合成数据集，这是自动构建的游戏视频。我们的工作是出于这样一个事实，即现有的模型通常只在需要非常高层次的推理或主要包含通过单帧推理可访问的实例的数据集上进行测试。因此，难以测量训练模型的能力和灵活性，并且现有技术通常依赖于深度神经网络的ad-hoc实现，而没有对数据集和模型的清晰洞察。我们对理解视频事件之间的时间关系以解决VideoQA问题特别感兴趣;这是因为推理时间依赖性是视频中最明显的组成部分之一。为了实现这一目标，我们使用{ \em Super Mario Bros.}游戏视频，以便它包含不同层次的推理复杂性的事件。使用数据集，我们表明，正确构造的数据集与事件在不同的复杂程度是至关重要的学习有效的模型和提高整体性能。
Multi-granularity Contrastive Cross-modal Collaborative Generation for End-to-End Long-term Video Question Answering
School of Information Science and Technology, Hangzhou Normal University（杭州师范大学信息科学与技术学院）；School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）；School of Computer Science and Technology, Hangzhou Dianzi University（杭州电子科技大学计算机科学与技术学院）；Department of Computer Science and Technology, Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）计算机科学与技术系）
Redundancy-aware Transformer for Video Question Answering
National University of Singapore（新加坡国立大学）；University of Science and Technology of China（中国科学技术大学）；Institute of Dataspace, Hefei Comprehensive National Science Center（合肥综合性国家科学中心数据空间研究院）
Temporal Pyramid Transformer with Multimodal Interaction for Video Question Answering
Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences（中国科学院重庆绿色智能技术研究院）；Chongqing School, University of Chinese Academy of Sciences（中国科学院大学重庆学院）；University College London（伦敦大学学院）；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳人工智能与机器人学会）
