Training an effective video-and-language model intuitively requires multiple frames as model inputs. However, it is unclear whether using multiple frames is beneficial to downstream tasks, and if yes, whether the performance gain is worth the drastically-increased computation and memory costs resulting from using more frames. In this work, we explore single-frame models for video-and-language learning. On a diverse set of video-and-language tasks (including text-to-video retrieval and video question answering), we show the surprising result that, with large-scale pre-training and a proper frame ensemble strategy at inference time, a single-frame trained model that does not consider temporal information can achieve better performance than existing methods that use multiple frames for training. This result reveals the existence of a strong "static appearance bias" in popular video-and-language datasets. Therefore, to allow for a more comprehensive evaluation of video-and-language models, we propose two new retrieval tasks based on existing fine-grained action recognition datasets that encourage temporal modeling. Our code is available at https://github.com/jayleicn/singularity
Revealing the Gap in Human and VLM Scene Description through Counterfactual Semantic Saliency
Department of Computer Science, University of California, Santa Barbara（加州大学圣巴巴拉分校计算机科学系）；Department of Psychological and Brain Sciences, University of California, Santa Barbara（加州大学圣巴巴拉分校心理学与脑科学系）
Seeing the Goal, Missing the Truth: Human Accountability for AI Bias
University of Maryland（马里兰大学）；Robert H. Smith School of Business, University of Maryland, College Park（马里兰大学罗伯特·H·史密斯商学院）；Emory University（埃默里大学）；Lancaster University（兰卡斯特大学）
Benchmarking Gender and Political Bias in Large Language Models
School of Computing & Information Systems, The University of Melbourne（墨尔本大学计算与信息系统学院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Split Personality Training: Revealing Latent Knowledge Through Alternate Personalities
University of Bergen, Bergen, Norway（卑尔根大学）；Independent Researcher, London, UK（独立研究者）；University College London, London, UK（伦敦大学学院）；Independent Researcher, Berlin, Germany（独立研究者）；Independent Researcher（独立研究者）；Head of Research（研发负责人）
When Agents See Humans as the Outgroup: Belief-Dependent Bias in LLM-Powered Agents
Chongqing University（重庆大学）；The University of Queensland（昆士兰大学）；Virginia Polytechnic Institute and State University（弗吉尼亚理工大学）；The University of Manchester（曼彻斯特大学）
Fraunhofer Heinrich-Hertz-Institute（弗劳恩霍夫海因里希·赫兹研究所）；Technische Universität Berlin（柏林工业大学）；Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）
Applying traditional post-hoc attribution methods to segmentation or object detection predictors offers only limited insights, as the obtained feature attribution maps at input level typically resemble the models' predicted segmentation mask or bounding box. In this work, we address the need for more informative explanations for these predictors by proposing the post-hoc eXplainable Artificial Intelligence method L-CRP to generate explanations that automatically identify and visualize relevant concepts learned, recognized and used by the model during inference as well as precisely locate them …
Revealing Model Biases: Assessing Deep Neural Networks via Recovered Sample Analysis
University of Tehran（德黑兰大学）；University of Bremen（不来梅大学）；Okinawa Institute of Science and Technology (OIST)（冲绳科学技术大学院大学）；Institute For Research In Fundamental Sciences (IPM)（基础科学研究所）
The Chinese University of Hong Kong（香港中文大学）；University of Toronto（多伦多大学）；Megvii Inc（旷视科技有限公司）
Previous CNN-based video super-resolution approaches need to align multiple frames to the reference. In this paper, we show that proper frame alignment and motion compensation is crucial for achieving high quality results. We accordingly propose a `sub-pixel motion compensation' (SPMC) layer in a CNN framework. Analysis and experiments show the suitability of this layer in video SR. The final end-to-end, scalable CNN framework effectively incorporates the SPMC layer and fuses multiple frames to reveal image details. Our implementation can generate visually and quantitatively high-quality resul…
We consider the problem of temporal view synthesis, where the goal is to predict a future video frame from the past frames using knowledge of the depth and relative camera motion. In contrast to revealing the disoccluded regions through intensity based infilling, we study the idea of an infilling vector to infill by pointing to a non-disoccluded region in the synthesized view. To exploit the structure of disocclusions created by camera motion during their infilling, we rely on two important cues, temporal correlation of infilling directions and depth. We design a learning framework to predict …
