Understanding visual relationships involves identifying the subject, the object, and a predicate relating them. We leverage the strong correlations between the predicate and the (subj,obj) pair (both semantically and spatially) to predict the predicates conditioned on the subjects and the objects. Modeling the three entities jointly more accurately reflects their relationships, but complicates learning since the semantic space of visual relationships is huge and the training data is limited, especially for the long-tail relationships that have few instances. To overcome this, we use knowledge of linguistic statistics to regularize visual model learning. We obtain linguistic knowledge by mining from both training annotations (internal knowledge) and publicly available text, e.g., Wikipedia (external knowledge), computing the conditional probability distribution of a predicate given a (subj,obj) pair. Then, we distill the knowledge into a deep model to achieve better generalization. Our experimental results on the Visual Relationship Detection (VRD) and Visual Genome datasets suggest that with this linguistic knowledge distillation, our model outperforms the state-of-the-art methods significantly, especially when predicting unseen relationships (e.g., recall improved from 8.45% to 19.17% on VRD zero-shot testing set).
Thinking Without Images: Internalizing Visual Manipulation with On-Policy Self-Distillation
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（北京大学计算机学院多媒体信息处理国家重点实验室）；Central South University（中南大学）；University of Science and Technology of China（中国科学技术大学）；Peking University（北京大学）；Huawei Technologies（华为技术有限公司）
CL-HOI: Cross-Level Human-Object Interaction Distillation from Vision Large Language Models
School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）；Schaeffler Hub for Advanced Research at NTU（舍弗勒-南洋理工大学高级研究中心）
KD-VLP: Improving End-to-End Vision-and-Language Pretraining with Object Knowledge Distillation
Intel Labs（英特尔实验室）；Microsoft Research Asia（微软亚洲研究院）；Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences（中国科学院上海微系统与信息技术研究所）
Topic Scene Graph Generation by Attention Distillation from Caption
Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS)（中国科学院智能信息处理重点实验室）；Institute of Computing Technology, CAS（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Beijing Academy of Artificial Intelligence（北京智源人工智能研究院）
Overcoming Language Priors for Visual Question Answering Based on Knowledge Distillation
School of Computer Science and Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）；Joint Laboratory of HUST and Pingan Property & Casualty Research (HPL)（华中科技大学与平安财产保险研究联合实验室）
VAD: Attributing Visual Evidence for Target Reconstruction in Multimodal On-Policy Distillation
Shanghai Jiao Tong University（上海交通大学）；Xiaohongshu Inc.（小红书公司）；The Chinese University of Hong Kong（香港中文大学）；Zhejiang University（浙江大学）；Southeast University（东南大学）
GenEvolve: Self-Evolving Image Generation Agents via Tool-Orchestrated Visual Experience Distillation
The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；Meituan（美团）；The Hong Kong University of Science and Technology（香港科学与技术大学）；National University of Singapore（新加坡国立大学）
