This paper aims to analyze knowledge consistency between pre-trained deep neural networks. We propose a generic definition for knowledge consistency between neural networks at different fuzziness levels. A task-agnostic method is designed to disentangle feature components, which represent the consistent knowledge, from raw intermediate-layer features of each neural network. As a generic tool, our method can be broadly used for different applications. In preliminary experiments, we have used knowledge consistency as a tool to diagnose representations of neural networks. Knowledge consistency provides new insights to explain the success of existing deep-learning techniques, such as knowledge distillation and network compression. More crucially, knowledge consistency can also be used to refine pre-trained networks and boost performance.
Given a natural language query, a phrase grounding system aims to localize mentioned objects in an image. In weakly supervised scenario, mapping between image regions (i.e., proposals) and language is not available in the training set. Previous methods address this deficiency by training a grounding system via learning to reconstruct language information contained in input queries from predicted proposals. However, the optimization is solely guided by the reconstruction loss from the language modality, and ignores rich visual information contained in proposals and useful cues from external kno…
Yonsei University（延世大学）；Seoul National University（首尔大学）
Active learning can be defined as iterations of data labeling, model training, and data acquisition, until sufficient labels are acquired. A traditional view of data acquisition is that, through iterations, knowledge from human labels and models is implicitly distilled to monotonically increase the accuracy and label consistency. Under this assumption, the most recently trained model is a good surrogate for the current labeled data, from which data acquisition is requested based on uncertainty/diversity. Our contribution is debunking this myth and proposing a new objective for distillation. Fi…
Beyond Geometry: Benchmarking and Consistency Reasoning for 3D Logical Anomaly Detection
School of Artificial Intelligence and Robotics, Hunan University（湖南大学人工智能与机器人学院）；National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University（湖南大学机器人视觉感知与控制技术国家工程研究中心）
Human beings learn and accumulate hierarchical knowledge over their lifetime. This knowledge is associated with previous concepts for consolidation and hierarchical construction. However, current incremental learning methods lack the ability to build a concept hierarchy by associating new concepts to old ones. A more realistic setting tackling this problem is referred to as Incremental Implicitly-Refined Classification (IIRC), which simulates the recognition process from coarse-grained categories to fine-grained categories. To overcome forgetting in this benchmark, we propose Hierarchy-Consist…
Knowledge distillation has been used to transfer knowledge learned by a sophisticated model (teacher) to a simpler model (student). This technique is widely used to compress model complexity. However, in most applications the compressed student model suffers from an accuracy gap with its teacher. We propose extracurricular learning, a novel knowledge distillation method, that bridges this gap by (1) modeling student and teacher output distributions; (2) sampling examples from an approximation to the underlying data distribution; and (3) matching student and teacher output distributions over th…
Harbin Institute of Technology（哈尔滨工业大学）；Nanjing University of Science and Technology（南京理工大学）；Wuyi University（五邑大学）；Singapore Management University（新加坡管理大学）
Adaptive Semantic Consistency for Cross-domain Few-shot Classification
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）；School of Electronic Information and Communications, Huazhong University of Science and Technology（华中科技大学电子信息与通信学院）
Riiid! AI Research；Graduate School of AI, KAIST（韩国科学技术院人工智能研究生院）
Knowledge Tracing (KT), tracking a human's knowledge acquisition, is a central component in online learning and AI in Education. In this paper, we present a simple, yet effective strategy to improve the generalization ability of KT models: we propose three types of novel data augmentation, coined replacement, insertion, and deletion, along with corresponding regularization losses that impose certain consistency or monotonicity biases on the model's predictions for the original and augmented sequence. Extensive experiments on various KT benchmarks show that our regularization scheme consistentl…
Beyond Instance Consistency: Investigating View Diversity in Self-supervised Learning
Institute for Infocomm Research (I 2 R), A*STAR, Singapore（新加坡*STAR信息与通信研究所）；National University of Singapore, Singapore（新加坡国立大学）；Sichuan University, China（四川大学）；Southeast University, China（东南大学）；Shanghai Jiaotong University, China（上海交通大学）
University of Tübingen（图宾根大学）；IMPRS-IS（智能系统国际马克斯·普朗克研究学院）
A central problem in cognitive science and behavioural neuroscience as well as in machine learning and artificial intelligence research is to ascertain whether two or more decision makers (be they brains or algorithms) use the same strategy. Accuracy alone cannot distinguish between strategies: two systems may achieve similar accuracy with very different strategies. The need to differentiate beyond accuracy is particularly pressing if two systems are near ceiling performance, like Convolutional Neural Networks (CNNs) and humans on visual object recognition. Here we introduce trial-by-trial err…
Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models
Center for Language and Cognition, University of Groningen（格罗宁根大学语言与认知中心）；Institute for Logic, Language and Computation, University of Amsterdam（阿姆斯特丹大学逻辑、语言与计算研究所）
CheXPO-v2: Preference Optimization for Chest X-ray VLMs with Knowledge Graph Consistency
School of Computer Science and Technology, Xidian University（西安电子科技大学计算机科学与技术学院）；Warren Alpert Medical School of Brown University（布朗大学沃伦·阿尔珀特医学学院）；School of Biomedical Engineering, Capital Medical University（首都医科大学生物医学工程学院）
Learning the differential statistical dependency network between two contexts is essential for many real-life applications, mostly in the high dimensional low sample regime. In this paper, we propose a novel differential network estimator that allows integrating various sources of knowledge beyond data samples. The proposed estimator is scalable to a large number of variables and achieves a sharp asymptotic convergence rate. Empirical experiments on extensive simulated data and four real-world applications (one on neuroimaging and three from functional genomics) show that our approach achieves…
University of British Columbia（英属哥伦比亚大学）；Element AI
Knowledge graphs store facts using relations between two entities. In this work, we address the question of link prediction in knowledge hypergraphs where relations are defined on any number of entities. While techniques exist (such as reification) that convert non-binary relations into binary ones, we show that current embedding-based methods for knowledge graph completion do not work well out of the box for knowledge graphs obtained through these techniques. To overcome this, we introduce HSimplE and HypE, two embedding-based methods that work directly with knowledge hypergraphs. In both mod…
Knowledge tracing refers to a family of methods that estimate each student's knowledge component/skill mastery level from their past responses to questions. One key limitation of most existing knowledge tracing methods is that they can only estimate an \emph{overall} knowledge level of a student per knowledge component/skill since they analyze only the (usually binary-valued) correctness of student responses. Therefore, it is hard to use them to diagnose specific student errors. In this paper, we extend existing knowledge tracing methods beyond correctness prediction to the task of predicting …
To Know or Not To Know? Analyzing Self-Consistency of Large Language Models under Ambiguity
LMU Munich（慕尼黑大学）；Faculty of Philological and Cultural Studies, University of Vienna（维也纳大学语言与文化学院）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；UniVie Doctoral School Computer Science（维也纳大学计算机博士学院）
SEAM: Global consistency beyond local accuracy in scientific machine learning
Axiom Research Group（阿克西姆研究集团）；The Nelson Mandela African Institution of Science and Technology（纳尔逊·曼德拉非洲科学技术研究院）；African Institute for Mathematical Sciences（非洲数学科学研究所）；The University of Tokyo（东京大学）
Beihang University（北京航空航天大学）；University of Ottawa（渥太华大学）
The models developed to date for knowledge base embedding are all based on the assumption that the relations contained in knowledge bases are binary. For the training and testing of these embedding models, multi-fold (or n-ary) relational data are converted to triples (e.g., in FB15K dataset) and interpreted as instances of binary relations. This paper presents a canonical representation of knowledge bases containing multi-fold relations. We show that the existing embedding models on the popular FB15K datasets correspond to a sub-optimal modelling framework, resulting in a loss of structural i…
