知识图谱嵌入(KGE)是一项基础性任务,旨在学习知识图谱(KG)中实体和关系的表示,目标是构建足够全面的表示,以逼近实体之间的逻辑和符号互连。本文中,我们通过理论分析定义了一个度量 Z-counts,用于衡量 KG 中每个三元组(<头实体,关系,尾实体>)的训练难度。基于该度量,我们提出 CL4KGE,这是一种高效的、基于课程学习(Curriculum Learning)的 KGE 训练策略。该方法包含一个难度度量器和一个训练调度器,用于辅助 KGE 模型的训练。我们的方法具有灵活性,可作为插件用于多种 KGE 模型,并且具有适应大多数现有 KG 的额外优势。所提出的方法已在流行的 KGE 模型上进行评估,结果表明其能够增强最先进方法的性能。将 Z-counts 用作度量使得识别 KG 中具有挑战性的三元组成为可能,这有助于设计有效的训练策略。
英文摘要:
Knowledge graph embedding (KGE) constitutes a foundational task, directed towards learning representations for entities and relations within knowledge graphs (KGs), with the objective of crafting representations comprehensive enough to approximate the logical and symbolic interconnections among entities. In this paper, we define a metric Z-counts to measure the difficulty of training each triple ($<$head entity, relation, tail entity$>$) in KGs with theoretical analysis. Based on this metric, we propose \textbf{CL4KGE}, an efficient \textbf{C}urriculum \textbf{L}earning based training strategy for \textbf{KGE}. This method includes a difficulty measurer and a training scheduler that aids in the training of KGE models. Our approach possesses the flexibility to act as a plugin within a wide range of KGE models, with the added advantage of adaptability to the majority of KGs in existence. The proposed method has been evaluated on popular KGE models, and the results demonstrate that it enhances the state-of-the-art methods. The use of Z-counts as a metric has enabled the identification of challenging triples in KGs, which helps in devising effective training strategies.
Extensible Multi-Granularity Fusion Network and Transferable Curriculum Learning for Aspect-based Sentiment Analysis
School of Software Technology, Dalian University of Technology(大连理工大学软件学院);Graduate School of Information, Production and Systems, Waseda University(早稻田大学信息生产与系统研究生学校)
STEP: Stepwise Curriculum Learning for Context-Knowledge Fusion in Conversational Recommendation
School of Artificial Intelligence Shenzhen University(人工智能学院深圳大学);Beijing University of Posts and Telecommunications(北京邮电大学);Jinan University(暨南大学);Central South University(中南大学)
LLM-Assisted Knowledge Graph Completion for Curriculum and Domain Modelling in Personalized Higher Education Recommendations
University of Siegen(锡根大学);Institute for Knowledge-Based Systems and Knowledge Management(知识与知识管理研究所);Institute for Embedded Systems(嵌入式系统研究所);Department of Digital Health Sciences and Biomedicine(数字健康科学与生物医学系)
Adversarial Curriculum Graph-Free Knowledge Distillation for Graph Neural Networks
University of Electronic Science and Technology of China(电子科技大学);University of California San Diego(加利福尼亚大学圣迭戈分校);Westlake University(西湖大学);Wuhan University(武汉大学)
Graph Neural Network with Curriculum Learning for Imbalanced Node Classification
Aerospace Information Research Institute, Chinese Academy Of Sciences(中国科学院空天信息研究院);School of Software, Tsinghua University(清华大学软件学院);National University of Singapore(新加坡国立大学)
Curriculum-Enhanced Residual Soft An-Isotropic Normalization for Over-smoothness in Deep GNNs
Fuzhou University(福州大学);HKUST (Guangzhou)(香港科技大学(广州));Shandong Provincial Key Laboratory of Computer Networks(山东省计算机网络重点实验室);Shandong Fundamental Research Center for Computer Science(山东计算机科学基础研究中心)
A Knowledge-Injected Curriculum Pretraining Framework for Question Answering
University of Science and Technology of China(中国科学技术大学);Guangdong OPPO Mobile Telecommunications Corp., Ltd(广东欧珀移动通信有限公司);State Key Laboratory of Cognitive Intelligence(认知智能国家重点实验室)
An Effective Incorporating Heterogeneous Knowledge Curriculum Learning for Sequence Labeling
The Hong Kong Polytechnic University(香港理工大学);Department of Information Management, Peking University(北京大学信息管理系);School of Foreign Languages, Peking University(北京大学外语学院)
From Atoms to Chains: Divergence-Guided Reasoning Curriculum for Unlabeled LLM Domain Adaptation
Beijing Key Laboratory of Intelligent Information Technology, School of Computer Science and Technology, Beijing Institute of Technology(北京智能信息科技重点实验室,计算机科学与技术学院,北京理工大学);ByteDance China(字节跳动中国)
TeachMateGPT: A Multi-Agent Knowledge-Grounded Framework for Pedagogical Assessment Generation from Science Curriculum Materials
Ahsanullah University of Science and Technology(阿赫桑乌拉科技大学);American International University - Bangladesh(孟加拉国美国国际大学);Jashore University of Science and Technology(杰索尔科技大学)
Self-Guided Curriculum Learning for Neural Machine Translation
Nagoya University(名古屋大学);The University of Sydney(悉尼大学);Johns Hopkins University(约翰斯·霍普金斯大学);Carnegie Mellon University(卡内基梅隆大学)
在机器学习领域,假设经过良好训练的模型能够恢复训练标签,即模型预测的合成标签应该尽可能接近地面真实标签。受此启发,我们提出了一个自我指导的课程策略,以鼓励神经机器翻译(NMT)模型的学习遵循上述恢复标准,其中我们投每个训练样本的恢复程度作为其学习难度。具体来说,我们采用句子级别的BLEU分数作为恢复程度的代理。与现有的课程依赖于语言先验知识或第三方语言模型不同,我们选择的学习难度更适合衡量NMT模型的知识掌握程度。在WMT 14 English $\Rightarrow$ German和WMT 17 Chinese $\Rightarrow$ English两个翻译基准测试上的实验表明,该方法可以在强基线Transformer上持续提高翻译性能。
TSCL:Multi-party loss Balancing scheme for deep learning Image steganography based on Curriculum learning
Qianan College, North China University of Science and Technology(华北科技大学千安学院);School of Cyberspace Security, Beijing University of Posts and Telecommunications(北京邮电大学网络安全学院);College of Science, North China University of Science and Technology(华北科技大学理学院)
Enhancing Spatio-temporal Quantile Forecasting with Curriculum Learning: Lessons Learned
University of New South Wales(新南威尔士大学);HKGAI(香港人工智能学会);Southern University of Science and Technology(南方科技大学);The University of Tokyo(东京大学);School of Artificial Intelligence of Jilin University(吉林大学人工智能学院)