Prompting Disentangled Embeddings for Knowledge Graph Completion with Pre-trained Language Model
Powerchina Huadong Engineering Corporation Limited(中国电力建设集团华东勘测设计研究院有限公司);School of Computer Science, Hangzhou Dianzi University(杭州电子科技大学计算机学院);The University of Manchester(曼彻斯特大学);Zhejiang University(浙江大学);The University of Edinburgh(爱丁堡大学)
Yuxia Geng, Jiaoyan Chen, Yuhang Zeng, Zhuo Chen, Wen Zhang, Jeff Z. Pan, Yuxiang Wang, Xiaoliang Xu
备注:
published in Expert Systems With Applications, 2025
发表信息:
Expert Systems With Applications 268 (2025) 126175
Both graph structures and textual information play a critical role in Knowledge Graph Completion (KGC). With the success of Pre-trained Language Models (PLMs) such as BERT, they have been applied for text encoding for KGC. However, the current methods mostly prefer to fine-tune PLMs, leading to huge training costs and limited scalability to larger PLMs. In contrast, we propose to utilize prompts and perform KGC on a frozen PLM with only the prompts trained. Accordingly, we propose a new KGC method named PDKGC with two prompts -- a hard task prompt which is to adapt the KGC task to the PLM pre-training task of token prediction, and a disentangled structure prompt which learns disentangled graph representation so as to enable the PLM to combine more relevant structure knowledge with the text information. With the two prompts, PDKGC builds a textual predictor and a structural predictor, respectively, and their combination leads to more comprehensive entity prediction. Solid evaluation on three widely used KGC datasets has shown that PDKGC often outperforms the baselines including the state-of-the-art, and its components are all effective. Our codes and data are available at https://github.com/genggengcss/PDKGC.
Rubric-Aligned Disentangled Evaluation of Human Simultaneous Interpreting
School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)数据科学学院);School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)人工智能学院);Shenzhen Loop Area Institute(深圳河套学院)
$k$NN Prompting: Beyond-Context Learning with Calibration-Free Nearest Neighbor Inference
University of Science and Technology of China(中国科学技术大学);Beijing University of Posts and Telecommunications(北京邮电大学);Baidu Inc.(百度公司);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥综合性国家科学中心人工智能研究院)
Bidirectional Language Models Are Also Few-shot Learners
University of Pennsylvania(宾夕法尼亚大学);Microsoft(微软);Google Research(谷歌研究院);UNC Chapel Hill(北卡罗来纳大学教堂山分校)
大语言模型,例如GPT-3(Brown等人,2020)可以执行任意任务,而无需在仅使用几个标记示例进行提示后进行微调。一个任意的任务可以被重新表述为一个自然语言提示,一个语言模型可以被要求生成完成,间接地执行任务的范式称为基于语义的学习。到目前为止,基于突现的学习能力主要是针对单向语言模型来证明的。然而,在去噪目标上预训练的双向语言模型(例如掩蔽语言建模)为迁移学习产生更强的学习表示。这激发了促进双向模式的可能性,但它们的预培训目标使它们在很大程度上与现有的激励范式不相容。我们提出了SAP(顺序自回归拟合),一种技术,使双向模型的提示。利用机器翻译任务作为案例研究,我们提出了双向mT5模型(Xue et al.,2021)与SAP进行比较,并证明其Few-Shot和zero-shot转换优于单向模型(如GPT-3和XGLM)的Few-Shot转换(Lin et al.,2021),尽管mT5的参数减少了约50%。我们进一步表明SAP在问答和摘要方面是有效的。这是第一次,我们的研究结果表明,基于语义的学习是一个更广泛的一类语言模型,而不仅仅是单向模型的新兴属性。
FoodTaxo: Generating Food Taxonomies with Large Language Models
ADAPT Centre, School of Computing, Dublin City University(ADAPT中心、计算学院、都柏林城市大学);Lucerne School of Computer Science and Information Technology (HSLU)(卢塞恩计算机科学与信息科技学院(HSLU))
Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models
Bonn-Aachen International Center for Information Technology (b-it)(波恩-埃森国际信息科技中心);University of Bonn(波恩大学);Lamarr Institute for Machine Learning and Artificial Intelligence(拉马尔机器学习与人工智能研究所);Research Center Trustworthy Data Science and Security (RC-Trust)(可信数据科学与安全研究中心);University of Duisburg-Essen(埃森-杜伊斯堡大学)
本研究通过扰动任务和链式推理提示探讨大语言模型的理论 of mind能力,发现其鲁棒性受扰动影响显著,CoT提示虽提升整体表现,但对某些扰动类别却降低准确性。
llmNER: (Zero|Few)-Shot Named Entity Recognition, Exploiting the Power of Large Language Models
University of Chile(智利大学);Millennium Institute for Foundational Research on Data(数据基础研究千年研究所);Catholic University of Chile(智利天主教大学);Faculty of Physical and Mathematical Sciences, University of Chile(智利大学物理与数学学院)
本文提出 llmNER 库,通过简化提示构建与结果解析,支持基于大型语言模型的零样本和少样本命名实体识别,并在两个 NER 任务上验证其灵活性与有效性。
Large Language Models as Code Executors: An Exploratory Study
Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Xinjiang University(新疆大学);University of Melbourne(墨尔本大学);Tsinghua University(清华大学);Shanghai Jiao Tong University(上海交通大学);IBM Research(IBM研究院);Nantong University(南通大学);Alibaba(阿里巴巴)
Bridging Perception and Language: A Systematic Benchmark for LVLMs' Understanding of Amodal Completion Reports
The University of Tokyo(东京大学);Keio University(庆应大学);NICT(国家信息与通信技术研究所);Osaka University(大阪大学);Hokkaido University(北海道大学);CiNET(信息网络研究中心);Kobe University(Kobe 大学)
Towards a Mathematics Formalisation Assistant using Large Language Models
Microsoft Research(微软研究院);Indian Institute of Science(印度科学学院);London School of Geometry and Number Theory(伦敦几何与数论学院);Indian Institute of Science Education and Research(印度科学教育与研究院)
Bridging Auxiliary Constraints to Resolve Instruction Following in Large Reasoning Models
The Chinese University of Hong Kong(香港中文大学);University of International Relations(国际关系大学);Tencent Jarvis Lab(腾讯Jarvis实验室);Westlake University(西湖大学);King’s College London(伦敦国王学院)
Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in Large Language Models
National University of Singapore(新加坡国立大学);Nanyang Technological University(南洋理工大学);University College London(伦敦大学学院);Yale University(耶鲁大学);The Hong Kong University of Science and Technology(香港科技大学);University of Washington(华盛顿大学)
提出Uncertainty of Thoughts (UoT)算法,通过不确定性感知模拟、基于不确定性的奖励和奖励传播机制,增强大型语言模型主动提问寻求信息的能力,在医疗诊断等任务中平均成功率提升38.1%。