Knowledge graphs (KGs) provide reliable external knowledge for a wide variety of AI tasks in the form of structured triples. Knowledge graph pre-training (KGP) aims to pre-train neural networks on large-scale KGs and provide unified interfaces to enhance different downstream tasks, which is a key direction for KG management, maintenance, and applications. Existing works often focus on purely research questions in open domains, or they are not open source due to data security and privacy in real scenarios. Meanwhile, existing studies have not explored the training efficiency and transferability of KGP models in depth. To address these problems, We propose a framework MuDoK to achieve multi-domain collaborative pre-training and efficient prefix prompt tuning to serve diverse downstream tasks like recommendation and text understanding. Our design is a plug-and-play prompt learning approach that can be flexibly adapted to different downstream task backbones. In response to the lack of open-source benchmarks, we constructed a new multi-domain KGP benchmark called KPI with two large-scale KGs and six different sub-domain tasks to evaluate our method and open-sourced it for subsequent research. We evaluated our approach based on constructed KPI benchmarks using diverse backbone models in heterogeneous downstream tasks. The experimental results show that our framework brings significant performance gains, along with its generality, efficiency, and transferability.
Prompt-R1: Collaborative Automatic Prompting Framework via End-to-end Reinforcement Learning
Hainan University(海南大学);Nanyang Technological University(南洋理工大学);Hithink Research(慧思研究);HKUST (Guangzhou)(香港科技大学(广州));IDEA Research(IDEA研究院);Tsinghua University(清华大学);National University of Singapore(新加坡国立大学);Nanjing University of Science and Technology(南京理工大学)
Enhancing Collaborative Filtering Recommender with Prompt-Based Sentiment Analysis
New York University(纽约大学)
协同过滤推荐系统是在线市场和电子商务中的一个重要应用。然而,CF推荐已被证明遭受持续的问题,与稀疏的用户评级,这将进一步导致冷启动问题。现有方法通过应用标记级情感分析来解决数据稀疏问题,该标记级情感分析将文本评论转换为情感分数作为用户评级的补充。在本文中,我们试图优化情感分析与先进的NLP模型,包括BERT和RoBERTa,并实验是否CF推荐已进一步增强。我们在Amazon US Reviews数据集上构建了训练器,并使用传统的微调范式以及新的基于自适应的学习范式来调整预训练的BERT和RoBERTa。实验结果表明,基于RoBERTa预测的情感评分的推荐系统性能最优,与基线推荐系统相比,MAP、NDCG和K点的精确度提高了30.7%.基于情感分析的学习范式虽然在纯情感分析方面优于传统的微调范式,但不能进一步改进CF推荐系统。
Whispers of Many Shores: Cultural Alignment through Collaborative Cultural Expertise
Arizona State University(亚利桑那州立大学);National Taiwan University(台湾国立大学);Carnegie Mellon University(卡内基梅隆大学);New York University(纽约大学);Indian Institute of Technology Hyderabad(印度海得拉巴理工学院);Minitab(Minitab公司);Trinity College Dublin(都柏林圣三一学院)
Unveiling the Lexical Sensitivity of LLMs: Combinatorial Optimization for Prompt Enhancement
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
BayesPrompt: Prompting Large-Scale Pre-Trained Language Models on Few-shot Inference via Debiased Domain Abstraction
University of Chinese Academy of Sciences(中国科学院大学);Tsinghua University(清华大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Aligning the Spectrum: Hybrid Graph Pre-training and Prompt Tuning across Homophily and Heterophily
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Pennsylvania State University(宾夕法尼亚州立大学)
Improving Task Generalization via Unified Schema Prompt
Sun Yat-sen University(中山大学);Chinese University of Hong Kong(香港中文大学);Tsinghua University(清华大学);Langboat Technology(澜舟科技);Microsoft Research Asia(微软亚洲研究院)