HyperBERT：融合超图感知层与语言模型用于文本属性超图上的节点分类
HyperBERT: Mixing Hypergraph-Aware Layers with Language Models for Node Classification on Text-Attributed Hypergraphs
AI总结：
提出HyperBERT架构，通过在预训练BERT中融合超图感知层引入高阶结构归纳偏置，同时建模超图关系结构与文本语义，在五个文本属性超图节点分类基准上达到最优。
摘要：
超图以复杂的拓扑结构为特征，通过超边表示多个实体间的高阶交互。近期，针对文本属性超图节点分类问题，利用基于超图的深度学习方法学习信息丰富的数据表示，已引起越来越多的研究关注。然而，现有方法难以同时捕获超图结构信息的全貌和节点属性中固有的丰富语言属性，这极大阻碍了其有效性与泛化能力。为克服这些挑战，我们探索通过专门的超图感知层进一步增强预训练BERT模型，以用于节点分类任务。这些层将高阶结构归纳偏置引入语言模型，从而提升模型同时利用超图结构高阶上下文信息与文本语义信息的能力。本文提出新架构HyperBERT，这是一种混合文本-超图模型，能同时建模超图关系结构并保持预训练BERT的高质量文本编码能力。值得注意的是，HyperBERT在五个具有挑战性的文本属性超图节点分类基准上取得了新的最优结果。
英文摘要：
Hypergraphs are characterized by complex topological structure, representing higher-order interactions among multiple entities through hyperedges. Lately, hypergraph-based deep learning methods to learn informative data representations for the problem of node classification on text-attributed hypergraphs have garnered increasing research attention. However, existing methods struggle to simultaneously capture the full extent of hypergraph structural information and the rich linguistic attributes inherent in the nodes attributes, which largely hampers their effectiveness and generalizability. To overcome these challenges, we explore ways to further augment a pretrained BERT model with specialized hypergraph-aware layers for the task of node classification. Such layers introduce higher-order structural inductive bias into the language model, thus improving the model's capacity to harness both higher-order context information from the hypergraph structure and semantic information present in text. In this paper, we propose a new architecture, HyperBERT, a mixed text-hypergraph model which simultaneously models hypergraph relational structure while maintaining the high-quality text encoding capabilities of a pre-trained BERT. Notably, HyperBERT presents results that achieve a new state-of-the-art on five challenging text-attributed hypergraph node classification benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
