LightHGNN：将超图神经网络蒸馏为 MLP 以实现 $100\ imes$ 更快的推理
LightHGNN: Distilling Hypergraph Neural Networks into MLPs for $100\times$ Faster Inference
AI总结：
本文提出 LightHGNN 和 LightHGNN$^+$，通过知识蒸馏将超图神经网络转化为 MLP，消除高阶结构依赖，在保持性能的同时实现百倍推理加速。
摘要：
超图神经网络（HGNNs）最近引起了广泛关注，并因其在高阶相关性建模方面的优势而表现出令人满意的性能。然而，人们注意到超图的高阶建模能力也带来了增加的计算复杂性，这阻碍了其在工业中的实际部署。在实践中，我们发现高效部署 HGNNs 的一个关键障碍是推理过程中的高阶结构依赖性。在本文中，我们提出弥合 HGNNs 与推理高效的多层感知机（MLPs）之间的差距，以消除 HGNNs 的超图依赖性，从而降低计算复杂性并提高推理速度。具体而言，我们引入了 LightHGNN 和 LightHGNN$^+$ 以实现低复杂度的快速推理。LightHGNN 通过软标签直接将知识从教师 HGNNs 蒸馏到学生 MLPs，而 LightHGNN$^+$ 则进一步将可靠的高阶相关性显式注入学生 MLPs，以实现拓扑感知蒸馏并抵抗过度平滑。在八个超图数据集上的实验表明，即使没有超图依赖性，所提出的 LightHGNNs 仍能取得与 HGNNs 相当甚至更好的性能，并且平均比普通 MLPs 高出 $16.3$。在三个图数据集上的广泛实验进一步表明，与所有其他方法相比，我们的 LightHGNNs 具有平均最佳性能。在具有 5.5w 个顶点的合成超图上的实验表明，LightHGNNs 的运行速度比 HGNNs 快 $100\ imes$，展示了其在延迟敏感部署中的能力。
英文摘要：
Hypergraph Neural Networks (HGNNs) have recently attracted much attention and exhibited satisfactory performance due to their superiority in high-order correlation modeling. However, it is noticed that the high-order modeling capability of hypergraph also brings increased computation complexity, which hinders its practical industrial deployment. In practice, we find that one key barrier to the efficient deployment of HGNNs is the high-order structural dependencies during inference. In this paper, we propose to bridge the gap between the HGNNs and inference-efficient Multi-Layer Perceptron (MLPs) to eliminate the hypergraph dependency of HGNNs and thus reduce computational complexity as well as improve inference speed. Specifically, we introduce LightHGNN and LightHGNN$^+$ for fast inference with low complexity. LightHGNN directly distills the knowledge from teacher HGNNs to student MLPs via soft labels, and LightHGNN$^+$ further explicitly injects reliable high-order correlations into the student MLPs to achieve topology-aware distillation and resistance to over-smoothing. Experiments on eight hypergraph datasets demonstrate that even without hypergraph dependency, the proposed LightHGNNs can still achieve competitive or even better performance than HGNNs and outperform vanilla MLPs by $16.3$ on average. Extensive experiments on three graph datasets further show the average best performance of our LightHGNNs compared with all other methods. Experiments on synthetic hypergraphs with 5.5w vertices indicate LightHGNNs can run $100\times$ faster than HGNNs, showcasing their ability for latency-sensitive deployments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
