Graph Neural Networks (GNNs) have shown satisfying performance on various graph learning tasks. To achieve better fitting capability, most GNNs are with a large number of parameters, which makes these GNNs computationally expensive. Therefore, it is difficult to deploy them onto edge devices with scarce computational resources, e.g., mobile phones and wearable smart devices. Knowledge Distillation (KD) is a common solution to compress GNNs, where a light-weighted model (i.e., the student model) is encouraged to mimic the behavior of a computationally expensive GNN (i.e., the teacher GNN model). Nevertheless, most existing GNN-based KD methods lack fairness consideration. As a consequence, the student model usually inherits and even exaggerates the bias from the teacher GNN. To handle such a problem, we take initial steps towards fair knowledge distillation for GNNs. Specifically, we first formulate a novel problem of fair knowledge distillation for GNN-based teacher-student frameworks. Then we propose a principled framework named RELIANT to mitigate the bias exhibited by the student model. Notably, the design of RELIANT is decoupled from any specific teacher and student model structures, and thus can be easily adapted to various GNN-based KD frameworks. We perform extensive experiments on multiple real-world datasets, which corroborates that RELIANT achieves less biased GNN knowledge distillation while maintaining high prediction utility.
Toward Fair Graph Neural Networks Via Dual-Teacher Knowledge Distillation
School of Computer Science（计算机科学学院）；Guangxi Key Lab of MSMI（广西 MSMI 重点实验室）；MOE Key Lab of EBIT, Guangxi Normal University, Guilin, 541004, Guangxi, China（教育部 EBIT 重点实验室，广西师范大学，桂林，541004，广西，中国）；School of Computer Science and Technology（计算机科学与技术学院）；Hainan University（海南大学）；China University of Mining and Technology（中国矿业大学）
Model-to-Data Distillation for Graph Neural Networks
Department of Computer Science, Rice University, Houston, TX 77005, USA（计算机科学系，里士大学，休斯顿，德克萨斯州，77005，美国）；Department of Computer Science & Ken Kennedy Institute, Rice University, Houston, TX 77005, USA（计算机科学系及肯尼迪学院，里士大学，休斯顿，德克萨斯州，77005，美国）
College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；Key Laboratory of Advanced Design and Intelligent Computing, Dalian University（大连大学先进设计与智能计算重点实验室）；School of Computing Technologies, RMIT University（RMIT大学计算技术学院）；Guangdong Institute of Smart Education, Jinan University（暨南大学广东智慧教育研究院）
Debias the Black-box: A Fair Ranking Framework via Knowledge Distillation
Ping An Technology (Shenzhen) Co., Ltd.（平安科技（深圳）有限公司）；University of Science and Technology of China（中国科学技术大学）；Shanghai International Studies University（上海外国语大学）；Peking University（北京大学）
Graph-based Knowledge Distillation: A survey and experimental evaluation
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Zhejiang University（浙江大学）；China University of Geosciences (Beijing)（中国地质大学（北京））
Mitigating Gender Bias in Distilled Language Models via Counterfactual Role Reversal
University of Southern California（南加州大学）；Information Sciences Institute（信息科学研究所）；Amazon Alexa（亚马逊Alexa）；Harvard University（哈佛大学）；University of California, Los Angeles（加州大学洛杉矶分校）
Towards Collaborative Fairness in Federated Learning Under Imbalanced Covariate Shift
The Pennsylvania State University（宾夕法尼亚州立大学）；State University of New York at Albany（纽约州立大学阿尔巴尼分校）；University of Minnesota Twin Cities（明尼苏达大学双城分校）；Dalian University of Technology（大连理工大学）
Beyond Static Knowledge Messengers: Towards Adaptive, Fair, and Scalable Federated Learning for Medical AI
Department of Computer Science and Software Engineering, Auburn University（计算机科学与软件工程系，阿伯丁大学）；Department of Information and Communication Technology, Bangladesh University of Professionals（信息与通信技术系，孟加拉国专业大学）；Mymensingh Medical College and Hospital（迈明辛医疗学院和医院）
Nix-TTS: Lightweight and End-to-End Text-to-Speech via Module-wise Distillation
Amazon（亚马逊）
几种轻量级TTS的解决方案已经显示出有希望的结果。尽管如此，它们要么依赖于手工设计，达到非最佳尺寸，要么使用神经架构搜索，但通常会付出培训成本。我们提出了Nix-TTS，一个轻量级的TTS实现通过知识蒸馏到一个高质量的，但大尺寸的，非自回归，和端到端（无声码器）的TTS教师模型。具体来说，我们提供模块式蒸馏，使编码器和解码器模块能够灵活和独立地蒸馏。由此产生的Nix-TTS继承了教师的非自回归和端到端的有利特性，但尺寸明显更小，只有5.23M参数或高达89.34%的教师模型减少;它还在英特尔上实现了超过3.04倍和8.36倍的推理加速，i7 CPU和Raspberry Pi 3B分别，与教师模型相比，仍然保留了公平的声音自然度和可理解性。我们提供Nix-TTS的预训练模型和音频样本。
Partial Knowledge Distillation for Alleviating the Inherent Inter-Class Discrepancy in Federated Learning
College of Informatics Huazhong Agricultural University（华中农业大学信息学院）；Department of Electronic and Computer Engineering The Hong Kong University of Science and Technology（香港科技大学电子与计算机工程系）；NVIDIA Corporation（NVIDIA公司）
Guangdong Provincial Key Laboratory of Interdisciplinary Research and Application for Data Science（广东省数据科学交叉研究与应用重点实验室）；BNU-HKBU United International College（北京师范大学-香港浸会大学联合国际学院）；Faculty of Science, Hong Kong Baptist University（香港浸会大学理学院）；Trinity College Dublin（都柏林三一学院）
Adversarial-Based Knowledge Distillation for Multi-Model Ensemble and Noisy Data Refinement
Carnegie Mellon University（卡内基梅隆大学）；University of California San Diego（加利福尼亚大学圣迭戈分校）；Shanghai University of Finance and Economics（上海财经大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
