In the domains of dataset construction and crowdsourcing, a notable challenge is to aggregate labels from a heterogeneous set of labelers, each of whom is potentially an expert in some subset of tasks (and less reliable in others). To reduce costs of hiring human labelers or training automated labeling systems, it is of interest to minimize the number of labelers while ensuring the reliability of the resulting dataset. We model this as the problem of performing $K$-class classification using the predictions of smaller classifiers, each trained on a subset of $[K]$, and derive bounds on the number of classifiers needed to accurately infer the true class of an unlabeled sample under both adversarial and stochastic assumptions. By exploiting a connection to the classical set cover problem, we produce a near-optimal scheme for designing such configurations of classifiers which recovers the well known one-vs.-one classification approach as a special case. Experiments with the MNIST and CIFAR-10 datasets demonstrate the favorable accuracy (compared to a centralized classifier) of our aggregation scheme applied to classifiers trained on subsets of the data. These results suggest a new way to automatically label data or adapt an existing set of local classifiers to larger-scale multiclass problems.
Scalable Heterogeneous Graph Learning via Heterogeneous-aware Orthogonal Prototype Experts
National Engineering Research Center for Big Data Technology；System Lab, Cluster；Grid Computing Lab, School of Computer Science；Technology, Huazhong University of Science；DIRO, Université de Montréal \& Mila \& Canada CIFAR AI Chair, Canada
N-BEATS-MOE: N-BEATS with a Mixture-of-Experts Layer for Heterogeneous Time Series Forecasting
Faculdade de Engenharia da Universidade do Porto, Porto, Portugal（葡萄牙波尔图大学工程学院）；Laboratory for Artificial Intelligence and Computer Science (LIACC)（人工智能与计算机科学实验室）
Adaptive Graph Mixture of Residual Experts: Unsupervised Learning on Diverse Graphs with Heterogeneous Specialization
School of New Media and Communication, Tianjin University（新媒体与传播学院，天津大学）；Baidu（百度）；School of Computer Science and Engineering, Beihang University（计算机科学与工程学院，北航大学）
CHoE: Cross-Domain Heterogeneous Graph Prompt Learning via Structure-Conditioned Experts
School of Computer Science and Technology, Tianjin University（天津大学计算机科学与技术学院）；Department of Health Technology and Informatics, and Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University（香港理工大学健康科技与信息学系、数据科学与人工智能系）
Unified Multi-Domain Graph Pre-training for Homogeneous and Heterogeneous Graphs via Domain-Specific Expert Encoding
School of Computer Science；Technology Tianjin University Tianjin China；North China University of Science；School of Economics；Management Beijing University of Posts；Departments of Health Technology \& Informatics；Computing The Hong Kong Polytechnic University Kowloon Hong Kong；Tianjin University；Beijing University of Posts；The Hong Kong Polytechnic University
Effective MoE-based LLM Compression by Exploiting Heterogeneous Inter-Group Experts Routing Frequency and Information Density
Department of Computer Science, Stevens Institute of Technology, Hoboken, NJ, USA（计算机科学系，史蒂文斯理工学院，新泽西州霍博肯）；Northeastern University, Boston, MA, USA（东北大学，波士顿，马萨诸塞州）
UB-SMoE: Universally Balanced Sparse Mixture-of-Experts for Resource-adaptive Federated Fine-tuning of Foundation Models
School of Computer Science and Statistics, Trinity College Dublin, Ireland（特里尼蒂学院都柏林分校计算机科学与统计学学院）；School of Computer Science, University College Dublin, Ireland（都柏林大学学院计算机科学学院）
Analysis of an adaptive lead weighted ResNet for multiclass classification of 12-lead ECGs
University of Manchester（曼彻斯特大学）；Xi’an Jiaotong University（西安交通大学）；University of Exeter（埃克塞特大学）；University of Leeds（利兹大学）；Loughborough University（拉夫堡大学）
混合专家模型（Mixture of Experts，MoE）是在回归、聚类和分类等统计学习问题中为异构数据建模的成功模型。通常通过期望最大化算法用极大似然估计拟合，因此它们在高维问题中的应用仍然具有挑战性。我们考虑在MoE模型的拟合和特征选择的问题，并提出了一个正则化的极大似然估计的方法，鼓励稀疏解决方案的异质回归数据模型与潜在的高维预测。与最先进的正则化MLE MoE，所提出的建模不需要近似的惩罚函数。本文提出了两种混合期望最大化算法：期望-优化-最大化（EM/MM）算法和带有坐标上升算法的期望最大化算法。所提出的算法允许自动获得稀疏的解决方案，而无需阈值，并避免矩阵求逆，允许单变量参数更新。实验研究表明，该算法在恢复实际稀疏解、参数估计和异质回归数据聚类等方面具有良好的性能。
CausalMoE: A Billion-Scale Multimodal Foundation Model for Granger Causal Discovery with Pattern-Routed Heterogeneous Experts
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院通用人工智能国家重点实验室）；National Institute of Health Data Science, and Institute for Artificial Intelligence, Peking University（北京大学健康医疗大数据国家研究院、人工智能研究院）
Personalized Federated Learning on Heterogeneous and Long-Tailed Data via Expert Collaborative Learning
Xiamen University（厦门大学）；University College London（伦敦大学学院）；A*STAR Institute of High Performance Computing（新加坡高性能计算研究院）；Guangdong University of Technology（广东工业大学）；Guangdong Laboratory of Artificial Intelligence and Digital Economy（广东省人工智能与数字经济实验室）
Mosaic: Data-Free Knowledge Distillation via Mixture-of-Experts for Heterogeneous Distributed Environments
School of Computer Science and Technology, Tongji University（同济大学计算机科学与技术学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；The City University of New York（纽约城市大学）；Shenzhen University of Advanced Technology（深圳先进技术大学）
Exploiting Heterogeneous Graph Neural Networks with Latent Worker/Task Correlation Information for Label Aggregation in Crowdsourcing
Zhejiang University（浙江大学）；IBM T. J. Watson Research Center（IBM托马斯·J·沃森研究中心）；JD.COM Silicon Valley Research Center（京东硅谷研发中心）；Squirrel AI Learning（松鼠AI智适应教育）
Linking heterogeneous microstructure informatics with expert characterization knowledge through customized and hybrid vision-language representations for industrial qualification
McGill University（麦吉尔大学）；National Research Council Canada（加拿大国家研究委员会）；Apollo-Clad Laser Cladding, a division of Apollo Machine and Welding Ltd.（Apollo-Clad激光熔覆，Apollo机械与焊接有限公司旗下部门）；Fraunhofer Institute for Laser Technology ILT（弗劳恩霍夫激光技术研究所）
Mixture of Weight-shared Heterogeneous Group Attention Experts for Dynamic Token-wise KV Optimization
University of Chinese Academy of Sciences（中国科学院大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Macau（澳门大学）；Shenzhen University of Advanced Technology（深圳先进技术大学）
