用于可解释机器学习的异构混合专家框架
A Heterogeneous Mixture of Experts Framework for Interpretable Machine Learning
浏览论文内容
中文总结 AI 辅助
该研究提出异构混合专家框架，将决策树、线性SVM等异构专家与概率门控结合，在保证可解释性的同时，在基准数据集上达到与同构MoDT、随机森林相当的预测性能。
中文摘要 AI 辅助
混合专家（MoE）模型通过依赖输入的门控机制，为将复杂预测问题划分为更简单的局部学习任务提供了灵活框架。现有可解释MoE方法，如决策树混合（MoDT），通过采用同构决策树专家实现透明性，但这限制了模型在特征空间所有区域仅使用单一归纳偏置。我们扩展MoDT框架，在通用概率门控机制下引入由决策树、线性支持向量机和二次判别分析组成的异构专家族。为确保基于似然的推理具有一致性，非概率专家被校准以生成条件类概率，使参数估计可在MoDT的广义期望-最大化框架内进行。我们进一步为所提出的异构门控更新建立理论单调上升保证，为优化过程提供依据。在多样的合成及真实基准数据集上的实验表明，所提框架能根据局部数据几何结构自适应地使专家专业化，在实现可解释专家分配的同时，达到与同构MoDT和随机森林相当的预测性能。该方法在统一混合专家框架内结合了可解释性、自适应归纳偏置选择和概率一致性。
英文摘要
Mixture-of-Experts (MoE) models provide a flexible framework for partitioning complex prediction problems into simpler local learning tasks through an input-dependent gating mechanism. Existing interpretable MoE approaches, such as Mixture of Decision Trees (MoDT), achieve transparency by employing homogeneous decision-tree experts, but this restricts the model to a single inductive bias across all regions of the feature space. We extend the MoDT framework by introducing heterogeneous expert families comprising decision trees, linear support vector machines, and quadratic discriminant analysis under a common probabilistic gating mechanism. To ensure coherent likelihood-based inference, non-probabilistic experts are calibrated to produce conditional class probabilities, allowing parameter estimation within the generalized Expectation-Maximization framework of MoDT. We further establish theoretical monotone ascent guarantees for the proposed heterogeneous gating updates, providing a justification for the optimization procedure. Experiments on a diverse collection of synthetic and real-world benchmark datasets demonstrate that the proposed framework adaptively specializes experts according to local data geometry, yielding interpretable expert assignments while achieving predictive performance competitive with homogeneous MoDT and Random Forests. The proposed approach combines interpretability, adaptive inductive bias selection, and probabilistic coherence within a unified mixture-of-experts framework.
发表机构
- Indian Statistical Institute（印度统计研究所）
- Interdisciplinary Statistical Research Unit, Indian Statistical Institute（印度统计研究所跨学科统计研究单元）
机构由 AI 辅助整理，请以论文原文为准。
