稀疏决策树的全罗生门集探索
Exploring the Whole Rashomon Set of Sparse Decision Trees
摘要：
在任何给定的机器学习问题中，可能有许多模型几乎可以同样好地解释数据。然而，大多数学习算法只返回这些模型中的一个，使得从业者没有实用的方法来探索可能具有超出损失函数内所能表达的期望属性的替代模型。罗生门集是这些几乎最优模型的集合。罗生门集可能非常复杂，特别是对于允许复杂交互项的高度非线性函数类，例如决策树。我们提供了第一个技术，完全枚举的罗生门集稀疏决策树，事实上，我们的工作提供了第一个完整的枚举的任何罗生门集的非平凡的问题，一个高度非线性的离散函数类。这使得用户能够在几乎同样好的所有模型中对模型选择进行前所未有的控制。我们表示罗生门集在一个专门的数据结构，支持有效的查询和采样。我们展示了罗生门集的三个应用：1）它可以用来研究几乎最优树集合的变量重要性（而不是单个树），2）准确度的罗生门集可以枚举平衡准确度和F1得分的罗生门集，3）完整数据集的罗生门集可以用来生成仅用数据集的子集构造的罗生门集。因此，我们能够用一个新的镜头来检查罗生门集合，使用户能够选择模型，而不是受只产生单个模型的算法的支配。
英文摘要：
In any given machine learning problem, there may be many models that could explain the data almost equally well. However, most learning algorithms return only one of these models, leaving practitioners with no practical way to explore alternative models that might have desirable properties beyond what could be expressed within a loss function. The Rashomon set is the set of these all almost-optimal models. Rashomon sets can be extremely complicated, particularly for highly nonlinear function classes that allow complex interaction terms, such as decision trees. We provide the first technique for completely enumerating the Rashomon set for sparse decision trees; in fact, our work provides the first complete enumeration of any Rashomon set for a non-trivial problem with a highly nonlinear discrete function class. This allows the user an unprecedented level of control over model choice among all models that are approximately equally good. We represent the Rashomon set in a specialized data structure that supports efficient querying and sampling. We show three applications of the Rashomon set: 1) it can be used to study variable importance for the set of almost-optimal trees (as opposed to a single tree), 2) the Rashomon set for accuracy enables enumeration of the Rashomon sets for balanced accuracy and F1-score, and 3) the Rashomon set for a full dataset can be used to produce Rashomon sets constructed with only subsets of the data set. Thus, we are able to examine Rashomon sets across problems with a new lens, enabling users to choose models rather than be at the mercy of an algorithm that produces only a single model.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
