Predictive modeling applications increasingly use data representing people's behavior, opinions, and interactions. Fine-grained behavior data often has different structure from traditional data, being very high-dimensional and sparse. Models built from these data are quite difficult to interpret, since they contain many thousands or even many millions of features. Listing features with large model coefficients is not sufficient, because the model coefficients do not incorporate information on feature presence, which is key when analysing sparse data. In this paper we introduce two alternatives for explaining predictive models by listing important features. We evaluate these alternatives in terms of explanation "bang for the buck,", i.e., how many examples' inferences are explained for a given number of features listed. The bottom line: (i) The proposed alternatives have double the bang-for-the-buck as compared to just listing the high-coefficient features, and (ii) interestingly, although they come from different sources and motivations, the two new alternatives provide strikingly similar rankings of important features.
异常检测有许多应用，并已被广泛研究。我们考虑一个互补的问题，有一个稀疏的文献：异常描述。对异常的解释对于从业者进行意义判断、故障排除和计划行动至关重要。为此，我们提出了一种新的方法称为x-PACS（eXplaining Patterns of Anomalies with Characterizing Subspaces），它通过识别（1）它们形成的组（或模式）以及（2）将每个异常模式与正常实例分开的特征子空间和特征规则来“逆向工程”已知异常。分组解释异常不仅可以节省分析时间并深入了解各种类型的异常，还可以引起对潜在关键的重复异常的注意。在发展x-PACS，我们首先构建一个异常描述问题的desiderata。从描述性数据采掘的角度来看，我们的方法表现出五个所需的属性在我们的desiderata。也就是说，它可以发现异常模式（i）多种不同类型，（ii）隐藏在高维空间的任意子空间中，（iii）可由分析师解释，（iv）不同于数据的正常模式，以及最后（v）简洁，提供最短的数据描述。此外，x-PACS是高度并行化的，并在数据大小方面线性扩展。没有现有的工作异常描述同时满足所有这些属性。虽然不是我们的主要目标，但我们发现的异常模式可以作为可解释的“签名”，并可用于检测。我们展示了x-PACS在解释和检测现实世界的数据集相比，国家的最先进的有效性。
High-Dimensional Analysis of Single-Layer Attention for Sparse-Token Classification
Department of Statistics, Harvard University（哈佛大学统计学系）；Center of Mathematical Sciences and Applications, Harvard University（哈佛大学数学科学中心）；John A. Paulson School of Engineering and Applied Sciences, Harvard University（哈佛大学工程与应用科学学院）
