基于草图的相关异常点快速恢复
Fast Sketch-based Recovery of Correlation Outliers
摘要:
许多数据源可以被解释为时间序列,关键问题是识别大量信号中哪些对是高度相关的。我们预计将有很少的,大的,有趣的相关性,而大多数信号对没有任何强相关性。我们将其抽象为在n个主要是两两不相关的随机变量的集合中识别高度相关的对的问题,其中变量的观测值作为流到达。简化可以消除对观测数量的依赖,但需要进一步的技术来驯服通过所有可能的对搜索的二次(n)成本。我们开发了一种新的算法,用于快速找到大的相关性的基础上草图技术与一个额外的扭曲:我们快速生成草图的随机组合的信号,并使用这些与编码理论的想法,解码相关对的身份。我们证明了正确性,并与最好的LSH(局部敏感哈希)为基础的方法的性能和有效性进行比较。
英文摘要:
Many data sources can be interpreted as time-series, and a key problem is to identify which pairs out of a large collection of signals are highly correlated. We expect that there will be few, large, interesting correlations, while most signal pairs do not have any strong correlation. We abstract this as the problem of identifying the highly correlated pairs in a collection of n mostly pairwise uncorrelated random variables, where observations of the variables arrives as a stream. Dimensionality reduction can remove dependence on the number of observations, but further techniques are required to tame the quadratic (in n) cost of a search through all possible pairs. We develop a new algorithm for rapidly finding large correlations based on sketch techniques with an added twist: we quickly generate sketches of random combinations of signals, and use these in concert with ideas from coding theory to decode the identity of correlated pairs. We prove correctness and compare performance and effectiveness with the best LSH (locality sensitive hashing) based approach.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。