arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

支持度阈值而非算法限制了共购网络中稀有关联的恢复

Support Thresholds, Not Algorithms, Limit Rare-Association Recovery in Co-Purchase Networks

Xiao Han, Zhen Zhang, Xin Zhao, Jiechun Lei, Moxuan Zheng, Youting Wang

arXiv 2609.20171首次发表:更新:

AI 中文总结

本研究比较五种共购网络边过滤方法,发现基于提升度的top-K和噪声校正方法比Apriori恢复更多稀有高提升度关联,且两者选择的边有显著差异。

AI 中文摘要

Apriori算法的支持度阈值在进行市场购物篮分析时涉及一个权衡:高阈值能够识别频繁出现的关联,而低阈值则会导致生成大量规则。本文在两个杂货数据集上比较了五种共购边过滤方法:即Instacart(320万个购物篮)和Dunnhumby(20.8万个购物篮),包括Apriori、Apriori+提升度后过滤、基于提升度的前K排名(top-$K$),以及两种基于网络的方法:噪声校正(NC)和差异过滤(DF)。top-$K$方法确保了最大的平均提升度,而NC通过单一显著性参数($\alpha$)值达到了类似的提升度水平。这两种方法比Apriori恢复了更多的稀有高提升度关联(80-100%对比22-28%)。NC和top-$K$选择了显著不同的边(18-29%不重叠):NC保留了统计上有效的配对,而top-$K$保留了高提升度但统计显著性低的稀有配对。滚动起点留出法评估显示,top-$K$边在每个分割点上的重现率更高,但NC边在留出网络中保持统计显著性的可能性高出约12个百分点。

英文摘要

The support threshold of the Apriori algorithm involves a trade-off in conducting market basket analysis: the associations that occur frequently are noted with high threshold; however, the low ones lead to generating the large amount of rules. The paper compares five methods for co-purchase edge filtration on two grocery datasets: i.e., Instacart (3.2 million baskets) and Dunnhumby (208 thousand baskets), including Apriori, Apriori + lift post-filtering, top-$K$ ranking based on lift, and two methods based on networks, noise-corrected (NC) and disparity filter (DF). The top-$K$ method ensures the maximum average lift, while the NC achieves similar lift level by means of a single value of the significance parameter ($α$). These two methods recover substantially more rare high-lift associations than Apriori (80-100% against 22-28%). NC and top-$K$ select meaningfully different edges (18-29% non-overlapping): NC retains statistically validated pairs, while top-$K$ retains rare pairs with high lift but low statistical significance. A rolling-origin holdout evaluation shows that top-$K$ edges recur at higher rates at every split, but NC edges are ~12 pp more likely to remain statistically significant in the held-out network.

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑