arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2407.10000cs.LGstat.ML
收藏

神经符号学习中的不平衡问题:特征化与缓解策略

Imbalances in Neurosymbolic Learning: Characterization and Mitigating Strategies

Huawei Labs(华为实验室);University of Pennsylvania(宾夕法尼亚大学)

Kaifu Wang, Efthymia Tsamoura, Dan Roth

AI总结:

本文提出了一种缓解神经符号学习中学习不平衡问题的方法,通过估计隐藏标签的边际并将其作为约束,提升模型性能达14%。

摘要:

我们研究了神经符号学习(NSL)中最受欢迎的问题,即在仅给定对向量x中元素的黄金标签应用符号组件σ的结果的情况下,学习神经分类器。学习者不知道x中元素的黄金标签。我们做出了多个贡献,理论和实践上的,以解决一个此前在该背景下未被研究的问题,即特征化和缓解学习不平衡,即在分类不同类别的实例时出现的显著误差差异(即类别特定风险)。我们的理论分析揭示了一个独特现象:σ可以极大地影响学习不平衡。这一结果与之前对监督和弱监督学习的研究形成鲜明对比,后者仅在数据不平衡的情况下研究学习不平衡。在实践中,我们介绍了一种利用弱监督数据估计隐藏黄金标签边际的方法。然后,我们介绍了在训练和测试时通过将隐藏标签的边际视为约束来缓解不平衡的算法。我们使用NSL和长尾学习中的强大基线展示了我们技术的有效性,表明性能可提高高达14%。

英文摘要:

We study one of the most popular problems in **neurosymbolic learning** (NSL), that of learning neural classifiers given only the result of applying a symbolic component $σ$ to the gold labels of the elements of a vector $\mathbf x$. The gold labels of the elements in $\mathbf x$ are unknown to the learner. We make multiple contributions, theoretical and practical, to address a problem that has not been studied so far in this context, that of characterizing and mitigating *learning imbalances*, i.e., major differences in the errors that occur when classifying instances of different classes (aka **class-specific risks**). Our theoretical analysis reveals a unique phenomenon: that $σ$ can greatly impact learning imbalances. This result sharply contrasts with previous research on supervised and weakly supervised learning, which only studies learning imbalances under data imbalances. On the practical side, we introduce a technique for estimating the marginal of the hidden gold labels using weakly supervised data. Then, we introduce algorithms that mitigate imbalances at training and testing time by treating the marginal of the hidden labels as a constraint. We demonstrate the effectiveness of our techniques using strong baselines from NSL and long-tailed learning, suggesting performance improvements of up to 14%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑