运用对比预训练促进对数学问题的整体理解
Towards a Holistic Understanding of Mathematical Questions with Contrastive Pre-training
摘要:
有效地理解数学问题是一项重要的任务,它可以使许多应用受益,例如难度估计。由于人类注释的缺乏,研究人员已经引起了对设计用于问题表示的预训练模型的极大关注(例如,标签困难)。然而,与一般的自由格式文本(例如,用户评论),数学问题通常设计有明确的目的和数学逻辑,并且通常包括更复杂的内容,例如公式,以及相关的数学知识(例如,函数)。因此,整体表示数学问题的问题仍然没有得到充分的探索。为此,在本文中,我们提出了一种新的数学问题表征的对比预训练方法,即QuesCo,它试图使具有更相似目的的问题更接近。具体来说,我们首先设计了两个层次的问题增强,包括内容级和结构级,产生字面上不同的问题对相似的目的。然后,为了充分利用知识概念的层次信息,我们提出了一个知识层次感知的排名策略(KHAR),排名问题之间的相似性在细粒度的方式。接下来,我们采用排名对比学习任务,以优化我们的模型的基础上增强和排名的问题。我们在两个真实世界的数学数据集上进行了广泛的实验。实验结果证明了该模型的有效性。
英文摘要:
Understanding mathematical questions effectively is a crucial task, which can benefit many applications, such as difficulty estimation. Researchers have drawn much attention to designing pre-training models for question representations due to the scarcity of human annotations (e.g., labeling difficulty). However, unlike general free-format texts (e.g., user comments), mathematical questions are generally designed with explicit purposes and mathematical logic, and usually consist of more complex content, such as formulas, and related mathematical knowledge (e.g., Function). Therefore, the problem of holistically representing mathematical questions remains underexplored. To this end, in this paper, we propose a novel contrastive pre-training approach for mathematical question representations, namely QuesCo, which attempts to bring questions with more similar purposes closer. Specifically, we first design two-level question augmentations, including content-level and structure-level, which generate literally diverse question pairs with similar purposes. Then, to fully exploit hierarchical information of knowledge concepts, we propose a knowledge hierarchy-aware rank strategy (KHAR), which ranks the similarities between questions in a fine-grained manner. Next, we adopt a ranking contrastive learning task to optimize our model based on the augmented and ranked questions. We conduct extensive experiments on two real-world mathematical datasets. The experimental results demonstrate the effectiveness of our model.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。