arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2110.08438cs.CLcs.AIcs.LG
收藏

基于PHL三元组生成的无监督自然语言推理

Unsupervised Natural Language Inference Using PHL Triplet Generation

Arizona State University(亚利桑那州立大学)

Neeraj Varshney, Pratyay Banerjee, Tejas Gokhale, Chitta Baral

摘要:

基于transformer的模型在各自的训练数据集上训练时,在众多自然语言推理(NLI)基准测试中取得了令人印象深刻的性能。然而,在某些情况下,训练样本可能不可用,或者收集它们可能是耗时和资源密集型的。在这项工作中,我们解决了上述挑战,并提出了一个探索性的研究无监督NLI,一个范例,其中没有人类注释的训练样本。我们在三种设置下研究它:PH,P和NPH,它们在可用于学习的未标记数据的程度上有所不同。作为一种解决方案,我们提出了一个程序化的数据生成方法,利用一组句子转换来收集PHL(假设,假设,标签)三元组用于训练NLI模型,绕过了对人类注释的训练数据的需要。在多个NLI数据集上的综合实验表明,该方法在PH、P和NPH设置下的准确率分别高达66.75%、65.9%和65.39%,优于所有现有的无监督基线。此外,使用人工注释的训练数据集(500个实例)的约0.1%来微调我们的模型,比在相同的500个实例上从头开始训练的模型的准确率高出12.2%。在这种卓越性能的支持下,我们总结了收集高质量特定任务数据的建议。

英文摘要:

Transformer-based models achieve impressive performance on numerous Natural Language Inference (NLI) benchmarks when trained on respective training datasets. However, in certain cases, training samples may not be available or collecting them could be time-consuming and resource-intensive. In this work, we address the above challenge and present an explorative study on unsupervised NLI, a paradigm in which no human-annotated training samples are available. We investigate it under three settings: PH, P, and NPH that differ in the extent of unlabeled data available for learning. As a solution, we propose a procedural data generation approach that leverages a set of sentence transformations to collect PHL (Premise, Hypothesis, Label) triplets for training NLI models, bypassing the need for human-annotated training data. Comprehensive experiments with several NLI datasets show that the proposed approach results in accuracies of up to 66.75%, 65.9%, 65.39% in PH, P, and NPH settings respectively, outperforming all existing unsupervised baselines. Furthermore, fine-tuning our model with as little as ~0.1% of the human-annotated training dataset (500 instances) leads to 12.2% higher accuracy than the model trained from scratch on the same 500 instances. Supported by this superior performance, we conclude with a recommendation for collecting high-quality task-specific data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑