波兰语自然语言推理与事实性——一个基于专家的数据集和基准
Polish Natural Language Inference and Factivity -- an Expert-based Dataset and Benchmarks
AI总结：
针对波兰语自然语言推理中的事实性现象，构建了包含2,432个动词-补语对的数据集，并评估了BERT模型与基于语言特征的方法，发现BERT模型接近专家特征性能，但复杂案例仍待研究。
摘要：
尽管机器学习在自然语言处理领域近期取得了突破，自然语言推理（NLI）问题仍然构成一项挑战。为此，我们贡献了一个新数据集，该数据集专门聚焦于事实性现象；然而，我们的任务与其他NLI任务相同，即预测蕴含、矛盾或中性（ECN）。该数据集包含完全自然语言的波兰语话语，并收集了2,432个动词-补语对和309个独特动词。该数据集基于波兰语国家语料库（NKJP），在主动词频率和其他语言特征（如内部否定的出现）方面具有代表性样本。我们发现，基于句子的Transformer BERT模型取得了相对较好的结果（约89%的F1分数）。尽管使用语言特征取得了更好的结果（约91%的F1分数），但该模型需要更多人力（人在回路中），因为特征是由专家语言学家手动准备的。仅消费输入句子的基于BERT的模型表明，它们捕获了NLI/事实性的大部分复杂性。该现象中的复杂案例——例如涉及权利（E）和非事实性动词的案例——仍然是进一步研究的开放问题。
英文摘要：
Despite recent breakthroughs in Machine Learning for Natural Language Processing, the Natural Language Inference (NLI) problems still constitute a challenge. To this purpose we contribute a new dataset that focuses exclusively on the factivity phenomenon; however, our task remains the same as other NLI tasks, i.e. prediction of entailment, contradiction or neutral (ECN). The dataset contains entirely natural language utterances in Polish and gathers 2,432 verb-complement pairs and 309 unique verbs. The dataset is based on the National Corpus of Polish (NKJP) and is a representative sample in regards to frequency of main verbs and other linguistic features (e.g. occurrence of internal negation). We found that transformer BERT-based models working on sentences obtained relatively good results ($\approx89\%$ F1 score). Even though better results were achieved using linguistic features ($\approx91\%$ F1 score), this model requires more human labour (humans in the loop) because features were prepared manually by expert linguists. BERT-based models consuming only the input sentences show that they capture most of the complexity of NLI/factivity. Complex cases in the phenomenon - e.g. cases with entitlement (E) and non-factive verbs - remain an open issue for further research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
