PESTS：波斯语-英语跨语言语义文本相似度语料库
PESTS: Persian_English Cross Lingual Corpus for Semantic Textual Similarity
AI总结：
本文首次构建波斯语-英语语义文本相似度语料库 PESTS（含5375个句子对），并微调多种 transformer 模型，使 XLM ROBERTa 的 Pearson 相关系数从 85.87% 提升至 95.62%。
摘要：
语义文本相似度是自然语言处理中近期受到大量研究的组成部分之一。在计算语言学和自然语言处理中，评估词语、短语、段落和文本的语义相似度至关重要。计算两个文本片段、段落或短语之间的语义相似程度（同时提供单语和跨语言版本）被称为语义相似度。跨语言语义相似度需要语料库，其中包含源语言和目标语言的句子对，并且句子对之间具有某种程度的语义相似度。由于缺乏跨语言语义相似度数据集，许多现有的跨语言语义相似度模型使用机器翻译，而机器翻译错误的传播会降低模型的准确性。另一方面，当我们想要将语义相似度特征用于机器翻译时，不应将相同的机器翻译用于语义相似度。对于作为低资源语言之一的波斯语，在这方面尚未做出任何努力，并且比以往任何时候都更需要一个能够理解两种语言上下文的模型。本文首次借助语言学专家生成了波斯语和英语句子之间的语义文本相似度语料库。我们将该数据集命名为 PESTS（Persian English Semantic Textual Similarity）。该语料库包含 5375 个句子对。此外，使用该数据集对基于 transformer 的不同模型进行了微调。结果表明，使用 PESTS 数据集，XLM ROBERTa 模型的 Pearson 相关系数从 85.87% 提高到 95.62%。
英文摘要：
One of the components of natural language processing that has received a lot of investigation recently is semantic textual similarity. In computational linguistics and natural language processing, assessing the semantic similarity of words, phrases, paragraphs, and texts is crucial. Calculating the degree of semantic resemblance between two textual pieces, paragraphs, or phrases provided in both monolingual and cross-lingual versions is known as semantic similarity. Cross lingual semantic similarity requires corpora in which there are sentence pairs in both the source and target languages with a degree of semantic similarity between them. Many existing cross lingual semantic similarity models use a machine translation due to the unavailability of cross lingual semantic similarity dataset, which the propagation of the machine translation error reduces the accuracy of the model. On the other hand, when we want to use semantic similarity features for machine translation the same machine translations should not be used for semantic similarity. For Persian, which is one of the low resource languages, no effort has been made in this regard and the need for a model that can understand the context of two languages is felt more than ever. In this article, the corpus of semantic textual similarity between sentences in Persian and English languages has been produced for the first time by using linguistic experts. We named this dataset PESTS (Persian English Semantic Textual Similarity). This corpus contains 5375 sentence pairs. Also, different models based on transformers have been fine-tuned using this dataset. The results show that using the PESTS dataset, the Pearson correlation of the XLM ROBERTa model increases from 85.87% to 95.62%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
