首页/论文/1506.03487

arXiv 1506.034872015-06cs.CL

From Paraphrase Database to Compositional Paraphrase Model and Back

University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Toyota Technological Institute at Chicago（芝加哥丰田技术研究所）

John Wieting, Mohit Bansal, Kevin Gimpel, Karen Livescu, Dan Roth

备注：: 2015 TACL paper updated with an appendix describing new 300 dimensional embeddings. Submitted 1/2015. Accepted 2/2015. Published 6/2015
发表信息：: TACL Vol 3 (2015) pg 345-358

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

The Paraphrase Database (PPDB; Ganitkevitch et al., 2013) is an extensive semantic resource, consisting of a list of phrase pairs with (heuristic) confidence estimates. However, it is still unclear how it can best be used, due to the heuristic nature of the confidences and its necessarily incomplete coverage. We propose models to leverage the phrase pairs from the PPDB to build parametric paraphrase models that score paraphrase pairs more accurately than the PPDB's internal scores while simultaneously improving its coverage. They allow for learning phrase embeddings as well as improved word embeddings. Moreover, we introduce two new, manually annotated datasets to evaluate short-phrase paraphrasing models. Using our paraphrase model trained using PPDB, we achieve state-of-the-art results on standard word and bigram similarity tasks and beat strong baselines on our new short phrase paraphrase tasks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

ParaAMR：基于AMR回译的大规模句法多样化复述数据集

ParaAMR: A Large-Scale Syntactically Diverse Paraphrase Dataset by AMR Back-Translation

University of California Los Angeles（加州大学洛杉矶分校）；University of Illinois Chicago（伊利诺伊大学芝加哥分校）；University of Southern California（南加州大学）；Amazon Alexa AI（亚马逊Alexa人工智能部门）

本文提出ParaAMR数据集，通过抽象语义表示回译生成大规模句法多样化复述，并验证其在句子嵌入、句法可控复述生成和少样本数据增强等任务上的改进效果。

2023-05-26cs.CL

ARPA: Armenian Paraphrase Detection Corpus and Models

Russian-Armenian University（俄罗斯-亚美尼亚大学）；Ivannikov Laboratory for System Programming（伊万尼科夫系统编程实验室）

In this work, we employ a semi-automatic method based on back translation to generate a sentential paraphrase corpus for the Armenian language. The initial collection of sentences is translated from Armenian to English and back twice, resulting in pairs of lexically distant but semantically similar sentences. The generated paraphrases are then manually reviewed and annotated. Using the method train and test datasets are created, containing 2360 paraphrases in total. In addition, the datasets are used to train and evaluate BERTbased models for detecting paraphrase in Armenian, achieving results…

2020-09-26cs.CL

PAWS: Paraphrase Adversaries from Word Scrambling

Google AI Language（谷歌人工智能语言）

Existing paraphrase identification datasets lack sentence pairs that have high lexical overlap without being paraphrases. Models trained on such data fail to distinguish pairs like flights from New York to Florida and flights from Florida to New York. This paper introduces PAWS (Paraphrase Adversaries from Word Scrambling), a new dataset with 108,463 well-formed paraphrase and non-paraphrase pairs with high lexical overlap. Challenging pairs are generated by controlled word swapping and back translation, followed by fluency and paraphrase judgments by human raters. State-of-the-art models trai…

2019-04-01cs.CL

Paraphrase-Supervised Models of Compositionality

Carnegie Mellon University（卡内基梅隆大学）；eBay Inc.（易贝公司）

Compositional vector space models of meaning promise new solutions to stubborn language understanding problems. This paper makes two contributions toward this end: (i) it uses automatically-extracted paraphrase examples as a source of supervision for training compositional models, replacing previous work which relied on manual annotations used for the same purpose, and (ii) develops a context-aware model for scoring phrasal compositionality. Experimental results indicate that these multiple sources of information can be used to learn partial semantic supervision that matches previous technique…

2018-01-31cs.CL

越南语句子复述的大规模基准数据集

A Large-Scale Benchmark for Vietnamese Sentence Paraphrases

University of Information Technology（信息科技大学）；Vietnam National University（越南国立大学）

本文构建首个大规模越南语句子复述高质量数据集ViSP（含120万句对），通过混合方法保障质量，用多种模型实验验证，为越南语复述研究提供基础。

2025-02-11cs.CL

Learning Phrase Embeddings from Paraphrases with GRUs

Rensselaer Polytechnic Institute（伦斯勒理工学院）

Learning phrase representations has been widely explored in many Natural Language Processing (NLP) tasks (e.g., Sentiment Analysis, Machine Translation) and has shown promising improvements. Previous studies either learn non-compositional phrase representations with general word embedding learning techniques or learn compositional phrase representations based on syntactic structures, which either require huge amounts of human annotations or cannot be easily generalized to all phrases. In this work, we propose to take advantage of large-scaled paraphrase database and present a pair-wise gated r…

2017-10-13cs.CL

更多相关论文收起更多论文

Phrase-BERT: Improved Phrase Embeddings from BERT with an Application to Corpus Exploration

University of Massachusetts, Amherst（马萨诸塞大学阿默斯特分校）

Phrase representations derived from BERT often do not exhibit complex phrasal compositionality, as the model relies instead on lexical similarity to determine semantic relatedness. In this paper, we propose a contrastive fine-tuning objective that enables BERT to produce more powerful phrase embeddings. Our approach (Phrase-BERT) relies on a dataset of diverse phrasal paraphrases, which is automatically generated using a paraphrase generation model, as well as a large-scale dataset of phrases in context mined from the Books3 corpus. Phrase-BERT outperforms baselines across a variety of phrase-…

2021-10-13cs.CL

Distributional semantics beyond words: Supervised learning of analogy and paraphrase

National Research Council Canada（加拿大国家研究委员会）

There have been several efforts to extend distributional semantics beyond individual words, to measure the similarity of word pairs, phrases, and sentences (briefly, tuples; ordered sets of words, contiguous or noncontiguous). One way to extend beyond words is to compare two tuples using a function that combines pairwise similarities between the component words in the tuples. A strength of this approach is that it works with both relational similarity (analogy) and compositional similarity (paraphrase). However, past work required hand-coding the combination function for different tasks. The m…

2013-10-18cs.AIcs.CLcs.IR

多模态模型中的序列组合泛化

Sequential Compositional Generalization in Multimodal Models

Hacettepe University（哈杰泰佩大学）；Koç University（科克大学）；KUIS AI Center（KUIS人工智能中心）；University of Copenhagen（哥本哈根大学）；Pioneer Centre for AI（先锋人工智能中心）

本研究通过CompAct数据集探究多模态模型的序列组合泛化能力，发现双模态和三模态模型优于纯文本模型，强调了多模态在泛化中的关键作用。

2024-04-18cs.CL

Visual Information Guided Zero-Shot Paraphrase Generation

Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机研究所）；Center for Data Science, Peking University（北京大学数据科学中心）；The MOE Key Laboratory of Computational Linguistics, Peking University（北京大学计算语言学教育部重点实验室）

Zero-shot paraphrase generation has drawn much attention as the large-scale high-quality paraphrase corpus is limited. Back-translation, also known as the pivot-based method, is typical to this end. Several works leverage different information as "pivot" such as language, semantic representation and so on. In this paper, we explore using visual information such as image as the "pivot" of back-translation. Different with the pipeline back-translation method, we propose visual information guided zero-shot paraphrase generation (ViPG) based only on paired image-caption data. It jointly trains an …

2022-09-22cs.AIcs.CL

Learning Paraphrastic Sentence Embeddings from Back-Translated Bitext

Toyota Technological Institute at Chicago（芝加哥丰田工业研究所）；University of Edinburgh（爱丁堡大学）

We consider the problem of learning general-purpose, paraphrastic sentence embeddings in the setting of Wieting et al. (2016b). We use neural machine translation to generate sentential paraphrases via back-translation of bilingual sentence pairs. We evaluate the paraphrase pairs by their ability to serve as training data for learning paraphrastic sentence embeddings. We find that the data quality is stronger than prior work based on bitext and on par with manually-written English paraphrase pairs, with the advantage that our approach can scale up to generate large training sets for many langua…

2017-06-06cs.CL

TrackList: 回溯查询语言多样性以探究开放式大语言模型中的头部与尾部知识

TrackList: Tracing Back Query Linguistic Diversity for Head and Tail Knowledge in Open Large Language Models

ATILF, University of Lorraine - CNRS, France（ATILF，洛林大学 - CNRS，法国）；Center for Data Science in Humanities, Chosun University, South Korea（人文数据科学中心，全州大学，韩国）

TrackList研究了预训练数据对LLMs回答多样化语言查询的影响，发现LLMs在定义型问题表现最佳，但在例子型问题表现最差，且更倾向于改写流行知识而非尾部和技术知识。

2025-11-27cs.CL

Revisiting Iterative Back-Translation from the Perspective of Compositional Generalization

Microsoft Research Asia（微软亚洲研究院）

Human intelligence exhibits compositional generalization (i.e., the capacity to understand and produce unseen combinations of seen components), but current neural seq2seq models lack such ability. In this paper, we revisit iterative back-translation, a simple yet effective semi-supervised method, to investigate whether and how it can improve compositional generalization. In this work: (1) We first empirically show that iterative back-translation substantially improves the performance on compositional generalization benchmarks (CFQ and SCAN). (2) To understand why iterative back-translation is …

2020-12-08cs.AIcs.CL

pair2vec: Compositional Word-Pair Embeddings for Cross-Sentence Inference

University of Washington（华盛顿大学）；Facebook AI Research（脸书人工智能研究院）

Reasoning about implied relationships (e.g., paraphrastic, common sense, encyclopedic) between pairs of words is crucial for many cross-sentence inference problems. This paper proposes new methods for learning and using embeddings of word pairs that implicitly represent background knowledge about such relationships. Our pairwise embeddings are computed as a compositional function on word representations, which is learned by maximizing the pointwise mutual information (PMI) with the contexts in which the two words co-occur. We add these representations to the cross-sentence attention layer of e…

2019-04-05cs.CL