首页/论文/2004.14914

arXiv 2004.149142020-04cs.CL

Tired of Topic Models? Clusters of Pretrained Word Embeddings Make for Fast and Good Topics too!

Johns Hopkins University（约翰斯·霍普金斯大学）；Department of Computer Science（计算机科学系）；Baltimore, MD, USA（美国马里兰州巴尔的摩）

Suzanna Sia, Ayush Dalmia, Sabrina J. Mielke

备注：: Published as a short paper at EMNLP 2020

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

Topic models are a useful analysis tool to uncover the underlying themes within document collections. The dominant approach is to use probabilistic topic models that posit a generative story, but in this paper we propose an alternative way to obtain topics: clustering pre-trained word embeddings while incorporating document information for weighted clustering and reranking top words. We provide benchmarks for the combination of different word embeddings and clustering algorithms, and analyse their performance under dimensionality reduction with PCA. The best performing combination for our approach performs as well as classical topic models, but with lower runtime and computational complexity.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Understand customer reviews with less data and in short time: pretrained language representation and active learning

GMF Assurances（GMF保险公司）

In this paper, we address customer review understanding problems by using supervised machine learning approaches, in order to achieve a fully automatic review aspects categorisation and sentiment analysis. In general, such supervised learning algorithms require domain-specific expert knowledge for generating high quality labeled training data, and the cost of labeling can be very high. To achieve an in-production customer review machine learning enabled analysis tool with only a limited amount of data and within a reasonable training data collection time, we propose to use pre-trained language…

2019-10-29cs.CLcs.LGstat.ML

LINSPECTOR WEB: A Multilingual Probing Suite for Word Representations

Technische Universität Darmstadt（达姆施塔特工业大学）

We present LINSPECTOR WEB, an open source multilingual inspector to analyze word representations. Our system provides researchers working in low-resource settings with an easily accessible web based probing tool to gain quick insights into their word embeddings especially outside of the English language. To do this we employ 16 simple linguistic probing tasks such as gender, case marking, and tense for a diverse set of 28 languages. We support probing of static word embeddings along with pretrained AllenNLP models that are commonly used for NLP downstream tasks such as named entity recognition…

2019-10-15cs.CL

多语言到英语的机器翻译工具、数据与预训练模型

Many-to-English Machine Translation Tools, Data, and Pretrained Models

Information Sciences Institute（信息科学研究所）；University of Southern California（南加州大学）；Texas Advanced Computing Center（德克萨斯高级计算中心）；University of Texas（德克萨斯大学）；NASA Jet Propulsion Laboratory（美国国家航空航天局喷气推进实验室）；California Institute of Technology（加州理工学院）

本文提出MTData、NLCodec和RTG三个工具，并构建了一个支持500种源语言到英语的多语言神经机器翻译模型，该模型可下载使用或作为低资源语言迁移学习的父模型。

2021-07-01cs.AIcs.CLcs.LG

Pretrained Language Models are Symbolic Mathematics Solvers too!

Sharif University of Technology（谢里夫理工大学）；University of Tartu（塔尔图大学）；University of South Carolina（南卡罗来纳大学）

Solving symbolic mathematics has always been of in the arena of human ingenuity that needs compositional reasoning and recurrence. However, recent studies have shown that large-scale language models such as transformers are universal and surprisingly can be trained as a sequence-to-sequence task to solve complex mathematical equations. These large transformer models need humongous amounts of training data to generalize to unseen symbolic mathematics problems. In this paper, we present a sample efficient way of solving the symbolic tasks by first pretraining the transformer model with language …

2023-03-14cs.CLcs.LGstat.ML

基于通用嵌入的词典用于透明且可复现的文本评分

Generic Embedding-Based Lexicons for Transparent and Reproducible Text Scoring

提出用通用预训练词嵌入构建透明且高性能的文本评分词典，兼顾模型性能与可解释性。

2024-11-01cs.CL

Transformer过早停止思考，一个微小的LoRA即可修复

Transformers Stop Thinking Too Early, and a Tiny LoRA Fixes It

Georgia Institute of Technology（佐治亚理工学院）

预训练Transformer仅用少量深度处理上下文引用，在早期层添加任务训练的秩8 LoRA（冻结其余权重）可显著扩展链式推理能力，如Qwen3-8B准确率从15.5%升至99%，并提升MuSiQue表现。

2026-09-30cs.AIcs.CLcs.LG

更多相关论文收起更多论文

基于扩散预训练的密集和上下文嵌入

Diffusion-Pretrained Dense and Contextual Embeddings

本文提出pplx-embed模型，通过多阶段对比学习在扩散预训练语言模型基础上实现多语言嵌入，其中pplx-embed-v1在多个基准测试中表现优异，pplx-embed-context-v1在ConTEB基准中创纪录。

2026-02-13cs.CLcs.IRcs.LG

Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations

University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）

Topic models have been the prominent tools for automatic topic discovery from text corpora. Despite their effectiveness, topic models suffer from several limitations including the inability of modeling word ordering information in documents, the difficulty of incorporating external linguistic knowledge, and the lack of both accurate and efficient inference methods for approximating the intractable posterior. Recently, pretrained language models (PLMs) have brought astonishing performance improvements to a wide variety of tasks due to their superior representations of text. Interestingly, there…

2022-02-09cs.CLcs.IRcs.LG

Wikipedia2Vec: An Efficient Toolkit for Learning and Visualizing the Embeddings of Words and Entities from Wikipedia

Studio Ousia；RIKEN AIP（理化学研究所先进智能项目中心）；University of Washington（华盛顿大学）；The University of Tokyo（东京大学）；Nara Institute of Science and Technology（奈良先端科学技术大学院大学）；National Institute of Informatics（国立情报学研究所）；Keio University（庆应义塾大学）

The embeddings of entities in a large knowledge base (e.g., Wikipedia) are highly beneficial for solving various natural language tasks that involve real world knowledge. In this paper, we present Wikipedia2Vec, a Python-based open-source tool for learning the embeddings of words and entities from Wikipedia. The proposed tool enables users to learn the embeddings efficiently by issuing a single command with a Wikipedia dump file as an argument. We also introduce a web-based demonstration of our tool that allows users to visualize and explore the learned embeddings. In our experiments, our tool…

2020-09-26cs.CLcs.LG

GPT 也懂理解

GPT Understands, Too

Tsinghua University（清华大学）；Massachusetts Institute of Technology（麻省理工学院）

针对离散提示性能不稳定的问题，提出 P-Tuning 方法，通过结合可训练的连续提示嵌入与离散提示，稳定训练过程并在多种 NLU 任务上大幅提升性能。

2023-10-25cs.CLcs.LG

WordBias: An Interactive Visual Tool for Discovering Intersectional Biases Encoded in Word Embeddings

Stony Brook University（石溪大学）

Intersectional bias is a bias caused by an overlap of multiple social factors like gender, sexuality, race, disability, religion, etc. A recent study has shown that word embedding models can be laden with biases against intersectional groups like African American females, etc. The first step towards tackling such intersectional biases is to identify them. However, discovering biases against different intersectional groups remains a challenging task. In this work, we present WordBias, an interactive visual tool designed to explore biases against intersectional groups encoded in static word embe…

2021-09-06cs.AIcs.CLcs.HC

通过句子嵌入预测进行潜在推理

Latent Reasoning via Sentence Embedding Prediction

KAIST（韩国科学技术院）；Carnegie Mellon University（卡内基梅隆大学）；University College London（伦敦大学学院）；LG AI Research（LG人工智能研究）

本研究提出一种通过自回归预测下一句连续嵌入使预训练语言模型在句子空间进行潜在推理的框架，实验表明连续推理下的上下文嵌入性能媲美CoT且推理时FLOPs减半。

2025-10-11cs.AIcs.CL