首页/论文/2103.04044

arXiv 2103.040442021-03-06cs.CLcs.AIcs.HCcs.LG

Putting Humans in the Natural Language Processing Loop: A Survey

College of Computing, Georgia Institute of Technology（佐治亚理工学院计算学院）

Zijie J. Wang, Dongjin Choi, Shenyu Xu, Diyi Yang

备注：: The paper is accepted to the HCI+NLP workshop at EACL 2021

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

How can we design Natural Language Processing (NLP) systems that learn from human feedback? There is a growing research body of Human-in-the-loop (HITL) NLP frameworks that continuously integrate human feedback to improve the model itself. HITL NLP research is nascent but multifarious -- solving various NLP problems, collecting diverse feedback from different people, and applying different methods to learn from collected feedback. We present a survey of HITL NLP work from both Machine Learning (ML) and Human-Computer Interaction (HCI) communities that highlights its short yet inspiring history, and thoroughly summarize recent frameworks focusing on their tasks, goals, human interactions, and feedback learning methods. Finally, we discuss future directions for integrating human feedback in the NLP development loop.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

将“自然”放回自然语言处理中

Putting Natural in Natural Language Processing

Tilburg University（蒂尔堡大学）

本文指出NLP过度依赖文本而忽视口语，主张借助深度学习的进展统一语音处理与主流NLP，将口语视为人类交流的主要模式，从而构建更高效、拟人且超越文本模态的系统。

2023-05-23cs.AIcs.CLeess.AS

Does Putting a Linguist in the Loop Improve NLU Data Collection?

New York University（纽约大学）；IIIT-Delhi（德里印度理工学院）；Columbia University（哥伦比亚大学）

Many crowdsourced NLP datasets contain systematic gaps and biases that are identified only after data collection is complete. Identifying these issues from early data samples during crowdsourcing should make mitigation more efficient, especially when done iteratively. We take natural language inference as a test case and ask whether it is beneficial to put a linguist `in the loop' during data collection to dynamically identify and address gaps in the data by introducing novel constraints on the task. We directly compare three data collection protocols: (i) a baseline protocol, (ii) a linguist-…

2021-04-15cs.CL

将人类纳入图像描述生成循环

Putting Humans in the Image Captioning Loop

German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心（DFKI））；Oldenburg University（奥尔登堡大学）

本文提出在MS COCO预训练图像描述模型中引入用户反馈，通过数据增强、逐步更新与稀疏记忆回放实现个性化适配并缓解灾难性遗忘。

2023-06-06cs.CLcs.CV

PyTAIL: Interactive and Incremental Learning of NLP Models with Human in the Loop for Online Data

University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）

Online data streams make training machine learning models hard because of distribution shift and new patterns emerging over time. For natural language processing (NLP) tasks that utilize a collection of features based on lexicons and rules, it is important to adapt these features to the changing data. To address this challenge we introduce PyTAIL, a python library, which allows a human in the loop approach to actively train NLP models. PyTAIL enhances generic active learning, which only suggests new instances to label by also suggesting new features like rules and lexicons to label. Furthermor…

2022-11-24cs.AIcs.CLcs.HC

QueerBench：量化语言模型对酷儿身份的歧视程度

QueerBench: Quantifying Discrimination in Language Models Toward Queer Identities

本研究针对现有NLP偏见研究对酷儿群体关注不足的问题，提出基于模板和MLM任务的评估框架QueerBench，量化英文大语言模型对LGBTQIA+群体的歧视，发现模型对该群体的有害性得分差距达7.2%。

2024-06-18cs.AIcs.CLcs.CY

利用基于GPT的视觉语言模型生成牙槽囊肿的诊断发现：一种构建具有结构化输出（SLSO）框架的两阶段自校正循环的初步研究

Generating Findings for Jaw Cysts in Dental Panoramic Radiographs Using a GPT-Based VLM: A Preliminary Study on Building a Two-Stage Self-Correction Loop with Structured Output (SLSO) Framework

Gifu University（岐阜大学）；EyeTech Co., Ltd（EyeTech有限公司）；Asahi University（朝日大学）；Shiga University（滋贺大学）；The University of Osaka Dental Hospital（大阪大学齿科医院）

本文提出SLSO框架，通过整合图像分析、结构化数据生成等步骤，提升牙槽囊肿的AI诊断准确性。实验显示SLSO在牙数识别、牙移动检测等方面优于传统方法，但对多牙病变的识别仍有局限。

2026-04-20cs.AIcs.CV

更多相关论文收起更多论文

见你所需：通过推理-感知循环实现查询感知的视觉智能

See What You Need: Query-Aware Visual Intelligence through Reasoning-Perception Loops

College of Computer, National University of Defense Technology（计算机学院，国防科技大学）

针对长视频问答系统推理与感知解耦的局限，提出无需训练的闭环框架CAVIA，通过分层推理引导帧定位、跨模态语义桥接目标提取及置信度驱动迭代合成，在EgoSchema等三个基准上取得性能提升。

2025-08-25cs.AIcs.CV

Librarian-in-the-Loop: A Natural Language Processing Paradigm for Detecting Informal Mentions of Research Data in Academic Literature

University of Michigan（密歇根大学）；ICPSR（校际政治与社会研究联盟）

Data citations provide a foundation for studying research data impact. Collecting and managing data citations is a new frontier in archival science and scholarly communication. However, the discovery and curation of research data citations is labor intensive. Data citations that reference unique identifiers (i.e. DOIs) are readily findable; however, informal mentions made to research data are more challenging to infer. We propose a natural language processing (NLP) paradigm to support the human task of identifying informal mentions made to research datasets. The work of discovering informal da…

2022-03-10cs.CLcs.DLcs.HC

Human-in-the-Loop Refinement of Word Embeddings

Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）

Word embeddings are a fixed, distributional representation of the context of words in a corpus learned from word co-occurrences. Despite their proven utility in machine learning tasks, word embedding models may capture uneven semantic and syntactic representations, and can inadvertently reflect various kinds of bias present within corpora upon which they were trained. It has been demonstrated that post-processing of word embeddings to apply information found in lexical dictionaries can improve the semantic associations, thus improving their quality. Building on this idea, we propose a system t…

2021-10-06cs.CLcs.LG

HilMeMe: A Human-in-the-Loop Machine Translation Evaluation Metric Looking into Multi-Word Expressions

The University of Manchester（曼彻斯特大学）

With the fast development of Machine Translation (MT) systems, especially the new boost from Neural MT (NMT) models, the MT output quality has reached a new level of accuracy. However, many researchers criticised that the current popular evaluation metrics such as BLEU can not correctly distinguish the state-of-the-art NMT systems regarding quality differences. In this short paper, we describe the design and implementation of a linguistically motivated human-in-the-loop evaluation metric looking into idiomatic and terminological Multi-word Expressions (MWEs). MWEs have played a bottleneck in m…

2022-11-09cs.CLcs.HC

Human in the loop: How to effectively create coherent topics by manually labeling only a few documents per class

TU Clausthal（克劳斯塔尔工业大学）；University of Göttingen（哥廷根大学）

Few-shot methods for accurate modeling under sparse label-settings have improved significantly. However, the applications of few-shot modeling in natural language processing remain solely in the field of document classification. With recent performance improvements, supervised few-shot methods, combined with a simple topic extraction method pose a significant challenge to unsupervised topic modeling methods. Our research shows that supervised few-shot learning, combined with a simple topic extraction method, can outperform unsupervised topic modeling techniques in terms of generating coherent …

2022-12-19cs.CL