arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2311.05754cs.CLcs.AI
收藏

面向可解释预测的深度自然语言特征学习

Deep Natural Language Feature Learning for Interpretable Prediction

Centro Nacional de Inteligencia Artificial(国家人工智能中心);Universidad de Chile(智利大学)

Felipe Urrutia, Cristian Buc, Valentin Barriere

AI总结:

该研究提出将复杂任务拆解为自然语言二元子任务的NLLF特征学习方法,用LLM生成弱标签训练BERT生成可解释特征,可增强分类器性能并支撑高表现可解释决策树,已在两类任务中验证有效。

摘要:

我们提出一种通用方法,可将一项复杂的主任务拆解为一系列更简单的中间子任务,这些子任务以自然语言形式被表述为与最终目标任务相关的二元问题。该方法能够用由这些问题的答案组成的向量来表示每个样本,我们将这种表示命名为Natural Language Learned Features(自然语言学习特征,简称NLLF)。NLLF由小型Transformer语言模型(例如BERT)生成,该模型以Natural Language Inference(自然语言推理,简称NLI)的方式训练,所用的弱标签由Large Language Model(大语言模型,简称LLM)自动获取。我们发现,LLM在通过上下文学习处理主任务时通常表现不佳,但能够处理这些更简单的子任务,并生成有用的弱标签以训练BERT。对BERT的类NLI训练使其能够应对任意二元问题的零样本推理,而无需局限于训练期间见过的问题。我们证明,该NLLF向量不仅能通过增强任意分类器来提升性能,还可作为决策树这类易解释的机器学习模型的输入。这种决策树具备可解释性,同时还能达到很高的性能,在部分场景中超过预训练Transformer的表现。我们已将该方法成功应用于两项完全不同的任务:检测学生在开放式数学考试问题作答中的不一致性,以及为气候变化与农业生态学相关科学论文的系统文献综述筛选摘要。

英文摘要:

We propose a general method to break down a main complex task into a set of intermediary easier sub-tasks, which are formulated in natural language as binary questions related to the final target task. Our method allows for representing each example by a vector consisting of the answers to these questions. We call this representation Natural Language Learned Features (NLLF). NLLF is generated by a small transformer language model (e.g., BERT) that has been trained in a Natural Language Inference (NLI) fashion, using weak labels automatically obtained from a Large Language Model (LLM). We show that the LLM normally struggles for the main task using in-context learning, but can handle these easiest subtasks and produce useful weak labels to train a BERT. The NLI-like training of the BERT allows for tackling zero-shot inference with any binary question, and not necessarily the ones seen during the training. We show that this NLLF vector not only helps to reach better performances by enhancing any classifier, but that it can be used as input of an easy-to-interpret machine learning model like a decision tree. This decision tree is interpretable but also reaches high performances, surpassing those of a pre-trained transformer in some cases.We have successfully applied this method to two completely different tasks: detecting incoherence in students' answers to open-ended mathematics exam questions, and screening abstracts for a systematic literature review of scientific papers on climate change and agroecology.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑