Transformer-based QA models use input-wide self-attention -- i.e. across both the question and the input passage -- at all layers, causing them to be slow and memory-intensive. It turns out that we can get by without input-wide self-attention at all layers, especially in the lower layers. We introduce DeFormer, a decomposed transformer, which substitutes the full self-attention with question-wide and passage-wide self-attentions in the lower layers. This allows for question-independent processing of the input text representations, which in turn enables pre-computing passage representations reducing runtime compute drastically. Furthermore, because DeFormer is largely similar to the original model, we can initialize DeFormer with the pre-training weights of a standard transformer, and directly fine-tune on the target QA dataset. We show DeFormer versions of BERT and XLNet can be used to speed up QA by over 4.3x and with simple distillation-based losses they incur only a 1% drop in accuracy. We open source the code at https://github.com/StonyBrookNLP/deformer.
Tel-Aviv University（特拉维夫大学）；Allen Institute for AI（艾伦人工智能研究所）
Answering questions that involve multi-step reasoning requires decomposing them and using the answers of intermediate steps to reach the final answer. However, state-of-the-art models in grounded question answering often do not explicitly perform decomposition, leading to difficulties in generalization to out-of-distribution examples. In this work, we propose a model that computes a representation and denotation for all question spans in a bottom-up, compositional manner using a CKY-style parser. Our model induces latent trees, driven by end-to-end (the answer) supervision only. We show that t…
Facebook AI Research（脸书人工智能研究院）；New York University（纽约大学）；University College London（伦敦大学学院）；CIFAR Azrieli Global Scholar（加拿大高级研究院阿兹列利全球学者项目）
We aim to improve question answering (QA) by decomposing hard questions into simpler sub-questions that existing QA systems are capable of answering. Since labeling questions with decompositions is cumbersome, we take an unsupervised approach to produce sub-questions, also enabling us to leverage millions of questions from the internet. Specifically, we propose an algorithm for One-to-N Unsupervised Sequence transduction (ONUS) that learns to map one hard, multi-hop question to many simpler, single-hop sub-questions. We answer sub-questions with an off-the-shelf QA model and give the resulting…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Science and Technology of China（中国科学技术大学）；Bytedance Inc（字节跳动公司）
Answering complex questions involving multiple entities and relations is a challenging task. Logically, the answer to a complex question should be derived by decomposing the complex question into multiple simple sub-questions and then answering those sub-questions. Existing work has followed this strategy but has not attempted to optimize the order of how those sub-questions are answered. As a result, the sub-questions are answered in an arbitrary order, leading to larger search space and a higher risk of missing an answer. In this paper, we propose a novel reinforcement learning(RL) approach …
keqing: knowledge-based question answering is a nature chain-of-thought mentor of LLM
National Key Laboratory of Radar Signal Processing（雷达信号处理国家重点实验室）；Xidian University（西安电子科技大学）；School of Computer Science and Engineering（计算机科学与工程学院）；Nanyang Technological University（南洋理工大学）
Effective multi-hop question answering (QA) requires reasoning over multiple scattered paragraphs and providing explanations for answers. Most existing approaches cannot provide an interpretable reasoning process to illustrate how these models arrive at an answer. In this paper, we propose a Question Decomposition method based on Abstract Meaning Representation (QDAMR) for multi-hop QA, which achieves interpretable reasoning by decomposing a multi-hop question into simpler sub-questions and answering them in order. Since annotating the decomposition is expensive, we first delegate the complexi…
Facebook AI（脸书人工智能研究院）；University of Cambridge（剑桥大学）
Policy compliance detection is the task of ensuring that a scenario conforms to a policy (e.g. a claim is valid according to government rules or a post in an online platform conforms to community guidelines). This task has been previously instantiated as a form of textual entailment, which results in poor accuracy due to the complexity of the policies. In this paper we propose to address policy compliance detection via decomposing it into question answering, where questions check whether the conditions stated in the policy apply to the scenario, and an expression tree combines the answers to o…
Georgia Institute of Technology（佐治亚理工学院）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）
Multi-hop Question Answering (QA) is a challenging task since it requires an accurate aggregation of information from multiple context paragraphs and a thorough understanding of the underlying reasoning chains. Recent work in multi-hop QA has shown that performance can be boosted by first decomposing the questions into simpler, single-hop questions. In this paper, we explore one additional utility of the multi-hop decomposition from the perspective of explainable NLP: to create explanation by probing a neural QA model with them. We hypothesize that in doing so, users will be better able to pre…
Multi-hop Question Answering (QA) requires the machine to answer complex questions by finding scattering clues and reasoning from multiple documents. Graph Network (GN) and Question Decomposition (QD) are two common approaches at present. The former uses the "black-box" reasoning process to capture the potential relationship between entities and sentences, thus achieving good performance. At the same time, the latter provides a clear reasoning logical route by decomposing multi-hop questions into simple single-hop sub-questions. In this paper, we propose a novel method to complete multi-hop QA…
We propose a general framework called Text Modular Networks(TMNs) for building interpretable systems that learn to solve complex tasks by decomposing them into simpler ones solvable by existing models. To ensure solvability of simpler tasks, TMNs learn the textual input-output behavior (i.e., language) of existing models through their datasets. This differs from prior decomposition-based approaches which, besides being designed specifically for each complex task, produce decompositions independent of existing sub-models. Specifically, we focus on Question Answering (QA) and show how to train a…
University of California, Irvine（加州大学欧文分校）；Microsoft Semantic Machines（微软语义机器）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）
Answering complex questions that require making latent decisions is a challenging task, especially when limited supervision is available. Recent works leverage the capabilities of large language models (LMs) to perform complex question answering in a few-shot setting by demonstrating how to output intermediate rationalizations while solving the complex question in a single pass. We introduce ``Successive Prompting'', where we iteratively break down a complex task into a simple task, solve it, and then repeat the process until we get the final solution. Successive prompting decouples the superv…
T$^2$: An Adaptive Test-Time Scaling Strategy for Contextual Question Answering
The Chinese University of Hong Kong（香港中文大学）；University of International Relations（国际关系大学）；Jarvis Research Center, Tencent YouTu Lab（腾讯YouTu实验室）；Westlake University（西湖大学）
Imperial College London（帝国理工学院）；University of Cambridge（剑桥大学）；Meta AI
Policy Compliance Detection (PCD) is a task we encounter when reasoning over texts, e.g. legal frameworks. Previous work to address PCD relies heavily on modeling the task as a special case of Recognizing Textual Entailment. Entailment is applicable to the problem of PCD, however viewing the policy as a single proposition, as opposed to multiple interlinked propositions, yields poor performance and lacks explainability. To address this challenge, more recent proposals for PCD have argued for decomposing policies into expression trees consisting of questions connected with logic operators. Ques…
Facebook AI Research（脸书人工智能研究院）；University College London（伦敦大学学院）；LORIA（洛林计算机及其应用实验室）；University of Cambridge（剑桥大学）
Fact checking at scale is difficult -- while the number of active fact checking websites is growing, it remains too small for the needs of the contemporary media ecosystem. However, despite good intentions, contributions from volunteers are often error-prone, and thus in practice restricted to claim detection. We investigate how to increase the accuracy and efficiency of fact checking by providing information about the claim before performing the check, in the form of natural language briefs. We investigate passage-based briefs, containing a relevant passage from Wikipedia, entity-centric ones…
Educational Question Generation of Children Storybooks via Question Type Distribution Learning and Event-Centric Summarization
IBM Research Europe（IBM欧洲研究院）；IBM Research（IBM研究院）；WeChat AI（微信AI）；The Hong Kong University of Science and Technology（香港科技大学）；Nanjing University of Information Science and Technology（南京信息工程大学）；Nankai University（南开大学）
Decomposing and Revising What Language Models Generate
School of Computer and Information Technology, Shanxi University, Taiyuan, China（计算机与信息科技学院，山西大学，太原，中国）；Department of Computer Science, University of Manchester, Manchester, England（计算机科学系，曼彻斯特大学，曼彻斯特，英国）；Beijing University of Technology, Beijing, China（北京理工大学，北京，中国）；Singapore University of Technology and Design（新加坡科技与设计大学）；ILCC, School of Informatics, University of Edinburgh, Edinburgh, England（ILCC，信息学院，爱丁堡大学，爱丁堡，英国）
