This paper studies the performances of BERT combined with tree structure in short sentence ranking task. In retrieval-based question answering system, we retrieve the most similar question of the query question by ranking all the questions in datasets. If we want to rank all the sentences by neural rankers, we need to score all the sentence pairs. However it consumes large amount of time. So we design a specific tree for searching and combine deep model to solve this problem. We fine-tune BERT on the training data to get semantic vector or sentence embeddings on the test data. We use all the sentence embeddings of test data to build our tree based on k-means and do beam search at predicting time when given a sentence as query. We do the experiments on the semantic textual similarity dataset, Quora Question Pairs, and process the dataset for sentence ranking. Experimental results show that our methods outperform the strong baseline. Our tree accelerate the predicting speed by 500%-1000% without losing too much ranking accuracy.
George Mason University（乔治梅森大学）；University of Virginia（弗吉尼亚大学）
Recent work by Søgaard (2020) showed that, treebank size aside, overlap between training and test graphs (termed leakage) explains more of the observed variation in dependency parsing performance than other explanations. In this work we revisit this claim, testing it on more models and languages. We find that it only holds for zero-shot cross-lingual settings. We then propose a more fine-grained measure of such leakage which, unlike the original measure, not only explains but also correlates with observed performance variation. Code and data are available here: https://github.com/miriamwanner/…
Though linguistic knowledge emerges during large-scale language model pretraining, recent work attempt to explicitly incorporate human-defined linguistic priors into task-specific fine-tuning. Infusing language models with syntactic or semantic knowledge from parsers has shown improvements on many language understanding tasks. To further investigate the effectiveness of structural linguistic priors, we conduct empirical study of replacing parsed graphs or trees with trivial ones (rarely carrying linguistic knowledge e.g., balanced tree) for tasks in the GLUE benchmark. Encoding with trivial gr…
Northwestern University（西北大学）；AWS AI Labs（亚马逊云科技人工智能实验室）
Large language models have achieved high performance on various question answering (QA) benchmarks, but the explainability of their output remains elusive. Structured explanations, called entailment trees, were recently suggested as a way to explain and inspect a QA system's answer. In order to better generate such entailment trees, we propose an architecture called Iterative Retrieval-Generation Reasoner (IRGR). Our model is able to explain a given hypothesis by systematically generating a step-by-step explanation from textual premises. The IRGR model iteratively searches for suitable premise…
Improving TCM Question Answering through Tree-Organized Self-Reflective Retrieval with LLMs
School of Basic Medical Sciences, Zhejiang Chinese Medical University（浙江中医药大学基础医学院）；Breast Disease Specialist Hospital of Guangdong Provincial Hospital of Chinese Medicine（广东省中医院乳腺疾病专科医院）；Chinese Medicine Artificial Intelligence Joint Engineering Center, Zhejiang Chinese Medical University（浙江中医药大学中医药人工智能联合工程中心）；Faculty of Chinese Medicine, Macau University of Science and Technology（澳门科技大学中医药学院）
Open-domain question answering (QA) is the tasl of identifying answers to natural questions from a large corpus of documents. The typical open-domain QA system starts with information retrieval to select a subset of documents from the corpus, which are then processed by a machine reader to select the answer spans. This paper describes Mindstone, an open-domain QA system that consists of a new multi-stage pipeline that employs a traditional BM25-based information retriever, RM3-based neural relevance feedback, neural ranker, and a machine reading comprehension stage. This paper establishes a ne…
UC San Diego（加州大学圣地亚哥分校）；Princeton University（普林斯顿大学）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）；UMass Amherst（马萨诸塞大学阿默斯特分校）；Tencent（腾讯）；MIT（麻省理工学院）；WeChat AI（微信人工智能）
Text games present opportunities for natural language understanding (NLU) methods to tackle reinforcement learning (RL) challenges. However, recent work has questioned the necessity of NLU by showing random text hashes could perform decently. In this paper, we pursue a fine-grained investigation into the roles of text in the face of different RL challenges, and reconcile that semantic and non-semantic language representations could be complementary rather than contrasting. Concretely, we propose a simple scheme to extract relevant contextual information into an approximate state hash as extra …
The University of Edinburgh（爱丁堡大学）；University of Aveiro（阿维罗大学）；National Technical University of Athens（雅典技术大学）；Archimedes/Athena Research Centre（阿基米德/雅典研究中心）；The Artificial Intelligence Applications Institute, The University of Edinburgh（人工智能应用研究所，爱丁堡大学）
