Despite the successes of pretrained language models, there are still few high-quality, general-purpose QA systems that are freely available. In response, we present Macaw, a versatile, generative question-answering (QA) system that we are making available to the community. Macaw is built on UnifiedQA, itself built on T5, and exhibits strong performance, zero-shot, on a wide variety of topics, including outperforming GPT-3 by over 10% (absolute) on Challenge300, a suite of 300 challenge questions, despite being an order of magnitude smaller (11 billion vs. 175 billion parameters). In addition, Macaw allows different permutations ("angles") of its inputs and outputs to be used, for example Macaw can take a question and produce an answer; or take an answer and produce a question; or take an answer and question, and produce multiple-choice options. We describe the system, and illustrate a variety of question types where it produces surprisingly good answers, well outside the training setup. We also identify question classes where it still appears to struggle, offering insights into the limitations of pretrained language models. Macaw is freely available, and we hope that it proves useful to the community. Macaw is available at https://github.com/allenai/macaw
Ontology-Guided Reverse Thinking Makes Large Language Models Stronger on Knowledge Graph Question Answering
Harbin Institute of Technology(哈尔滨工业大学);Beijing University of Posts and Telecommunications(北京邮电大学);Joint Laboratory of HIT and iFLYTEK(哈工大与科大讯飞联合实验室);University of Science and Technology of China(中国科学技术大学);Pengcheng Laboratory(鹏城实验室)
Empowering LLMs with Logical Reasoning: A Comprehensive Survey
Institute for Logic, Language and Computation, University of Amsterdam(阿姆斯特丹大学逻辑、语言与计算研究所);Center for Data Science, Peking University(北京大学数据科学中心);Tsinghua-UvA JRC for Logic, Department of Philosophy, Tsinghua University(清华大学哲学系);Machine Learning Department, MBZUAI(MBZUAI机器学习部门);Department of Philosophy, CMU(卡内基梅隆大学哲学系);State Key Lab of General AI, School of Intelligence Science and Technology, Peking University(北京大学人工智能通用实验室);Institute for Artificial Intelligence, Peking University(北京大学人工智能研究所);Pazhou Laboratory (Huangpu), Guangzhou(广州黄埔实验室)
Leveraging Synthetic Data for Question Answering with Multilingual LLMs in the Agricultural Domain
CSIR-Central Scientific Instruments Organisation, India(印度CSIR-中央科学仪器组织);Academy of Scientific and Innovative Research (AcSIR), Ghaziabad, U.P, India(印度科学与创新研究院(AcSIR));School of Mathematical and Computational Sciences, Massey University, New Zealand(梅西大学数学与计算科学学院)
Question Answering as Global Reasoning over Semantic Abstractions
University of Illinois, Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Pennsylvania(宾夕法尼亚大学);Allen Institute for Artificial Intelligence (AI2)(艾伦人工智能研究所(AI2))
RGAR: Recurrence Generation-augmented Retrieval for Factual-aware Medical Question Answering
King’s College London(伦敦国王学院);School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University(上海交通大学电子信息与电气工程学院);School of Electrical and Computer Engineering, Carnegie Mellon University(卡内基梅隆大学电气与计算机工程学院)
First Token Probability Guided RAG for Telecom Question Answering
Shenzhen Research Institute of Big Data(深圳大数据研究院);School of Science and Engineering, The Chinese University of Hong Kong (Shenzhen)(香港中文大学(深圳)理工学院);School of Computer Science and Engineering, Sun Yat-sen University(中山大学计算机科学与工程学院)
Answering real-world clinical questions using large language model based systems
Atropos Health(阿特罗波斯健康公司);University of California, Los Angeles(加州大学洛杉矶分校);The Hospital for Sick Children(病童医院);University of California, San Diego(加州大学圣地亚哥分校);University of Michigan(密歇根大学);Columbia University(哥伦比亚大学);Stanford University(斯坦福大学)
CoE-Ops: Collaboration of LLM-based Experts for AIOps Question-Answering
SKLCCSE, Institute of Artificial Intelligence, Beihang University(SKLCCSE,人工智能学院,北航);Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University(未来区块链与隐私计算先进创新中心,北航)
Automated Regulatory Compliance Question Answering in Financial Services with Domain-Adapted Retrieval-Augmented Generation
University of Bonn(波恩大学);Lamarr-Institute for Machine Learning and Artificial Intelligence(拉马尔机器学习和人工智能研究所);Universitat Rovira i Virgili(罗维拉-维尔吉利大学);Fraunhofer IAIS(弗劳恩霍夫智能分析和信息系统研究所)
Structured Outputs Enable General-Purpose LLMs to be Medical Experts
University of Bristol(布里斯托大学);Lehigh University(利哈伊大学);The University of Queensland(昆士兰大学);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);University of California, Merced(加利福尼亚大学默塞德分校)
A Systematic Study of Retrieval Pipeline Design for Retrieval-Augmented Medical Question Answering
Department of Mechatronics & Industrial Engineering, Chittagong University of Engineering & Technology(吉大港工程与技术大学机电与工业工程系);Department of Mechanical Engineering, Chittagong University of Engineering & Technology(吉大港工程与技术大学机械工程系)
HealthBranches: Synthesizing Clinically-Grounded Question Answering Datasets via Decision Pathways
University of Calabria(卡拉布里亚大学);University of Catanzaro(卡坦扎罗大学);University of Eastern Piedmont(东皮埃蒙特大学);University of Pisa(比萨大学);University of Cambridge(剑桥大学)
IMB: An Italian Medical Benchmark for Question Answering
University of Naples Federico II, Department of Electrical Engineering(那不勒斯费德里科二世大学电子工程系);Consorzio Interuniversitario Nazionale per l'Informatica (CINI) - ITEM National Lab(国家信息计算联合研究中心(CINI)- ITEM国家实验室);Northwestern University, Department of Computer Science, McCormick School of Engineering(西北大学计算机科学系,工程学院)
Polish Medical Visual Question Answering: Vision-Language Models Underutilize Visual Evidence
ARAAI Poland(波兰ARAAI);NASK National Research Institute(NASK国家研究院);Poznań University of Medical Sciences(波兹南医科大学);T. Marciniak Lower Silesian Specialist Hospital(T.马尔奇尼亚克下西里西亚专科医院);Adam Mickiewicz University(亚当·密茨凯维奇大学)