In this paper, we explore the challenges inherent to Large Language Models (LLMs) like GPT-4, particularly their propensity for hallucinations, logic mistakes, and incorrect conclusions when tasked with answering complex questions. The capacity of LLMs to present erroneous answers in a coherent and semantically rigorous manner further complicates the detection of factual inaccuracies. This issue is especially pronounced in fields that require specialized expertise. Our work delves into these challenges, aiming to enhance the understanding and mitigation of such errors, thereby contributing to the improvement of LLM accuracy and reliability in scientific and other specialized domains. Our findings reveal a non-linear relationship between the context's relevancy and the answers' measured quality. In addition, we demonstrate that with the correct calibration, it is possible to automate the grading procedure -- a finding suggesting that, at least to some degree, the LLMs can be used to self-examine the quality of their own performance. Finally, we describe an experimental platform that can be seen as a proof-of-concept of the techniques described in this work.
DALK: Dynamic Co-Augmentation of LLMs and KG to answer Alzheimer's Disease Questions with Scientific Literature
School of Computing, and Augmented Intelligence, Arizona State University(亚利桑那州立大学计算与增强智能学院);University of Pennsylvania Perelman School of Medicine(宾夕法尼亚大学佩雷尔曼医学院);Department of Computer Science, The University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校计算机系);Department of Mathematics, United States Naval Academy(美国海军学院数学系);School of Information, The University of Texas at Austin(德克萨斯大学奥斯汀分校信息学院)
Order Doesn't Matter, But Reasoning Does: Training LLMs with Order-Centric Augmentation
Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University(上海数据科学关键实验室,计算机科学学院,复旦大学);School of Data Science, Fudan University(数据科学学院,复旦大学);Ant Group(蚂蚁集团)
Reconnecting Fragmented Citation Networks with Semantic Augmentation
Digital Data and Information for Society, Science, and Culture, Zuse Institute Berlin(数字数据与信息社会、科学与文化,柏林祖布研究所);Institute of Mathematics, Vietnam Academy of Science and Technology(越南科学技术 academy 数学研究所);Software and Algorithms for Discrete Optimization, Technische Universität Berlin(离散优化软件与算法,柏林技术大学);Applied Optimization, Zuse Institute Berlin(应用优化,柏林祖布研究所);Kooperativer Bibliotheksverbund Berlin-Brandenburg (KOBV), Zuse Institute Berlin(柏林-勃兰登堡合作图书馆联合会 (KOBV),柏林祖布研究所)
Skeletons Matter: Dynamic Data Augmentation for Text-to-Query
School of Data Science, Fudan University(复旦大学数据科学学院);School of Computer Science and Technology, Donghua University(东华大学计算机科学与技术学院);College of Computer Science and Artificial Intelligence, Fudan University(复旦大学计算机科学与人工智能学院);Ant Group(蚂蚁集团);Shanghai Key Laboratory of Data Science(上海数据科学 key laboratory)
Improving Scientific Document Retrieval with Academic Concept Index
Korea University Seoul South Korea;University of Illinois Urbana-Champaign Champaign United States;Texas A\&M University College Station United States;Yonsei University Seoul South Korea;Pohang University of Science;Korea University;University of Illinois Urbana-Champaign;Texas A\&M University;Yonsei University
Mining Reasons For And Against Vaccination From Unstructured Data Using Nichesourcing and AI Data Augmentation
Universidad de Buenos Aires(布宜诺斯艾利斯大学);Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)(国家科学技术研究委员会(CONICET));Ludwig-Maximilians-Universität München(慕尼黑大学);Quantit;Universidad Torcuato Di Tella(托库阿托·迪·特拉大学)
OhioState at SemEval-2018 Task 7: Exploiting Data Augmentation for Relation Classification in Scientific Papers using Piecewise Convolutional Neural Networks
Artificial Intuition: Efficient Classification of Scientific Abstracts
Information Sciences Institute, Viterbi School of Engineering, University of Southern California(南加州大学维特比工程学院信息科学研究所);Sol Price School of Public Policy, University of Southern California(南加州大学索尔·普莱斯公共政策学院)
本文提出利用 LLM 生成元数据以模拟人类直觉补充知识的工作流程,实现对 NASA 奖项摘要等短篇科学文本的粗略领域分类,并配套新评估工具与既有指标。
MatSciBench: Benchmarking the Reasoning Ability of Large Language Models in Materials Science
University of California, Los Angeles Computer Science Department(加州大学洛杉矶分校计算机科学系);University of Pennsylvania Department of Materials Science and Engineering(宾夕法尼亚大学材料科学与工程系);Virginia Tech Department of Computer Science(弗吉尼亚理工大学计算机科学系)
Artificial Intelligence Algorithms for the Detection of Pathologies Related to Lung Cancer through Image Analysis using Convolutional Neural Networks and Data Augmentation: a systematic mapping of the literature
The Impact of Synthetic Data Augmentation on Discourse-Pragmatic Function Classification
Institute of Linguistics(语言学研究所);College of Liberal Arts and Sciences, University of Florida(佛罗里达大学文理学院);Faculty of Arts and Humanities, Heinrich Heine University Düsseldorf(杜塞尔多夫海因里希·海涅大学艺术与人文学院);Sun Yat-sen University(中山大学)