This paper introduces a novel approach to enhancing closed-domain Question Answering (QA) systems, focusing on the specific needs of the Lawrence Berkeley National Laboratory (LBL) Science Information Technology (ScienceIT) domain. Utilizing a rich dataset derived from the ScienceIT documentation, our study embarks on a detailed comparison of two fine-tuned large language models and five retrieval-augmented generation (RAG) models. Through data processing techniques, we transform the documentation into structured context-question-answer triples, leveraging the latest Large Language Models (AWS Bedrock, GCP PaLM2, Meta LLaMA2, OpenAI GPT-4, Google Gemini-Pro) for data-driven insights. Additionally, we introduce the Aggregated Knowledge Model (AKM), which synthesizes responses from the seven models mentioned above using K-means clustering to select the most representative answers. The evaluation of these models across multiple metrics offers a comprehensive look into their effectiveness and suitability for the LBL ScienceIT environment. The results demonstrate the potential benefits of integrating fine-tuning and retrieval-augmented strategies, highlighting significant performance improvements achieved with the AKM. The insights gained from this study can be applied to develop specialized QA systems tailored to specific domains.
Optimizing Medical Question-Answering Systems: A Comparative Study of Fine-Tuned and Zero-Shot Large Language Models with RAG Framework
Department of Electrical Engineering;Computer Science University of Toledo Toledo, USA;Institute of Mathematical Sciences Claremont Graduate University Claremont, USA;Department of Bioengineering University of Toledo Toledo, USA;Department of Computer Science Bowling Green State University Bowling Green, USA
Retrieval-Augmented Detection of Potentially Abusive Clauses in Chilean Terms of Service
School of Computer Engineering, Pontificia Universidad Católica de Valparaíso(Pontificia Universidad Católica de Valparaíso计算机工程学院);Faculty of Law, Universidad Adolfo Ibáñez(Adolfo Ibáñez大学法学院)
MuISQA: Multi-Intent Retrieval-Augmented Generation for Scientific Question Answering
Zhongke Zidong Taichu (Beijing)(中科紫东太初(北京));Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Wuhan AI Research(武汉人工智能研究院)
TAGS: A Test-Time Generalist-Specialist Framework with Retrieval-Augmented Reasoning and Verification
Mohamed bin Zayed University of Artificial Intelligence(莫莫德·本·扎耶德人工智能大学);Monash University(墨尔本大学);Xi’an Jiaotong-Liverpool University(西安交通大学利物浦大学);University of Southampton(南安普顿大学)
Large Language Models (LLMs) as Agents for Augmented Democracy
University of Toulouse(图卢兹大学);Corvinus University of Budapest(布达佩斯考文纽斯大学);IRIT, Université Toulouse Capitole(图卢兹 Capitole 大学图卢兹计算机研究所);Toulouse School of Economics(图卢兹经济学院);Université Toulouse Capitole(图卢兹 Capitole 大学)
KG2QA: Knowledge Graph-enhanced Retrieval-augmented Generation for Communication Standards Question Answering
School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, China(香港中文大学(深圳)科学与工程学院);School of Microelectronics, Xi'an Jiaotong University, China(西安交通大学微电子学院)
LaB-RAG: Label Boosted Retrieval Augmented Generation for Radiology Report Generation
Department of Computer Science(计算机科学系);Center for Translational Data Science(转化数据科学中心);Medical Scientist Training Program(医学科学家培训计划);Section of Biomedical Data Science, Department of Medicine(医学部生物医学数据科学部门)
RAG-BioQA: A Retrieval-Augmented Generation Framework for Long-Form Biomedical Question Answering
Department of CSE Emory University Georgia, USA(计算机科学与工程系 埃默里大学 美国佐治亚州);Department of CSE SRM University AP Andhra Pradesh, India(计算机科学与工程系 SRM大学AP 安得拉邦印度);Department of CSE Trine University Michigan, USA(计算机科学与工程系 特林大学 密歇根州美国)
Domain Fine-Tuning vs. Retrieval-Augmented Generation for Medical Multiple-Choice Question Answering: A Controlled Comparison at the 4B-Parameter Scale
Department of Information Science and Applied Artificial Intelligence(信息科学与应用人工智能系)
VisRAG2.0: Mitigating Visual Hallucinations via Evidence-Guided Multi-Image Reasoning in Visual Retrieval-Augmented Generation
School of Software and Microelectronics, Peking University, China(北京大学软件与微电子学院);School of Computer Science and Engineering, Northeastern University, China(东北大学计算机科学与工程学院);Department of Computer Science and Technology, Institute for AI, Tsinghua University, China(清华大学人工智能研究院计算机科学与技术系)
Proof-of-TBI -- Fine-Tuned Vision Language Model Consortium and OpenAI-o3 Reasoning LLM-Based Medical Diagnosis Support System for Mild Traumatic Brain Injury (TBI) Prediction
Old Dominion University(旧 Dominion 大学);McDonald Army Health Center(麦克唐纳陆军医疗中心);BRAINBox Solutions(BRAINBox 解决方案);UVA Health(弗吉尼亚大学健康系统)