Enterprise searches require users to have complex knowledge of queries, configurations, and metadata, rendering it difficult for them to access information as needed. Most go-to-market (GTM) platforms utilize advanced search, an interface that enables users to filter queries by various fields using categories or keywords, which, historically, however, has proven to be exceedingly cumbersome, as users are faced with seemingly hundreds of options, fields, and buttons. Consequently, querying with natural language has long been ideal, a notion further empowered by Large Language Models (LLMs).
In this paper, we implement and evaluate a solution for the Zoominfo product for sellers, which prompts the LLM with natural language, producing search fields through entity extraction that are then converted into a search query. The intermediary search fields offer numerous advantages for each query, including the elimination of syntax errors, simpler ground truths, and an intuitive format for the LLM to interpret.
We paired this pipeline with many advanced prompt engineering strategies, featuring an intricate system message, few-shot prompting, chain-of-thought (CoT) reasoning, and execution refinement. Furthermore, we manually created the ground truth for 500+ natural language queries, enabling the supervised fine-tuning of Llama-3-8B-Instruct and the introduction of sophisticated numerical metrics.
Comprehensive experiments with closed, open source, and fine-tuned LLM models were conducted through exact, Jaccard, cosine, and semantic similarity on individual search entities to demonstrate the efficacy of our approach. Overall, the most accurate closed model had an average accuracy of 97% per query, with only one field performing under 90%, with comparable results observed from the fine-tuned models.
University of Warsaw(华沙大学);Warsaw University of Technology(华沙理工大学);Centre for Credible Artificial Intelligence (CCAI)(可信人工智能中心(CCAI));Group for Research in Applied Economics (GRAPE)(应用经济学研究组(GRAPE))
Harnessing the Potential of Large Language Models in Modern Marketing Management: Applications, Future Directions, and Strategic Recommendations
Iran University of Science & Technology(伊朗科学技术大学);Iran University of Medical Sciences(伊朗医科大学);Shahid Beheshti University of Medical Sciences(沙希德贝赫什提医科大学);Islamic Azad University(伊斯兰阿扎德大学);Alborz Vocational Technical University(厄尔布尔士职业技术大学)
Leveraging Synthetic Data for Question Answering with Multilingual LLMs in the Agricultural Domain
CSIR-Central Scientific Instruments Organisation, India(印度CSIR-中央科学仪器组织);Academy of Scientific and Innovative Research (AcSIR), Ghaziabad, U.P, India(印度科学与创新研究院(AcSIR));School of Mathematical and Computational Sciences, Massey University, New Zealand(梅西大学数学与计算科学学院)
Extract Information from Hybrid Long Documents Leveraging LLMs: A Framework and Dataset
Microsoft(微软公司);School of Software & Microelectronics, Peking University(北京大学软件与微电子学院);Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);Ant Group(蚂蚁集团)
Slide2Text: Leveraging LLMs for Personalized Textbook Generation from PowerPoint Presentations
East China Normal University(华东师范大学);Shanghai Institute of Artificial Intelligence for Education(上海人工智能教育研究院);School of Computer Science and Technology, East China Normal University(华东师范大学计算机科学与技术学院)
Strategic Innovation Management in the Age of Large Language Models Market Intelligence, Adaptive R&D, and Ethical Governance
Iran University of Science & Technology(伊朗科技大学);Sharif University of Technology(谢里夫理工大学);Iran University of Medical Sciences(伊朗医科大学);Shahid Beheshti University of Medical Sciences(沙希德·贝赫什提医科大学);Alborz Vocational Technical University(阿尔伯兹职业技术大学);Islamic Azad University(伊斯兰阿扎德大学)