Small Language Model Helps Resolve Semantic Ambiguity of LLM Prompt
School of Computer Science and Engineering, University of Electronic Science and Technology of China(电子科技大学计算机科学与工程学院);Department of Electronic Engineering, Kyung Hee University(庆熙大学电子工程系)
Zhenzhen Huang, Chaoning Zhang, Fachrina Dewi Puspitasari, Jiaquan Zhang, Yitian Zhou, Shuxu Chen, Yang Yang
Large language models (LLMs) are increasingly utilized in various complex reasoning tasks due to their excellent instruction following capability. However, the model's performance is highly dependent on the open-ended characteristics of the users' input prompt. Natural prompts often do not follow proper syntactic rules, which creates ambiguous queries that yield multiple interpretations. Such ambiguous prompts confuse the model in choosing the correct reasoning paths to answer questions. Prior works address this challenge by applying query editing during the LLM inference process without explicitly solving the root cause of the ambiguity. To address this limitation, we propose a pre-inference prompt optimization mechanism via explicit prompt disambiguation. Particularly, we identify semantic risks in the prompt, check their multi-perspective consistency, and resolve any semantic conflicts that arise. Finally, we organize the resolved ambiguities in a logically structured manner as a clean input to the LLM. By explicitly resolving semantic ambiguity, our method can produce a more focused attention distribution to the semantically essential tokens. We also leverage small language models (SLMs) as the main executor of prompt disambiguation to benefit from their efficient computation. Through comprehensive experiments on multiple benchmarks, we demonstrate that our method improves reasoning performance by 2.5 points at a cost of only \$0.02. Our study promotes explicit prompt disambiguation as an effective prompt optimization method without disturbing the internal mechanism of LLM inference.
Augmenters at SemEval-2023 Task 1: Enhancing CLIP in Handling Compositionality and Ambiguity for Zero-Shot Visual WSD through Prompt Augmentation and Text-To-Image Diffusion
University of Maryland, College Park(马里兰大学帕克分校);Duolingo, Inc.(多邻国公司)
To Know or Not To Know? Analyzing Self-Consistency of Large Language Models under Ambiguity
LMU Munich(慕尼黑大学);Faculty of Philological and Cultural Studies, University of Vienna(维也纳大学语言与文化学院);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);UniVie Doctoral School Computer Science(维也纳大学计算机博士学院)
Evaluating Chinese Ambiguity Understanding in Large Language Models
Graduate School of Information Science and Technology, The University of Tokyo(东京大学信息科学与技术研究生院);School of Software Engineering, South China University of Technology(华南理工大学软件学院)
Jury Duty: Calibration and Orientation Failures in MLLM-as-a-Judge Under Cultural Ambiguity
Salesforce AI Research(Salesforce AI 研究院);University of Cambridge(剑桥大学);University of Colorado Boulder(科罗拉多大学博尔德分校);Carnegie Mellon University(卡内基梅隆大学);UIUC(伊利诺伊大学厄巴纳-香槟分校)
BioCoref: Benchmarking Biomedical Coreference Resolution with LLMs
Computational Bioscience Program University of Colorado Anschutz Medical Campus(科学生物学程序,科罗拉多大学安舒茨医学校区);Department of Pediatrics University of Chicago(儿科学系,芝加哥大学)
EchoPilot: Training-Free Ultrasound Video Segmentation via Scale-Space Semantic Prompting and Reliability-Gated Memory
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Third Affiliated Hospital of Sun Yat-Sen University(中山大学第三附属医院);Hong Kong Metropolitan University(香港 Metropolitan 大学)
Trick or Neat: Adversarial Ambiguity and Language Model Evaluation
University of Copenhagen(哥本哈根大学);Technische Universität Berlin(柏林技术大学);Massachusetts Institute of Technology(麻省理工学院);Aleph Alpha Research(Aleph Alpha研究)
Calibrated Ambiguity in Multimodal Language Models: Humans reach for cultural references, while models describe the picture
The Alan Turing Institute(艾伦·图灵研究所);Technical University of Darmstadt(达姆施塔特工业大学);University of Southampton(南安普顿大学);University of Chicago(芝加哥大学);University of Sheffield(谢菲尔德大学);University of Edinburgh(爱丁堡大学)
Visual Word Sense Disambiguation with CLIP through Dual-Channel Text Prompting and Image Augmentations
2 The Bredesen Center for Interdisciplinary Research;3 Department of Electrical Engineering \& Computer Science, University of Tennessee, Knoxville, TN 37916;4 Oak Ridge National Laboratory (ORNL), Oak Ridge, TN 37830