Prompt engineering is a challenging and important task due to the high sensitivity of Large Language Models (LLMs) to the given prompt and the inherent ambiguity of a textual task instruction. Automatic prompt engineering is essential to achieve optimized performance from LLMs. Recent studies have demonstrated the capabilities of LLMs to automatically conduct prompt engineering by employing a meta-prompt that incorporates the outcomes of the last trials and proposes an improved prompt. However, this requires a high-quality benchmark to compare different prompts, which is difficult and expensive to acquire in many real-world use cases. In this work, we introduce a new method for automatic prompt engineering, using a calibration process that iteratively refines the prompt to the user intent. During the optimization process, the system jointly generates synthetic data of boundary use cases and optimizes the prompt according to the generated dataset. We demonstrate the effectiveness of our method with respect to strong proprietary models on real-world tasks such as moderation and generation. Our method outperforms state-of-the-art methods with a limited number of annotated samples. Furthermore, we validate the advantages of each one of the system's key components. Our system is built in a modular way, facilitating easy adaptation to other tasks. The code is available $\href{https://github.com/Eladlev/AutoPrompt}{here}$.
Enhancing Speech Instruction Understanding and Disambiguation in Robotics via Speech Prosody
IT University of Copenhagen（丹麦技术大学）；University of Florida（佛罗里达大学）；Brown University（布朗大学）；Intelligent Agents Research Group（智能代理研究组）；Intelligent Robot Lab（智能机器人实验室）
ReflectDiffu:Reflect between Emotion-intent Contagion and Mimicry for Empathetic Response Generation via a RL-Diffusion Framework
University of Shanghai for Science and Technology（上海理工大学）；East China Normal University（华东师范大学）；Nanjing University of Information Science and Technology（南京信息工程大学）；Macquarie University（麦考瑞大学）
InterChat: Enhancing Generative Visual Analytics using Multimodal Interactions
University of California, Davis（加利福尼亚大学戴维斯分校）；Bosch Research North America（博世北美研究院）；Bosch Center for Artificial Intelligence (BCAI)（博世人工智能中心）；Robert Bosch GmbH（罗伯特·博世有限公司）
Should I Have Expressed a Different Intent? Counterfactual Generation for LLM-Based Autonomous Control
Department of Engineering, King's College London, London, UK（伦敦大学金史密斯学院工程系）；Intelligent Networked Systems Institute (INSI), Northeastern University, Boston, MA, USA（东北大学智能网络系统研究所）；Intelligent Networked Systems Institute (INSI), Northeastern University, London, UK（伦敦大学东北大学智能网络系统研究所）
Samsung R&D Institute（三星研发院）；Indian Institute of Technology（印度理工学院）
Recent joint intent detection and slot tagging models have seen improved performance when compared to individual models. In many real-world datasets, the slot labels and values have a strong correlation with their intent labels. In such cases, the intent label information may act as a useful feature to the slot tagging model. In this paper, we examine the effect of leveraging intent label features through 3 techniques in the slot tagging task of joint intent and slot detection models. We evaluate our techniques on benchmark spoken language datasets SNIPS and ATIS, as well as over a large priva…
LLM-Guided Semantic Relational Reasoning for Multimodal Intent Recognition
Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）；School of Information Science and Engineering, Hebei University of Science and Technology（河北科技大学信息科学与工程学院）
Data-driven methods have achieved notable performance on intent detection, which is a task to comprehend user queries. Nonetheless, they are controversial for over-confident predictions. In some scenarios, users do not only care about the accuracy but also the confidence of model. Unfortunately, mainstream neural networks are poorly calibrated, with a large gap between accuracy and confidence. To handle this problem defined as confidence calibration, we propose a model using the hyperspherical space and rebalanced accuracy-uncertainty loss. Specifically, we project the label vector onto hypers…
Intent classification is a major task in spoken language understanding (SLU). Since most models are built with pre-collected in-domain (IND) training utterances, their ability to detect unsupported out-of-domain (OOD) utterances has a critical effect in practical use. Recent works have shown that using extra data and labels can improve the OOD detection performance, yet it could be costly to collect such data. This paper proposes to train a model with only IND data while supporting both IND intent classification and OOD detection. Our method designs a novel domain-regularized module (DRM) to r…
Enhancing Target-Guided Proactive Dialogue Systems via Conversational Scenario Modeling and Intent-Keyword Bridging
School of Computer Science and Technology, Soochow University, China（苏州大学计算机科学与技术学院）；Jiangsu Key Lab of Language Computing, Suzhou（江苏省语言计算重点实验室）；School of Computer and Information Engineering, Henan Normal University, China（河南师范大学计算机与信息工程学院）
Intent classification has been widely researched on English data with deep learning approaches that are based on neural networks and word embeddings. The challenge for Chinese intent classification stems from the fact that, unlike English where most words are made up of 26 phonologic alphabet letters, Chinese is logographic, where a Chinese character is a more basic semantic unit that can be informative and its meaning does not vary too much in contexts. Chinese word embeddings alone can be inadequate for representing words, and pre-trained embeddings can suffer from not aligning well with the…
