The future of conversational agents will provide users with personalized information responses. However, a significant challenge in developing models is the lack of large-scale dialogue datasets that span multiple sessions and reflect real-world user preferences. Previous approaches rely on experts in a wizard-of-oz setup that is difficult to scale, particularly for personalized tasks. Our method, LAPS, addresses this by using large language models (LLMs) to guide a single human worker in generating personalized dialogues. This method has proven to speed up the creation process and improve quality. LAPS can collect large-scale, human-written, multi-session, and multi-domain conversations, including extracting user preferences. When compared to existing datasets, LAPS-produced conversations are as natural and diverse as expert-created ones, which stays in contrast with fully synthetic methods. The collected dataset is suited to train preference extraction and personalized response generation. Our results show that responses generated explicitly using extracted preferences better match user's actual preferences, highlighting the value of using extracted preferences over simple dialogue history. Overall, LAPS introduces a new method to leverage LLMs to create realistic personalized conversational data more efficiently and effectively than previous methods.
ConvSDG: Session Data Generation for Conversational Search
Université de Montréal（蒙特利尔大学）；RALI（RALI（机器翻译与计算语言学研究实验室））；Renmin University of China（中国人民大学）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；Beijing Jiaotong University（北京交通大学）
A personalized conversational sales agent could have much commercial potential. E-commerce companies such as Amazon, eBay, JD, Alibaba etc. are piloting such kind of agents with their users. However, the research on this topic is very limited and existing solutions are either based on single round adhoc search engine or traditional multi round dialog system. They usually only utilize user inputs in the current session, ignoring users' long term preferences. On the other hand, it is well known that sales conversion rate can be greatly improved based on recommender systems, which learn user pref…
Leibniz Universität Hannover（汉诺威莱布尼茨大学）；RMIT University（皇家墨尔本理工大学）；MLU Halle（哈雷马丁路德大学）；University of Tsukuba（筑波大学）；Bauhaus-Universität Weimar（魏玛包豪斯大学）
Dagstuhl Seminar 19461 "Conversational Search" was held on 10-15 November 2019. 44~researchers in Information Retrieval and Web Search, Natural Language Processing, Human Computer Interaction, and Dialogue Systems were invited to share the latest development in the area of Conversational Search and discuss its research agenda and future directions. A 5-day program of the seminar consisted of six introductory and background sessions, three visionary talk sessions, one industry talk session, and seven working groups and reporting sessions. The seminar also had three social events during the prog…
ChatRetriever: Adapting Large Language Models for Generalized and Robust Conversational Dense Retrieval
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Université de Montréal（蒙特利尔大学）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；Waseda University（早稻田大学）
University of Chinese Academy of Sciences（中国科学院大学）；Tencent（腾讯）；Beihang University（北京航空航天大学）；Institute of Intelligent Computing Technology（智能计算技术研究所）
Conversational recommender systems (CRS) aim to provide highquality recommendations in conversations. However, most conventional CRS models mainly focus on the dialogue understanding of the current session, ignoring other rich multi-aspect information of the central subjects (i.e., users) in recommendation. In this work, we highlight that the user's historical dialogue sessions and look-alike users are essential sources of user preferences besides the current dialogue session in CRS. To systematically model the multi-aspect information, we propose a User-Centric Conversational Recommendation (…
Rapid response, namely low latency, is fundamental in search applications; it is particularly so in interactive search sessions, such as those encountered in conversational settings. An observation with a potential to reduce latency asserts that conversational queries exhibit a temporal locality in the lists of documents retrieved. Motivated by this observation, we propose and evaluate a client-side document embedding cache, improving the responsiveness of conversational search systems. By leveraging state-of-the-art dense retrieval models to abstract document and query semantics, we cache the…
Where to Look and What to Use: Retrieve-Localize-Generate for Long-Term Conversational Memory Question Answering
University of Electronic Science and Technology of China（电子科技大学）；Tsinghua University（清华大学）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Xiaomi Inc.（小米公司）；MiLM Plus
University of Strathclyde（思克莱德大学）；University of Amsterdam（阿姆斯特丹大学）
Various conceptual and descriptive models of conversational search have been proposed in the literature -- while useful, they do not provide insights into how interaction between the agent and user would change in response to the costs and benefits of the different interactions. In this paper, we develop two economic models of conversational search based on patterns previously observed during conversational search sessions, which we refer to as: Feedback First where the agent asks clarifying questions then presents results, and Feedback After where the agent presents results, and then asks fol…
University of Montreal（蒙特利尔大学）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；Renmin University of China（中国人民大学）；Beihang University（北京航空航天大学）；University of Copenhagen（哥本哈根大学）；Beijing Jiaotong University（北京交通大学）
MACS: A Hybrid Multi-Agent Framework for Reliable Conversational E-Commerce Recommendation
Stanford University（斯坦福大学）；Google DeepMind（谷歌DeepMind）；University of Southern California（南加州大学）；Amazon（亚马逊公司）；Massachusetts Institute of Technology（麻省理工学院）；MIT Sloan School of Management（麻省理工学院斯隆管理学院）
Università della Svizzera italiana（瑞士意大利语区大学）；University of Amsterdam（阿姆斯特丹大学）
Clarifying the underlying user information need by asking clarifying questions is an important feature of modern conversational search system. However, evaluation of such systems through answering prompted clarifying questions requires significant human effort, which can be time-consuming and expensive. In this paper, we propose a conversational User Simulator, called USi, for automatic evaluation of such conversational search systems. Given a description of an information need, USi is capable of automatically answering clarifying questions about the topic throughout the search session. Throug…
University of Nottingham（诺丁汉大学）；Nokia Bell Labs（诺基亚贝尔实验室）
Conversational search systems, such as Google Assistant and Microsoft Cortana, enable users to interact with search systems in multiple rounds through natural language dialogues. Evaluating such systems is very challenging given that any natural language responses could be generated, and users commonly interact for multiple semantically coherent rounds to accomplish a search task. Although prior studies proposed many evaluation metrics, the extent of how those measures effectively capture user preference remains to be investigated. In this paper, we systematically meta-evaluate a variety of co…
University of Amsterdam（阿姆斯特丹大学）；University of Strathclyde（思克莱德大学）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；Microsoft（微软）
Information seeking conversations between users and Conversational Search Agents (CSAs) consist of multiple turns of interaction. While users initiate a search session, ideally a CSA should sometimes take the lead in the conversation by obtaining feedback from the user by offering query suggestions or asking for query clarifications i.e. mixed initiative. This creates the potential for more engaging conversational searches, but substantially increases the complexity of modelling and evaluating such scenarios due to the large interaction space coupled with the trade-offs between the costs and b…
