Conversational search systems increasingly employ clarifying questions to refine user queries and improve the search experience. Previous studies have demonstrated the usefulness of text-based clarifying questions in enhancing both retrieval performance and user experience. While images have been shown to improve retrieval performance in various contexts, their impact on user performance when incorporated into clarifying questions remains largely unexplored. We conduct a user study with 73 participants to investigate the role of images in conversational search, specifically examining their effects on two search-related tasks: (i) answering clarifying questions and (ii) query reformulation. We compare the effect of multimodal and text-only clarifying questions in both tasks within a conversational search context from various perspectives. Our findings reveal that while participants showed a strong preference for multimodal questions when answering clarifying questions, preferences were more balanced in the query reformulation task. The impact of images varied with both task type and user expertise. In answering clarifying questions, images helped maintain engagement across different expertise levels, while in query reformulation they led to more precise queries and improved retrieval performance. Interestingly, for clarifying question answering, text-only setups demonstrated better user performance as they provided more comprehensive textual information in the absence of images. These results provide valuable insights for designing effective multimodal conversational search systems, highlighting that the benefits of visual augmentation are task-dependent and should be strategically implemented based on the specific search context and user characteristics.
Asking Clarifying Questions in Open-Domain Information-Seeking Conversations
Università della Svizzera italiana (USI)（瑞士意大利语区大学（USI））；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
用户往往无法在一个查询中表达其复杂的信息需求。因此，他们可能需要扫描多个结果页面或重新制定他们的查询，这可能是一个令人沮丧的经历。或者，系统可以通过主动询问用户的问题来澄清他们的信息需求，从而提高用户满意度。在会话系统中，提出澄清性问题尤其重要，因为它们只能返回有限数量的结果（通常只有一个）。在本文中，我们制定了开放域信息寻求会话系统中的问题澄清的任务。为此，我们提出了一种离线评估方法，并通过众包收集了一个名为Qulac的数据集。我们的数据集建立在TREC Web Track 2009-2012数据的基础上，由198个TREC主题的10 K多个问答对组成，涉及762个方面。我们在Oracle模型上的实验表明，只问一个好问题，就P@1而言，检索性能提高了170%以上，这清楚地表明了任务的潜在影响。我们进一步提出了一个检索框架，包括三个组成部分：问题检索，问题选择，和文档检索。特别是，我们的问题选择模型考虑到原始查询和以前的问答互动，同时选择下一个问题。我们的模型显著优于竞争基准。为了促进这一领域的研究，我们公开了Qulac。
TO-GATE: Clarifying Questions and Summarizing Responses with Trajectory Optimization for Eliciting Human Preference
School of Information Science and Engineering, Yunnan University, China（云南大学信息科学与工程学院）；Yunnan Key Laboratory of Intelligent Systems and Computing, China（云南省智能系统与计算重点实验室）
Clarify User Expertise: Towards Proactive Conversational Agents Tailoring Responses to User Proficiency
School of Computing and Data Science, The University of Hong Kong（香港大学计算与数据科学学院）；Institute of Data Science, National University of Singapore（新加坡国立大学数据科学研究所）；College of Computer Science, Sichuan University（四川大学计算机学院）
