融合人机交互与人工智能研究以评估对话式软件工程助手
Bridging HCI and AI Research for the Evaluation of Conversational SE Assistants
AI总结：
针对大型语言模型（LLMs）对话式软件工程助手的评估需求，本文主张融合人机交互（HCI）与人工智能（AI）研究，提出以人为中心的自动评估思路，明确评估需求与挑战，旨在构建符合用户需求的评估框架。
摘要：
随着大型语言模型（LLMs）在软件工程领域的应用日益广泛，尤其是最近以对话式助手的形式出现，确保这些技术符合开发者需求至关重要。传统以人为中心的方法在大规模评估基于LLM的工具时存在局限性，这凸显了自动评估的必要性。本文主张结合人机交互（HCI）和人工智能（AI）研究的见解，实现对基于LLM的对话式软件工程助手的以人为中心的自动评估。我们确定了此类评估的需求及未来面临的挑战，致力于构建一个确保这些助手的设计和部署符合用户需求的框架。
英文摘要：
As Large Language Models (LLMs) are increasingly adopted in software engineering, recently in the form of conversational assistants, ensuring these technologies align with developers' needs is essential. The limitations of traditional human-centered methods for evaluating LLM-based tools at scale raise the need for automatic evaluation. In this paper, we advocate combining insights from human-computer interaction (HCI) and artificial intelligence (AI) research to enable human-centered automatic evaluation of LLM-based conversational SE assistants. We identify requirements for such evaluation and challenges down the road, working towards a framework that ensures these assistants are designed and deployed in line with user needs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
