基于LLM的对话代理的自动化调查收集
Automated Survey Collection with LLM-based Conversational Agents
AI总结：
本研究提出一种由对话大语言模型驱动的端到端电话调查收集框架，通过LLM代理实施调查并用GPT-4o分析转录本，实验证明其提取调查响应准确率达98%，有效降低了人力成本并具备可扩展性。
摘要：
目的：传统的基于电话的调查是收集生物医学和医疗保健数据最易获取且使用最广泛的方法之一，然而，它们通常成本高昂、劳动密集，且难以有效扩展。为了克服这些局限性，我们提出了一种由对话大语言模型（LLMs）驱动的端到端调查收集框架。材料与方法：我们的框架由一名负责设计调查并招募参与者的研究人员、一个由LLM驱动并负责拨打参与者电话及实施调查的对话电话代理、一个用于分析调查期间生成的对话转录本的第二个LLM（GPT-4o），以及一个用于存储和组织结果的数据库组成。为了测试我们的框架，我们招募了8名参与者（包括5名母语和3名非母语英语使用者），并实施了40次调查。我们评估了LLM生成的对话转录本的正确性、GPT-4o推断出的调查响应的准确性以及参与者的整体体验。结果：尽管转录本的平均每行单词错误率为7.7%，GPT-4o仍成功从对话转录本中提取了调查响应，平均准确率达到98%。虽然参与者注意到对话LLM代理偶尔会犯错，但他们报告称该代理有效地传达了调查的目的，表现出良好的理解力，并保持了引人入胜的互动。结论：我们的研究突显了LLM代理在为医疗保健应用进行和分析电话调查方面的潜力。通过减轻人类访谈者的工作量并提供可扩展的解决方案，这种方法为现实世界中的端到端AI驱动的电话调查收集系统铺平了道路。
英文摘要：
Objective: Traditional phone-based surveys are among the most accessible and widely used methods to collect biomedical and healthcare data, however, they are often costly, labor intensive, and difficult to scale effectively. To overcome these limitations, we propose an end-to-end survey collection framework driven by conversational Large Language Models (LLMs). Materials and Methods: Our framework consists of a researcher responsible for designing the survey and recruiting participants, a conversational phone agent powered by an LLM that calls participants and administers the survey, a second LLM (GPT-4o) that analyzes the conversation transcripts generated during the surveys, and a database for storing and organizing the results. To test our framework, we recruited 8 participants consisting of 5 native and 3 non-native english speakers and administered 40 surveys. We evaluated the correctness of LLM-generated conversation transcripts, accuracy of survey responses inferred by GPT-4o and overall participant experience. Results: Survey responses were successfully extracted by GPT-4o from conversation transcripts with an average accuracy of 98% despite transcripts exhibiting an average per-line word error rate of 7.7%. While participants noted occasional errors made by the conversational LLM agent, they reported that the agent effectively conveyed the purpose of the survey, demonstrated good comprehension, and maintained an engaging interaction. Conclusions: Our study highlights the potential of LLM agents in conducting and analyzing phone surveys for healthcare applications. By reducing the workload on human interviewers and offering a scalable solution, this approach paves the way for real-world, end-to-end AI-powered phone survey collection systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
