在线疫苗关注点的层次化多标签分类
Hierarchical Multi-Label Classification of Online Vaccine Concerns
AI总结：
本研究利用大型语言模型在零样本设置下对在线疫苗关注点进行层次化多标签分类，通过多遍布尔提问的提示策略，GPT-4在VaxConcerns数据集上达到78.7%的F1分数，超越了众包工人。
摘要：
疫苗关注点是一个不断演变的目标，并且可能迅速变化，正如在COVID-19大流行期间所观察到的那样。识别疫苗关注点和错误信息的纵向趋势可能通过帮助公共卫生工作战略性地分配资源或信息宣传活动来为医疗保健领域提供信息。我们探索了在零样本设置下使用大型语言模型（LLMs）检测在线话语中疫苗关注点的任务，无需昂贵的训练数据集。由于对在线来源的实时监控需要大规模推理，我们探讨了不同提示策略的成本-准确性权衡，并为当前应用中的系统设计选择提供了具体的建议。对不同提示策略的分析表明，通过多次LLM遍历对关注点进行分类（每次遍历包含一个关于文本是否提及疫苗关注点的布尔问题）效果最佳。我们的结果表明，与最近引入的VaxConcerns数据集上专家提供的地面真实标注相比，GPT-4可以大幅超越众包工人的准确性，实现了78.7%的总体F1分数。
英文摘要：
Vaccine concerns are an ever-evolving target, and can shift quickly as seen during the COVID-19 pandemic. Identifying longitudinal trends in vaccine concerns and misinformation might inform the healthcare space by helping public health efforts strategically allocate resources or information campaigns. We explore the task of detecting vaccine concerns in online discourse using large language models (LLMs) in a zero-shot setting without the need for expensive training datasets. Since real-time monitoring of online sources requires large-scale inference, we explore cost-accuracy trade-offs of different prompting strategies and offer concrete takeaways that may inform choices in system designs for current applications. An analysis of different prompting strategies reveals that classifying the concerns over multiple passes through the LLM, each consisting a boolean question whether the text mentions a vaccine concern or not, works the best. Our results indicate that GPT-4 can strongly outperform crowdworker accuracy when compared to ground truth annotations provided by experts on the recently introduced VaxConcerns dataset, achieving an overall F1 score of 78.7%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
