将偏倚标注者能力估计(BACE)模型应用于COVID-19疫苗Twitter数据:针对潜在消息特征的人工标注
Adoption and implication of the Biased-Annotator Competence Estimation (BACE) model into COVID-19 vaccine Twitter data: Human annotation for latent message features
AI总结:
本文针对传统人工编码假设编码者同等准确的弱点,提出将BACE模型应用于COVID-19疫苗Twitter数据,通过贝叶斯建模考虑编码者偏倚并将真实标签视为潜在参数,以改进潜在消息特征的人工编码。
摘要:
传统的定量内容分析方法(人工编码方法)存在弱点,例如假设一旦训练中的编码者间信度达到阈值分数,所有人工编码者就具有同等的准确性。我们应用了偏倚标注者能力估计(BACE)模型(Tyler, 2021),该模型利用贝叶斯建模来改进人工编码。该模型的一个重要贡献在于,它考虑了每个编码者的潜在偏倚和可靠性,并将每条消息的“真实”标签视为潜在参数,具有可量化的估计不确定性。相比之下,在传统人工编码中,每条消息将获得一个固定标签,而没有对测量不确定性的估计。在这篇扩展摘要中,我们首先总结了传统人工编码的弱点;然后将BACE模型应用于COVID-19疫苗Twitter数据,并将BACE与其他统计模型进行比较;最后,我们讨论了如何应用BACE模型来改进潜在消息特征的人工编码。
英文摘要:
Traditional quantitative content analysis approach (human coding method) has weaknesses, such as assuming all human coders are equally accurate once the intercoder reliability for training reaches a threshold score. We applied the Biased-Annotator Competence Estimation (BACE) model (Tyler, 2021), which draws on Bayesian modeling to improve human coding. An important contribution of this model is it takes each coder's potential biases and reliability into consideration and treats the "true" label of each message as a latent parameter, with quantifiable estimation uncertainties. In contrast, in conventional human coding, each message will receive a fixed label without estimates for measurement uncertainties. In this extended abstract, we first summarize the weaknesses of conventional human coding; and then apply the BACE model to COVID-19 vaccine Twitter data and compare BACE with other statistical models; finally, we discuss how the BACE model can be applied to improve human coding of latent message features.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。