使用三元嵌入生成主观结构回归的标签
Generating Labels for Regression of Subjective Constructs using Triplet Embeddings
摘要:
人类注释在计算模型中扮演着重要的角色,其中研究中的目标结构是隐藏的,例如情感维度。这在机器学习中尤其相关,其中从相关的可观察信号(例如,音频、视频、文本)来支持模型训练和测试。目前的研究趋势集中在纠正人工制品和偏见的注释过程中引入的注释,同时融合到一个单一的注释。在这项工作中,我们提出了一种新的注释方法,使用三重嵌入。通过将绝对注释过程提升到相对注释,其中注释者以三元组比较各个目标构造,我们利用了人类注释者的绝对评级比较的准确性。然后,我们在欧几里得空间中构建一个一维嵌入,该嵌入在时间上进行索引,并作为回归的标签。在这种设置中,注释融合自然地发生为不同注释器之间的采样三元组比较的集合的联合。我们表明,通过使用我们提出的采样方法来找到一个嵌入,我们能够准确地表示合成隐藏的结构在噪声采样条件下的时间。我们使用从Mechanical Turk收集的人类注释进一步验证了这种方法,并表明我们可以恢复隐藏结构的底层结构,直到偏差和缩放因子。
英文摘要:
Human annotations serve an important role in computational models where the target constructs under study are hidden, such as dimensions of affect. This is especially relevant in machine learning, where subjective labels derived from related observable signals (e.g., audio, video, text) are needed to support model training and testing. Current research trends focus on correcting artifacts and biases introduced by annotators during the annotation process while fusing them into a single annotation. In this work, we propose a novel annotation approach using triplet embeddings. By lifting the absolute annotation process to relative annotations where the annotator compares individual target constructs in triplets, we leverage the accuracy of comparisons over absolute ratings by human annotators. We then build a 1-dimensional embedding in Euclidean space that is indexed in time and serves as a label for regression. In this setting, the annotation fusion occurs naturally as a union of sets of sampled triplet comparisons among different annotators. We show that by using our proposed sampling method to find an embedding, we are able to accurately represent synthetic hidden constructs in time under noisy sampling conditions. We further validate this approach using human annotations collected from Mechanical Turk and show that we can recover the underlying structure of the hidden construct up to bias and scaling factors.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。