结构化数据自然语言双射的探索路径及LLM标注的作用
Inroads to a Structured Data Natural Language Bijection and the role of LLM annotation
AI总结：
本文探究多任务训练与LLM标注数据对序列到序列语言模型的影响，发现多任务t5-small在F1上优于单任务模型，但逆任务和合成数据未带来显著总体提升，提示存在模型容量瓶颈。
摘要：
本研究发现有限证据支持以下理论：对序列到序列Transformer语言模型使用多任务训练可以在某些指标上提升性能。具体而言，多任务通才模型t5-small以0.771的F1分数优于专才模型t5-small的0.692，这可能表明存在潜在的跨任务知识泛化。这进一步说明，即使使用相同的网络，以不同方式“复用”相同数据也可能在某些指标上带来更高性能。然而，仅靠逆任务本身很可能只是一种优化策略，因为在本工作所探索的模型规模下，它并未带来显著的总体改进。此外，与不含合成数据的相同t5-small模型相比，加入约4500条LLM标注记录（与12800条WebNLG训练记录交错）并未显著改变自动指标性能。这可能是由于模型规模导致的学习容量瓶颈，而观察到的性能下降可能是由于语料库的分布差异所致。未来需要使用更大模型或进行人工评估的研究，以更充分地解释影响这些任务性能的机制。
英文摘要：
This work finds limited evidence supporting the theory that using multiple tasks with sequence-to-sequence transformer language models can improve performance on some metrics. In particular, the multi-task generalist t5-small outperforms the specialist t5-small with a $F_1$ of $0.771$ up from $0.692$, which may point to underlying cross-task knowledge generalization. This further suggests that even with the same network, "re-using" the same data in a different way may lead to higher performance in some metrics. However, the inverse task alone is likely only an optimization strategy, since it does not yield a significant general improvement at the model sizes explored in this work. Also, adding $\approx 4500$ LLM annotated records (interlaced with the $12800$ WebNLG training records) does not substantially change automatic metric performance compared to the same t5-small model without the synthetic data. This may be due to a learning capacity bottleneck on account of model size, and decreases observed may be due to distributional differences in the corpora. Future research using larger models or human evaluation is required to more fully explain the mechanisms contributing to performance on these tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
