UZH_CLyp at SemEval-2023 Task 9：用于推文亲密性预测跨语言学习的头优先微调与ChatGPT数据生成
UZH_CLyp at SemEval-2023 Task 9: Head-First Fine-Tuning and ChatGPT Data Generation for Cross-Lingual Learning in Tweet Intimacy Prediction
AI总结：
本文提出用于多语言推文亲密性预测的头优先微调方法HeFiT与ChatGPT数据生成策略，在SemEval-2023 Task 9的10种语言中取得第二名，有效稳定了训练并提升了零样本跨语言性能。
摘要：
本文描述了UZH_CLyp针对SemEval 2023 Task 9“多语言推文亲密性分析”的提交方案。根据官方的Pearson相关回归评估指标，我们在所有10种语言中均取得了第二好的成绩。我们的跨语言迁移学习方法探索了使用头优先微调方法（HeFiT）的益处，该方法首先仅更新回归头参数，随后以降低的学习率更新预训练transformer编码器参数。此外，我们还研究了在没有人工标注数据的低资源环境下，使用少量自动生成示例（在我们的案例中来自ChatGPT）的影响。我们的研究表明，HeFiT稳定了训练过程，并持续提升了缺乏对推文领域适应的预训练模型的结果。研究还表明，使用合成数据时跨语言学习的性能有显著提升，证实了当前文本生成系统在改善零样本基线结果方面的有效性。最后，我们检查了标注数据中可能存在的不一致性如何导致跨语言干扰问题。
英文摘要：
This paper describes the submission of UZH_CLyp for the SemEval 2023 Task 9 "Multilingual Tweet Intimacy Analysis". We achieved second-best results in all 10 languages according to the official Pearson's correlation regression evaluation measure. Our cross-lingual transfer learning approach explores the benefits of using a Head-First Fine-Tuning method (HeFiT) that first updates only the regression head parameters and then also updates the pre-trained transformer encoder parameters at a reduced learning rate. Additionally, we study the impact of using a small set of automatically generated examples (in our case, from ChatGPT) for low-resource settings where no human-labeled data is available. Our study shows that HeFiT stabilizes training and consistently improves results for pre-trained models that lack domain adaptation to tweets. Our study also shows a noticeable performance increase in cross-lingual learning when synthetic data is used, confirming the usefulness of current text generation systems to improve zero-shot baseline results. Finally, we examine how possible inconsistencies in the annotated data contribute to cross-lingual interference issues.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
