arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2212.10522cs.CL
收藏

Transformer 去搞笑了:从科学摘要端到端生成(幽默)标题

Transformers Go for the LOLs: Generating (Humourous) Titles from Scientific Abstracts End-to-End

University of Mannheim(曼海姆大学)

Yanran Chen, Steffen Eger

AI总结:

本研究探索端到端摘要生成标题,微调七个Transformer模型,并首次构建幽默标题数据集,发现最佳系统接近人类但幽默生成更难,ChatGPT无需微调即达最佳水平。

摘要:

我们考虑端到端的摘要到标题生成问题,探索了七个最近的基于 Transformer 的模型(包括 ChatGPT),这些模型在来自 NLP 和机器学习(ML)会议/期刊的超过 3 万对摘要-标题对上进行了微调。作为扩展,我们还考虑了生成幽默论文标题这一更难的问题。对于后者,我们编制了第一个针对 NLP/ML 领域科学论文的大规模幽默标注数据集,包含近约 2.6 千个标题。我们使用人工和自动指标评估所有模型。我们的人工评估表明,我们最好的端到端系统表现与人类作者相当(但可以说略差)。然而,生成有趣的标题更加困难,我们的自动系统明显不如人类,并且经常学习到数据集中的幽默伪影。最后,未经任何微调的 ChatGPT 表现与我们最好的微调系统相当。

英文摘要:

We consider the end-to-end abstract-to-title generation problem, exploring seven recent transformer based models (including ChatGPT) fine-tuned on more than 30k abstract-title pairs from NLP and machine learning (ML) venues. As an extension, we also consider the harder problem of generating humorous paper titles. For the latter, we compile the first large-scale humor annotated dataset for scientific papers in the NLP/ML domains, comprising almost ~2.6k titles. We evaluate all models using human and automatic metrics. Our human evaluation suggests that our best end-to-end system performs similarly to human authors (but arguably slightly worse). Generating funny titles is more difficult, however, and our automatic systems clearly underperform relative to humans and often learn dataset artefacts of humor. Finally, ChatGPT, without any fine-tuning, performs on the level of our best fine-tuned system.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑