arXiv 2302.142292023-10-24cs.CLcs.AI

基于大型语言模型的零样本跨语言摘要

Zero-Shot Cross-Lingual Summarization via Large Language Models

Soochow University（苏州大学）；Beijing Jiaotong University（北京交通大学）；Pattern Recognition Center, WeChat AI, Tencent Inc（腾讯公司微信AI模式识别中心）；Princeton University（普林斯顿大学）

Jiaan Wang, Yunlong Liang, Fandong Meng, Beiqi Zou, Zhixu Li, Jianfeng Qu, Jie Zhou

备注：: Both first authors contributed equally. Technical Report, 12 pages. Accepted to the 4th New Frontiers in Summarization Workshop (NewSumm@EMNLP 2023)

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文实证研究了GPT-3.5、ChatGPT和GPT-4等大型语言模型在零样本跨语言摘要任务中的表现，发现通过交互式提示可显著提升模型性能，其中GPT-4达到最先进水平。

摘要：

给定一篇源语言文档，跨语言摘要（CLS）旨在生成不同目标语言的摘要。近期，大型语言模型（LLMs）如GPT-3.5、ChatGPT和GPT-4的出现，引起了计算语言学界的广泛关注。然而，LLMs在CLS上的表现尚不明确。本报告中，我们实证性地使用各种提示从不同范式（即端到端和流水线）引导LLMs执行零样本CLS，并对生成的摘要进行初步评估。我们发现ChatGPT和GPT-4原本倾向于生成包含详细信息的长摘要。在交互式提示的帮助下，这两种LLMs能进一步平衡信息量和简洁性，显著提升其CLS性能。在三个广泛使用的CLS数据集上的实验结果表明，GPT-4实现了最先进的零样本CLS性能，与微调后的mBART-50相比表现相当。此外，我们还发现一些多语言和双语LLMs（即BLOOMZ、ChatGLM-6B、Vicuna-13B和ChatYuan）的零样本CLS能力有限。由于CLS的复合性质要求模型同时执行摘要和翻译，以零样本方式完成此任务对LLMs而言仍是挑战。因此，我们真诚希望并建议未来的LLM研究能将CLS作为测试平台。

英文摘要：

Given a document in a source language, cross-lingual summarization (CLS) aims to generate a summary in a different target language. Recently, the emergence of Large Language Models (LLMs), such as GPT-3.5, ChatGPT and GPT-4, has attracted wide attention from the computational linguistics community. However, it is not yet known the performance of LLMs on CLS. In this report, we empirically use various prompts to guide LLMs to perform zero-shot CLS from different paradigms (i.e., end-to-end and pipeline), and provide a preliminary evaluation on the generated summaries. We find that ChatGPT and GPT-4 originally prefer to produce lengthy summaries with detailed information. These two LLMs can further balance informativeness and conciseness with the help of an interactive prompt, significantly improving their CLS performance. Experimental results on three widely-used CLS datasets show that GPT-4 achieves state-of-the-art zero-shot CLS performance, and performs competitively compared with the fine-tuned mBART-50. Moreover, we also find some multi-lingual and bilingual LLMs (i.e., BLOOMZ, ChatGLM-6B, Vicuna-13B and ChatYuan) have limited zero-shot CLS ability. Due to the composite nature of CLS, which requires models to perform summarization and translation simultaneously, accomplishing this task in a zero-shot manner is even a challenge for LLMs. Therefore, we sincerely hope and recommend future LLM research could use CLS as a testbed.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。