AdTEC：一个用于评估搜索引擎广告中文本质量的统一基准
AdTEC: A Unified Benchmark for Evaluating Text Quality in Search Engine Advertising
AI总结：
提出首个搜索引擎广告文本质量评估公开基准AdTEC，定义五项任务并构建日语数据集，验证预训练语言模型与人类评估者表现，发现模型在部分任务已达实用水平但人类在特定领域仍具优势。
摘要：
随着自然语言生成技术自动创建的广告文本流畅度不断提高，在真实场景中验证这些创意内容的质量有着强烈需求。我们提出了AdTEC（CyberAgent的广告文本评估基准），这是首个在实际广告运营中从多视角评估广告文本的公开基准。我们的贡献如下：(i) 定义了五项评估广告文本质量的任务，并基于广告代理商的实际运营经验构建了一个日语数据集，这些经验通常被内部保留；(ii) 在该数据集上验证了现有的预训练语言模型（PLMs）和人类评估者的表现；(iii) 分析了基准的特征并提出了其面临的挑战。结果表明，尽管PLMs在几项任务中已达到实用水平，但人类在某些领域仍优于模型，这意味着该领域仍有巨大的改进空间。
英文摘要：
With the increase in the fluency of ad texts automatically created by natural language generation technology, there is high demand to verify the quality of these creatives in a real-world setting. We propose AdTEC (Ad Text Evaluation Benchmark by CyberAgent), the first public benchmark to evaluate ad texts from multiple perspectives within practical advertising operations. Our contributions are as follows: (i) Defining five tasks for evaluating the quality of ad texts, as well as building a Japanese dataset based on the practical operational experiences of building a Japanese dataset based on the practical operational experiences of advertising agencies, which are typically kept in-house. (ii) Validating the performance of existing pre-trained language models (PLMs) and human evaluators on the dataset. (iii) Analyzing the characteristics and providing challenges of the benchmark. The results show that while PLMs have already reached practical usage level in several tasks, humans still outperform in certain domains, implying that there is significant room for improvement in this area.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
