arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.08379cs.CLcs.IRcs.LG
收藏

迈向真实的合成用户生成内容:一种生成在线讨论的脚手架方法

Towards Realistic Synthetic User-Generated Content: A Scaffolding Approach to Generating Online Discussions

Google(谷歌公司)

Krisztian Balog, John Palowitch, Barbara Ikica, Filip Radlinski, Hamidreza Alvari, Mehdi Manshadi

AI总结:

针对直接用大语言模型生成在线讨论结构还原度低、可控性差的问题,提出基于脚手架(讨论串紧凑表示)的多步生成框架,适配不同平台,并配套评估指标验证合成数据的代表性与真实性。

摘要:

合成数据的出现代表了现代机器学习领域的关键转变,为满足真实数据稀缺、高度隐私或难以获取的领域对海量数据的需求提供了一种解决方案。我们探究了创建真实、大规模的用户生成内容合成数据集的可行性,注意到这类内容正日益普及,也是高频信息获取的来源。大语言模型(LLMs)凭借其生成符合在线互动特征的多样化回复的能力,为合成社交媒体讨论串提供了基础。但我们的研究表明,直接应用LLM在捕捉在线讨论的复杂结构方面成效有限,且标准提示机制缺乏足够的控制能力。因此我们提出了一种多步骤生成流程,其核心思路是创建讨论串的紧凑表示,称为scaffolds(脚手架)。我们的框架具备通用性,同时可适配特定社交媒体平台的独特特征。我们利用两个不同在线讨论平台的数据验证了该框架的可行性。为应对确保合成数据代表性与真实性这一核心挑战,我们提出了一套评估指标体系,用于对比我们框架的多种实现方案。

英文摘要:

The emergence of synthetic data represents a pivotal shift in modern machine learning, offering a solution to satisfy the need for large volumes of data in domains where real data is scarce, highly private, or difficult to obtain. We investigate the feasibility of creating realistic, large-scale synthetic datasets of user-generated content, noting that such content is increasingly prevalent and a source of frequently sought information. Large language models (LLMs) offer a starting point for generating synthetic social media discussion threads, due to their ability to produce diverse responses that typify online interactions. However, as we demonstrate, straightforward application of LLMs yields limited success in capturing the complex structure of online discussions, and standard prompting mechanisms lack sufficient control. We therefore propose a multi-step generation process, predicated on the idea of creating compact representations of discussion threads, referred to as scaffolds. Our framework is generic yet adaptable to the unique characteristics of specific social media platforms. We demonstrate its feasibility using data from two distinct online discussion platforms. To address the fundamental challenge of ensuring the representativeness and realism of synthetic data, we propose a portfolio of evaluation measures to compare various instantiations of our framework.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑