arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.10941cs.CLcs.AIcs.LG
收藏

Text2Data:基于文本控制的低资源数据生成

Text2Data: Low-Resource Data Generation with Textual Control

Salesforce(赛富时)

Shiyu Wang, Yihao Feng, Tian Lan, Ning Yu, Yu Bai, Ran Xu, Huan Wang, Caiming Xiong, Silvio Savarese

AI总结:

本文提出Text2Data方法,通过无监督扩散模型学习无标签数据分布,并结合基于约束优化的可控微调,在分子、运动和时间序列等低资源场景下实现文本可控的数据生成,有效缓解灾难性遗忘并提升可控性。

摘要:

自然语言是人类与机器无缝交互的一种常见且直接的信号。认识到这一接口的重要性,机器学习社区正在投入大量精力生成与文本指令语义一致的数据。尽管文本到数据生成在图像编辑、音频合成、视频创建等领域取得了进展,但以昂贵标注或复杂数据结构为特征的低资源领域(如分子、运动动力学和时间序列)往往缺乏文本标签。这一不足阻碍了监督学习,从而限制了先进生成模型在文本到数据任务中的应用。针对低资源场景中的这些挑战,我们提出了Text2Data,这是一种新方法,利用无标签数据通过无监督扩散模型来理解底层数据分布。随后,它通过一种新颖的基于约束优化的学习目标进行可控微调,该目标确保可控性并有效对抗灾难性遗忘。综合实验表明,与现有基线相比,Text2Data能够在分子、运动和时间序列等多种模态上实现更强的可控性表现。

英文摘要:

Natural language serves as a common and straightforward signal for humans to interact seamlessly with machines. Recognizing the importance of this interface, the machine learning community is investing considerable effort in generating data that is semantically coherent with textual instructions. While strides have been made in text-to-data generation spanning image editing, audio synthesis, video creation, and beyond, low-resource areas characterized by expensive annotations or complex data structures, such as molecules, motion dynamics, and time series, often lack textual labels. This deficiency impedes supervised learning, thereby constraining the application of advanced generative models for text-to-data tasks. In response to these challenges in the low-resource scenario, we propose Text2Data, a novel approach that utilizes unlabeled data to understand the underlying data distribution through an unsupervised diffusion model. Subsequently, it undergoes controllable finetuning via a novel constraint optimization-based learning objective that ensures controllability and effectively counteracts catastrophic forgetting. Comprehensive experiments demonstrate that Text2Data is able to achieve enhanced performance regarding controllability across various modalities, including molecules, motions and time series, when compared to existing baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑