arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2005.00663cs.CL
收藏

具有复杂结构的多模态正则表达式合成的基准测试

Benchmarking Multimodal Regex Synthesis with Complex Structures

The University of Texas at Austin（德克萨斯大学奥斯汀分校）

Xi Ye, Qiaochu Chen, Isil Dillig, Greg Durrett

备注：
ACL2020
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

摘要：

现有的从自然语言生成正则表达式（regex）的数据集在复杂性上是有限的;与用户在StackOverflow上发布的regex任务相比，这些数据集中的regex很简单，用于描述它们的语言并不多样。我们介绍了StructuredRegex，一个新的正则表达式合成数据集不同于以前的三个方面。首先，为了获得结构复杂和现实的正则表达式，我们使用概率语法生成正则表达式，并从真实世界的StackOverflow帖子中观察到预定义的宏。第二，为了获得语言多样的自然语言描述，我们向众包工作者展示底层正则表达式的抽象解释，并要求他们描述他们看到的模式，而不是让他们转述合成语言。第三，我们用一组字符串来增加每个正则表达式的例子，这些字符串与地面真值正则表达式匹配或不匹配，类似于真实用户给出例子的方式。我们的定量和定性分析证明了StructuredRegex相对于先前数据集的优势。使用各种多模态合成技术的进一步实验结果突出了我们的数据集所带来的挑战，包括非局部约束和多模态输入。

英文摘要：

Existing datasets for regular expression (regex) generation from natural language are limited in complexity; compared to regex tasks that users post on StackOverflow, the regexes in these datasets are simple, and the language used to describe them is not diverse. We introduce StructuredRegex, a new regex synthesis dataset differing from prior ones in three aspects. First, to obtain structurally complex and realistic regexes, we generate the regexes using a probabilistic grammar with pre-defined macros observed from real-world StackOverflow posts. Second, to obtain linguistically diverse natural language descriptions, we show crowdworkers abstract depictions of the underlying regex and ask them to describe the pattern they see, rather than having them paraphrase synthetic language. Third, we augment each regex example with a collection of strings that are and are not matched by the ground truth regex, similar to how real users give examples. Our quantitative and qualitative analysis demonstrates the advantages of StructuredRegex over prior datasets. Further experimental results using various multimodal synthesis techniques highlight the challenge presented by our dataset, including non-local constraints and multi-modal inputs.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑