基于自然语言命令的场景图修改
Scene Graph Modification Based on Natural Language Commands
摘要：
图和解析树等结构化表示在许多自然语言处理系统中起着至关重要的作用。近年来，多圈用户界面的进步使得需要在给定新的信息源的情况下控制和更新这些结构化表示。虽然已经有很多努力集中在提高性能的解析器，映射文本到图或解析树，很少有探索的问题，直接操纵这些表示。在本文中，我们探讨了新的问题的图形修改，系统需要学习如何更新现有的场景图给出了一个新的用户的命令。我们的新模型基于图的稀疏Transformer和交叉注意信息融合优于以前的系统改编自机器翻译和图形生成文献。我们进一步将我们的大型图修改数据集贡献给研究社区，以鼓励未来对这个新问题的研究。
英文摘要：
Structured representations like graphs and parse trees play a crucial role in many Natural Language Processing systems. In recent years, the advancements in multi-turn user interfaces necessitate the need for controlling and updating these structured representations given new sources of information. Although there have been many efforts focusing on improving the performance of the parsers that map text to graphs or parse trees, very few have explored the problem of directly manipulating these representations. In this paper, we explore the novel problem of graph modification, where the systems need to learn how to update an existing scene graph given a new user's command. Our novel models based on graph-based sparse transformer and cross attention information fusion outperform previous systems adapted from the machine translation and graph generation literature. We further contribute our large graph modification datasets to the research community to encourage future research for this new problem.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
