arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.06174cs.AIcs.SE
收藏

面向无监督实体解析的高成本效益提示工程

Cost-Efficient Prompt Engineering for Unsupervised Entity Resolution

University of Southern California(南加州大学);Information Sciences Institute(信息科学研究所)

Navapat Nananukul, Khanin Sisaengsuwanchai, Mayank Kejriwal

AI总结:

本文系统评估面向无监督实体解析的低成本提示工程方法,在两个真实数据集上验证 GPT3.5 的有效性,并发现复杂提示未必优于简单方法。

摘要:

实体解析(ER)是指半自动地判定两个实体何时指向同一底层实体的问题,其应用范围从医疗保健到电子商务。传统 ER 解决方案需要大量人工专业知识,包括面向特定领域的特征工程,以及训练数据的识别与整理。近期发布的大型语言模型(LLM)为让 ER 更加无缝且独立于具体领域提供了机会。然而,众所周知,LLM 也可能带来风险,并且其输出质量可能取决于提示的工程设计方式。遗憾的是,迄今为止,针对使用 ChatGPT 等 LLM 解决无监督 ER 时不同提示方法的效果,仍缺乏系统性实验研究。本文旨在通过开展此类研究来弥补这一空白。我们考虑了若干相对简单且具有成本效益的 ER 提示工程方法,并将其应用于社区广泛使用的两个真实数据集上的 ER。我们通过大量实验结果表明,像 GPT3.5 这样的 LLM 可用于高性能的无监督 ER;而且有趣的是,更复杂、更详细(因而也更昂贵)的提示方法并不一定优于更简单的方法。我们还简要讨论了定性分析和错误分析,包括研究不同提示方法之间的一致性,以确定它们是否产生稳定输出。最后,我们考虑了 LLM 应用于 ER 时的一些局限性。

英文摘要:

Entity Resolution (ER) is the problem of semi-automatically determining when two entities refer to the same underlying entity, with applications ranging from healthcare to e-commerce. Traditional ER solutions required considerable manual expertise, including domain-specific feature engineering, as well as identification and curation of training data. Recently released large language models (LLMs) provide an opportunity to make ER more seamless and domain-independent. However, it is also well known that LLMs can pose risks, and that the quality of their outputs can depend on how prompts are engineered. Unfortunately, a systematic experimental study on the effects of different prompting methods for addressing unsupervised ER, using LLMs like ChatGPT, has been lacking thus far. This paper aims to address this gap by conducting such a study. We consider some relatively simple and cost-efficient ER prompt engineering methods and apply them to ER on two real-world datasets widely used in the community. We use an extensive set of experimental results to show that an LLM like GPT3.5 is viable for high-performing unsupervised ER, and interestingly, that more complicated and detailed (and hence, expensive) prompting methods do not necessarily outperform simpler approaches. We provide brief discussions on qualitative and error analysis, including a study of the inter-consistency of different prompting methods to determine whether they yield stable outputs. Finally, we consider some limitations of LLMs when applied to ER.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑