关注数据鸿沟：将大语言模型桥接至企业数据集成
Mind the Data Gap: Bridging LLMs to Enterprise Data Integration
AI总结：
本文发布企业数据集成基准GOBY Benchmark，揭示LLMs在真实企业暗数据上性能显著下降的问题，并提出分层标注、运行时类别学习和本体合成三项技术，使LLMs在企业数据上的表现达到与公开数据相当的水平。
摘要：
领先的大语言模型（LLMs）基于公开数据进行训练。然而，世界上大部分数据是以私有组织或企业数据为主的暗数据，并不公开可获取。我们表明，基于LLMs的方法在真实企业数据集上测试时性能严重下降。当前基于公开数据的基准高估了LLMs的性能。我们发布了一个新的基准数据集——GOBY Benchmark，以推动企业数据集成领域的探索。基于我们在该企业基准上的经验，我们提出了提升LLMs在企业数据上表现的技术，包括：（1）分层标注，（2）运行时类别学习，以及（3）本体合成。我们表明，一旦部署这些技术，LLMs在企业数据上的性能可与公开数据上的性能相媲美。GOBY Benchmark可在 https://goby-benchmark.github.io/ 获取。
英文摘要：
Leading large language models (LLMs) are trained on public data. However, most of the world's data is dark data that is not publicly accessible, mainly in the form of private organizational or enterprise data. We show that the performance of methods based on LLMs seriously degrades when tested on real-world enterprise datasets. Current benchmarks, based on public data, overestimate the performance of LLMs. We release a new benchmark dataset, the GOBY Benchmark, to advance discovery in enterprise data integration. Based on our experience with this enterprise benchmark, we propose techniques to uplift the performance of LLMs on enterprise data, including (1) hierarchical annotation, (2) runtime class-learning, and (3) ontology synthesis. We show that, once these techniques are deployed, the performance on enterprise data becomes on par with that of public data. The Goby benchmark can be obtained at https://goby-benchmark.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
