CONE：用于复杂数值数据的嵌入方法，保留单位和变量语义
CONE: Embeddings for Complex Numerical Data Preserving Unit and Variable Semantics
AI总结：
CONE通过复合嵌入方法提升复杂数值数据的语义编码，实现跨领域高精度数值推理。
摘要：
大型预训练模型（LMs）和大型语言模型（LLMs）通常在捕捉语言语义和上下文关系方面表现良好。然而，这些模型在处理涉及数字的任务时面临挑战。盲目地将数值或结构化数据视为术语是不够的——它们的语义必须被模型充分理解和编码。在本文中，我们提出了CONE，一种混合的Transformer编码器预训练模型，该模型将数字、范围和高斯分布编码到一个嵌入向量空间中，以保持距离。我们引入了一种新的复合嵌入构建算法，该算法将数值值、范围或高斯分布及其相关的单位和属性名称结合起来，以精确捕捉其复杂的语义。我们在跨不同领域（网络、医疗、金融和政府）的大型数据集上进行了广泛的实验评估，证明了CONE强大的数值推理能力，在DROP数据集上取得了87.28%的F1分数，比最先进的（SOTA）基线提高了9.37%的F1分数，并在Recall@10方面显著优于主要的SOTA模型，最高提升了25%。
英文摘要：
Large pre-trained models (LMs) and Large Language Models (LLMs) are typically effective at capturing language semantics and contextual relationships. However, these models encounter challenges in maintaining optimal performance on tasks involving numbers. Blindly treating numerical or structured data as terms is inadequate -- their semantics must be well understood and encoded by the models. In this paper, we propose CONE, a hybrid transformer encoder pre-trained model that encodes numbers, ranges, and gaussians into an embedding vector space preserving distance. We introduce a novel composite embedding construction algorithm that integrates numerical values, ranges or gaussians together with their associated units and attribute names to precisely capture their intricate semantics. We conduct extensive experimental evaluation on large-scale datasets across diverse domains (web, medical, finance, and government) that justifies CONE's strong numerical reasoning capabilities, achieving an F1 score of 87.28% on DROP, a remarkable improvement of up to 9.37% in F1 over state-of-the-art (SOTA) baselines, and outperforming major SOTA models with a significant Recall@10 gain of up to 25%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
