arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.15337cs.CLcs.LG
收藏

使用LLM在概念空间维度上对实体进行排序:微调策略分析

Ranking Entities along Conceptual Space Dimensions with LLMs: An Analysis of Fine-Tuning Strategies

Cardiff University(卡迪夫大学)

Nitesh Kumar, Usashi Chatterjee, Steven Schockaert

AI总结:

本文研究使用LLMs在概念空间维度上对实体排序,因真实排序稀缺,采用易获取特征微调,发现其排序能力可部分迁移至感知和主观特征,且训练数据包含少量此类特征对最佳结果至关重要。

摘要:

概念空间根据实体的基本语义特征来表示实体。这种表示非常有价值,但众所周知很难学习,尤其是在建模感知和主观特征时。从大型语言模型(LLMs)中提取概念空间最近成为一种有前景的策略,但现有工作仅限于使用相对简单的零样本策略探测预训练的LLMs。我们特别关注根据给定的概念空间维度对实体进行排序的任务。遗憾的是,我们无法在此任务上直接微调LLMs,因为概念空间维度的真实排序很罕见。因此,我们使用更容易获取的特征作为训练数据,并分析所得模型的排序能力是否能迁移到感知和主观特征上。我们发现确实在一定程度上如此,但在训练数据中包含至少一些感知和主观特征似乎对于获得最佳结果至关重要。

英文摘要:

Conceptual spaces represent entities in terms of their primitive semantic features. Such representations are highly valuable but they are notoriously difficult to learn, especially when it comes to modelling perceptual and subjective features. Distilling conceptual spaces from Large Language Models (LLMs) has recently emerged as a promising strategy, but existing work has been limited to probing pre-trained LLMs using relatively simple zero-shot strategies. We focus in particular on the task of ranking entities according to a given conceptual space dimension. Unfortunately, we cannot directly fine-tune LLMs on this task, because ground truth rankings for conceptual space dimensions are rare. We therefore use more readily available features as training data and analyse whether the ranking capabilities of the resulting models transfer to perceptual and subjective features. We find that this is indeed the case, to some extent, but having at least some perceptual and subjective features in the training data seems essential for achieving the best results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑