一点信心大有裨益
A Little Confidence Goes a Long Way
AI总结:
本文提出利用LLM隐藏状态探针的二分类方法,性能媲美最先进模型,但计算资源极少且无需标注数据,通过语义描述、对称破缺和熵最大化生成置信度,在四个数据集上验证。
摘要:
我们介绍了一组用于二分类任务的相关方法,这些方法利用大型语言模型(LLMs)中隐藏状态激活的探针。其性能与目前可用的最大、最先进的LLMs相当,但所需的计算资源少几个数量级,且不需要标注数据。该方法包括:将类别标签转化为语义丰富的描述;利用多层感知机探针的自发对称破缺进行无监督学习和推理;训练探针以从隐藏状态激活中生成置信度分数(先验概率),并通过熵最大化使其受已知约束;以及从集成中选择最自信的探针模型进行预测。这些技术在四个数据集上使用五个基础LLM进行了评估。
英文摘要:
We introduce a group of related methods for binary classification tasks using probes of the hidden state activations in large language models (LLMs). Performance is on par with the largest and most advanced LLMs currently available, but requiring orders of magnitude fewer computational resources and not requiring labeled data. This approach involves translating class labels into a semantically rich description, spontaneous symmetry breaking of multilayer perceptron probes for unsupervised learning and inference, training probes to generate confidence scores (prior probabilities) from hidden state activations subject to known constraints via entropy maximization, and selecting the most confident probe model from an ensemble for prediction. These techniques are evaluated on four datasets using five base LLMs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。