arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.03901cs.ITcs.LGmath.ITstat.ML
收藏

批量通用预测

Batch Universal Prediction

School of Computer and Communication Sciences EPFL(洛桑联邦理工学院计算机与通信科学学院)

Marco Bondaschi, Michael Gastpar

AI总结:

本文从通用预测视角评估大语言模型,提出批量遗憾指标,并分析其在无记忆源和一阶马尔可夫源下加常数预测器的渐近值。

摘要:

大型语言模型(LLMs)近期因其生成类似人类英语句子的惊人能力而广受欢迎。LLMs本质上是预测器,根据过去的词序列估计下一个词的概率。因此,从通用预测的角度评估其性能是自然的。为了公平地进行评估,我们引入了批量遗憾(batch regret)的概念,作为经典平均遗憾的修正,并研究了在无记忆源和一阶马尔可夫源情况下,加常数预测器的批量遗憾的渐近值。

英文摘要:

Large language models (LLMs) have recently gained much popularity due to their surprising ability at generating human-like English sentences. LLMs are essentially predictors, estimating the probability of a sequence of words given the past. Therefore, it is natural to evaluate their performance from a universal prediction perspective. In order to do that fairly, we introduce the notion of batch regret as a modification of the classical average regret, and we study its asymptotical value for add-constant predictors, in the case of memoryless sources and first-order Markov sources.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑