一种用于去中心化大语言模型推理的多维质量评分框架
A Multi-Dimensional Quality Scoring Framework for Decentralized LLM Inference with Proof of Quality
AI总结：
本文提出一种多维质量评分框架，用于去中心化LLM推理中的质量评估，通过分解输出质量为多个模块化维度，并通过实验验证其在不同任务中的有效性。
摘要：
去中心化的大语言模型（LLM）推理网络可以通过汇集异构计算来扩展服务规模，但它们需要轻量且激励兼容的机制来评估输出质量。先前的工作引入了成本感知的证明质量（PoQ）和自适应鲁棒PoQ，以在评估者异质性和对抗性行为下分配奖励。在本文中，我们关注质量信号本身，并提出了一种多维质量评分框架，将输出质量分解为模块化维度，包括模型和成本先验、结构质量、语义质量、查询-输出对齐以及一致性/不确定性。利用记录的QA和摘要任务的输出，我们系统地审计维度可靠性，并显示看似合理的维度可能具有任务依赖性，甚至在未经校准的情况下与参考质量呈负相关。虽然默认复合体在强单个语义评估者下表现不佳，但消融实验表明，移除不可靠的维度并重新归一化权重可以得到一个校准的复合体，其表现可与最佳单评估者和共识基线相媲美。最后，我们将复合评分整合为PoQ中的可插拔质量信号，并在对抗性评估者攻击下展示了与鲁棒聚合和自适应信任加权的互补优势。
英文摘要：
Decentralized large language model (LLM) inference networks can pool heterogeneous compute to scale serving, but they require lightweight and incentive-compatible mechanisms to assess output quality. Prior work introduced cost-aware Proof of Quality (PoQ) and adaptive robust PoQ to allocate rewards under evaluator heterogeneity and adversarial behavior. In this paper, we focus on the quality signal itself and propose a multi-dimensional quality scoring framework that decomposes output quality into modular dimensions, including model and cost priors, structure quality, semantic quality, query-output alignment, and agreement/uncertainty. Using logged outputs from QA and summarization tasks, we systematically audit dimension reliability and show that seemingly reasonable dimensions can be task-dependent and even negatively correlated with reference quality without calibration. While the default composite underperforms a strong single semantic evaluator, ablations reveal that removing unreliable dimensions and re-normalizing weights yields a calibrated composite that matches or exceeds the best single- evaluator and consensus baselines. Finally, we integrate the composite score as a drop-in quality signal in PoQ and demonstrate complementary benefits with robust aggregation and adaptive trust weighting under adversarial evaluator attacks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
