StableEval Arena：面向稳定币价格稳定性预测的成本感知智能体基准
StableEval Arena: A Cost-Aware Agentic Benchmark for Stablecoin Price Stability Prediction
- Duke Kunshan University（昆山杜克大学）
- Tenorshare（天锐时空）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
StableEval Arena是一个成本感知基准，用于评估LLM智能体在稳定币挂钩风险预测中的表现，揭示协议遵循与金融风险可靠性之间的差距，并发布数据集与代码。
AI中文摘要：
我们引入了StableEval Arena，一个用于评估智能体AI系统在稳定币挂钩风险预测方面的成本感知基准框架。StableEval Arena评估基于LLM的智能体系统在诊断挂钩压力和预测隐藏七天期限内偏离一美元挂钩的情况，使用具有泄漏安全的历史回放，包含交易所价格-成交量数据和市场背景特征。我们报告了两个互补的实验模块：一个包含120个案例的压力增强验证模块和一个包含507个案例的自然分布全竞技场评估模块。在六个基于LLM的智能体配置和基线中，StableEval Arena衡量预测质量、校准标签行为、结构化输出可靠性、延迟、令牌消耗和估计推理成本。该框架不仅根据准确性对智能体进行排名，还将可信度视为预测质量、操作可靠性和计算成本的联合属性。结果显示，协议遵循可靠性与金融风险可靠性之间存在差距：智能体以适度的测量成本可靠地生成有效的结构化输出，但仍会遗漏大多数罕见的严重压力和持续脱钩案例。为了支持审计和复现，我们在Hugging Face上发布了基准数据集，并在GitHub上发布了源代码。
英文摘要：
We introduce StableEval Arena, a cost-aware benchmark framework for evaluating agentic AI systems on stablecoin peg-risk prediction. StableEval Arena evaluates LLM-backed agentic systems on diagnosing peg stress and forecasting deviations from the one-dollar peg over a hidden seven-day horizon, using leakage-safe historical replay with exchange price-volume data and market-context features. We report two complementary experiment blocks: a 120-case stress-enriched validation block and a 507-case natural-distribution full-arena evaluation block. Across six LLM-backed agent configurations and baselines, StableEval Arena measures prediction quality, calibrated-label behavior, structured-output reliability, latency, token consumption, and estimated inference cost. Rather than ranking agents by accuracy alone, the framework treats trustworthiness as a joint property of forecast quality, operational reliability, and computational cost. The results show a gap between protocol-following reliability and financial-risk reliability: agents reliably produce valid structured outputs at modest measured cost, but still miss most rare severe-stress and sustained-depeg cases. To support auditing and replication, we release the benchmark dataset on Hugging Face and the source code on GitHub.
