可重播的金融代理：用于工具使用LLM代理的确定性-忠实性保证框架
Replayable Financial Agents: A Determinism-Faithfulness Assurance Harness for Tool-Using LLM Agents
AI总结：
本文提出DFAH框架，用于评估金融代理的确定性和忠实性，发现决策确定性与准确性无显著相关性，强调需独立测量两者的多维评估方法。
摘要：
LLM代理在监管审计重播中面临困难：当被要求用相同的输入重现一个被标记的交易决策时，许多部署无法返回一致的结果。我们介绍了确定性-忠实性保证框架（DFAH），这是一个用于测量轨迹确定性、决策确定性和证据条件下的忠实性的框架，用于在金融服务中部署的使用工具的代理。在4700多个代理运行（7个模型，4个提供商，3个金融基准，每个基准50个案例，T=0.0）中，我们发现决策确定性和任务准确性之间没有可检测的相关性（r = -0.11，95%置信区间[-0.49, 0.31]，p = 0.63，n = 21配置）：模型可以是确定性的而不准确，也可以是准确的而不确定。由于在我们的样本中这两个指标并不预测彼此，因此必须独立测量，这正是DFAH所提供的。小型模型（7-20B）通过在模式匹配上严格一致来实现接近完美的确定性，但以准确性（20-42%）为代价，而前沿模型则显示出中等程度的确定性（50-96%）和可变的准确性。没有模型能同时实现完美的确定性和高准确性，支持DFAH的多维测量方法。我们提供了三个金融基准（合规审查、投资组合限制和DataOps异常；每个基准50个案例）以及一个开源的压力测试框架。在这些基准和DFAH评估设置中，一级模型（以模式优先的架构）实现了与审计重播要求一致的确定性水平。
英文摘要：
Tool-using agents can repeat a final decision while changing their recorded execution. We introduce the Determinism-Faithfulness Assurance Harness (DFAH), a framework that distinguishes decision repeatability, trajectory agreement, and evidence-conditioned faithfulness. Task correctness requires separately qualified labels and evaluation; evidence-conditioned faithfulness was not evaluated in the historical v2 agentic experiments. The original v2 study reported 4,705 agentic runs in three synthetic financial tasks and a decision-determinism/task-label-match correlation of r = -0.11 across 21 model-benchmark configuration summaries. This statistic is reproducible from the historical configuration table, but includes a subsequently excluded portfolio fixture. It is retained as a historical description, not evidence of statistical independence, predictive uselessness, or an architectural determinism-accuracy tradeoff. Recorded decision concentration and tool-path variation do not identify hidden model strategy. This correction qualifies the historical evidence and removes the deployment recommendations derived from those unsupported interpretations. A separate corrected study, DFAH-Bench (arXiv:2607.20491), provides qualified evidence of decision/path disagreement. The contribution retained here is a measurement framework: repeatability, observable execution, evidence alignment, and correctness require distinct evidence, with explicit capture and study boundaries.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
