大型语言模型API网关的行为一致性与透明度分析
Behavioral Consistency and Transparency Analysis on Large Language Model API Gateways
AI总结：
本文提出GateScope框架，用于评估LLM网关的行为一致性和操作透明度，发现多个网关存在模型降级、响应不一致、计费不准确等问题。
摘要：
第三方大型语言模型（LLM）API网关正迅速成为多供应商模型的统一接入点。然而，这些网关的内部路由、缓存和计费政策大多不透明，使用户无法确定请求是否由广告模型服务，响应是否忠实于上游API，或发票是否准确反映公开定价政策。为解决这一差距，我们引入GateScope，一种轻量级的黑盒测量框架，用于评估商业LLM网关的行为一致性和操作透明性。GateScope旨在通过审计四个关键维度：响应内容分析、多轮对话性能、计费准确性和延迟特性，检测关键误操作，包括模型降级或切换、沉默截断、计费不准确和延迟不稳定。在对10个真实商业LLM API网关的测量中，揭示了预期与实际行为之间的频繁差距，包括沉默模型替换、记忆退化、偏离公开定价以及平台间显著的延迟稳定性变化。
英文摘要：
Third-party Large Language Model (LLM) API gateways are rapidly emerging as unified access points to models offered by multiple vendors. However, the internal routing, caching, and billing policies of these gateways are largely undisclosed, leaving users with limited visibility into whether requests are served by the advertised models, whether responses remain faithful to upstream APIs, or whether invoices accurately reflect public pricing policies. To address this gap, we introduce GateScope, a lightweight black-box measurement framework for evaluating behavioral consistency and operational transparency in commercial LLM gateways. GateScope is designed to detect key misbehaviors, including model downgrading or switching, silent truncation, billing inaccuracies, and instability in latency by auditing gateways along four critical dimensions: response content analysis, multi-turn conversation performance, billing accuracy, and latency characteristics. Our measurements across 10 real-world commercial LLM API gateways reveal frequent gaps between expected and actual behaviors, including silent model substitutions, degraded memory retention, deviations from announced pricing, and substantial variation in latency stability across platforms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
