重新思考LLMOps用于欺诈和反洗钱：构建合规级LLM服务栈
Rethinking LLMOps for Fraud and AML: Building a Compliance-Grade LLM Serving Stack
AI总结：
本文提出针对欺诈和反洗钱工作负载的LLMOps栈，通过自托管开源模型提升服务性能，优化吞吐量、降低延迟并提高GPU利用率，展示合规级LLM服务的关键挑战。
摘要：
欺诈检测和反洗钱（AML）合规性是大型语言模型（LLMs）的高价值领域，但其服务需求与通用聊天工作负载差异显著。合规提示通常包含大量前缀、受约束的模式和丰富的证据，结合可重用的政策指令、风险分类法、交易或文档上下文以及短结构化输出如JSON标签或风险因素。这些特性使得前缀重用、KV缓存效率、运行时调优、模型编排和输出验证成为首要关注点。本文介绍了一种工作负载感知的LLMOps栈，利用自托管的开源模型如Meta Llama和Alibaba Qwen，结合vLLM风格的运行时调优、分页注意力、自动前缀缓存、多适配器服务、适配器和提示长度感知的批处理、睡眠/唤醒生命周期管理、推测解码以及可选的预填/解码拆分。为避免暴露机构特定数据，可重复性跟踪将公共合成AML数据集，包括IBM AML和SAML-D，转换为包含可重用政策文本、交易证据、类型定义和受约束输出的前缀密集型合规提示。我们还整合了LLM-as-judge质量门，使用确定性合规检查、参考指标、专家裁定校准数据（如可用）和多裁判评分标准。在公共合成AML工作负载和受控服务基准测试中，工作负载感知调优将吞吐量从612-650提升到3,600请求/小时，将P99延迟从31-38秒降至6.4-8.7秒，并将GPU利用率从12%提升到78%。这些结果表明，受监管的LLM性能是工作负载设计、服务优化和质量门控问题，而不仅仅是模型选择问题。
英文摘要：
Fraud detection and anti-money-laundering (AML) compliance are high-value domains for large language models (LLMs), but their serving requirements differ sharply from generic chat workloads. Compliance prompts are often prefix-heavy, schema-constrained, and evidence-rich, combining reusable policy instructions, risk taxonomies, transaction or document context, and short structured outputs such as JSON labels or risk factors. These properties make prefix reuse, KV-cache efficiency, runtime tuning, model orchestration, and output validation first-order systems concerns. This paper introduces a workload-aware LLMOps stack for fraud and AML workloads using self-hosted open-weight models such as Meta Llama and Alibaba Qwen. The stack combines vLLM-style runtime tuning, PagedAttention, Automatic Prefix Caching, multi-adapter serving, adapter and prompt-length-aware batching, sleep/wake lifecycle management, speculative decoding, and optional prefill/decode disaggregation. To avoid exposing institution-specific data, the reproducibility track converts public synthetic AML datasets, including IBM AML and SAML-D, into prefix-heavy compliance prompts with reusable policy text, transaction evidence, typology definitions, and schema-constrained outputs. We also incorporate an LLM-as-judge quality gate using deterministic compliance checks, reference metrics, expert-adjudicated calibration data where available, and multi-judge rubric scoring. Across public-synthetic AML workloads and controlled serving benchmarks, workload-aware tuning improved throughput from 612-650 to 3,600 requests/hour, reduced P99 latency from 31-38 seconds to 6.4-8.7 seconds, and increased GPU utilization from 12% to 78%. These results show that regulated LLM performance is a workload-design, serving-optimization, and quality-gating problem, not only a model-selection problem.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
