FlashRT：用于引导智能体部署实时多模态应用的智能体框架
FlashRT: Agent Harness for Guiding Agents to Deploy Real-Time Multimodal Applications
查看机构详情
- Carnegie Mellon University（卡内基梅隆大学）
- AMD（超威半导体公司）
- University at Buffalo（纽约州立大学水牛城分校）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
研究实时多模态应用部署难题，提出FlashRT智能体框架。通过新范式引导编码智能体多阶段转换，将参考实现转为高效部署，在不同GPU上显著提升性能，尤其在专家优化不成熟平台更具扩展性。
中文摘要 AI 辅助
实时多模态应用，如语音智能体和交互式视频生成，将异构模型组合成管道，其高效部署需要针对放置、流和模型内并行性做出特定于应用的决策。现有服务系统和自动并行编译器的转换有限且工作量假设固定。我们提出了FlashRT，一种智能体框架，它引导编码智能体将简单的开发者编写的参考实现提升为优化的多GPU部署，灵活权衡延迟和吞吐量等目标指标。通过新的程序链范式，FlashRT指导通用编码智能体进行多阶段转换过程，包括将参考转换为中间表示以捕获数据依赖和持久状态范围，通过顺序解释器验证此IR，并进行静态分析以识别候选转换。然后，智能体在测量门控优化循环下迭代实现、验证和基准测试每个候选，以产生跨越不同硬件预算的有效部署。在各种应用中，FlashRT将参考实现转换为高效部署，在NVIDIA B200 GPU上延迟降低高达约70倍，吞吐量提高2.8倍。在AMD MI355X GPU上，FlashRT在匹配峰值延迟降低的同时将峰值吞吐量提高到3.6倍，表明在专家优化不太成熟的平台上，智能体驱动的优化更具可扩展性。实际上，对于Qwen3 - Omni文本到音频推理，与AMD MI355X上的专家vLLM - Omni实现相比，FlashRT将响应延迟降低了65%。
英文摘要
Real-time multimodal applications, including voice agents and interactive video generation, compose heterogeneous models into pipelines whose efficient deployment requires application-specific decisions about placement, streaming, and intra-model parallelism. Existing serving systems and auto-parallelism compilers commit to limited transformations and fixed workload assumptions, so achieving high performance on a new application requires hand-crafting an efficient implementation. We present FlashRT, an agent harness that guides coding agents to lift simple developer-written reference implementations into optimized multi-GPU deployments that flexibly weigh target metrics like latency and throughput. Using a new chain-of-program paradigm, FlashRT directs a generic coding agent through a multi-pass transformation process where an agent transforms the reference into an intermediate representation (IR) to capture data dependencies and persistent-state scopes, validates this IR via a sequential interpreter, and performs static analyses to identify candidate transformations. Then, the agent iteratively implements, verifies, and benchmarks each candidate under a measurement-gated optimization loop to produce effective deployments that span different hardware budgets. Across various applications, including video world models and multimodal LLMs, FlashRT converts reference implementations into highly efficient deployments, delivering up to ~70x latency reduction and 2.8x throughput improvement on NVIDIA B200 GPUs. On AMD MI355X GPUs, FlashRT matches the peak latency reduction while increasing peak throughput improvement to 3.6x, demonstrating that agent-driven optimization can be more scalable on platforms with less mature expert optimization. In fact, for Qwen3-Omni text-to-audio inference, FlashRT reduces response latency by 65% compared to the expert vLLM-Omni implementation on AMD MI355X.
