TSNBench：评估大语言模型在时间敏感网络中能力的基准测试
TSNBench: Benchmarking LLM Proficiency in Time-Sensitive Networking
AI总结：
本文提出TSNBench，首个评估大语言模型在时间敏感网络中的能力的基准测试，包含939道专家验证的多项选择题和100道开放性问题，揭示LLM在安全关键网络领域的能力不足。
摘要：
本文提出了TSNBench，首个用于评估大语言模型在时间敏感网络中的能力的基准测试。该基准测试包含939道由专家验证的多项选择题和100道开放性问题，涵盖了多样化的TSN机制，以及针对Credit-Based Shaper (CBS)和Cyclic Queuing and Forwarding (CQF)的Worst-Case Delay (WCD)计算任务。多项选择题的答案由领域专家验证，而开放性问题的地面真实WCD值通过已验证的网络计算（Network Calculus, NC）求解器计算得出。我们评估了16个大语言模型，发现尽管模型在多项选择题上达到了67至95%的准确率，但在开放性WCD计算上表现不佳。对于CBS，只有GPT-5的MAPE为36.2%，而大多数模型超过80%。对于CQF，最佳模型的MAPE为41.8%，大多数模型集中在80%至100%之间。这些误差相对于TSN延迟预算较大，可能导致实时约束的违反和不安全的配置。TSNBench表明，多项选择题基准可能高估大语言模型在安全关键网络领域的能力。
英文摘要：
We present TSNBench, the first benchmark for evaluating large language model (LLM) proficiency in Time-Sensitive Networking (TSN), a suite of IEEE 802.1 standards for deterministic communication with bounded latency in safety-critical domains such as autonomous vehicles, aviation, defense, and industrial automation. While LLMs have been extensively evaluated on general knowledge tasks, their capabilities in safety-critical networking domains remain largely unexplored. TSNBench comprises 939 expert-validated multiple-choice questions (MCQs) covering diverse TSN mechanisms, along with 100 open-ended Worst-Case Delay (WCD) computation tasks for each of Credit-Based Shaper (CBS) and Cyclic Queuing and Forwarding (CQF), across varying topologies and traffic flows. The ground-truth (GT) WCD values are computed using a verified Network Calculus (NC) solver for CBS and closed-form upper bounds for CQF. We evaluate 16 LLMs and find that although models achieve 67 to 95% accuracy on MCQs, they fail substantially on open-ended WCD computation. For CBS, the best-performing model, GPT-5, deviates from the GT by 150.2 $μ$s on average (mean absolute error, MAE), and its predicted WCD is within 50 $μ$s of the GT for only 39.4% of flows, while all other models exceed an MAE of 250 $μ$s. Providing the WCD equations in the prompt lowers the error for 9 of 10 models on CQF but only 2 of 9 on CBS. Such errors are large relative to TSN latency budgets and can lead to violations of real-time constraints and unsafe configurations. TSNBench demonstrates that MCQ benchmarks may overestimate LLM capabilities in safety-critical networking.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
