TSNBench: Benchmarking LLM Proficiency in Time-Sensitive Networking
Rubi Debnath, Daniel Bujosa Mateu, Luxi Zhao, Silviu S. Craciunas, Paul Pop, Sebastian Steinhorst
We present TSNBench, the first benchmark for evaluating large language model (LLM) proficiency in Time-Sensitive Networking (TSN), a suite of IEEE 802.1 standards for deterministic communication with bounded latency in safety-critical domains such as autonomous vehicles, aviation, defense, and industrial automation. While LLMs have been extensively evaluated on general knowledge tasks, their capabilities in safety-critical networking domains remain largely unexplored. TSNBench comprises 939 expert-validated multiple-choice questions (MCQs) covering diverse TSN mechanisms, along with 100 open-ended Worst-Case Delay (WCD) computation tasks for each of Credit-Based Shaper (CBS) and Cyclic Queuing and Forwarding (CQF), across varying topologies and traffic flows. The ground-truth (GT) WCD values are computed using a verified Network Calculus (NC) solver for CBS and closed-form upper bounds for CQF. We evaluate 16 LLMs and find that although models achieve 67 to 95% accuracy on MCQs, they fail substantially on open-ended WCD computation. For CBS, the best-performing model, GPT-5, deviates from the GT by 150.2 $μ$s on average (mean absolute error, MAE), and its predicted WCD is within 50 $μ$s of the GT for only 39.4% of flows, while all other models exceed an MAE of 250 $μ$s. Providing the WCD equations in the prompt lowers the error for 9 of 10 models on CQF but only 2 of 9 on CBS. Such errors are large relative to TSN latency budgets and can lead to violations of real-time constraints and unsafe configurations. TSNBench demonstrates that MCQ benchmarks may overestimate LLM capabilities in safety-critical networking.
尚无 AI 解读，展示原文摘要