← 研踪 PaperTrailarXiv:2605.09481 · v2
TSNBench: Benchmarking LLM Proficiency in Time-Sensitive Networking
首次提交 2026-05-10 19:25
版本更新 2026-11-03 02:15
雷达观测 2026-11-04 23:41
摘要
We present TSNBench, the first benchmark for evaluating large language model (LLM) proficiency in Time-Sensitive Networking (TSN), a suite of IEEE 802.1 standards for deterministic communication with bounded latency in safety-critical domains such as autonomous vehicles, aviation, defense, and industrial automation. While LLMs have been extensively evaluated on general knowledge tasks, their capabilities in safety-critical networking domains remain largely unexplored. TSNBench comprises 939 expert-validated multiple-choice questions (MCQs) covering diverse TSN mechanisms, along with 100 open-ended Worst-Case Delay (WCD) computation tasks for each of Credit-Based Shaper (CBS) and Cyclic Queuing and Forwarding (CQF), across varying topologies and traffic flows. The ground-truth (GT) WCD values are computed using a verified Network Calculus (NC) solver for CBS and closed-form upper bounds for CQF. We evaluate 16 LLMs and find that although models achieve 67 to 95% accuracy on MCQs, they fail substantially on open-ended WCD computation. For CBS, the best-performing model, GPT-5, deviates from the GT by 150.2 $μ$s on average (mean absolute error, MAE), and its predicted WCD is within 50 $μ$s of the GT for only 39.4% of flows, while all other models exceed an MAE of 250 $μ$s. Providing the WCD equations in the prompt lowers the error for 9 of 10 models on CQF but only 2 of 9 on CBS. Such errors are large relative to TSN latency budgets and can lead to violations of real-time constraints and unsafe configurations. TSNBench demonstrates that MCQ benchmarks may overestimate LLM capabilities in safety-critical networking.
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-03 02:15 · 最新观测版本
TSNBench: Benchmarking LLM Proficiency in Time-Sensitive Networking
We present TSNBench, the first benchmark for evaluating large language model (LLM) proficiency in Time-Sensitive Networking (TSN), a suite of IEEE 802.1 standards for deterministic communication with bounded latency in safety-critical domains such as autonomous vehicles, aviation, defense, and industrial automation. While LLMs have been extensively evaluated on general knowledge tasks, their capabilities in safety-critical networking domains remain largely unexplored. TSNBench comprises 939 expert-validated multiple-choice questions (MCQs) covering diverse TSN mechanisms, along with 100 open-ended Worst-Case Delay (WCD) computation tasks for each of Credit-Based Shaper (CBS) and Cyclic Queuing and Forwarding (CQF), across varying topologies and traffic flows. The ground-truth (GT) WCD values are computed using a verified Network Calculus (NC) solver for CBS and closed-form upper bounds for CQF. We evaluate 16 LLMs and find that although models achieve 67 to 95% accuracy on MCQs, they fail substantially on open-ended WCD computation. For CBS, the best-performing model, GPT-5, deviates from the GT by 150.2 $μ$s on average (mean absolute error, MAE), and its predicted WCD is within 50 $μ$s of the GT for only 39.4% of flows, while all other models exceed an MAE of 250 $μ$s. Providing the WCD equations in the prompt lowers the error for 9 of 10 models on CQF but only 2 of 9 on CBS. Such errors are large relative to TSN latency budgets and can lead to violations of real-time constraints and unsafe configurations. TSNBench demonstrates that MCQ benchmarks may overestimate LLM capabilities in safety-critical networking.
查看此版本解读 →查看此版本原文 ↗