Reliability Engineering for AI Systems: Challenges, Methods, and Directions

摘要变化

AI reliability concerns whether an AI system performs its intended function dependably over a stated period and under stated operating conditions, with statedclear evidence. As these systems become more autonomous, thatthis function includesshould include more than only a correct output. Retrieval, memory, tool use, permissions, human oversight, and interactions among systems must operate consistently and safely, and, for a generative systemsAI, so must the reasoning process that produces the output. Average benchmark accuracy only measures system capability; it does not quantify this broader reliability claim. This paper adapts established reliability engineering methods, from failure definitions and operational envelopes to FMEA, accelerated testing, field monitoring, and reliability growth, to AI systems. A four-level diagnostic framework classifies failures as component, operational-loop, agentic-conduct, or network and governance failures. Test, evaluation, verification, and validation (TEVV), sequential monitoring, and FRACAS create and refresh evidence. The SMART framework provides statistical guidance for measurement, analysis, assessment, and test planning;, and the NIST's AI Riskrisk Managementmanagement Frameworkframework provides organizational guidance for governance, evaluation, monitoring, and mitigation. Three cases illustrateare thediscussed program:to adversarialillustrate testingthese ofprograms. aIt convolutionalis neuralshown network,that perception-errorwhile propagation,the andestablished autonomous-vehiclereliability disengagements.engineering Establishedprinciples reliabilitycan engineeringstill providesprovide a usable foundation;, new measurements and safety guardrails are stillurgently needed as these systems are self-evolving at a fast pace.