Organizations training highly capable AI agents are encouraged to adopt technical safeguards, including rapid kill switches and external audits, to prevent system failures. This comes as NIST CAISSI, NSA SISC, and CISA issued voluntary guidance for managing “SI control incidents,” defined as instances where an AI agent takes significant uninstructed actions that bypass operator policies, oversight, or controls. The framework covers both actual failures and near-misses, urging companies to employ red-teaming and hire independent auditors to verify that monitoring systems are effective.
The guidelines require monitoring all layers of an agent’s reasoning and outputs through programmatic and SI-based tools that agents cannot access or modify. In the event of an incident, organizations must preserve weights and intermediate checkpoints and pause the deployment of similar agents until controls are re-verified. Final reports are required for all incidents, with cyber events reported to CISA and SI-specific information shared with CAISSI and NSA SISC.