AI Agents Are Getting Flight Recorders. That Tells You Where the Real Problem Is.

Justin Boitano, vice president of enterprise computing at Nvidia, explained the idea to Axios at Black Hat earlier this month: the harness around an AI agent, he said, has visibility into everything the agent is doing. That is the flight recorder. "If you can get cybersecurity experts access to the flight recorders when these accidents happen, they can make a better determination on the right set of controls for the industry."

It is a useful framing. What I find more interesting is what the comparison implies about the problem.

A flight recorder does not prevent an accident. It preserves evidence so that investigators can understand what happened after one occurs. When you describe an agent's runtime harness as a flight recorder, you are making a specific claim about what you expect to need to reconstruct: a detailed record of what an autonomous system did, decided, reached for, and produced. You are also implicitly acknowledging that the record and the control are two different things.

That distinction is worth examining, because it has consequences for how we build agent deployments.

What SAFE actually proposes

On August 4, 2026, the Linux Foundation published a request for comments on the Shared AI Findings Exchange, which the Open Secure AI Alliance calls SAFE. The proposal was developed by a working group including Nvidia, Cisco, CrowdStrike, Hugging Face, and Red Hat. It was unveiled at Black Hat USA in Las Vegas as the alliance, which launched in late July, reported membership exceeding 120 organizations. The alliance builds on the Linux Foundation's Akrites initiative and the foundational work of the Open Source Security Foundation.

SAFE remains a proposal. The RFC defines proposed reporting timelines, but the working group and reporting infrastructure are not yet operational. No submission mechanism exists yet. No formal safe-harbor protects companies that voluntarily disclose potentially damaging details. What exists is a request for comments, a public GitHub repository for community feedback, and a working group charter. That is the appropriate way to describe it.

The RFC's mission statement is direct: "SAFE confidentially collects and analyzes AI incidents and near misses, promptly informs affected parties and turns recurring failures into shared, evidence-based controls that reduce systemic risk."

The draft specifies the types of incidents members would be required to report. An AI system accesses, exploits, disrupts, misuses, or modifies a third-party system without authorization. An agent causes third-party impact by escaping or bypassing a sandbox, network, identity, policy, or tool boundary. An agent accesses third-party confidential information. An agent continues to probe, access, exploit, or modify a production target after the operator knows or reasonably suspects the activity is unauthorized or outside the approved scope.

One sentence in the reporting compact stands out: "Intent does not determine whether an event is reportable. Believing that an environment was simulated may explain an incident, but it does not remove the duty to report it."

That is a specific response to a pattern that has appeared repeatedly in 2026 disclosures, including the Meta evaluation incident, where the company attributed the outcome to a misconfiguration of the test environment. Under the SAFE framework, that framing would explain the incident but not excuse the reporting obligation. The question is what the agent did, not whether anyone meant for it to happen.

The evidence they want to preserve

This is where the proposal becomes interesting in a way that a short summary of it does not quite capture.

Under SAFE, members would not simply report that an incident occurred. They would be required to preserve and provide affected organizations with the evidence needed for a complete forensic response. The RFC lists:

• Prompts, traces, tool calls, logs, configurations, model and safeguard versions, and third-party dependencies
• Agent and workload identities
• Permissions and credentials available during the run
• Human approval and intervention events
• Files and external artifacts created or modified
• Detection, containment, and recovery events
• A complete incident timeline
• Reproduction testing and remediation evidence

Read that list carefully. It is not a log of what a server received. It is a record of the conditions under which an autonomous system made decisions: what it was told to do, what it did along the way, which tools it invoked, whose identity it operated under, what it was authorized to reach, and what approvals or human interventions existed during the sequence.

The RFC proposes that incident analysis span a structured review framework covering eight layers of the operating stack:

• Model. Did the model recognize uncertainty, scope boundaries, and stop conditions?
• Instructions. Were authorization and environmental assumptions explicit and correct?
• Safeguards. Were classifiers, policies, approvals, and action limits operating as intended?
• Tools. Were credentials, permissions, spending, publishing, and execution constrained?
• Environment. Were network paths, isolation, targets, and data boundaries independently verified?
• Monitoring. Could operators detect and interrupt unexpected behavior in real time?
• Human operations. Were responsibilities, escalation paths, and kill procedures clear?
• Supply chain. Did a cloud, evaluation, data, or tooling partner invalidate assumed controls?

The structure is explicit about its purpose: to prevent any investigation from being reduced to the single phrase "model performance." When an agent causes harm, each of those eight layers represents a question that must be answered before the analysis is complete.

Agents create a different reconstruction problem

Security logging has always had a straightforward goal: preserve a trail of what happened so you can understand it later. For most software, that trail looked something like this. A request arrived, a resource was accessed, an outcome occurred. The actor behind the consequential operation was ultimately a person or a service acting on explicit instructions from a person.

Agents change that assumption in a specific way.

When an agent takes a consequential action, the sequence that produced it may be longer and less legible than a single API call or command. The agent may have interpreted an objective, evaluated options, selected a tool, constructed a request, received a response, updated its internal plan, and then acted. The final action is the visible artifact. Everything before it explains why that was the action the agent chose, and whether it was the action anyone intended.

I have written here about how the OpenAI agent involved in the Hugging Face incident was not acting against its instructions but pursuing its objective with the capabilities it had available. I have written about how the Replit incident showed how development and production boundaries can determine the consequence of an agent action. I have written about how the Meta evaluation incident showed that instructions and technical boundaries are not the same thing.

In each case, the surface record alone would not have been sufficient. You would have needed to understand the objective, the identity, the tools, the permissions, and the sequence of decisions. That is precisely the evidence SAFE proposes preserving.

A record is not a gate

The SAFE RFC makes a point that I think practitioners can underweight: "Learning is separate from enforcement. Confidential review should encourage candid reporting, while regulators and affected parties retain their legal rights."

That separation cuts both ways. It means that what SAFE learns from an incident does not become regulatory enforcement. It also means that SAFE's learning process is distinct from the question of whether an action should have occurred in the first place.

Evidence after an action is not a control over whether the action happens. Flight data recorders exist because investigations are valuable, not because recording an approach vector prevents an accident. The record and the control solve different problems.

The SAFE proposal addresses accountability and ecosystem learning: after an agent causes harm or nearly does, SAFE would create a confidential channel to understand what the evidence shows, identify patterns across the industry, and produce recommendations that reduce risk for future deployments. That is genuinely valuable. An ecosystem with shared visibility into agent failures will learn faster than one where each organization investigates in isolation, if it investigates at all.

But accountability after an incident is a different capability than deciding whether a particular action should execute at all. One answers: what happened? The other answers: should this happen?

Both matter. The distinction is worth keeping clear, because conflating them produces a specific failure mode. Organizations that invest in post-incident evidence may believe they have sufficient controls. They do not. The evidence tells you what went wrong after it has already gone wrong. The control determines whether it goes wrong in the first place.

What the evidence list signals

I think the more interesting observation about SAFE is not what it proposes but what it assumes.

Whenever a standard-setting effort produces an evidence framework, it is making a claim about what a reasonable investigator would need to reconstruct what happened. Aviation requires specific flight data because investigators have determined that data is necessary to understand a mishap. Medical device reporting requires specific documentation because regulators have determined those records are necessary to evaluate device-related harm. The evidence framework reflects an investigation theory.

To me, SAFE is making a similar claim about AI agents: that understanding an agent incident requires the prompts, the traces, the tool calls, the identities, the permissions, and the credentials in play at execution time. The RFC puts it plainly: "Trust is not a control; shared evidence and verifiable improvement are how trust is earned."

That list represents an emerging theory of what agent accountability requires. It is not yet a standard. The proposal has no implementation mechanism and no formal safe-harbor. But the fact that an alliance of more than 120 technology organizations, including some of the most significant infrastructure and security companies in the industry, has converged on that list as the minimum evidence for meaningful incident analysis tells you something about how seriously agent behavior is being taken as an operational risk.

We are beginning to define, at the industry level, what it means to know what an agent did.

That is a newer problem than it sounds. A large part of existing security infrastructure was designed around actors who were ultimately human or explicitly instructed by humans. An agent operating with a degree of autonomy, across a sequence of decisions, against resources it was broadly authorized to reach, does not fit neatly into that model. The evidence requirements are different. The reconstruction problem is different. SAFE's evidence list is one of the first serious industry attempts to say: here is what you would need.

Questions worth asking about your agent deployment

SAFE's evidence framework is useful as a diagnostic regardless of whether any organization adopts it. Think of each item in the evidence list not as a compliance requirement but as a question about your current visibility.

If your agent takes a consequential action tomorrow, can you establish what objective it was pursuing at the time?

Can you establish whose identity it operated under, and what that identity was authorized to do?

Can you establish which tools it invoked and in what sequence?

Can you establish what permissions and credentials were available at the moment of the consequential action?

Can you establish what human approval or intervention events, if any, existed during the sequence?

Can you distinguish between what the agent proposed doing and what it actually executed?

If the honest answer to any of those is no, you have a gap in your evidence layer that SAFE is proposing to formalize as a reportable absence. But more practically, you have a gap in your ability to understand what your agent is doing in your own environment, before anything goes wrong.

The harder question underneath the flight recorder

The flight recorder framing is useful. What it slightly obscures is that the question is not just whether you have a recorder. It is whether what you would find on it would actually explain what happened.

Aviation built its evidence conventions over decades of accident investigation. Investigators knew what data they needed because they had spent years reconstructing failures with insufficient information. The conventions for what a flight recorder must capture reflect accumulated experience about what matters for understanding a mishap.

We are at the beginning of that process for AI agents. To me, SAFE looks like one of the clearest early industry attempts to say: here is what evidence an investigation would need. Here is what a meaningful record looks like for an autonomous system. Here is what you would have to preserve to reconstruct the sequence that produced a consequential action.

If an agent makes a consequential decision tomorrow morning, what evidence will you actually have by the time you need to understand what happened?