# Engineering Safe Autonomy: Governing AI Agents on Critical Infrastructure
The question of when it is safe to let an autonomous system operate machinery is becoming urgent. As organizations integrate artificial intelligence into their infrastructure, the boundaries of control must be clearly defined. There is no universal rule for this, but at the operating system level, a specific architectural approach is emerging that balances capability with safety.
### The Illusion of Unrestricted Power
Giving an AI root access to a machine feels empowering, allowing it to perform complex migrations or system configurations effortlessly. However, power breeds nondeterminism. You cannot guarantee outcomes, even with strict instructions. Just as we restrict human administrators from uncontrolled root access in production environments, we must deny the same privileges to AI agents. The goal is reproducibility and auditability—knowing exactly what changed and why. Unrestricted root access makes tracing failures impossible and turns the system into a black box.
### Redefining the Host’s Role
Historically, applications were tightly coupled to the underlying machine. If an app needed a library, it was installed on the base system, meaning changing the application also meant changing the machine underneath it. Modern architectures separate the application’s dependencies from the host. The host’s responsibility narrows to providing reliable boot times, the kernel, and essential runtime capabilities. This separation allows the platform to manage the applications securely and predictably, while the applications carry their own dependencies.
### Treating the System as a Versioned Artifact
Preventing configuration drift is important, but the true value lies in state management. By treating the base system as a versioned image rather than a mutable collection of packages, we ensure the machine transitions between explicitly defined states. Runtime is where these predefined states are executed, not where the next state is improvised on the fly. This approach allows us to trace operational issues back to the exact commit and build that introduced them. The system moves from one known state to another, rather than drifting silently through ad-hoc modifications.
### The Boundaries of Protection
An image-based system does not exist in a vacuum. AI agents often introduce external software—language packages, standalone binaries, or container images. While the base system is locked down, the agent might still have broad access to the host’s filesystem, network, or credentials. Protecting the operating system is not the same as protecting the entire machine from a compromised or overly permissive agent. Immutability alone does not shield the machine from a process that has unrestricted host, kernel, or disk access. The mistake is assuming that because the base system is protected, the entire runtime environment is secure.
### Isolation Through Ephemeral Environments
To mitigate this risk, the AI agent itself must be treated as a workload. Each session should execute within a temporary, isolated environment—such as a container or a virtual machine—created for the task’s duration and destroyed afterward. The level of isolation should match the threat model. Crucially, the agent does not need direct host access to observe system behavior or suggest modifications. By executing the agent in an isolated boundary, we limit the blast radius of any potential errors or malicious actions.
### The Proposal Lifecycle
The most valuable capability of an AI agent is its potential for self-healing and self-improvement. Self-healing involves executing pre-authorized responses, like restarting a failed service or rolling back a deployment. Self-improving, however, involves the AI proposing a new system state that has not yet been authorized. For this to be safe, the agent’s role must be strictly limited to observation and proposal. Any changes must flow through a formal pipeline—code review, automated testing, image building, cryptographic signing, and controlled deployment. The agent must never have the authority to approve its own changes, alter the pipeline, or force the host to consume an unverified image.
### A Layered Security Approach
Securing autonomous infrastructure requires a layered model. Protect the base system from runtime modifications, isolate the agent according to its required permissions, and ensure every proposed change undergoes a rigorous, reproducible pipeline. We must trust the architecture and the process, not the AI model itself. The boundaries around the model are what turn an agent’s observation into a safe, reproducible system state.
***
### FAQ
**Q1: Why can’t an AI agent just be given root access if it seems to work fine?**
A: Root access removes traceability and makes the system nondeterministic. Even if the agent succeeds 99% of the time, the one failure is unrecoverable or unauditable. Just like human admins, unrestricted root leads to unreproducible states and makes debugging impossible when things go wrong.
**Q2: How does immutability protect the machine if the AI can still access the filesystem?**
A: Immutability protects the base system from package drift and unauthorized runtime modifications. However, if the AI has host filesystem access, it can still alter application data or configurations outside the base image. Immutability is a boundary, not a complete sandbox.
**Q3: What is the difference between self-healing and self-improving in this context?**
A: Self-healing is the ability to restart, replace, or roll back services based on pre-defined, authorized responses. Self-improving involves the AI proposing structural changes to the system, which must go through a software factory pipeline for review, testing, and signing before being deployed.
**Q4: Should we trust the AI model to make safe infrastructure changes?**
A: No. We must trust the boundaries and the pipeline. The model can suggest, but the process must govern. Trusting the model alone is a security risk; trusting the architectural boundaries and the software delivery process is the only safeguard.
***
### Conclusion
Moving AI agents into critical infrastructure requires a fundamental shift in how we manage system state. By rejecting unrestricted root access, embracing immutable base systems, isolating AI workloads, and enforcing a strict proposal-and-review lifecycle, we can harness the benefits of autonomous systems without compromising security or stability. The goal is an architecture where the AI contributes to the evolution of the system without being allowed to rewrite the rules directly.
Thank you for reading
