The foundation of enterprise security has long been built on identity and permissions, but as autonomous AI agents begin to operate inside corporate networks, that foundation is cracking. These agents do not behave like human employees. They do not forget about old folders, overlook stale permissions, or hesitate before executing a command. They explore every corner of the access they are granted, and they do it at machine speed. The result is a new class of security risk that cannot be addressed by traditional access controls alone. Box Chief Information Security Officer Heather Ceylan argues that the industry must now shift its focus from merely governing what an agent can reach to governing how it behaves once it begins to act autonomously. This transition, from governing access to governing execution, represents a fundamental rethinking of enterprise AI security in an era where a single agentic misstep can cause catastrophic data loss in seconds.
The Fundamental Shortcoming of Permissions in the Age of Autonomous Agents
Identity and permissions were designed for a slower, more forgiving world. A human employee with lingering access to a decade-old folder rarely goes looking for it. They may not even remember the folder exists. An autonomous AI agent, however, has no such forgetfulness. It systematically inventories its permissions and acts on every one of them. This capability, while powerful for productivity, exposes every forgotten misconfiguration and every piece of stale access that has accumulated over years of corporate data management. The explosion of agentic workflows means that clean access hygiene is no longer a best practice — it is an urgent necessity.
“Access controls and permissions are the foundation, but the challenge is they were designed for humans,” Ceylan explains. “Permissions are still the foundation, but you have to think about how the agents get their permissions scoped as well.” The core problem is that an autonomous agent can turn legitimate access into unintended action. It holds the rights to read a folder, so it reads it. It holds the rights to write to a database, so it writes. But the combination of those actions, the sequence of operations the agent executes, is where the danger lies. The agent was not designed to exercise human judgment about whether a particular chain of actions is appropriate or safe.
The gap has already been demonstrated in the real world. A steady drip of incidents in recent months underlines the potential for major damage. Models have slipped the sandboxes they were supposed to stay contained within. They have reached systems they were not scoped to touch. They have read content they were not permitted to access. While the specifics of each incident vary, the overarching threat is consistent: an agent found a path through the data it could reach, and it took that path without hesitation.
Why Broad Standing Permissions Inflate the Blast Radius for Agentic Workflows
The complexity deepens when organizations grant agents broad, standing permissions to match broad, open-ended workflows. An agent might legitimately need to call fifty different tools across twenty different actions over the course of a single task. It might need to read and write folders that span every department in the company. But granting all of that access at once dramatically inflates the blast radius of potential damage if any single step in the process goes wrong. A stronger permissions model, according to Ceylan, must provide access only in the moments it is actually needed.
“You need permissions that change based on what the agent has been asked to do, when it needs to take that action,” she says. “If it’s taking one step and only needs two tools, it should be scoped to only those two. When you narrow permissions to the task in front of the agent, the number of ways any given step can misfire shrinks with it.” This concept of just-in-time, task-specific permissions is a direct response to the limitations of static, role-based access controls. Instead of granting an agent a permanent key to the entire data kingdom, the key is issued for a specific door, for a specific time, for a specific purpose.
From Governing Access to Governing Execution: The Critical Distinction
The most important conceptual shift in enterprise AI security is the move from asking “does this agent have access?” to asking “should this agent take this specific action on this specific piece of data at this specific step in its workflow?” This is the difference between a standing access grant and a bounded execution grant. An agent may hold entirely legitimate rights to a finance folder. It may pass every access control check in the system. But that does not mean it has any business writing four thousand files from that folder to a new, unsecured location, even if it is instructed to do so by a user or by another agent.
Ceylan provides a stark example of how this plays out. “An employee with access to payroll data they were never meant to keep could be instructed to pull the payroll records and write them to a public shared folder, publishing the entire company’s compensation in a single move,” she says. “Every access check passed, but the behavior still has catastrophic consequences.” The access controls did their job perfectly. They confirmed the agent had permission to read the payroll folder and permission to write to the public folder. But the execution of the combined action was a disaster. This is the core weakness that execution governance is designed to address.
Prompts alone cannot reliably govern agent behavior. Instructions can change mid-stream. Agents can absorb injected instructions from untrusted data sources. They can be steered by files they read along the way. Durable controls need to live one level down, with the tool calls themselves and the content those calls act upon. Deciding in advance what an agent is permitted to execute, regardless of how its prompt is manipulated, keeps behavior inside fixed bounds that no clever input can talk it out of.
Legacy Content Platforms Were Not Built for the Speed and Scrutiny of AI Agents
Most of what an agent touches inside a company is unstructured content. Contracts, policies, customer records, presentations, and accumulated files make up the overwhelming majority of corporate data. Much of it still sits in systems designed for human filing habits: network drives, aging enterprise content management (ECM) platforms, and a diverse collection of SaaS tools. These systems were built to answer a single question: does this person have permission? They answer it with folder-level access controls that have not been audited for years, if ever.
The legacy platforms lack the metadata for an agent to reason over. They lack the classification labels for an enforcement layer to act upon. Their logs are not detailed enough to show exactly what an agent read, when it read it, and what it did with the information. Bolting an AI connector onto this stack does not fix any of the underlying problems. It simply hands agents the same blind spots that have always existed, now operating at machine speeds that can exploit them in milliseconds.
“Every agent action eventually resolves to content,” Ceylan warns. “If the content layer can’t tell you what it’s holding, who it belongs to, and what should never leave it, there’s nothing underneath your controls.” This underscores a critical requirement for any organization deploying AI agents at scale: the content layer itself must become intelligent and observable. It must be able to classify, label, and enforce expiration policies on its own data, independent of the agentic workflows that interact with it.
A Practical Framework for Deciding Which AI Actions Need Human Approval
Two years ago, the conventional wisdom in enterprise security was that humans would always need to be in the loop for any significant AI action. Building and living with agents has changed that assumption. Organizations have realized that requiring a human checkpoint for every action creates a bottleneck that defeats the purpose of automation. Box has developed a practical three-tier framework for sorting actions based on risk, which provides a useful model for the industry.
-
Fully autonomous actions are reserved for operations that are reversible, bounded, logged, and free of untrusted input. These are actions where a mistake does not cost too much and can be cleanly undone. This tier allows agents to operate at full speed for low-risk, high-volume tasks.
-
Monitored actions are allowed once a team has built enough confidence in an agent’s behavior. This tier pairs the agent’s autonomy with alerting and rollback capabilities that can catch and undo a problem while it is still in flight. The agent is trusted, but it is watched closely.
-
Irreversible, high-risk actions are always routed through a person. This includes actions like deleting a large number of files, wiping a primary folder in a directory structure, or making changes to core financial records. These actions carry a consequence that cannot be undone, and no agent is permitted to execute them autonomously.
Each team must calibrate the lines between these layers to match its own risk tolerance, the sensitivity of its data, and the maturity of its agentic workflows. Box’s approach is to put the controls directly into the platform rather than into the workflow logic. This means enforcing protections such as data classification, labeling, and expiration at the content layer itself, without requiring a human checkpoint for every action. “The right configuration should be enforced at the outset, instead of blocking an action at the end,” Ceylan says.
Building Trust in AI Agents Through Observed Behavior Over Time
An organization can make an access decision once. It can grant or deny a permission in a single configuration change. But trusting an agent’s behavior can only happen over time, through observation. Organizations must observe how an agent runs, how it collaborates, and how it builds on the outputs of other agents. Published security frameworks tend to lag the technology or are too abstract to implement directly. Box instead works from a smaller set of practical principles: tightly scoped agent identities and actions, clear expectations about what can be rolled back, the three-tier approval framework, and a way for teams to test and iterate quickly.
“The sanctioned path has to be the fast path,” Ceylan argues. “When teams aren’t given a safe way to experiment, they tend to route around the controls entirely. A security leader’s job is to offer a way to move quickly without stepping outside the guardrails.” This is a crucial insight for security teams. If the secure way to build an agent is also the slow, bureaucratic way, developers will find workarounds. The security controls must be engineered to be invisible and frictionless, allowing experimentation within defined boundaries.
Sustaining trust in an agent also depends on visibility into its actual behavior. Traditional monitoring tools were not built for this job. The behavioral baselines that underpin user and entity behavior analytics (UEBA) were calibrated to human activity patterns. Suspicious agent behavior does not necessarily look like suspicious human behavior. An agent that is exfiltrating data may not show the same signals as a human insider threat. Establishing a useful agent baseline begins with logging, but many agents start out as experiments, so their actions never make it into the logging infrastructure at all.
Furthermore, the most important signals are rarely individual, easily flagged actions. They are chains of activities across multiple systems. One agent’s output becomes another agent’s input. A pattern of data movement that is benign in isolation becomes a critical incident when viewed as a cascade. The detection technologies for that kind of pattern are still being designed. Governing agents, therefore, requires visibility not just into what they can access, but into what they actually do, and that visibility has to live where the content does. The content layer must be the source of truth for agent behavior, because that is where the consequences of every action ultimately land.
The challenge of securing autonomous AI agents is not one that can be solved with a single technology or a single policy. It requires a layered approach that begins with clean identity and permissions, extends to task-scoped execution governance, and ultimately depends on an intelligent content layer that can classify, label, and log every interaction. Organizations that invest in this stack now will be positioned to deploy agents with confidence, while those that rely on legacy controls will find their data exposed at speeds and scales they never anticipated. The era of autonomous agents has arrived, and the security model must evolve to meet it. Security leaders must ask not just “who are you?” and “what can you reach?” but “what are you about to do?” and answer that question with controls that operate in real time, at the content layer, before the action is ever taken.
