Follow ZDNET: Add us as a preferred source on Google.
ZDNET’s key takeaways
- Strike a balance between AI agent autonomy and control.
- Context and purpose must be integrated into agent design.
- Evaluate agent configurations and the data they can access.
AI agents are advancing beyond basic chatbots, becoming fully capable digital workers empowered to act on applications and data. However, these expanded abilities bring significant security and governance challenges.
Think of your AI agents as enthusiastic but inexperienced interns—they need the same level of supervision and direction as human interns, according to experts at a panel during the recent Snowflake Summit in San Francisco. AI agents require clear instructions and close monitoring by human supervisors.
Also: How to build better AI agents for your business – without creating trust issues
An agent without boundaries can cause serious problems, the panelists from AI security firms agreed. “You might ask the agent to purchase shoes, and suddenly it has bought you a car,” said Mayank Agarwal, founder and CTO of Resolve AI.
Control, context, and purpose
“You need to carefully consider what permissions you grant the agent. You can’t simply assume an agent will behave appropriately. You must establish strict boundaries to restrict its actions.”
Beyond control, context and purpose are essential principles for creating and managing agents. “It’s not sufficient to understand what this agent was built to do. You also need to know details like whose authority it operates under and how it will handle the data it accesses,” said Nancy Wang, chief technology officer for 1Password.
Also: What you’ll pay for AI agents will be wildly variable and unpredictable
Professionals should abandon traditional software development approaches, as building and deploying agents today differs significantly from past software practices, Agarwal noted.
“Just two years ago, an engineer knew precisely how they would connect APIs across various systems,” he said. “Everything was highly predictable: A would call API B, B would process that data and call C, and so on. In the agentic world, it’s entirely unpredictable. The agent connects things dynamically. Give it a goal—solve this problem—and it explores every available path.”
This approach can create new kinds of problems that professionals and managers aren’t equipped to handle. The agent is “interacting with tools that can act on your behalf, so you can’t be sure whether these tools are leaking data,” Agarwal said. “The agent might retrieve information from one tool and use another tool to send it somewhere unauthorized.”
The threat of hidden AI
This concern highlights the threat of hidden AI, operating without oversight. “We had a client with 12 OpenClaw instances running in their environment, with access to API feeds, source code, and a contractor communicating via Telegram,” said Jason Merrick, senior vice president of product at Tenable. “What could possibly go wrong?”
Also: AI agents of chaos? New research shows how bots talking to bots can go sideways fast
Because of these challenges, understanding what agents do behind the scenes can be difficult. Questions will emerge, such as “Who actually performed this action on the system? Was it a human? A service account? Or an agent?” Wang said. “Your team likely doesn’t know, or there’s no definitive answer. Because today, agents appear like humans, but they might also resemble a service account, since they hold all your permissions.”
Therefore, a balance must be found between governance and access, as AI is a powerful productivity and innovation tool that needs some degree of independence. “You don’t want to block everything or create impenetrable firewalls,” Wang advised.
That need for balance also underscores why thorough human oversight is critical. “Examine the user-generated content employees are creating—through Copilot, Claude Chat, or Gemini,” Merrick recommended. “Review their configurations. Is AI improperly configured? What data is it accessing? And be prepared to take corrective action. Also, analyze the prompts themselves. What information are the prompts conveying?”
Bottom line: Clear instructions
This is where safeguards and established identity management practices become vital, Wang said. The greatest risk comes “from an agent with excessive permissions and long-lived credentials.”
Also: Can a newbie really vibe code an app? I tried Cursor and Replit to find out
The challenge lies in designing security and governance for what are “non-deterministic entities,” Wang continued. “It’s about allowing them to be creative while applying traditional instruction sets through SDKs. You want predictable controls, but you also don’t want to restrict them so much that you lose the productivity benefits.”
The essential takeaway for professionals is that agents, like interns, need “extremely clear instructions,” Wang said. “Even then, they may still deviate from the intended path. Whether you focus on governing agents or maintaining complete agent traces, it all comes down to full visibility, remediation, and ensuring you establish the right purpose from the start—and that purpose must carry through every step, every action the agent takes.”
