SPECIAL FEATURE | AI agent security needs more than prompt protection, study says
A technical report released by AI security and sandboxing platform KOSA8 Labs, argues that organizations deploying autonomous AI systems need security controls that remain effective even when an agent’s execution environment has been compromised.
AI agents that can write code, access company data and execute commands introduce a security problem that conventional chatbot safeguards cannot fully address: What happens when an agent is manipulated into doing something it should not?
A technical report released by AI security and sandboxing platform KOSA8 Labs, argues that organizations deploying autonomous AI systems need security controls that remain effective even when an agent’s execution environment has been compromised. These controls include hardware-based isolation, restricted network access, protected credentials, human approval for sensitive actions and tamper-evident audit records.
The 46-page report, titled Sealed Machines for Autonomous Agents: Design, Security Analysis and Evaluation of kosa8, a Local-First Security Infrastructure for AI Agents, evaluates the company’s security platform against 13 requirements for AI agents operating in regulated environments.
The authors argue that security must extend beyond the language model itself to the infrastructure that determines what an agent can access, execute and change.
The report describes its central concern this way: “The practical conclusion is uncomfortable and, we argue, unavoidable: a deployed agent must be assumed to be steerable by an adversary.”
The findings are relevant as organizations move from conversational AI assistants toward systems that can independently execute software, interact with business applications, retrieve sensitive information and call external services.
However, the report’s findings require context. KOSA8 Labs conducted the technical evaluation of its own software, and its comparison with competing products relied on publicly available documentation rather than equivalent hands-on testing across all systems.
From answering questions to taking action
Traditional chatbots primarily generate responses to user prompts. AI agents, by contrast, can use tools, execute commands, modify files and interact with external services to complete tasks.
That expanded capability also increases the consequences of a security failure. An agent that processes untrusted content may encounter malicious instructions embedded in documents, websites, software repositories or tool descriptions. If the agent has access to sensitive information or operational systems, successful manipulation could lead to unauthorized actions.
The report examines this problem through the possibility that an attacker could gain control of an agent’s inputs and execution environment. Rather than relying exclusively on the model to recognize malicious instructions, KOSA8’s approach seeks to restrict what the agent can do after a compromise.
This shifts the security focus from preventing every possible prompt injection to containing its consequences.
“The practical conclusion is uncomfortable and, we argue, unavoidable: a deployed agent must be assumed to be steerable by an adversary,” the report states.
Containing a compromised agent
KOSA8 runs each agent inside a hardware-virtualized microvirtual machine, or microVM, with its own kernel. The system is designed to isolate the agent from the host environment and enforce network restrictions outside the guest machine.
Under this architecture, network traffic is controlled by the host rather than relying solely on restrictions implemented inside the agent’s environment. This is intended to prevent a compromised agent from bypassing its assigned network policy.
The report says testing on an Apple M4 host found that the system blocked all tested network probes outside the configured policy across four policy modes. The tests included IPv6, cloud metadata and gateway addresses.
These results demonstrate the reported behavior under the test conditions, but they do not establish that every possible attack can be prevented. The report acknowledges residual risks, including the possibility of reaching unintended services at an allowed address and using wildcard domain rules to create a DNS-based covert channel.
The distinction matters because network controls can limit an agent’s access without necessarily determining whether every permitted action is safe.
Keeping credentials away from AI agents
Another component is a credential broker designed to keep API keys out of the agent’s virtual machine. Instead of giving the agent direct access to a secret, the host adds the credential when an authorized request passes through a controlled network route.
“The real key never enters guest memory because the guest never receives it; it is added by a host process the guest can reach only through the pinhole,” the report states.
The design aims to reduce the risk that an agent manipulated by malicious instructions could extract and disclose credentials stored in its execution environment.
However, keeping a key out of guest memory does not eliminate the risk of misuse. An agent may still use a credential through an authorized route to perform actions permitted by that credential.
The report also identifies a limitation involving one built-in GitHub Copilot integration. Because the integration cannot accept a base URL, it still passes its token into the environment, a behavior the report flags with a warning.
This illustrates a broader implementation challenge: Security controls depend not only on the underlying architecture but also on how individual tools and integrations handle credentials.
Human approval and reversible execution
KOSA8 also provides a mechanism for requiring one or two designated people to approve sensitive actions. The approval is tied to the specific request, while the system records the decision in an audit log.
The aim is to prevent an agent from independently carrying out certain consequential operations without the required human authorization.
The system also supports checkpointing, rewinding and forking running virtual machines. These features are intended to help operators recover from mistakes or investigate suspicious activity without necessarily rebuilding an agent’s environment from scratch.
In its tests, the report measured a median checkpoint time of 638 milliseconds and a median rewind time of 520 milliseconds.
However, reversing a virtual machine’s state cannot undo every consequence of an agent’s actions. An email that has already been sent, a code commit that has been pushed or an external API call that has been completed may remain effective even after the environment is rewound.
The report therefore distinguishes between restoring an execution environment and reversing actions that have already affected external systems.
What the technical evaluation found
KOSA8 Labs reported that all 1,142 unit tests for the evaluated software release passed, including 449 tests across 20 security-critical packages.
The evaluation covered version 0.9.36, released Oct. 16, and used an Apple M4 host running macOS 26.0.1. The report also describes fuzzing campaigns and measurements of network enforcement and sandbox performance.
Full sandbox creation took about four seconds, considerably longer than the measured checkpoint and rewind operations. The authors also observed an unexplained slower mode when checkpointing and rewinding a recently restored machine.
These limitations are important when evaluating the platform for production use, particularly in environments where agents need to start quickly or recover predictably.
The report’s performance results are based on the stated test environment. They should not automatically be generalized to other hardware, operating systems or deployment configurations.
Vendor comparison needs independent testing
The report compares KOSA8 with 10 other production systems against a framework of 13 security requirements. Based on publicly available documentation, the authors concluded that KOSA8 was the only system among the 11 evaluated that fully met all 13 requirements. The next most complete systems fully met four.
The comparison is a claim made by the report’s authors, not the result of an independent, same-hardware benchmark of all the products. Differences in documentation, evaluation methods and product configurations may affect the results.
The report itself identifies independent comparative testing as an important missing step.
It also acknowledges that KOSA8 has not yet undergone an external penetration test and does not yet have a SOC 2 report or a product certificate under the Cryptographic Module Validation Program. The authors list these as future assurance priorities.
Other limitations include platform differences, incomplete identity features and the need for further testing. The report also notes that person-created sandboxes have a default network policy that permits access broadly, while agent-created and brokered sandboxes default to denying access. It recommends changing the broader default before general availability.
These disclosures help define the current scope of the system’s security claims and the work that remains before organizations can assess its suitability for particular regulatory or operational requirements.
Security depends on controlling consequences
The report’s central argument is that AI security should not depend entirely on whether a model can recognize and resist malicious instructions. Organizations also need controls around the systems that agents use to execute tasks, access information and affect business operations.
Local-first deployment can give organizations greater control over where agents run and how their activity is restricted. But running software on customer-controlled hardware does not, by itself, guarantee security. Isolation, credential handling, network policy, human oversight and independent assurance all remain relevant.
KOSA8 Labs concludes: “Agents will be steered by adversaries; the only question is what they can do when it happens.”
For organizations considering autonomous AI systems, the practical question is whether their infrastructure can limit an agent’s privileges, establish accountability for sensitive actions and contain the damage when safeguards fail. The report offers one technical approach to that problem, while its own disclosed limitations underscore the need for independent validation.
Full disclosure: All news articles published on the TechSabado website are written by human journalists, unless otherwise specified. Final text editing is also performed by human editors, with artificial intelligence (AI) used only to assist with additional grammar and style guide corrections..
————————————————————————-
TEN YEARS OF TECHSABADO!
PLEASE LIKE our FACEBOOK PAGE and SUBSCRIBE to OUR YOUTUBE & SPOTIFY channels.
WATCH TECHSABADO ON OUR YOUTUBE CHANNEL:
WE ARE ALSO ON SPOTIFY
WATCH OUR OTHER YOUTUBE CHANNELS:
PLEASE LIKE our FACEBOOK PAGE and SUBSCRIBE to OUR YOUTUBE CHANNEL.
