Before I Trust an Agent, I Try to Break Its Box

Software engineer Kunal Kushwaha released testing architecture details for Koala, a sandboxing system that runs one dedicated Linux virtual machine per autonomous AI agent across macOS and Linux hosts. Koala isolates network policies and API credentials on the host instead of relying on guest honesty. According to kunalkushwaha.github.io's report, validating security boundaries required hostile internal attacks against the hypervisor rather than trusting status telemetry from the guest itself.

This technical analysis details the adversarial test suite, crash injections, containment mitigations, and implementation bugs discovered while probing Koala from compromised virtual machines. It is aimed at platform engineers, system architects, and developers who deploy autonomous agents that consume untrusted third-party data and require verified virtualization boundaries before delegating host system access.

01Real hypervisors and host witness listeners

Every adversarial test executes against production hypervisors rather than simulated mocks. The test framework drives Apple Virtualization framework on macOS hardware, while Linux test cycles target Kernel-based Virtual Machine and Cloud Hypervisor. Because agents ingest untrusted messages, web pages, and files, test suites assume an attacker already controls root inside the guest VM.

Each hostile run divides roles among three distinct system components. An attacker virtual machine runs as root to simulate a fully compromised agent. Beside it, an uncompromised bystander virtual machine responds to operational health checks every half second, ensuring neighboring workloads stay functional even during aggressive penetration attempts.

Host witnesses evaluate test outcomes to prevent compromised guests from faking results. Network listeners record every transit packet, while cryptographic file hashes are captured before and after each run alongside planted canary secrets. Because guest code can report access denied while escaping, every outcome is judged strictly by host observations.

02Command port races and poisoned symlink probes

During penetration testing, an attacker virtual machine attempted to connect to ten host ports opened for command input and output, but all ten connections were blocked. The attacker also launched 80,000 to 86,000 parallel race attempts per command port against Koala, winning zero times. Probes attempting to inspect neighboring disk partitions or attach victim volumes were similarly refused.

The hostile guest planted a symlink inside a shared folder pointing directly at a host credential, waiting for the file copy command to fetch it. Koala refused the symlink and left the host credential untouched, although mid-copy swap scenarios remain untested. This test illustrates that virtualization boundaries frequently fail where the host reaches inside the guest rather than where guests reach outward.

Network fuzzing subjected the boundary to spoofed packets, fragmented IPv6 payloads, router impersonation, and DNS rebinding attacks. In all cases, only two authorized host listeners accepted traffic. Under a 2,000-query DNS flood limited to sixteen concurrent lookups, the helper answered 486 queries, immediately rejected remaining requests, and survived without crashing.

03Manager SIGKILL checks and endurance runs

Adversarial tests combined high traffic volume with aggressive process terminations to verify system resilience. During the DNS flood, test scripts killed the network helper and terminated the manager process outright with SIGKILL. Memory consumption peaked at 325 MiB with zero file descriptor leaks, while more than 40 fuzz targets and roughly 1,400 crash checks targeted data parsers and background helpers per platform.

The adjacent bystander virtual machine maintained a 97 percent response rate during process terminations, with missed queries falling strictly within two seconds of a manager kill on Linux. The test revealed two operational bugs: virtual machines lost networking after manager crashes, and one helper process became orphaned. Both failures failed safe and were resolved.

Extended soak testing evaluated Koala over eight hours of continuous execution with injected runtime faults. On macOS, all 9,884 operations finished successfully. On Linux, a race condition in the event journal that halted command execution following VM restarts was fixed, and command execution overhead dropped from 6.1 seconds to 0.35 seconds after processing 67,000 operations.

04Nested Linux sandboxes and the four-byte bug

To defend against hypervisor escapes, every Linux process touching a virtual machine runs inside a dedicated sandbox. An escape probe simulated an adversary breaking out of the hypervisor into that intermediate host process layer, attempting to read SSH keys, access neighboring disks, open network sockets, launch shells, and freeze the manager. Every attempt was refused.

Running the identical escape probe outside the sandbox resulted in seventeen successful compromises, confirming that the sandbox prevented real host escalation. This layered defense addresses vulnerabilities like the Firecracker escape, which traversed the host kernel to seize an unconfined root hypervisor. By running non-root sandboxed processes with seccomp filtering, Koala blocks the initial escalation path.

The escape probe also exposed a four-byte bug in Koala's Linux seccomp interception mechanism. When using process_vm_readv through the variadic syscall interface, a 64-bit flags argument was passed as a 32-bit integer zero, leaving four bytes of leftover stack memory. Non-zero stack values caused the read to fail closed, blocking legitimate subsequent connections until resolved with a dedicated C regression test.

05Release candidate status and domain grant testing

The recorded attack suites were compiled in October 2026 on macOS, and the identical test suites run and pass on Linux platforms. Host listeners continue to treat guest telemetry as unverified because virtual machines can forge their own execution logs. The testing harness confirms that boundary validation must rely entirely on external host measurements.

Current release candidate testing maintains specific documented limits. Domain grants introduced in the newest build have not yet been evaluated against the root-in-the-guest test suite, and the implementation has not undergone independent third-party security audits. Furthermore, physical testing remains limited to macOS 26 systems and Ubuntu 24.04 Linux environments.

Koala remains in release candidate status as developers expand coverage across hypervisors and operating system releases. The project plans to make its source code fully public upon reaching version 0.1.0. Until then, engineering teams evaluating autonomous agent sandboxes should verify whether boundary claims were tested through aggressive host attacks rather than vendor assurances.