Benchmarking OpenClaw Agents Exposes Lifecycle Security Risks

Enterprises are racing to deploy Large Language Model (LLM) agents that plan tasks, call tools and keep working context. That combination turns a chat model into a system component with real reach. A new systematic evaluation of six OpenClaw-series agent frameworks shows how much that shift matters for security: agents are measurably riskier than the backbone models they wrap.

The researchers built a 205‑case benchmark spanning 13 attack categories across the agent lifecycle, then ran more than 1,200 adversarial interactions against OpenClaw, AutoClaw, QClaw, KimiClaw, MaxClaw and ArkClaw, paired with multiple backbone models. They tracked how risks propagate from input ingestion through planning and tool execution to final output.

Three findings stand out. First, agentisation enlarges the attack surface. Tool invocation, multi‑step planning and persistent state increased the probability that a risky action would succeed compared with the model used in isolation. Second, reconnaissance and discovery were the most common weaknesses, often succeeding above 65%. Early probes surfaced environment details that later enabled privilege changes, lateral movement or data theft. Third, each framework exhibited a distinct high‑risk profile: QClaw was prone to credential access (85.71%) and data exfiltration (80.00%); KimiClaw showed higher lateral movement (66.67%); AutoClaw reached 70.00% in privilege escalation and 71.43% in resource development. MaxClaw was comparatively resilient with an overall 16.10% success rate; ArkClaw and OpenClaw variants sat between these poles (ArkClaw overall 29.76%).

The study also catalogues concrete vectors that cut across frameworks. Symbolic‑link and path traversal bugs let agents escape sandboxes. Some agents echoed credentials in plaintext outputs. Encoded or semantically obfuscated payloads bypassed input filters, only to be decoded during planning and executed by tools. Several frameworks pulled unaudited external resources, importing hostile code. Once an agent holds execution capability and state, a small early lapse becomes a systemic failure.

For security teams, the message is practical: treat agent runtimes as untrusted execution environments and design controls for the whole lifecycle, not just the prompt. The authors point to specific mitigations that reduce amplification risk. Multi‑layer input decoding and semantic inspection make it harder for encoded payloads to slip through plan stages. Task‑intent centred plan reviews, with mandatory human confirmation for high‑privilege actions, slow down dangerous chains before tools fire. Strict physical path resolution and sandbox enforcement on tool execution contain file‑system abuse, while dynamic output desensitisation and outbound controls prevent credential leakage and covert exfiltration.

These are engineering steps, but they are also governance steps. If an agent can write files, spawn processes or reach external networks, those capabilities should be permissioned, logged and bounded. High‑risk plans should face explicit approval, not implicit trust. In time, I expect procurement and audit standards to ask for exactly this evidence: lifecycle testing across planning and execution, red‑teaming beyond prompt injection, and telemetry that proves containment works in production.

There are caveats. The benchmark covers six related frameworks and selected model pairings. Different architectures, tools or configurations may shift the risk profile. Still, the pattern is clear: coupling model capability with tools and orchestration drives new failure modes that model‑only evaluations miss.

The policy implication is straightforward. We cannot regulate agent safety by focusing on input prompts alone. Assurance needs to follow the agent from intent formation to tool calls and outputs, with controls that assume early‑stage reconnaissance will happen and seek to prevent its amplification. If we build and require that kind of lifecycle discipline, agents can earn their place in sensitive environments without asking for blind trust.