Pentesting Agents Expand Attack Surface, Guardrails Fall Short

Problem

This paper studies security risks introduced by autonomous large language model powered penetration testing agents. These agents combine long‑horizon reasoning, persistent memory, retrieval and direct tool execution to perform multi‑step offensive workflows with minimal human supervision. That capability increases the attack surface beyond conversational LLMs because agents accept untrusted runtime context, store and retrieve knowledge across sessions, coordinate across agents, and translate output into real‑world commands. The paper argues existing guardrails for chat systems are insufficient and examines threats that target both the underlying LLM lifecycle and agent architecture.

Approach

The authors perform an architecture‑centred security survey. They analyse representative single‑agent and multi‑agent pentesting frameworks, identify core architectural capabilities (planning, persistent memory, RAG, reflection, tool integration, multi‑agent cooperation), define three trust boundaries (model, context, execution), propose a two‑axis threat taxonomy (LLM lifecycle attacks and agent architecture attacks), map attack surfaces to guardrail categories, and produce qualitative vulnerability and guardrail‑insufficiency assessments based on framework designs rather than live exploitation.

Key Findings

• Autonomous pentesting agents materially expand attack surfaces: persistent memory and retrieval enable long‑term poisoning, inter‑agent channels enable worm‑like propagation, and tool execution converts textual outputs to real‑world consequences.
• The two‑axis taxonomy separates inherited LLM lifecycle attacks (pre‑training, fine‑tuning/alignment, inference‑time) from emergent agent architecture attacks (memory/RAG poisoning, multi‑agent coordination attacks, and tool/execution exploitation), clarifying where defences must operate.
• Three architectural trust boundaries structure attacks: the model boundary (compromise of training or alignment), the context boundary (poisoned runtime inputs, memory and retrieval), and the execution boundary (unsafe tool calls, malicious plugins and workflow manipulation).
• Existing guardrails map to a mirrored six‑category taxonomy but are largely component‑specific and brittle; many defences address single layers and therefore fail against hybrid attacks that cross boundaries.
• Qualitative assessments of representative frameworks show widespread exposure: single‑agent systems are vulnerable to prompt injection and execution abuse, while multi‑agent systems add exposure to inter‑agent message poisoning and coordination attacks.
• The authors identify eight research gaps including the need for integrated architecture‑aware guardrails, runtime model integrity verification, execution‑aware safety reasoning, trustworthy memory management, secure multi‑agent coordination, policy‑aware execution control, standardised benchmarks, and governance and auditability.

Limitations

The analysis is a qualitative, architecture‑based survey rather than an empirical benchmark of exploit success rates. It excludes vulnerabilities within target infrastructures, physical or hardware side‑channels outside agent execution environments, and depends on published framework descriptions which may not reflect operational deployments.

Implications

From an offensive perspective, the survey highlights practical avenues an attacker could exploit: poisoning pre‑training or fine‑tuning data to embed persistent malicious behaviours; crafting runtime observations, prompts or retrieved documents to hijack agent goals; injecting malicious memory entries or index items to bias long‑term planning; compromising inter‑agent channels to propagate control across a system; and tampering with tools, plugins or execution metadata to cause unauthorised commands and unintended real‑world effects. Such attacks can divert assessments to wrong targets, cause unsafe exploit chains, leak sensitive data, or make autonomous agents act under adversarial control.