LLM Agents Tackle Lateral Movement, Still Brittle

Problem

The paper examines how Language Model Agents (LMAs) can augment offensive cyber operations, focusing on lateral movement, a multi-step capability central to advanced persistent threat activity. It asks how reliably LMAs can plan, execute and verify chained intrusion steps in a realistic enterprise Active Directory environment, and what failure modes and governance challenges arise when agents act autonomously or with varying degrees of scaffolding.

Approach

The authors implement a controlled multi-host Windows Active Directory testbed and formalise two lateral‑movement scenarios as ordered task chains with explicit verification predicates (Scenario‑1: nine tasks; Scenario‑2: ten tasks). A three-agent architecture separates planning (Orchestrator LMA), execution (Cyber Agent) and evaluation (Judge LMA using an LLM-as-a-Judge). They compare three operational modalities: expert-defined action plans, self-scaffolded planning by the agent, and fully autonomous operation. Experiments ran across five large language models under a fixed token budget per run and instrumented feedback to measure partial progress, retries and behavioural signals such as premature progression and loss‑of‑control loops.

Key Findings

• Expert-defined plans produced the highest task‑completion rates: Claude Sonnet 4.5, GPT‑5.1 and Claude Opus 4.5 each completed the full nine‑task chain in the expert‑defined mode, while Gemini 3 Pro stalled at task 3 and DeepSeek v3.2 stalled at task 1.
• Failures remained frequent across modes and models, driven by three recurring bottlenecks: unreliable command invocation and execution context maintenance, credential and identity transition errors, and verification ambiguity where partial telemetry is misinterpreted as success.
• Agent operation often incurred large resource burn through repeated low‑level retries and looping behaviour. Examples include an environment setup task that required four attempts, a reverse‑shell deployment that took nine attempts consuming 11 minutes and 53k tokens, and a credential‑dumping task in a fully autonomous run that expended about 2.6×10^5 tokens without yielding verifiable artifacts.
• Self‑scaffolded runs showed high variance: some completed nearly all proposed tasks while others failed early; fully autonomous runs exhibited still greater variance in both success count and token efficiency, illustrating that task success alone does not capture operational quality.
• Agents sometimes advanced on weak or unverified signals, reusing invalid credentials or assuming privileged access despite judge verdicts marking prerequisite tasks unmet, producing end states that appear successful but lack correct intermediate evidence.

Limitations

The evaluation reflects upper‑bound performance by reporting best runs per configuration and uses a single attempt per run under a fixed token limit. Judge LMA verdicts can be noisy when telemetry is partial, and model behaviour and failure modes vary by implementation and trigger conditions. The testbed emphasises post‑compromise lateral movement and does not cover initial access vectors in the same depth. Results depend on the specific models and tooling interfaces evaluated and may not generalise to all deployments.

Implications

Offensive implications are twofold: LMAs can automate multi‑step adversary emulation and scale lateral‑movement workflows, reducing the labour and coordination costs of red teams or attackers. With adequate scaffolding, current LMAs can execute complex kill chains end‑to‑end, but operational brittleness remains exploitable; attackers may waste resources in retry loops or produce unreliable access when credential handling fails. The traces produced by LMAs—recurrent PowerShell execution, LSASS access attempts, Pass‑the‑Hash activity and writable‑share abuse—align with standard EDR and SIEM primitives and could be generated at scale by automated agents during intrusions or red‑team exercises.