Safe LLM agents fail together in multi-agent systems

Problem

This paper examines security failures that arise when multiple LLM-based agents interact as a system. Individually safe agents can produce system-level harms when messages, state, decisions or authority cross principal boundaries; local checks can miss such cross-boundary effects. The work seeks a unified, execution-centred account of how adversarial influence enters a multi-agent LLM system, traverses interaction interfaces, and produces system-level risks so researchers and practitioners can distinguish interaction-dependent failures from inherited single-agent vulnerabilities.

Approach

The authors systematize 197 public works with an execution-oriented representation that traces adversarial influence from an adversary position (A) through interaction interfaces (I) to a resulting system-level risk (R). They define eight configuration dimensions (C1–C8) that condition executions, six interaction interfaces (admission, message transfer, state propagation, authority transfer and action, collective commitment, detection and intervention), four adversary starting positions, seven risk families and eight recurring attack paths. They also audit 44 evaluation and benchmark works to assess current measurement and reproducibility practices. Defences are organised by a five-part contract: path target, observation, intervention, trust boundary and recovery.

Key Findings

• The A→I→R framework unifies disparate attacks by locating where influence begins, which interfaces it crosses and which system-level failure it causes, covering eight recurring attack paths such as compositional injection, propagation and persistence, routing manipulation, collective manipulation, delegation and shared-resource interference.
• Seven system-level risk families are synthesised: containment failure, loss of independence, collective-decision integrity failure, trajectory and goal integrity failure, cross-principal confidentiality breach, transitive authorisation failure, and availability/resource-isolation failure.
• Defences map to four control primitives: identity/policy enforcement, trust and interaction-structure control, detection/attribution/containment, and state/provenance governance. Each defence should state its path target, what it can observe, when it can intervene, its trust assumptions and what recovery it enables.
• Three cross-cutting defence limits: local controls do not guarantee end-to-end path closure; defence effectiveness depends on observation and timing; and containment is not equivalent to recovery because propagated state, spawned principals or delegated authority may persist.
• Evaluation audit shows gaps: many studies do not isolate interaction effects from inherited single-agent failures; commonly used metrics like Attack Success Rate lack standardised units or stage-level breakdowns; benchmarks are often not configurable or extensible across MAS designs; and most evaluations assume fixed, closed memberships rather than open, dynamic operation.

Limitations

The synthesis relies on public literature and treats demonstrated vulnerabilities in controlled settings as evidence without asserting deployment prevalence. Some evidence areas remain thin: systematic recovery methods, adversarial resource-exhaustion studies, runtime admission and dynamic-membership threats, and controlled provenance graphs for causal attribution are underexplored. Many evaluations lack provenance traces or relation-focused counterfactuals needed to prove interaction dependence.

Implications

From an offensive perspective, the work highlights multiple avenues an attacker could exploit: submitting compositional inputs that become harmful only after later combination, abusing routing or role placement to reach privileged actors, leveraging shared or persistent state to propagate influence, coordinating malicious principals to subvert collective decisions, escalating delegated authority to perform privileged actions, and extracting confidential internal artefacts by combining partial views. The analysis suggests attackers need not compromise a model directly to cause system-level failure; manipulating relations, protocols, topology, state persistence or admission rules can suffice.