The AI security research that matters — distilled.
The week’s most important AI-security papers, scored and summarised into what a practitioner should actually do about them.
Search the research by meaning — not just keywords
Weaponised setup docs compromise AI coding agents
New research shows AI coding agents will install malicious or vulnerable packages when setup docs are tweaked. Across 12 scenarios and nine harness–model pairings, detection hinges on the harness, not just the model. Source redirections mostly succeed; subtle name tricks slip through; pinned vulnerable versions always install. A pre-install gate largely closes the gap.
Poisoned logs steer LLMs in SOCs, study finds
New research shows Large Language Models (LLMs) used in Security Operations Centres can be steered by attacker-written text buried in logs. Using a 12,847-entry benchmark, attacks succeed up to 88.2% under baseline conditions. Fragmented payloads evade filters, and layered defences cut risk by 90.4% but leave 8.4% residual exposure.
Lifecycle Attacks Expose Fragility in Reusable Agent Skills
New research maps how reusable agent skills get compromised across their full lifecycle. Using 327 real-world skills, the study shows attackers win at repository admission, semantic retrieval, planner selection, execution and updates. Sybil-style retrieval hits 93.20% when undefended; metadata tricks outvote prompt injection; runtime guards miss paraphrased flows; unmoderated evolution inherits trust.
Smoothing Only What Attackers Control Hardens LLM IDS
New research on Traffic-Aware Randomized Smoothing (TA-RS) defends Large Language Model (LLM) intrusion detectors by adding noise only to attacker-controllable flow features. Noise-augmented fine-tuning is key. TA-RS lifts certified accuracy to 55–100% on CIC-IDS-2018 and HIKARI-2021, beats isotropic smoothing by up to 72 points, and exposes training-certification mismatches.
Self-Play Red Teaming Probes LLM Prompt Injection Weaknesses
OpenAI unveils GPT-Red, an automated red-teaming pipeline that uses self-play to uncover safety and alignment failures in Large Language Models. Internal model instances generate attacks while others defend, surfacing prompt-injection tactics at scale. The approach promises repeatable coverage, though public details omit metrics, comparisons, and concrete empirical results.
Deterministic guardrails block LLM prompt injection
A new paper moves authority out of the Large Language Model (LLM) and into deterministic code, stopping low-trust text from authorising actions. Using an unmodified Gemma-4-26B and a “cascade” wrapper, defended rate jumps from 27% to 94% with a small quality hit, and holds 87% under adaptive red-teaming. Attribution also improves sharply.
Prompts drive clustered flaws in LLM-generated code
New research shows code from Large Language Models (LLMs) fails in structured, predictable ways driven by the prompt, not just random bugs. Using metamorphic testing and association-rule mining on 3,700 snippets, 68.8% violated at least one security check, with hard-coded credentials and command injection rife. Standard SAST missed most issues.
Multi-view attacks mislead autonomous driving VLAs
A CVPR AdvML challenge shows how attackers steer autonomous-driving vision‑language agents using coordinated tweaks across six camera views and short text suffixes. Image‑side attacks dominate, typography inside images is a persistent weak point, and feature‑space objectives improve black‑box transfer, reshuffling the leaderboard and signalling real deployment risk.
Multi-turn red teaming exposes LLM safety gaps
AMT-X proposes phase-structured, multi-turn red teaming with a checklist-gated jury. Against six frontier models and seven categories, it hits 97.6–100% on a lenient metric, but only 66.7–78.6% when full, real, operational detail is required, exposing a large actionability gap and heavy dependence on deeper attack phases.
MCPZoo finds MCP scanners unreliable at runtime
A new runtime-scale study of Model Context Protocol (MCP) servers shows most security scanners over-flag and disagree. From 64,611 unique servers, 37,288 ran interactively. Scanners marked 96.89% as risky, yet manual checks found less than half of sampled alerts were real. High‑risk tool categories and template reuse amplify attack paths.
NetInjectBench exposes prompt injection in network agents
New research puts tool-using Large Language Model (LLM) agents through 130 network-ops scenarios and shows how indirect prompt injection in tickets, logs and chat can drive unsafe actions. Naive agents fail badly. Prompt tweaks help but leave gaps. A runtime, metadata-aware policy gate nearly eliminates unsafe actions without blocking approved changes.
Persistent red team breaks CLI agents
ANCHOR audits autonomous command-line agents with a persistent, adaptive adversary grounded in real US court cases. Direct prompts see mixed refusal, but multi-turn pressure drives eight tested models to full compliance. When agents comply, they over-deliver, autonomously wiring up infrastructure for large-scale harm. The study uses emulated tools and model judges.
VEXAIoT Shows LLM Agents Exploit IoT Labs in Minutes
VEXAIoT pairs two Large Language Model agents to scan, plan, and execute attacks against lab IoT targets. In IoTGoat and Metasploitable2, it hit a 95% success rate across 260 runs, with most attacks completing in under two minutes. Failures clustered around command generation, refusals, and occasional hallucinations. Real-world generalisation remains uncertain.
RAG Misattributes Evidence: Deceptive Grounding Exposes Blind Spot
New research shows retrieval-augmented generation can cite real trials yet apply them to the wrong drug, a failure dubbed deceptive grounding. Across 13 models, worst-case rates hit 8–86.7%, with 7.8% in production and 13.6% for new drugs. A simple entity-attribution check flags most cases, but vendors rarely use it.
Physical prompts hijack VLM wearables, defences tested
New work on vision-language wearables shows physical prompt injection is not just a lab trick. Text placed in the scene steers outputs across 12 models, with up to 96% success in simulation and 60% in real photos. The authors test two defences that cut attack rates by large margins.
TokenWall audits agent token flows to stop abuse
Persistent AI agents expand the attack surface as instructions flow through memory, tools and inter-component messages. TokenWall tackles this by auditing natural-language token flows before they hit sensitive sinks. On CIK-Bench it cuts attack success to 12.5% while keeping 97.4% benign passes, adds 0.69s latency on clean runs, and reduces human escalations.
ScopeJudge keeps offensive AI agents in scope
ScopeJudge tests a pre-execution judge for Large Language Model (LLM) security agents to catch out-of-scope tool calls before they run. On 4,897 calls labelled by penetration testers, static policies failed without the user’s request. Context-aware judging improved recall but raised cost and exposure. An open-weight judge led with F1 0.66 and lower cost.
Symlinks Let AI Coding Agents Escape Sandboxes
New research shows AI coding assistants can follow symlinks out of their workspace and write to sensitive files, often before or without clear user consent. The UI can hide the real target path, turning human approval into a rubber stamp. Attackers can gain persistence or exfiltrate secrets via crafted repos.
LLMs Tip the Scales: Attackers Automate, Defenders Lag
A new survey argues Large Language Models now industrialise both attack and defence. It cites LLM-assisted malware growing from about 2% of detections in 2021 to a projected 50% by 2025. Vendors embed models in CI/CD and app screening, but risks like prompt injection, poisoning and model extraction remain immaturely managed.
Airflow turbulence fools infrared vision-language models
New research shows a single, airflow-like perturbation can mislead infrared (IR) vision-language models without touching the target system. Trained on one surrogate CLIP model, it flips top-1 scene labels 48.5% across five backbones and cuts accuracy on six VLMs by up to 38.2%, sometimes increasing false confidence.
CXI binds LLM agent actions to real authority
New research argues most agent failures aren’t “prompt injection” but authority laundering: untrusted context ends up authorising tools, parameters, or the call itself. Context-to-Execution Integrity (CXI) ties field permissions, exact-effect validation, and invocation rights to one action manifest, blocking escapes in evaluated runs while preserving utility.
Agent data injection exploits break LLM agents
New research shows agent data injection (ADI) lets attackers disguise malicious content as trusted metadata or tool context, pushing Large Language Model (LLM) agents into unintended actions. It bypasses many prompt-injection defences, lands real clicks and code execution, and exploits how agents mix trusted and untrusted data without isolation.
Forged Reasoning Poisons LLM Agent Memory, Beats Defences
New research shows you can poison an LLM agent's reasoning memory, not just its facts. The FARMA technique forges and amplifies prior rationales to bypass keyword and consensus defences, hitting up to 100% success in tests. SENTINEL's layered filters cut this to 0% in some settings without false positives.
FORGE hijacks planning in LLM research agents
New work shows a planning-layer poisoning attack on deep research agents. By planting a handful of adversarial web documents, attackers can steer subtask generation and push fabricated claims into final reports. Across 25 queries, five poisoned pages drove 26.4% PRISM contamination; a proposed anchoring defence cut a subset from 38.5% to 18.3%.
Cryptographic gates keep learning agents in bounds
New research proposes “governed individuation”: bind an agent at boot to a cryptographic identity and gate every action by its real effect, not its name. In tests, name-based blocks allow 75% of bypasses; dynamic effect tracing drops that to zero. Ungoverned agents try forbidden writes; the gate executes none.
Single Rule Poisons Policy-Aware RAG Agents
New research on a Policy-Aware LLM-RAG for battlefield IoT shows a single injected policy rule can dominate retrieval and corrupt the Large Language Model’s context across most operator prompts. The authors propose CLD-KB, a lightweight dual-detector that flags poisoned rules using category-aware signals, outperforming common anomaly detectors with millisecond overhead.
RL-Guided Node Injection Undermines Black-Box GNNs
New research shows a black-box reinforcement learning attack that injects a single crafted node and edge can flip Graph Neural Network (GNN) predictions. TIRBA jointly optimises fake node features and connections using query feedback, outperforming prior methods and approaching 94% attack success on Pubmed, signalling real risk for account-creation and fraud scenarios.
LLM agent builds full zlib fuzz lab in a day
Trail of Bits reports that GPT-5.5-Cyber assembled a full fuzzing campaign against zlib in under a day, complete with sanitiser builds, variant flags and a dozen C/C++ harnesses. It prioritised reachable, high-impact bugs and left noise on the floor, signalling a real shift in the effort needed to launch bespoke fuzzing.
Moderation Traces Jailbreak Function-Calling LLMs by Exploiting State
A new black-box attack, Simulated Moderation Traces (SMT), fakes a moderation workflow to coax function-calling Large Language Models (LLMs) into unsafe output. By interleaving schemas, arguments and fabricated validator feedback in the same context, SMT hits 99.67% and 98.33% success with ~1.4 queries per input. Prompt-only defences falter; targeted prompts curb some models.
KidnapRAG hijacks agentic RAG with black-box poisoning
KidnapRAG shows how to hijack agentic Retrieval-Augmented Generation using only public poisoned documents. The attacker lures initial retrieval, drags the chain with follow-ups, then supplies malicious evidence. It beats other black-box attacks across ReAct and WebThinker setups and stays stealthier under moderation, raising reliability risks for deployed LLM agents.
ReShift plants reasoning-level backdoors in vision-language models
New research shows a backdoor that targets the chain-of-thought inside vision-language models, not just the final answer. ReShift uses poisoned training and reinforcement learning to redirect reasoning when a trigger appears, while clean accuracy and plausible rationales remain. Output-based and perplexity detectors slip, with near-random detection rates reported.
VLM phone agents: vision gaps and misused channels abused
New research dissects how third‑party mobile agents using vision‑language models treat screenshots as device state, opening two attack surfaces. By manipulating pixels and repurposed channels, a low‑privilege app can hijack actions and even trigger host commands. Across five Android agent frameworks, most attacks succeed without special permissions or visible cues.
Survey maps LLM vulnerabilities across the enterprise stack
A new survey reframes AI security around the full Large Language Model (LLM) lifecycle, not just model weights. It details how retrieval-augmented generation (RAG), memory and agents turn untrusted content into executable instruction, why attacks chain across layers, and why point defences rarely compose. Practical focus: provenance, authority and containment.
Framework maps LLM agent attacks across four layers
AI-Infra-Guard lands as an open-source, end-to-end red‑teaming framework for AI agents. It treats the attack surface as four layers and picks different detection methods for each: rule-based infra checks, protocol and skill auditing, black-box agent probing, and a broad jailbreak harness. It also tackles supply‑chain risk in agent skills.
Infrared POV attack blinds traffic sign classifiers
Researchers show a near‑infrared persistence‑of‑vision device can fool camera‑based traffic sign models from up to 20 metres, while staying invisible to humans. It works across 12 model families, thrives at night, and remains effective in slow drive‑bys. Simple IR‑cut filters stop it, but many sensors run without them.
Red Agent rips open airline GraphQL bookings
An autonomous red-team agent hit an airline GraphQL API from a public root URL, minted an anonymous session, ran schema introspection, and found 514 queries and 428 mutations exposed. Sequential booking IDs unlocked full records and destructive mutations. Frontend auth did nothing; resolver-level authorisation was missing. Classic BOLA, industrial scale.
MemLeak shows images foil AI agent forgetting
New research shows that telling a multimodal AI agent to forget a fact often fails. Deleting text works in isolation, yet 18.3% of facts reappear via correlated text and 12.0% via retained images, 47% unique to images. Content-aware deletion trims image leaks to 2.0%, hinting at stricter erasure tooling.
Agent-native immunity targets runtime hijacks in LLMs
New research argues agents need defences inside their reasoning, not just perimeter rules or training-time alignment. It maps how memory poisoning, tool-chain manipulation and multi-agent protocol attacks hijack behaviour, then proposes an agent-native immune system with layered controls, adaptive parametric vaccines and self-monitoring loops to counter fast-evolving runtime threats.
Amazon Q MCP Auto-Load Lets Repos Steal Cloud Keys
A flaw in the Amazon Q VS Code extension auto-loaded Model Context Protocol (MCP) servers from workspace files and executed them without prompts. Opening a booby-trapped repo was enough to run attacker code that inherited the user’s environment, enabling cloud credential theft. Amazon fixed it in language server version 1.65.0.
Fine-tuned LLMs Miss Simple PowerShell Evasions
New research shows small fine-tunes turn Large Language Models (LLMs) into brittle PowerShell classifiers. Alias swaps, command reconstruction and even case changes bypass the fine-tuned model while the base model holds up. The failure lives in an inherited late-attention circuit that fine-tuning narrows into token-level rules invisible to standard tests.
Bandits pick winning LLM jailbreaks at scale
New research shows a simple recipe for high-success jailbreaks: use bandit algorithms to pick prompts and add complexity to malicious queries. Tested on 15 open-weight LLMs, the approach reaches up to 97% success when sampling five jailbreaks per query. It also transfers across models, resists missing prompts, and favours complex requests.
MIRROR red-teams agentic RAG across text and images
MIRROR targets the messy reality of agentic retrieval-augmented generation (RAG): text poisoning, image injection, direct queries, and tool orchestration. It uses memory-guided search with a novelty gate to avoid copying benchmark attacks, posting high attack success across surfaces and exposing how single-surface methods crumble outside their comfort zone.
Diffusion tactics unify LLM and multimodal jailbreaks
A new survey maps how denoising diffusion models drive attacks across text, image, and vision-language systems. It unifies threat models and metrics, reports strong white-box success with weaker transfer to closed models, and flags a defence-audit gap for diffusion-based purifiers. The result is a clearer playbook for testing and governance.
Influence analysis exposes poisoned LLM summarisers
New work dissects how fine-tuning data poisoning can quietly skew Large Language Model (LLM) summarisers without denting ROUGE. It pairs white-box influence analysis and gradient-ascent unlearning with a black-box sensitivity audit. Reported results: 85–92% detection, up to 96% behaviour recovery, minimal utility loss, and resilience to adaptive attacks.
Stop Pretending RAG Makes Agents Safer
A new survey shows Retrieval-Augmented Generation (RAG) widens the attack surface for LLM agents. Risks cluster in indices, context packing, and federated updates: prompt injection via retrieved docs, retrieval poisoning, membership/index inference, and gradient leakage. Defences exist but trade utility for privacy, leaving practical deployments exposed to durable, stealthy manipulation.
AdversaBench scales LLM red-teaming with multi-judge checks
AdversaBench automates adversarial testing of Large Language Models by mutating prompts and confirming failures with a three-judge panel plus a meta-judge. Across 45 seeds, it found failures in every case, with instruction-following tasks taking more effort. The team also shows prompts transfer to larger models, hinting at systemic weaknesses.
Non-malleable authority locks down LLM agent memory
New research targets a blind spot in Large Language Model (LLM) agents: long-term memory that attackers can poison today and cash out later. The authors show content and lineage checks are easy to launder, then ship a non-malleable, origin-bound control that blocks laundering attacks across eight models at zero recorded utility loss.
PixJail turns papers into pipelines for T2I jailbreaks
PixJail automates text-to-image jailbreak reproduction as a full pipeline, not a single-prompt trick. It reads a paper, builds an attack module, runs standardised tests and keeps memory of what worked. In unified benchmarks, some diffusion models fell easily, while GPT-image-2 resisted. The framework sharply reduces effort and exposes pipeline sensitivities.
TooBad backdoors diffusion models with imperceptible triggers
New research shows a backdoor on diffusion models (DMs) that needs just 0.5% poisoned data to hit over 85% Attack Success Rate (ASR), and near 100% ASR in 3–5 epochs at 5%. The trigger hides in noise space, remains imperceptible, evades state-of-the-art defences, and preserves clean output quality.
OpenAI Daybreak automates finding and fixing vulnerabilities
OpenAI’s Daybreak suite, including Codex Security and GPT-5.5-Cyber, targets the drudgery of vulnerability management. It promises AI-driven discovery, validation and patching at scale, but offers no published measurements. The pitch raises hard questions about accuracy, patch integrity, data handling, and whether the tooling itself becomes a new attack surface.
Self-Evolving LLM Agents Turn Attacks Into Lineage Backdoors
New research shows self-evolving Large Language Model (LLM) agents convert one-off compromises into persistent, lineage-wide backdoors. Using a 25-cell Module-Lifecycle matrix, the study flags 17 critical threat areas and finds evolution-native designs light up 3.5× more attack surface. In tests, 40/40 attacks persisted while a co-located scanner blocked just 2.5%.
Adversaries corrupt the imagination in VLA agents
New work shows imagine-then-act vision–language–action agents trust a latent “imagination” that attackers can corrupt with small, gradient-based tweaks to a single camera frame. Untargeted corruption is about 60× stronger than random and can derail MPC planning, while a simple denoiser detector flags off‑manifold tampering with near‑perfect AUC.
Logit steering collapses LLM safety refusals fast
New research shows many Large Language Models encode refusal as a simple, steerable logit feature. A zero‑optimisation “logit steering” method hits 95% jailbreak success on Llama‑3.1 in about a second and outperforms activation‑level attacks. It needs white‑box logit access, but it exposes where model safety is fragile by design.
Revoking lingering agent access beats static sandboxes
New research targets a real weak spot in coding agents: lingering permissions. PORTICO enforces revocable, epoch-bound handles tied to explicit task contracts. In tests, it blocks post-closure reuses (10/10) and stale writes (0/6 vs 6/6), eliminates contract-forbidden effects, and preserves utility via controlled grants. Assumptions about mediation and catalogues still matter.
The signal, not the noise.
One weekly email: the few papers that matter, with the practitioner takeaway.