LLMs Tip the Scales: Attackers Automate, Defenders Lag
A new survey argues Large Language Models now industrialise both attack and defence. It cites LLM-assisted malware growing from about 2% of detections in 2021 to a projected 50% by 2025. Vendors embed models in CI/CD and app screening, but risks like prompt injection, poisoning and model extraction remain immaturely managed.
The security story around Large Language Models (LLMs) is not about sudden machine brilliance. It is about throughput. A new survey pulls together academic and industry work and estimates LLM-assisted malware jumped from roughly 2% of detections in 2021 to a projected 50% in 2025. If that trajectory holds, the centre of gravity has already moved.
Attackers are using generative tools to do the boring parts at machine speed: produce polymorphic code, rewrite loaders, alter control flow, and keep changing strings and structure until signature-based checks miss. The same models can be asked to reason about a target stack, propose likely weak points, and sketch exploit paths. None of this makes an operator smarter. It just lets an average operator try a hundred ideas before lunch and keep the two that work.
Social engineering benefits most from this scaling. Context-aware phishing that reflects local jargon, policy names, or vendor tooling reads like an insider wrote it. When you can customise tone and detail per inbox, at volume, you do not need perfect pretexting; you just need enough plausible hooks to net a few privileged clicks.
On the blue side, vendors are wiring LLMs into continuous integration and deployment pipelines and platform screening. The survey points to improved triage and fewer false positives when models apply semantic understanding of code and behaviour. That is progress, but it also centralises risk. An enterprise model that underpins code review, policy enforcement and app vetting becomes a single target. Prompt injection can bend it at inference time. Data poisoning in training or feedback loops can bias it in quieter ways. Model extraction threatens to clone capabilities defenders paid to build. Adversarial inputs can nudge classification without tripping alerts.
Explainability and privacy tooling exist, but the paper is blunt: they are essential and still immature at scale. Token-level salience maps and federated learning pipelines help with audit and compliance stories, yet they do not give you a verifiable account of why a specific detection flipped last Tuesday. Watermarking and cross-industry collaboration appear in the recommendations, useful but insufficient if your core control is a black box that can be steered.
Here is the uncomfortable read: the decisive change is speed and centralisation. Offence uses LLMs to flood the zone with variations. Defence concentrates decision-making into a few models. In that setup, the side that understands its own failure modes better will win more often. Today, that understanding still looks thin.
Additional analysis of the original ArXiv paper
📋 Original Paper Title and Abstract
Large Language Models (LLMs) and Generative AI in Cybersecurity and Privacy: A Survey of Dual-Use Risks, AI-Generated Malware, Explainability, and Defensive Strategies
🔍 ShortSpan Analysis of the Paper
Problem
This survey examines how Large Language Models (LLMs) and generative AI reshape cybersecurity through a pronounced dual-use dynamic: the same models that enable automated defence, secure code generation and zero-day detection also empower attackers to create malware, sophisticated phishing and vulnerability exploits at scale. The paper highlights a rapid rise in AI-assisted threats, reporting LLM-generated malware increasing from about 2% of detected threats in 2021 to a projected 50% in 2025, and argues that this shift requires new defensive frameworks, governance and auditability to manage systemic risks.
Approach
The authors present a comprehensive literature and industry survey synthesising over 70 academic papers, reports and technical documents alongside real-world case studies from platforms such as Google Play Protect, Microsoft Security Copilot, Amazon Web Services, Apple App Store, OpenAI Plugin Stores, Hugging Face Spaces and GitHub. The review covers defensive applications (zero-day detection, DevSecOps, federated learning, explainable AI), offensive misuse (AI-generated malware, social engineering), governance initiatives and recommended controls including watermarking, adversarial defence and cross-industry collaboration.
Key Findings
- Rapid growth of AI-enabled threats: industry data reviewed in the paper estimate LLM-assisted malware rose from roughly 2% of detections in 2021 to a projected 50% in 2025, indicating a paradigm shift in attack scalability.
- LLMs strengthen detection and triage: fine-tuned LLMs and domain-specific variants outperform traditional static analysers on semantic code understanding and can improve zero-day vulnerability detection, triage speed and plausible exploit-path hypothesis generation.
- Defensive integration at scale: major vendors are embedding LLMs into CI/CD and platform security workflows for real-time code scanning, policy enforcement and pre-deployment app screening, reducing false positives and time-to-detection in large ecosystems.
- Democratisation of offensive capability: generative models lower the skill barrier for creating malware, polymorphic code and context-aware phishing, enabling non-experts to produce functional malicious artefacts rapidly.
- Explainability and privacy are essential but immature: adapted XAI tools (for example token- and attention-based analyses) and federated learning pipelines show promise for auditability and GDPR-aligned deployment, yet scalable, verifiable explainability and privacy-preserving inference remain open challenges.
- Systemic risks to defender models: LLMs used for defence face threats including data poisoning, model extraction, prompt injection and adversarial evasion, creating single points of failure if not properly secured and monitored.
- Bias, scalability and compliance challenges: training data skew and regional regulatory differences can cause misclassification, higher false positive rates for underrepresented locales and operational overhead when deploying LLMs globally.
Limitations
The survey relies on heterogeneous sources including academic papers and industry reports, and notes scarcity of comprehensive public incident data on weaponised LLM usage. Benchmarks and standardised metrics for many security tasks remain limited, deployment studies often omit long-term robustness evaluation, and the black-box nature of large transformer models complicates causal attribution and compliance with transparency obligations.
Implications
Offensive security implications are significant: attackers can use LLMs to automate and scale malware creation, generate polymorphic code to evade signature detection, craft highly convincing spear-phishing at volume, discover and exploit zero-day vulnerabilities faster, and attempt to compromise defender models via poisoning, extraction or prompt injection. The democratization of capability means lower-skilled actors can mount sophisticated campaigns, and targeting of defender LLMs could create blind spots or backdoors that amplify adversary impact.
