Researchers Break Prompt Secrecy by Stealing Seeds

Problem

Diffusion models enable highly realistic text to image generation conditioned on prompts and seeds. The monetary and intellectual value embedded in prompts makes prompt theft a key security and privacy concern. The paper studies prompt stealing attacks targeting diffusion models and reveals that numerical optimization based prompt recovery is limited because it does not account for the initial random noise used during generation. It identifies a noise generation vulnerability (CWE-339) arising from PyTorch seed handling that restricts seed values to a 32 bit range on CPUs, enabling practical seed brute forcing. Through large scale analysis on images shared on CivitAI, the work shows that about 95 percent of seed values can be brute forced in roughly 140 minutes per seed with a seed recovery tool called SeedSnitch. With the recovered seed, the authors propose PromptPirate, a genetic algorithm based method for prompt stealing that outperforms existing approaches. The work also offers straightforward mitigations to render seed stealing ineffective and reports responsible disclosure with coordinated mitigations with developers.

Approach

The authors formalise a threat model in which an attacker aims to recover a prompt used with a diffusion model to reproduce the appearance of a target image. They show that seed knowledge is critical for accurate online prompt stealing and that seeds can be recovered by brute forcing a 32 bit seed space. SeedSnitch exploits the latent structure of generated images to recover the seed efficiently, without training a model. The recovered seed is then used by PromptPirate, a genetic algorithm based optimiser designed to reconstruct prompt modifiers. The GA searches modifier combinations by evaluating fitness as the latent space discrepancy between the target image and images generated from candidate modifiers with the recovered seed. The approach uses a constrained population with tournament selection, variable length crossover, mutation, and elitism, over multiple generations. The authors compare PromptPirate against Prompt Stealer, P2HP and CLIP Interrogator. They perform large scale real world evaluation on CivitAI data, and also provide a case study on seeds recovered from publicly available Stable Diffusion 3.5 images. They also discuss mitigation strategies including cryptographically secure random number generation and larger seed spaces.

Key Findings

• The seed used to initialise the diffusion process embeds a distinctive latent structure in images; online prompt stealing is unreliable without prior seed knowledge, and seeds can be brute-forced when within a 32 bit range.
• SeedSnitch can recover the seed for about 95 per cent of real world images from CivitAI, in approximately 140 minutes per seed, demonstrating practical seed vulnerability across platforms.
• In controlled experiments with 1000 prompts, seed recovery achieved 100 per cent accuracy, with an average seed recovery time of about 85 seconds per image, underscoring the reliability of seed extraction methods.
• A large real world case study on Stable Diffusion 3.5 variants showed seed recovery accuracy of 95 per cent across the dataset, with 94 per cent for Large, 98 per cent for Medium, and 100 per cent for Turbo variants, highlighting practical seed leakage risks in community generated content.
• Seed values in real data cluster into distinct ranges, with about 95 per cent of seeds lying within a 32 bit space, indicating a broad exposure to brute force attacks in practice.
• PromptPirate, using a genetic algorithm to optimise prompt modifiers once the seed is known, consistently outperforms state-of-the-art online and offline methods in visual similarity measures, achieving an 8 to 11 per cent improvement in LPIPS over competing methods; in Known Subject scenarios it yields superior perceptual similarity, while in Unknown Subject scenarios it matches or exceeds CLIP based similarities and maintains strong visual alignment.
• Computational costs differ across methods: Prompt Stealer is faster in offline settings, while PromptPirate, CLIP Interrogator and P2HP are online and slower, with PromptPirate requiring about 62 minutes per image on an NVIDIA A100, CLIP Interrogator about 1 minute, and P2HP about 140 minutes.
• Mitigations suggested include adopting cryptographically secure random number generators with large seed spaces, such as ChaCha20 with 256 bit seeds, which would significantly raise the cost of brute force attacks while preserving seed based reproducibility; these mitigations can be implemented with minimal overhead and kept compatible with seed setting.
• The work emphasises responsible disclosure and coordinated mitigation with developers, and notes that while mitigations can protect future images, already published images may still be at risk.

Limitations

The seed based attacks rely on CPU based RNG implementations and a constrained seed space; results may vary for models or configurations that use different RNGs or seed handling. Some seeds in real datasets were not identified, which may reflect image editing, alternative RNGs, or different generation pipelines. The mitigations discussed focus on diffusion pipelines and platforms hosting prompts; applicability to all diffusion based systems may vary. The study acknowledges that some attacks cannot retroactively secure already published content, and recommends coordinated disclosure and ongoing security enhancements. The authors also note that some results depend on the specific models and datasets used in evaluation.

Why It Matters

The research reveals a concrete and realistic attack vector against diffusion models by recovering initial seeds to enable prompt piracy, exploiting a CWE-339 weakness in seed handling. It demonstrates that seed knowledge significantly improves the success of online prompt stealing and highlights gaps in current defenses around seed management and prompt confidentiality. The practical seed recovery methods and GA based prompt extraction show the need for stronger seed security in diffusion pipelines and hosted prompt ecosystems, particularly for content creators and organisations reliant on unique prompts for IP and branding. The suggested mitigations offer concrete controls for diffusion pipelines and platforms, with potential broad impact on safeguarding intellectual property and access control in creative industries and model ecosystems. The work underlines that while it does not address weapons or surveillance, it has important security implications for protecting prompts, seeds and the authenticity of AI generated content.