New to ShortSpan? We distil the AI-security research that matters into practitioner takeaways — edited by Ben Williams (NCC Group).Get the weekly email
// Analysis
Local LLMs leak prompts via memory, logs and API
Published: Thu, 17 Sep 2026 • By Natalie Kestrel
Enterprise
New research shows local Large Language Model (LLM) setups leak prompts through memory residue, wrapper logs and faulty multi-tenant isolation. The team recovers plaintext from host memory post-inference, finds LM Studio logging prompts by default, and breaks llama.cpp tenant separation in 200/200 trials. A caching timing oracle stays visible over WAN.
Running a Large Language Model (LLM) on your own kit sounds private. No cloud, no third party, no problem. This paper pokes that myth and finds four places where confidentiality falls apart: model admission, runtime memory, persistence in wrappers and the serving interface. The headline is simple: locality is not confidentiality.
Runtime memory is a sieve
After each inference, the researchers repeatedly pulled multiple plaintext copies of prompts from host memory. Not one; typically thirteen or fourteen per run. In a sequence of twelve tenant sessions they recovered 110 plaintext copies totalling about 10.8 MB. Provenance tracing points to two culprits: the application duplicates prompt representations during parsing, template expansion and serialisation, then the allocator quietly preserves those freed buffers for reuse.
They tried sanitisation. It helped but did not fix it. Patching reduced recoverable copy counts by roughly 29 percent and plaintext volume by about 69 percent, yet prompts from eleven of twelve tenants remained recoverable. Zeroising the key–value (KV) cache barely moved latency, staying under roughly 0.5 percent. Partial GPU offload did not save you either; host-side copies still lingered across subsequent requests and idle periods.
Wrappers and the serving edge
Persistence is as boring as it is dangerous. Under the evaluated configuration, LM Studio’s default logging recorded plaintext prompts on disk until sensitive-data logging was disabled. Ollama, by contrast, produced no persistent prompt artefacts and showed no outbound transmissions in isolated tests. If your threat model includes later disk exfiltration, those defaults matter.
The serving boundary is where things get spicy. A previously undocumented authorisation flaw in llama.cpp let one authenticated client restore another tenant’s saved conversation slot in 200 out of 200 controlled trials. Once restored, the model’s continuation emitted the victim secret in 126 of those runs. You do not need raw memory access for that; you just need an account.
There is also a clean remote signal. Shared prompt-prefix caching created a timing oracle with perfect separability in measured samples (AUC 1.000) and it remained distinguishable under wide-area network conditions. If I can query the service and measure latency, I can infer cached membership of a prefix.
On the integrity front, they hammered GGUF model loading with structured malformed inputs and a 24-hour AFL++ campaign exceeding 12 million parser executions. No crashes, no sanitiser violations, no malformed loads in the explored space. Credit where due, but absence of evidence is not evidence of absence, and the authors are clear about scope limits.
The exploitable angles are concrete: scrape allocator-managed memory post-inference from an unprivileged local process; hunt wrapper logs for plaintext; abuse cross-tenant slot restore on llama.cpp; probe the timing oracle to test whether a prefix is already cached. The assumptions that broke are equally clear: offload does not mean erase, defaults are not benign, and local is not private by itself.
Additional analysis of the original ArXiv paper
📋Original Paper Title and Abstract
The Illusion of Local Privacy: Confidentiality Boundary Failures in Consumer LLM Serving Systems
Authors: Youssef Hamdi Zafan Ibrahim, Muhammad Ikram, and Mohammed Khalaf Salama
Running large language models (LLMs) locally is often considered more private than cloud-hosted inference because user prompts remain on the device. We ask whether keeping inference local is, by itself, sufficient to keep those prompts confidential. Our results show that it is not: prompt confidentiality also depends on how the surrounding serving software handles prompt data before, during, and after inference. We examine four boundaries at which prompt confidentiality can fail in consumer local-LLM serving systems: model loading, runtime memory, wrapper-level persistence, and the serving interface. To study these boundaries, we develop LLAnalyzer, a measurement framework that tests each boundary separately and traces observed failures to the responsible software component. Applying LLAnalyzer to four open-weight model families and two consumer deployment platforms, we find markedly different behaviour across boundaries. In a 24-hour AFL++ campaign with more than 12 million executions, we observe no parser crashes or successful malformed GGUF loads within the explored state space. Runtime memory tells a different story: we recover prompts after inference because multiple plaintext representations survive in allocator-managed memory, and sanitisation reduces this residue without eliminating it. We also find that consumer wrappers can extend prompt lifetime through plaintext persistence. At the serving boundary, we uncover a previously undocumented authorization flaw in llama.cpp that allows one authenticated client to restore another tenant's saved conversation state; the attack succeeds in 200/200 controlled trials. Separately, shared prompt-prefix caching exposes a remote timing oracle that remains distinguishable under WAN conditions. We argue that local LLM systems need explicit guarantees for prompt lifetime, persistent storage, and tenant isolation.
🔍ShortSpan Analysis of the Paper
Problem
The paper examines whether running large language models locally is sufficient to keep user prompts confidential. It argues that prompt confidentiality depends not only on local execution but on the surrounding serving software: model admission, runtime memory management, wrapper-level persistence, and the serving interface. The work asks where and how prompt data can survive or cross confidentiality boundaries in consumer local-LLM deployments.
Approach
The authors develop LLAnalyzer, a measurement framework that decomposes the local serving stack into four boundaries - Integrity, Lifetime, Persistence and Isolation - and tests each independently. Experiments use UUID-tagged prompts to establish ground truth, controlled configurations, four open-weight model families, two consumer wrappers, and repeated trials. Methods include structured malformed-file testing, a 24-hour coverage-guided AFL++ fuzzing campaign with sanitizers enabled, post-inference memory forensics, debugger-assisted provenance tracing, wrapper-file inspection, GPU offload tests, runtime sanitisation patches, and multi-client protocol experiments.
Key Findings
No Integrity-boundary bypass observed: structured malformed GGUF inputs and over 12 million parser executions produced no parser crash, sanitizer violation, or successful malformed model load in the explored state space.
Runtime memory retains prompt plaintext: every measured inference left multiple plaintext copies (typically 13–14) recoverable after response generation; twelve sequential tenant sessions produced 110 plaintext copies totalling about 10.8 MB.
Provenance shows two mechanisms: application-level duplication of prompt representations during parsing, template expansion and serialization, followed by allocator-managed memory reuse that preserves those buffers.
Sanitisation reduces residue but does not restore confidentiality: patching reduced recoverable copy count by about 29% and plaintext volume by about 69%, while prompts from 11 of 12 tenants remained recoverable; KV-cache zeroisation imposed negligible latency cost (under approximately 0.5%).
GPU offloading does not eliminate host-memory residue: partial CUDA offload left host-side prompt copies recoverable and surviving subsequent requests and idle intervals.
Wrapper persistence depends on configuration: LM Studio’s default logging setting recorded plaintext prompts on disk until sensitive-data logging was disabled; Ollama produced no persistent prompt artifacts under the evaluated configuration and showed no outbound transmissions in isolated tests.
Serving-interface isolation failures: a missing resource-level authorisation in llama.cpp allowed cross-tenant slot restoration in 200/200 trials; restored state produced the victim secret in 126/200 model continuations; shared prompt-prefix caching created a remote timing oracle with perfect separability in measured samples (AUC = 1.000).
Limitations
Results are bounded to the tested model artifacts, software versions, configurations and the explored fuzzing state space; absence of parser exploits in this campaign does not prove universal parser security. GPU findings apply to the evaluated partial-offload architecture and may not generalise to all fully offloaded systems. Tests used synthetic UUID canaries and researcher-controlled instances rather than production services.
Implications
Offensive-security implications are concrete: an unprivileged local process or forensic inspection of host memory can recover plaintext prompts left by other users; an authenticated client can exploit missing authorisation to restore another tenant’s conversation state and trigger model-mediated disclosure; remote attackers able to issue queries and measure timing can infer cached prompt-prefix membership. Persistence in wrapper logs permits later exfiltration from disk. The findings show that localisation of computation alone is insufficient; attackers can leverage memory residue, misconfigured wrappers, protocol authorisation bugs and timing side channels to obtain confidential prompts.
- System prompts can hold sensitive business logic; leakage via normal or adversarial queries is a real risk.
- Detects leakage using hypothesis testi...
- It shows system prompts, which encode core logic, are leaking in real-world LLM apps, exposing internal rules and even API keys.
- Across 1,200 apps...
- It shows that prompts used to tailor LLMs can be recovered from the model, potentially exposing confidential instructions or business logic even whe...
- It tackles a real risk: system prompts can leak sensitive information or rules used by AI agents.
- It introduces ProxyPrompt, which hides the origi...
- It tackles protecting sensitive data in prompts sent to external LLMs, addressing data leakage risk in AI workflows.
- Core idea: a cryptographical...
- It shows a new timing-based side channel in multi-tenant LLM serving stacks: shared KV and semantic caches, and GPU memory behavior, can leak privat...
- Why it matters: it tackles prompt leakage and prompt-injection risks in LLMs by adding a protective shield to system prompts, optimized automaticall...
- It shows a sneaky supply-chain style risk: malicious system prompts inside public prompt resources can steer LLMs to give targeted, compromised outp...
- It studies how prompts can manipulate LLM behavior (prompt injection and jailbreaks) across multiple open models.
- It shows wide variation in respo...
- It demonstrates a closed-box method (PLeak) to recover a target LLM application's confidential system prompt by crafting optimized queries, revealin...
Get the weekly digest
The few AI-security papers that matter, with the practitioner takeaway. No spam.
