Survey reveals users expose AI security risks

Society

Published: Mon, Nov 03, 2025 • By Dr. Marcus Halden

Survey reveals users expose AI security risks

Survey of 3,270 UK adults finds common behaviours that raise security and privacy risks when using conversational agents (CAs). A third use CAs weekly; among regular users up to a third engage in risky inputs, 28% attempt jailbreaking, and many are unaware their data may train models or that opt-outs exist.

New research examines how ordinary people use conversational agents (CAs) and what that means for safety. The study defines CAs as chat-style services powered by modern Large Language Models (LLMs). It does not hunt for exotic attacks; instead it asks a simple question: are everyday behaviours creating real attack surfaces? The short answer is yes, in ways that matter.

The authors surveyed 3,270 UK adults in 2024 using a representative recruitment pool. A prescreen identified people who use CAs at least weekly; 906 completed the main questionnaire. The instrument covered work and leisure use, file uploads, program access, redaction habits, and attempts to bypass safeguards (jailbreaking). The analysis is straightforward: proportions, contingency tests and ordinal regression to see which traits predict risky actions. That keeps the evidence tidy and interpretable.

Several findings jump out. About a third of UK adults report weekly CA use. Among those regular users, up to a third display behaviours that could enable attacks: uploading non-self-created documents, granting CAs access to other programs, or supplying inputs that prompt unsafe responses. In work settings 74.4% report loading content into CAs and 39.9% say they shared non-self-created material. A sizeable minority grant CAs access to office programs, calendars or email clients—23.6% at work and 15.6% in leisure.

Jailbreaking is common. Roughly 28% of respondents reported trying to bypass guardrails, typically for curiosity, entertainment or to obtain information. That matters because many academic threat models rely on users intentionally or accidentally providing inputs that exploit model behaviour. The survey shows those models are not theoretical; people do these things in the real world.

Privacy awareness is patchy. About half of participants say they edit or redact sensitive information, and few report sharing obvious secrets such as passwords, but some still do share bank or passport details. Crucially, 53.8% did not know their data could be used to train models and 76.5% were unsure whether opt-out options exist. That combination of sharing plus low awareness widens the practical risk surface.

Limitations are worth flagging. The study is UK only, relies on self-report and captures a fast-moving field at a single point in time. Definitions of CAs are shifting, and the authors note some questions around jailbreaks were reported as unclear by participants. Still, the core pattern is robust: common user behaviours line up with known risks.

Operational takeaways

Treat CA use as a measurable attack surface: audit uploads and program integrations and block non-essential access.
Push vendor transparency and easy opt-outs for data training; inform users that their inputs may be retained.
Train teams to redact sensitive fields and ban sharing of credentials; enforce secure-by-default settings for workplace CAs.

Additional analysis of the original ArXiv paper

📋 Original Paper Title and Abstract

Prevalence of Security and Privacy Risk-Inducing Usage of AI-based Conversational Agents

Authors: Kathrin Grosse and Nico Ebert

Recent improvement gains in large language models (LLMs) have lead to everyday usage of AI-based Conversational Agents (CAs). At the same time, LLMs are vulnerable to an array of threats, including jailbreaks and, for example, causing remote code execution when fed specific inputs. As a result, users may unintentionally introduce risks, for example, by uploading malicious files or disclosing sensitive information. However, the extent to which such user behaviors occur and thus potentially facilitate exploits remains largely unclear. To shed light on this issue, we surveyed a representative sample of 3,270 UK adults in 2024 using Prolific. A third of these use CA services such as ChatGPT or Gemini at least once a week. Of these ``regular users'', up to a third exhibited behaviors that may enable attacks, and a fourth have tried jailbreaking (often out of understandable reasons such as curiosity, fun or information seeking). Half state that they sanitize data and most participants report not sharing sensitive data. However, few share very sensitive data such as passwords. The majority are unaware that their data can be used to train models and that they can opt-out. Our findings suggest that current academic threat models manifest in the wild, and mitigations or guidelines for the secure usage of CAs should be developed. In areas critical to security and privacy, CAs must be equipped with effective AI guardrails to prevent, for example, revealing sensitive information to curious employees. Vendors need to increase efforts to prevent the entry of sensitive data, and to create transparency with regard to data usage policies and settings.

🔍 ShortSpan Analysis of the Paper

Problem

The paper examines security and privacy risks arising from ordinary users interacting with AI based conversational agents CA s. It focuses on how user behaviours such as providing insecure inputs jailbreaking and granting access to other programs may create practical attack surfaces for LLMs and LMMs and how aware users are of data usage and opt out options. Understanding the prevalence of these risk inducing behaviours is aimed at informing mitigations, guardrails and policy guidance for secure CA usage in personal and organisational settings.

Approach

The researchers conducted a three stage survey in 2024 of a representative sample of UK adults, using Prolific to recruit participants. From 3 270 respondents, a prescreen identified those who use CA services at least weekly, yielding 906 who completed the main questionnaire. The instrument covered demographics and general CA use, work and leisure time usage, content sharing with CA s, CA access to other programs, jailbreaking attempts and motivations, editing of sensitive inputs, and awareness of data usage and opt out options. Analyses included contingency tables and chi square tests with a Bonferroni correction for multiple testing, and ordinal regression to identify predictive features. The final cohort included 601 work users 763 leisure time users and 478 in both contexts. The most used CA was ChatGPT followed by Copilot and Gemini.

Key Findings

Approximately one third of UK adults use CA services at least weekly and among regular users up to a third exhibited behaviours that may enable attacks while a quarter reported jailbreaking for entertainment or information seeking. Half stated they sanitise data and most reported not sharing sensitive data, though a minority still shared highly sensitive information such as passwords.
Content sharing with CA s is common; in work settings 39.9 per cent shared non self created content and in leisure 34.9 per cent did so. A sizeable minority granted CA s access to other programs 23.6 per cent at work and 15.6 per cent in leisure, with about a third of those who shared content also providing access to programs across contexts.
Many participants loaded content into CA s; in work 74.4 per cent loaded content with text being most common, followed by documents and images. Of these, 53 per cent was non self created. In leisure times 64.7 per cent loaded content with a similar pattern, 34.9 per cent of shared content being non self created. Among those using CA s in both contexts, 95.8 per cent shared content and 19.7 per cent shared external content in both settings.
More participants gave CA s access to programs at work than in leisure time. The most frequent targets were office programs, calendars and email clients. A notable 12.0 per cent of at work and 8.3 per cent of leisure time users loaded non self created content into a CA connected to a program.
Jailbreaking is common with 28 per cent of participants having done so. Motivations include entertainment exploration and obtaining information; men and younger users jailbroke more often, though a predictive model did not identify strong demographic predictors. Early usage and CA savviness did not reliably predict jailbreak behaviour in ordinal regression.
Privacy related behaviours show a mixed picture. About half report editing or redacting sensitive information, with work settings showing higher editing rates. Across both contexts many avoid sharing sensitive data yet some do share, including passwords, bank details, and passport numbers. Awareness of data being used to train CA models and opt out options is low; 53.8 per cent did not know data could be used for model training, while 76.5 per cent were unsure whether opt out exists.
Overall, the study finds that academic threat models around prompt injections and data access realistically manifest in everyday CA use, reinforcing the need for robust guardrails, secure data handling, input controls and transparent data use policies. It also highlights a need for increased vendor transparency regarding data usage and opt out provisions.

Limitations

The authors acknowledge that CA definitions are evolving and often opaque, which may affect interpretation. Self reported data are subject to social desirability and recall biases, and the study focuses on a UK sample which may limit generalisability. The survey captures current crime and threat models but may not reflect unconscious CA usage or evolving technologies. Pre testing identified some questions related to jailbreaks as unclear, and the authors note that the fast changing CA landscape may render findings conservative over time.

Why It Matters

The findings reveal a substantial real world threat surface for CA usage driven by user behaviour including insecure inputs, jailbreaking and program access. They underscore the need for strong guardrails in security and privacy critical areas, secure data handling and minimisation, input controls and clear user friendly data usage policies including opt out options. For organisations, the work supports risk assessments and the deployment of governance and training to reduce sensitive data exposure. CA vendors are urged to increase efforts to prevent sensitive data entry and improve transparency around data usage and training practices. The study calls for secure by default designs and guidance on data handling and user education to mitigate privacy and security risks in the expanding use of AI based conversational agents.

Attribution Original paper on arXiv