ChatGPT Security Risks for Enterprise Teams (2026)

Apr 25, 2026

ChatGPT is now inside most enterprises, whether security teams approved it or not. The productivity gains are real, and so are the risks. Data leaves the building one prompt at a time. Hallucinated code ships to production. Prompt injection turns a helpful assistant into an exfiltration channel. Auditors notice. This is the security posture to understand before you draft another policy.

What are the security risks of ChatGPT at work?

ChatGPT security risks are the confidentiality, integrity, and compliance exposures that arise when employees use OpenAI’s chat interface, API, or Enterprise tier in the course of business. They range from accidental data leakage through prompts to manipulated outputs, hallucinated facts, and shadow AI sprawl that procurement has never approved.

The risks are not hypothetical. Samsung Semiconductor banned consumer ChatGPT in May 2023 after three separate employee incidents leaked proprietary code, meeting notes, and chip yield data to OpenAI in under a month. Apple, JPMorgan, Bank of America, Verizon, Amazon, Goldman Sachs, and Deutsche Bank followed with restrictions within weeks. A 2023 Cyberhaven analysis of 1.6 million knowledge workers found that 11% of the content employees pasted into ChatGPT contained confidential information.

Treat ChatGPT as another SaaS application that happens to receive unstructured text input. The same controls you already apply to cloud storage and code repositories need to apply here, adjusted for the fact that the input is natural language and the output is generated content that employees trust more than they should.

The 8 biggest ChatGPT risks for enterprises

1. Data leakage through copy-paste into prompts

The most common and most expensive ChatGPT risk. An engineer pastes a failing code snippet to debug it. A product manager drops the Q3 roadmap in to generate an exec summary. A customer success rep feeds in raw support tickets to draft a knowledge-base article. Each paste lands on OpenAI’s servers and, depending on the plan tier and account settings, may be retained for model improvement or operational review.

Consumer ChatGPT and ChatGPT Team retain prompts by default unless history is turned off. ChatGPT Enterprise and Edu do not use customer content to train OpenAI models per the current OpenAI Enterprise privacy documentation, but the data still flows to OpenAI infrastructure and sits in prompt logs. “We use ChatGPT Enterprise” is a better posture than unmanaged accounts, but it does not mean sensitive data can travel freely. See AI data leakage for employees for the fuller case.

2. Prompt injection in uploaded documents

Prompt injection is the highest-ranked risk in the OWASP Top 10 for LLM Applications, and ChatGPT is as vulnerable to it as any other LLM. An attacker embeds hidden instructions in a document, a web page, or a PDF. When an employee asks ChatGPT to summarize the file, the model follows the hidden instructions instead of, or in addition to, the user’s request.

Indirect injection is the dangerous variant. The attacker does not need access to the chat itself. They just need to place poisoned content somewhere the model will read it: a crafted Jira ticket, a doctored PDF shared over email, a response from a third-party API that a plugin consumes. Anthropic disclosed in November 2025 that a state-sponsored group weaponized Claude Code with these techniques against more than 30 organizations, and the same attack surface exists in ChatGPT. Walk the prompt injection exercise to see how the pattern actually works.

3. Hallucinated code with real vulnerabilities

ChatGPT produces code that compiles, passes review at a glance, and contains subtle security bugs. Insecure random number generation, hard-coded credentials, SQL string concatenation instead of parameterized queries, overly permissive CORS headers, and cryptographic antipatterns are all common outputs when developers ask for help without specifying security requirements.

A Stanford University study published in 2023 (“Do Users Write More Insecure Code with AI Assistants?”) found that developers using AI coding assistants produced less secure code while reporting greater confidence in that code’s quality. The combination is the worst possible outcome: more bugs shipping with less scrutiny. The fuller picture lives in AI coding assistant security risks.

4. Credential exposure in chat logs

Employees paste credentials into ChatGPT more often than anyone wants to admit. API keys inside error messages, database connection strings in debugging sessions, full .env files pasted for “help fixing a syntax error,” and bearer tokens copied with the rest of a curl command. GitGuardian’s annual secrets reports consistently find millions of leaked secrets each year across public repositories, and the same patterns show up inside prompts.

The exposure compounds over time. Even if OpenAI never sees a specific log entry, a single compromised OpenAI account with chat history enabled can leak months of secrets at once. That history is also subject to legal discovery in some jurisdictions.

5. IP leakage through training data

OpenAI’s public documentation is explicit that ChatGPT Enterprise and API traffic do not train foundation models. Consumer ChatGPT prompts can train models unless users manually opt out or turn off chat history. That distinction matters for intellectual property.

Proprietary text pasted into a consumer account can, in principle, influence future model behavior in ways that are impossible to fully audit. Even with opt-out, the data still lives in prompt logs, transits OpenAI infrastructure, and is accessible to authorized OpenAI personnel under defined circumstances. For regulated data, the correct model is “do not send,” not “trust the toggle.”

6. Shadow AI sprawl across the organization

ChatGPT does not arrive through procurement. It arrives through a browser tab. An engineer signs up with a work email. A marketer buys a $20/month Plus subscription on a personal card. A finance analyst connects a ChatGPT browser extension to their corporate Outlook. Each of those is shadow AI, and it is the fastest-growing category of shadow IT.

Shadow AI breaks three things at once. Procurement loses the ability to negotiate an enterprise contract with data protection terms. Security loses visibility into what data is leaving the organization and to whom. Compliance loses the paper trail regulators ask for during audits. None of those are theoretical. All of them happen quietly.

Attackers use ChatGPT to scale social engineering. LLMs strip out the typos, awkward phrasing, and generic greetings that employees were trained to spot. Our AI-powered phishing deep dive covers this in detail, and SlashNext’s 2025 State of Phishing report found a 4,151% rise in AI-generated phishing messages since ChatGPT’s public release, with click rates running about 14x higher than traditional mass campaigns.

The enterprise risk runs both ways. Internal users who do not understand the AI-phishing shift will treat well-written, contextually accurate emails as trustworthy by default. That misplaced trust is what AI-era human risk management has to address, because traditional “spot the typo” training no longer covers the attack.

Auditors have stopped asking whether AI is in the environment. They ask where it is, what data flows to it, and who approved that flow. For GDPR, sending personal data to a sub-processor the data subject was never informed about is a classic Article 28 problem. For HIPAA, protected health information pasted into a non-BAA-covered chat service is a disclosable breach. For SOC 2, the absence of an AI acceptable-use policy and supporting controls is now a common finding.

OpenAI offers BAAs on ChatGPT Enterprise for qualifying healthcare customers, and Data Processing Addenda on Enterprise and API tiers for GDPR. Those agreements only cover traffic that goes through the covered products. Any prompt submitted from an unmanaged consumer account falls outside them.

Real incidents worth studying

Samsung Semiconductor, 2023. Three engineers leaked confidential data to ChatGPT inside a month: source code, meeting transcripts, and chip testing measurements. Samsung responded with a company-wide ban on consumer ChatGPT and shipped an internal AI platform afterwards. The case is important because nothing about the behavior was malicious. Each employee was trying to work faster.

Mata v. Avianca, 2023 (S.D.N.Y.). A New York attorney used ChatGPT for case research and filed a brief citing six cases that did not exist. The model hallucinated the citations with convincing authority. Judge Castel imposed sanctions and the case is now a casebook example for every AI governance session. Official docket documents are public. The pattern is the same for any knowledge worker who treats ChatGPT as an authoritative source instead of a draft tool.

Open JumpCloud incident reports and GitGuardian research. Beyond the headline cases, research from GitGuardian and JumpCloud consistently shows employees pasting credentials, secrets, and customer data into AI chat tools at non-trivial rates. Those are not one-off stories, they are the base rate.

ChatGPT vs Claude vs Copilot risk comparison

All three are subject to the same core risk model, but the controls and defaults differ in practice.

Risk	ChatGPT Enterprise	Claude for Work	Microsoft 365 Copilot
Training on prompts	No	No	No
Default data retention	30 days, configurable	30 days, configurable	Tied to Microsoft 365 retention
BAA for HIPAA	Available	Available	Available under M365 BAA
Prompt injection defenses	Hardened, not eliminated	Hardened, not eliminated	Hardened, not eliminated
Source grounding	Browsing + user uploads	User uploads + MCP tools	M365 content and Graph
Hallucination risk	High on unverified claims	High on unverified claims	Lower when grounded in M365
Shadow use pressure	Very high (free tier)	Medium (no free public tier until 2024)	Lower (requires M365 license)
Admin controls	SSO, SCIM, DLP connectors	SSO, SCIM, admin console	Microsoft Purview + M365 DLP

The comparison is not about which tool is “safest” in isolation. It is about which tool fits the governance controls already in place. A Microsoft-heavy organization often reduces shadow AI pressure faster with Copilot, because licensing aligns with existing EA contracts. An engineering-heavy organization might prefer ChatGPT Enterprise or Claude for Work for coding and long-context tasks, with tighter DLP rules on the edge.

How to use ChatGPT safely at work

Deploy an enterprise tier with SSO. ChatGPT Enterprise, Team, or Edu remove training-on-prompts by default and route through your identity provider. That alone reduces a large share of shadow accounts and gives you a single place to enforce logging, retention, and DLP.

Apply DLP rules to AI domains. Add chat.openai.com, claude.ai, gemini.google.com, and similar AI services to your DLP policy. Block paste of tagged data classes (source code, PII, payment data, PHI) on unmanaged accounts. Allow it on managed enterprise accounts under logging. Modern CASBs from Netskope, Zscaler, and Microsoft Defender for Cloud Apps all support this.

Publish a prompt-data classification policy. Three classes are enough for most organizations: public, internal, and restricted. “Public” flows freely. “Internal” can go to approved enterprise AI with logging. “Restricted” (credentials, PII, PHI, regulated data, unreleased code) does not go to any AI tool, approved or not, until a documented exception is issued.

Require training for anyone who uses AI at work. Generic security awareness training does not cover the AI patterns. The AI Security catalogue includes exercises for prompt injection, AI data leakage, and AI-powered phishing. Short, scenario-based sessions with realistic prompts land harder than long videos.

Force human review of hallucinated output. Set a house rule that any fact, citation, or code block produced by ChatGPT must be verified against a primary source before it ships. For code, mandate security review when AI-written functions touch authentication, authorization, crypto, data handling, or external inputs.

Log and retain prompt history for audit. Enterprise tiers give you admin access to conversation logs. Retain them in line with your broader SIEM retention policy so you can investigate incidents, prove compliance, and respond to legal discovery. If your industry is regulated, tie these logs into the same pipeline as email and chat archives.

Guard plugins and third-party integrations. Each plugin, GPT, or agent that ChatGPT can call is another piece of attack surface. Approve them the same way you approve any OAuth integration with your identity provider. Watch for broad scopes (mailbox read, drive write) and revoke them the moment a business case ends.

Use browser policy to restrict consumer accounts. Group Policy, Jamf, or your MDM can enforce that chat.openai.com only accepts sign-in through your corporate IdP domain. That one setting quietly removes most consumer-account shadow use on managed devices.

Run tabletop drills for AI incidents. Assume that a prompt injection exfiltrates a mailbox, or that a hallucinated legal citation lands in a filing, or that a credential gets pasted into chat. Rehearse who pages who, who holds legal counsel, who talks to OpenAI support, and who notifies customers. The first time you run the drill is always the worst time to run the drill.

Pair policy with measured adoption. Draconian blocks push usage underground. The best-performing programs combine a visible, approved AI stack, clear data rules, and a simple exception process for edge cases. That combination drops shadow AI faster than any policy written in isolation.

Review vendor terms quarterly. OpenAI, Anthropic, Microsoft, and Google update data handling policies frequently. Subscribe to their changelogs, read the privacy and security pages, and check that your internal guidance still matches what the vendor actually promises.

Training employees on AI security

Policy without training does not move behavior. The teams that get ChatGPT risk right run three things in parallel.

First, they teach the patterns. The OWASP LLM Top 10 is a useful vocabulary for why risks exist. The AI-powered phishing deep dive covers what employees actually face in the inbox. The AI coding assistant security risks guide covers what developers need to know before they ship AI-written code.

Second, they rehearse under fire. Short, interactive exercises beat long videos. The RansomLeak AI Security catalogue includes scenarios for prompt injection, data leakage, and AI-assisted social engineering. Each one takes ten minutes and builds the specific reflex employees need.

Third, they measure. Incident reports, DLP triggers, and reported phishing metrics all tell you whether behavior is shifting. The goal is not zero AI use. It is clear signal on where AI is being used, by whom, for what, and under which controls.

FAQ

Is ChatGPT safe for enterprise use?

ChatGPT Enterprise and Edu are designed for enterprise use and do not train OpenAI models on customer prompts, but “safe” depends on the controls around it. Without DLP, policy, and training, even the enterprise tier can leak regulated data through careless prompts.

Does ChatGPT retain my prompts?

Consumer ChatGPT retains prompts by default unless a user turns off chat history. ChatGPT Enterprise and API retain prompts for up to 30 days for abuse monitoring under the current Enterprise privacy terms, and the retention window is configurable for Enterprise admins.

Can ChatGPT leak my company’s source code?

Yes, if employees paste code into a consumer account with history enabled, that code sits in OpenAI’s logs and, in principle, could influence model behavior. Enterprise and API traffic does not train models, but the code still traverses OpenAI infrastructure, which is why DLP on AI domains matters.

Is ChatGPT HIPAA compliant?

ChatGPT Enterprise can be used under a Business Associate Agreement with OpenAI for eligible healthcare customers. Consumer ChatGPT and ChatGPT Team are not covered by a BAA, and protected health information pasted into those tiers is a disclosable incident.

How do I detect shadow ChatGPT use?

Network monitoring for AI domain traffic, CASB policies, expense reports for consumer subscriptions, and employee surveys all help. The fastest signal is usually network DNS logs combined with a short, non-punitive survey of each department.

What data should never be pasted into ChatGPT?

Credentials, API keys, secrets, protected health information, payment card data, non-public financials, unreleased source code, customer PII, and anything regulated under NDA, HIPAA, or GDPR. When in doubt, treat the prompt like posting to a public forum.

Can prompt injection attacks really hurt my company?

Yes. Indirect prompt injection can exfiltrate data through ChatGPT’s own connectors and browsing features, manipulate outputs to mislead users, or cause AI-driven workflows to take unintended actions. The OWASP LLM Top 10 treats prompt injection as the number-one LLM risk for good reason.

How often should we audit AI use?

Quarterly at minimum. AI tools, plans, and integrations change fast, and the gap between the policy you wrote six months ago and the reality in your environment widens every week you are not paying attention.

Bottom line

ChatGPT is a productivity tool that sits on top of the same risk categories as any other SaaS, amplified by natural-language input and generative output. Enterprise tiers, DLP on AI domains, a prompt-data classification policy, and training that actually covers the AI patterns will move you from hope to control.

If you are ready to move the workforce past “spot the typo” training, the AI Security catalogue and the AI data leakage guide are the next two stops.