Shadow AI is AI tools being used for work without the company knowing. Here's what goes wrong, and what organizations and individuals should each do.
Meaning: the AI form of shadow IT
Shadow IT means employees using cloud services or devices for work that the company hasn't approved. Shadow AI is the part of it that involves AI tools. It is rarely malicious. It starts with reasons like "I want to write up meeting notes faster" or "I want to fix this bug faster", using whatever convenient tool is at hand.
Today's shadow AI is more than pasting text into a chatbot. Browser extensions read the page you have open, and AI agents connect to work accounts to read and act on mail and files.
Chatbots
pasting text or files into a personal account
Coding assistants
sending internal source code and config files
Meeting-note AI
recording meeting audio or screens in an outside service
AI browser extensions
can read the work pages you have open
OAuth grants
allowing an AI tool to read mail or drives
AI agents with API keys
operating business systems on your behalf
Why it happens
In most cases the reasons are convenience and the company not offering an alternative. There's no approved tool, approval takes too long, or the rules are vague enough that nobody knows what's prohibited. In that situation, people get the job done quickly with personal accounts.
In May 2023, Samsung Electronics was reported to have restricted internal use of generative AI after finding that staff had uploaded internal source code and meeting content to ChatGPT (reported by Bloomberg, Fortune and others). According to the reports, the company was concerned that data sent to outside services is hard to retrieve and delete, and said it would build its own internal tools for translation and summarization. It's widely cited as an example of data being entered out of a wish to work more efficiently.
What goes wrong
1. Entering confidential or personal data
The most common case is pasting customer data or internal documents into AI. How that content is handled depends on whether it's a consumer or business plan, and on settings. Summarizing the providers' official documentation as of October 2026:
| Provider | Consumer plans | Business plans and API |
|---|---|---|
| OpenAI | May be used for training; can be turned off in settings | Not used for training by default |
| Anthropic | The user chooses whether to allow training (2025 terms update) | Outside that update (covered by commercial terms) |
| Google (Gemini) | With "Keep Activity" on, may be used for training and human review | Work accounts have different data-handling terms |
This table only gives the headline. Retention periods, retention for abuse monitoring, and the handling of feedback you submit differ by provider, and terms change. Always check the official help pages and privacy policy before adopting a tool.
In its alert of June 2, 2023, Japan's Personal Information Protection Commission noted that if a business enters prompts containing personal data into a generative AI service without the person's consent, and that data is handled for purposes other than producing the response, the business may violate the Act on the Protection of Personal Information. It therefore asks businesses that do enter such prompts to confirm carefully that the provider won't use the data for machine learning. Other jurisdictions have their own rules, such as the GDPR in the EU.
2. OAuth grants and API keys
When someone clicks "Sign in with Google" or "Connect your Microsoft account" and allows an AI tool to read mail or drives, the tool can keep reading business data without the employee's password. Even after the employee leaves, the access remains until the grant is revoked. Giving an AI agent an API key for a business system works the same way: if the key leaks, everything that key can do is usable from outside (→ the basics of .env files and API keys).
3. AI browser extensions
Many AI extensions that summarize pages or polish writing ask for permission to read the content of the page you have open. That includes business-system screens, customer management screens and internal portals. Extensions can also change hands after release, or change behavior with an update.
4. Output errors, rights and records
AI output contains errors. If incorrect content goes straight to a customer, or code with an unclear license goes into a product, the organization that used it bears the responsibility. And when business exchanges sit in personal accounts, they fall outside record preservation for audits and lawsuits (legal hold), leaving the company unable to account for what happened.
How to prevent data leaks through generative AI
Data leaks through generative AI usually happen because data that was entered, or access that was granted, ends up outside the company's control. Here's what organizations and individuals can each do, route by route.
| How data leaks | What the organization does | What individuals do |
|---|---|---|
| Content entered is stored by the provider and used for training | Approve a tool on business terms that don't train on inputs | Keep work data out of personal accounts; turn off training in personal use |
| A personal account is taken over and chat history is read | Have people sign in with company accounts (SSO) and multi-factor authentication | Turn on multi-factor authentication and delete history you no longer need |
| A shared chat link makes a conversation visible outside the company | Cover sharing in the AI rule, and check whether the admin console can restrict it | Before creating a link, check that the chat contains nothing confidential |
| OAuth grants or extensions keep reading business data | Review authorized apps and extensions, and limit them with an allowlist | Read the permissions on the consent screen before allowing them |
| Other people's data or errors in the output go out as-is | Set a step where a person checks output before it leaves the company | Check facts and personal data in the output before using it |
What public bodies have said
- In its June 2, 2023 alert, Japan's Personal Information Protection Commission warned general users that personal data they enter may be used for machine learning and come back out in inaccurate form. It asks them to read the provider's terms and privacy policy and decide what to enter accordingly.
- The same alert asks businesses that enter prompts containing personal data to confirm it's within the stated purpose of use, and to confirm carefully that the provider won't use the data for machine learning.
- In "10 Major Information Security Threats 2026" (published January 29, 2026), Japan's IPA (Information-technology Promotion Agency) ranked "cyber risks around the use of AI" third for organizations, its first appearance. Unintended data leaks caused by a poor understanding of AI are part of it.
What to check in a business plan
The providers' official documentation, checked on October 6, 2026, says business-plan inputs aren't used for training by default.
- OpenAI: data from ChatGPT Business, Enterprise and Edu, and from the API, isn't used to train models by default
- Anthropic: inputs and outputs of commercial products such as Claude for Work and the API aren't used for training by default (exceptions include feedback or bug reports the user submits)
- Google: in Gemini for Google Workspace, prompts are treated as customer data and aren't used to train generative AI models without the customer's prior permission or instruction
Before signing, check more than training: retention periods and deletion, whether admins can see usage, and whether it supports SSO.
This site's view: a business plan still needs a data rule
"Not used for training" doesn't mean "not stored". Even on business plans, how long conversations are kept by the provider depends on the contract and settings, and they may be retained for a period to monitor abuse. And legal duties such as checking the purpose of use for personal data remain whatever the contract. So we think a business contract and a data rule only work when you have both.
For organizations: bring it into view
Japan's AI Guidelines for Business, published by the Ministry of Internal Affairs and Communications and the Ministry of Economy, Trade and Industry (latest: version 1.2, March 31, 2026), treat businesses that use AI at work as "AI business users" and call for management proportionate to risk. A practical first step for shadow AI is to work in this order.
Provide an approved AI tool
Choose a business contract that doesn't use inputs for training, and check whether you can control retention and whether admins can review history. Publish the list of tools people may use. Providing an alternative first is the precondition for every other measure.
Set a short rule on what data may be entered
Following your data classification, decide something that fits on one page, such as "public information: OK; internal documents: approved tools only; customer personal data, passwords, API keys and unpublished financials: never". A short table people can check when unsure is followed more than a long policy.
Connect it with SSO and admin controls
Have people sign in to the approved tool with company accounts (SSO) so access can be cut off when they leave. Choose a plan whose admin console lets you check usage and data-retention settings (→ an organization's security minimum).
Review OAuth grants and extensions
In the Google Workspace or Microsoft 365 admin console, regularly review the third-party apps employees have authorized, and revoke the ones you don't need. Where possible, allow connections only to apps an admin has approved. On managed devices, limit browser extensions with an allowlist too.
Detection and an ask-first channel
DLP (data loss prevention) and CASB (cloud access visibility) products, as well as proxy and DNS logs, can show traffic to AI services and large uploads. The aim isn't punishment but learning which tools are in use so they can become candidates for approval. Pair this with a channel where people can quickly ask "may I use this tool?" and short training.
A ban only
Banning AI outright without an alternative tends to push use onto personal phones and personal accounts. The company can't see the use, and when something goes wrong it often can't tell what was sent.
Approved tool plus a rule
A tool on terms that don't train on inputs is available through work accounts, and what may be entered is defined. Usage is visible in the admin console, and access ends with the account when someone leaves.
This site's view: start by finding out what's actually in use
We think shadow AI work is more effective when it starts with finding out which AI tools are actually in use, not with a ban notice. One look at the list of OAuth grants and the list of extensions often turns up tools nobody expected. Rather than banning what you find right away, consider whether it can move to a business contract or be replaced by another approved tool. Once people know "if I ask, I can use it", the reason to hide it goes away.
For individuals
Even if your company has no AI rule yet, avoid this
Don't paste customers' names, contact details or other personal data, passwords, API keys, or unpublished contracts and financial documents into AI on a personal account. If you enter an API key by mistake, don't just delete it: revoke the key and issue a new one (→ why leaked credentials must be revoked, not deleted).
- If your company has an approved AI tool, use it
- For personal AI accounts, check the setting that stops training on your content and the history settings
- On "Connect with Google" style consent screens, read the permissions requested (reading mail, access to the whole drive, and so on) before allowing them
- Watch for emails and fake login pages pretending to be AI tools (→ what phishing is)
Read next
- Learn: security for the AI era: a priority checklist (the basics to lock down before using AI tools)
- Guide: AI agents reaching real websites (how to think about handing access to AI agents)
- Learn: what's actually dangerous about .env and API keys (handling the keys you give AI tools)
- Guide: an organization's security minimum (priorities for account management and reviews)
- Glossary: what phishing is (defending against fake login pages posing as AI services)
Sources
- Personal Information Protection Commission (Japan), alert on the use of generative AI services (June 2, 2023): ppc.go.jp
- Personal Information Protection Commission (Japan), alert on the use of generative AI services, Attachment 1 (June 2, 2023, Japanese): ppc.go.jp (PDF)
- Ministry of Internal Affairs and Communications, AI Guidelines for Business (version 1.2, March 31, 2026): soumu.go.jp
- IPA, "10 Major Information Security Threats 2026" (January 29, 2026, Japanese): ipa.go.jp
- OpenAI Help Center, "How your data is used to improve model performance": help.openai.com
- OpenAI, "Business data privacy, security, and compliance" (checked October 6, 2026): openai.com
- Anthropic, "Updates to Consumer Terms and Privacy Policy" (August 28, 2025): anthropic.com
- Anthropic Privacy Center, "Is my data used for model training?" (commercial products, checked October 6, 2026): privacy.claude.com
- Google, Gemini Apps Privacy Hub: support.google.com
- Google, "Generative AI in Google Workspace Privacy Hub" (checked October 6, 2026): knowledge.workspace.google.com
- Bloomberg, "Samsung Bans Staff's AI Use After Spotting ChatGPT Data Leak" (May 2, 2023): bloomberg.com
- Fortune, "Samsung bans employee use of ChatGPT after data leak" (May 2, 2023): fortune.com