I have a Skynet scale I use to classify AI security stories. A 1 on the Skynet scale is a car company's chatbot agreeing to a user's trick prompt to sell a new vehicle for $1. A 5 on the Skynet scale is Arnold Schwarzenegger appearing at my office door and telling me to come with him if I want to live. Recently, OpenAI admitted to something that has me tracking Arnold's location on my celebrity tracker app.

While testing an unreleased model called GPT-5.6 Sol against a benchmark of hacking challenges, the model found a security hole nobody had mapped, used it to slip past its own sandbox, and reached into a real company, the AI platform Hugging Face, to pull the answers it needed to finish the test. Barely a week later, Anthropic disclosed something similar had happened to Claude three times since April, after a testing partner mistakenly gave the model live internet access during a cybersecurity exercise, and in one case Claude pulled several hundred rows of real production data out of a company that happened to share a name with a fictional target.

Nobody hacked OpenAI or Anthropic from the outside, cracked a password, or slipped through a firewall. Both companies had built a tool smart enough to solve the problem in front of it, and it solved its way straight through a door nobody had locked. It's an interesting case study because it's not really a story about frontier AI labs. It's a story about what happens when the people responsible for building trust with customers become the ones whose own AI tool breaks it, whether that's a lab's safety team or, more often, marketing.

I've spent my career in marketing, most of it in tech, and I've watched our function go from owning campaigns and content calendars to owning chatbots, personalization engines, AI-written email sequences, and tools that read customer data to decide what to say next. That's a lot of new responsibility, and most of us didn't sign up to be security officers, but when the tool that leaks the data, exposes the customer record, or hands a stranger something it shouldn't is a tool marketing chose and deployed, marketing owns the fallout.

The breach doesn't always look like a hack

When Samsung engineers wanted quick help debugging some source code and summarizing meeting notes in early 2023, they pasted the material straight into ChatGPT. They did it three separate times in twenty days, and each time, that proprietary information left the building and became part of a system Samsung didn't control. Samsung eventually banned generative AI tools on company devices. Nobody broke in and employees used the tool the way it was meant to be used: process whatever text you give it.

That pattern has a name now. Security researchers call it shadow AI, meaning AI tools employees use without formal approval or oversight. According to IBM's 2026 Cost of a Data Breach Report, shadow AI played a role in one out of every five breaches last year, and each of those breaches cost, on average, an extra $670,000 compared to breaches that didn't involve it. The same report found that one in four malicious breaches are now AI-enabled in some way, a 56% jump from the year before, pushing the average cost of those breaches to $6 million. And 63% of organizations still don't have a formal AI governance policy at all. We're plugging in tools faster than we're writing the rules for them.

There's a second category, and it doesn't involve an employee at all. Earlier this year, researchers disclosed a prompt injection vulnerability in Microsoft 365 Copilot called EchoLeak. Prompt injection means hiding instructions where an AI will read and obey them, like a hidden line in an email telling Copilot what to do. It let an attacker pull sensitive data out of a company's systems without the victim clicking a single link or opening a single attachment. The exploit rode in on a normal email and did its work quietly, in the background, through the AI assistant itself. That's a different category of risk than an employee pasting the wrong thing into a chat window. That's the tool having a hole in it that nobody, including the vendor, caught in time. You can train every employee on your team to be careful, and it won't matter, because the vulnerability lives in the software, not the person using it.

The AI risk math (I promise it's simple)

Marketers don't love equations, and I'm not about to throw calculus at you. But there's one idea from risk management worth knowing, because it explains why AI tools change the picture so much. It's just this:

Think of it like deciding whether to buy an umbrella. If rain's unlikely and getting wet is no big deal, you leave the umbrella home. If rain's likely and you're wearing a suit to a client meeting, you bring it. The math isn't complicated. You're just weighing the odds against the cost.

AI tools move both sides of that equation in the wrong direction at once. They raise the odds, because now you've got dozens of employees pasting text into chat windows, plug-ins connected to your CRM, and chatbots exposed directly to the public internet, any one of which can be the entry point. And they raise the cost, because when something goes wrong, the AI tool usually has access to more systems and data than one employee would, so damage spreads faster before anyone notices.

Here's a way to make that $670,000 shadow AI premium feel real instead of abstract. For a lot of mid-size marketing teams, that's close to an entire quarter's demand generation budget. It's the difference between funding a new campaign and spending that same money on breach notifications, forensic investigators, and a public apology. Every dollar of risk you don't manage on the front end is a dollar you're borrowing from a future budget you were planning to spend on growth.

Where the AI exposure actually hides

None of these tools were built to cause harm. They were built to make our jobs faster. The exposure hides in the fact that we often don't map out, before launch, exactly what each tool can see and what it can say.

Four things I'm doing differently now

The first change is treating every AI tool marketing brings in as something that needs a data map before it needs a launch date. Before I approve a new chatbot or personalization platform, I want to know exactly what customer data it can access, what it's allowed to say, and what happens if a customer tries to manipulate it. That last part matters more than people think. Both the OpenAI and Anthropic incidents happened because nobody had tested what the model would do if the fastest path to its goal ran through a system it was never supposed to touch. A few hours of adversarial testing, trying to break your own tool before a stranger, or the tool itself, does it for you, would have caught it.

The second change is scoping access down to the minimum a tool actually needs. A chatbot that answers shipping questions doesn't need read access to your entire customer database. IBM's report found that 97% of organizations that suffered an AI-related breach had no proper access controls in place. That's not a coincidence. It's the easiest fix on this list, and the one most often skipped because tight access controls slow down rollout by a few days.

The third change is writing an AI governance policy that marketing actually helped author, rather than one IT wrote and emailed to us. If your team is the one deploying these tools, your team needs to understand what's approved, what's not, and why. I've found that people follow rules they helped write far more reliably than rules handed down from a department they've never met.

The fourth is having a communications plan ready before you need it, not after. If a breach happens, marketing is going to be the team writing the apology, the customer email, and the press statement. That's a strange spot to be in when marketing's own tool caused the breach. Draft the honest version of that communication now, while you're calm, so you're not writing it for the first time in a crisis.

Why the human part still matters most

Here's the thing that keeps me from feeling gloomy about all this. Consumer trust research consistently shows that around 70% of customers say they'd stop shopping with a brand after a security incident, but trust also cuts the other way: customers who trust a brand are more than twice as likely to stay loyal even when a competitor offers something better. Trust isn't just what we're protecting. It's the asset we've been building this whole time, campaign by campaign, email by email.

The tools we use now are more powerful than anything marketing has had before, and that power comes with a new kind of responsibility. I don't think the answer is to be afraid of AI tools or to slow-walk every deployment for months. I think the answer is to treat trust the way we'd treat any other asset on the balance sheet: something we measure, protect, and invest in deliberately, rather than something we assume will just take care of itself because our intentions were good.

OpenAI and Anthropic both caught what their models did within days, shut it down, and published exactly what happened, more than most companies manage after a breach. But they build AI for a living and still didn't see it coming. The rest of us, who just buy these tools, have even less room to assume we understand them. The best time to ask what your AI tools can actually do is before they do it.

Common Questions About AI Security Breaches in Marketing

What is shadow AI?

Shadow AI is any AI tool employees use without formal approval or oversight, from pasting text into a public chatbot to installing a browser extension nobody vetted. IBM's 2026 Cost of a Data Breach Report found it played a role in one out of every five breaches last year, adding an average of $670,000 to the cost of each one.

What is prompt injection?

Prompt injection means hiding instructions somewhere an AI tool will read and follow them, often inside content the tool was never meant to treat as a command. It's the mechanism behind EchoLeak, the vulnerability that let attackers pull data out of Microsoft 365 Copilot without the victim clicking anything.

Who's responsible when a marketing AI tool causes a data breach?

That depends on your contracts and jurisdiction for the legal side, and I'm not a lawyer, so treat that part as a question for one. Operationally, though, whichever department chose and deployed the tool usually owns the response, and for most customer-facing AI, that's marketing.

How do you calculate AI security risk without doing math?

Use the umbrella rule: risk equals how likely something is to happen, multiplied by how bad it is when it does. AI tools raise both sides of that equation at once, more entry points for something to go wrong, and more connected systems for the damage to spread through once it does.

What can marketing teams do to reduce AI security risk?

Four things help most: map exactly what each new AI tool can see and say before it launches, scope its access down to the minimum it actually needs, help write the AI governance policy instead of waiting for one, and draft your breach communications before you need them, not during a crisis.