Skip to content

Stay connected

me [at] borghei [dot] me

article9 min read

Shadow AI Isn't a Security Problem. It's a Roadmap Failure.

17% of healthcare staff admit using unapproved AI. Only 29% can name their org's policy. That isn't a discipline problem. Your sanctioned tool lost on speed.

A nurse finishes a discharge summary at 11pm, opens ChatGPT on her personal phone, pastes in the patient note and asks it to tighten the language. No Business Associate Agreement. No audit trail. Four minutes saved. She does it again the next night, and so does most of her floor.

That scene is a statistic now, not an anecdote. Wolters Kluwer Health fielded a survey of more than 500 hospital and health system workers in December 2025 and published it in January. 17% admitted using an unauthorized AI tool at work. 41% said they'd watched a colleague do it.

IBM's 2026 Cost of a Data Breach report, built on 602 organizations across 16 countries and breaches between March 2025 and February 2026, priced the consequence. Shadow AI showed up in 43% of breached organizations, up from 20% the year before. Breaches involving it averaged $5.39 million against $4.63 million, so the shadow tool added about $760,000 to the bill. About one in five of those incidents drew a regulatory fine. 68% of the breached organizations had no AI governance policy at all, and only 40% applied access controls to their AI models and data.

Every few weeks another version of this story runs. The framing is always the same: staff are going rogue, here's the risk, here's the policy you need to write. I've read a dozen of these pieces this year, and I spent the same months building the compliant alternative for clinics across the EU and UAE. The policy was never the part that worked.

The tool was the part that worked.

Nobody breaks the rule for the thrill of it

The instinct when you find shadow AI in your organization is to treat it as a discipline problem. Write the acceptable-use policy. Run the training module. Block the domains at the firewall. I've had that instinct myself, standing in front of a compliance review with a list of tools I'd never approved showing up in the network logs.

Look at why people actually do it. In the same Wolters Kluwer data, 45% of care providers who'd used an unapproved tool said the reason was a faster workflow, and more than half of administrators said the same. Another 27% of providers said the approved tool had worse functionality or didn't exist for what they needed. More than a quarter were simply curious about what the thing could do.

Then there's the finding that should end the policy conversation on its own. Only 29% of providers said they're aware of their organization's main AI policies. Among administrators it's 17%.

Sit with both halves of that. Most staff can't tell you what the policy says, and the ones who can are choosing the faster tool anyway. The rulebook is losing to a free chatbot on two fronts at once: people don't know it exists, and knowing wouldn't change the math at 11pm, when the only question is how long until the note is done.

That's not a compliance failure. That's a product losing to a competitor, and the competitor happens to be a consumer chatbot with no idea it's in a regulated industry.

Once you see it that way, banning the competitor fixes nothing. It removes the fast option and leaves the slow one standing, which is the exact condition that produced the shadow usage in the first place. Write a second policy and you've doubled the volume of a document 71% of your providers have never read.

What I actually had to build

At Reviv, this problem showed up as clinic staff fielding a constant stream of patient questions across WhatsApp and email, at all hours, across markets with different languages and different rules about what a clinic may say to a patient before a clinician has reviewed the case.

Saying "just don't use ChatGPT for this" would have taken about a day to fail. So we built an agentic system instead. It answers routine questions on its own, routes anything resembling an emergency straight to the care team and triggers the right follow-up after treatment. Every exchange is logged, every escalation is traceable and every data flow is scoped to what a GDPR processing agreement actually permits. Alongside it, a compliant internal knowledge bot gives clinic staff the same instant-answer experience they were getting from consumer tools, except the underlying data never leaves a system we control.

The result wasn't a security win we could report to the board and forget about. Operations response time dropped 10%, and therapy adherence went up 33%, because the sanctioned tool had stopped being the slow option. Those are the numbers that predict whether shadow usage comes back. The audit trail is a side effect of building the fast thing properly, not the reason we built it.

You're not competing with a rulebook. You're competing with a chat window.

Three things decide whether staff stay inside the sanctioned tool, and none of them are "did we write the policy clearly enough."

Time to answer. If the approved tool takes longer than opening a personal phone and typing into a free chatbot, it loses every time, whatever the handbook says. Measure this, don't assume it. The comparison staff are making isn't your tool against nothing. It's your tool against the fastest thing they've ever used, and that bar moves every time a model ships.

Friction of access. A tool that's technically approved but sits behind three logins and a VPN might as well not exist at 11pm on a night shift. Every extra step is a vote for the tool already open in another tab. Single sign-on isn't an IT nicety here, it's the difference between a control that works and one that's decorative.

Cost of honesty. If admitting you used an unapproved tool starts a disciplinary conversation, people stop admitting it, and you lose the thing you need most: visibility into where the real gaps are. Run the amnesty window before the rollout, not after the incident. It's how you find out what your staff actually needed before you finish building the wrong thing. The 27% who said no approved product existed are telling you exactly which feature to build, and they'll only keep telling you while it's safe to.

Miss one of these and the policy is decorative. Get all three right and you rarely need the policy, because there's nothing left to route around.

The deadline moved. The nurse didn't.

Here's the part specific to where I operate, and it cuts against the argument you'd expect me to make.

The EU AI Act was supposed to bring high-risk obligations into force on 2 August 2026. It didn't. The Digital Omnibus, agreed politically on 6 May and confirmed by Member States on 13 May, pushed standalone Annex III high-risk systems out to 2 December 2027, and AI embedded in CE-marked medical devices under MDR and IVDR all the way to 2 August 2028. What did land this month is Article 50, the transparency duty to tell people they're talking to an AI, with a grace period on watermarking until 2 December 2026.

So the honest read is that the compliance gun to your head just moved sixteen months down the calendar. If shadow AI were really a compliance problem, that's sixteen months of relief.

It isn't relief, and this is the whole point. The nurse's deadline didn't move. She still has a discharge summary at 11pm tonight. Every month the sanctioned tool doesn't exist is another month of patient data going somewhere with no processing agreement, no log and no retention limit, and none of that becomes retroactively fine because a Brussels timetable slipped.

There's a second trap in the delay. Teams read "2027" and conclude they have time to write the documentation later. You can't. Human oversight, logging, data governance and a case for fitness for purpose are properties of how a system is built, not a layer you bolt on in month fourteen. I've written before about treating regulatory regimes as a product design problem, and this is the cleanest case of it I've seen: build the tool right and the conformity file mostly writes itself, because the evidence was generated as a side effect of running the thing. Build it wrong and December 2027 arrives with a working product you can't document.

The deeper issue is that the AI Act only governs systems you actually deployed. A nurse pasting a note into a consumer chatbot isn't a late-compliant high-risk system. It's invisible to the regime entirely. You can't log it, oversee it or assess it, because from your organization's point of view it never happened. Governance reaches exactly as far as your sanctioned surface does, and not one clinic further.

That reframes who should be losing sleep. Shadow AI isn't a job for the policy team. It's a backlog item, competing for priority against every other feature. And if it keeps losing that fight, the organization has already decided, just not on paper, that a free chatbot is an acceptable part of the clinical workflow.

The nurse was never the problem

She wasn't reckless. With her own behavior she was describing, accurately, where the roadmap fell short. She's also the best product researcher you have, because she tested the alternatives and picked a winner on the only criterion that mattered to her that night.

Ban the tool she found and she'll find a faster one by next week. The underlying need doesn't go away. It goes further underground, with less logging than before, into a tool your security team hasn't heard of yet.

Build the four-minute alternative and she stops needing to leave the building to get her four minutes back. That's the entire intervention. Everything else is paperwork about a problem you chose not to solve.

The measurement that tells you whether you've solved it isn't the count of policy acknowledgements collected. It's the same question I'd ask of any deployed system, the one I keep coming back to in the piece on verification layers: how does this behave when it's wrong, and who finds out. A tool nobody uses answers that question badly, because when it's wrong you never learn, and when it's right nobody was there.

If you're working out what a sanctioned alternative needs to look like for your own team, get in touch.


This is the tenth in a series on what I see in the market and hear from operators across the companies I've worked with. Next up: what a headless AI agent, one with no interface at all, does to the trust the interface used to carry.