How Cloudflare helps you democratize AI access for your teams, developers, and applications — without losing visibility, security, or your budget.

TL;DR — In part one of this series, I covered the AI traffic hitting your infrastructure from the outside — crawlers, training bots, the broken value exchange between content creators and AI platforms. This post is the mirror image: the AI your organization reaches out to, every day, across every team — and why that consumption is far less controlled than most organizations realize. Shadow AI usage, fragmented provider accounts, API keys scattered across codebases, and no centralized data governance are the norm, not the exception. The real challenge isn’t getting your teams to use AI — they already are. It’s building the infrastructure layer that makes that adoption sustainable, secure, and cost-controlled. Cloudflare’s AI Gateway, combined with the broader Cloudflare developer platform, is that layer. Not because of what it says on a product page, but because of what I see in the field.

In my previous post , I wrote about the AI traffic hitting your infrastructure from the outside — crawlers harvesting your content, training bots eroding your cache performance, and the tools Cloudflare is building to protect and monetize that exposure. The data that opened that post is worth holding onto here: 32% of all traffic across Cloudflare’s network is now automated, driven in large part by AI systems crawling the web at scale.

That number describes AI as something happening to you. But the same explosion of AI activity is also happening inside your organization — your teams, your developers, your applications, all reaching out to AI providers every day. Same trend. Two sides of the same wall.

This post is about that inward-facing side: the AI your organization reaches out to, and why that consumption is far less controlled than most organizations realize.

The Conversation I Keep Having

It usually starts with: “We want to build an AI strategy.”

A few questions in, the real picture comes out. I was in a conversation with a financial services company recently — mid-sized, serious security posture, the kind of team that has a process for everything. I asked their CISO how many active AI API keys existed across their environment. He looked at his security architect. The security architect looked at the lead developer. Nobody knew. Not even approximately. They had a formal AI policy. They had no idea what was actually happening underneath it.

That’s not an outlier. That’s the majority. One team expensing OpenAI on personal cards. Another team sharing a corporate API key over Slack. Developers hardcoding credentials directly into applications. Marketing using a third-party AI tool that bypassed procurement entirely. And somewhere in all of this, sensitive data — customer records, internal financials, source code — flowing into external models with no logging, no controls, no way to know what left and when.

This is not dysfunction. It is the entirely predictable result of genuinely useful technology arriving faster than organizations can govern it. People don’t wait for IT approval when a tool saves them two hours a day.

The uncomfortable truth: by the time most organizations decide they need an AI strategy, they already have an AI reality. The question is no longer how to start — it’s how to get control of something that started without them.

Shadow AI Is the New Shadow IT — And It Moves Faster

A decade ago, the same pattern played out with SaaS. Employees adopted Dropbox, Google Docs, and Slack before IT sanctioned them. It took years for the security and compliance implications to fully surface.

Shadow AI is the same problem, but the risk profile is different in a way that matters. When someone used an unsanctioned SaaS tool, the concern was mostly contractual — data residency, liability, vendor terms. When someone pastes a customer contract or internal financial projection into a public AI model to get a quick summary, the exposure is immediate and irreversible. There is no “retrieve the data” option once it has been processed externally. And unlike a file in someone’s personal Dropbox, you may never even know it happened.

I’ll be direct: I think most organizations are closer to an AI-related security incident than they realize. Not because AI is inherently dangerous, but because the gap between “we have a policy” and “we have technical controls that enforce that policy” is enormous right now — and that gap is where incidents happen. A policy document doesn’t stop a developer from pasting production credentials into ChatGPT to debug a problem at 11pm. Infrastructure does.

There’s another dimension here that connects back to something I described in part one. AI crawlers break CDN cache architectures because their access patterns are fundamentally unpredictable — bursty, broad, non-sequential, nothing like the human traffic those systems were designed for. Internal AI consumption has exactly the same property. Developers don’t call AI models in smooth, forecastable flows. They experiment in spikes, launch features that hit a model thousands of times overnight, chain multiple calls in ways that compound costs non-linearly. Static budget allocations and acceptable-use policies were not designed for this. You need infrastructure that enforces rules dynamically, at the point of execution.

Which brings me to the build-vs-buy question, because it comes up constantly. The first instinct when organizations decide to close the governance gap is to build something — proxy the AI traffic internally, write middleware that logs requests, stand up a key vault, add DLP scanning. I’ve seen teams spend three to six months doing this reasonably well. And then spend ongoing time maintaining it as provider APIs change, new models appear, and usage patterns shift in ways nobody anticipated.

That’s plumbing. It doesn’t appear in any roadmap. It consumes senior engineering time that could be building product. The honest question is: what is the actual value of owning this infrastructure yourself?

What Cloudflare Built — And Why It’s Different

Cloudflare’s AI Gateway is the answer to a specific question: what does the control plane for enterprise AI consumption look like if you build it the way you’d build internet infrastructure — once, at scale, for everyone?

The starting point is a single endpoint connecting to more than 350 models across the major providers: Anthropic, OpenAI, Google, Groq, xAI, and others. Your applications and tools point at one URL. Behind it, AI Gateway handles provider connectivity, request translation, and routing. Switching models or adding a fallback becomes configuration, not a code change.

But the endpoint is just the entry point. What matters is what sits around it.

The first thing AI Gateway gives you — and I want to be clear that this alone is often worth the conversation — is full observability over your AI consumption. Every request and response, logged. Usage broken down by application, model, provider. Real-time cost tracking. Before you can control costs, enforce security policies, or make intelligent routing decisions, you need to actually know what is happening. Who is calling what model, how often, with what payloads, at what latency.

This sounds basic. It is almost universally absent.

I’ve shown this view to customers who genuinely did not know three of their applications were calling AI models in production. The reaction when that dashboard loads for the first time is consistent — it’s somewhere between relief and alarm. Relief because there’s finally a picture. Alarm because of what the picture shows.

Once you have visibility, the billing picture usually produces its own strong reaction. Managing separate credit accounts across providers — each with its own dashboard, billing cycle, and top-up process — is one of those operational burdens that feels manageable until you’re doing it across five providers with a dozen teams. AI Gateway consolidates everything into a single Cloudflare account balance and one monthly invoice. You load credits once. Your finance team sees one line. Your procurement team has one contract.

More importantly: you cannot set organizational AI spending limits when the spending is scattered across systems with no common view. Unified billing is not convenience — it’s the prerequisite for cost governance.

The Zero Trust Approach to API Keys

Here is a pattern that should concern every security team: the API key that lives in a .env file, committed to a repository, shared with the whole engineering team, never rotated, originally created by a developer who left the company eight months ago.

This is not hypothetical. It is the default state of API key management in most development organizations — not because developers are careless, but because the tooling to do better has historically been friction-heavy enough that corners get cut under delivery pressure.

AI Gateway’s integration with Cloudflare Secrets Store applies Zero Trust principles directly to this problem. Provider keys are stored centrally, encrypted, and referenced by name rather than value in code. Developers can use a key without ever seeing its actual content. Role-based access controls separate those who manage keys — typically a security or platform team — from those who consume them. Every key operation is audit-logged.

When a developer leaves, you rotate one key in one place and it propagates automatically to every application referencing it. When a key is suspected of compromise, you have an audit trail that tells you exactly where and when it was used.

This is Zero Trust applied to your AI supply chain — the same principle that governs access to your internal applications: verify identity, enforce least privilege, log everything — applied to the credentials powering your AI features.

There is a deliberate parallel here to what I described in part one. Web Bot Auth solves the identity problem for external AI agents by replacing spoofable headers with cryptographic proof — you can only trust a bot’s behavior if you can trust who it claims to be. The same logic applies internally: you can only govern your organization’s AI consumption if you know, with certainty, which application used which key, when, and under whose authority. Secrets Store and audit logging are how that certainty is established on the outbound side.

The Data You Don’t Want Leaving the Building

The data leakage question is the one that makes security and compliance teams most anxious — and it should. An employee asks AI to summarize a customer contract. A developer debugs code that contains production credentials using a public model. A support agent pastes a ticket with PII into an AI tool to draft a response faster. None of these feel like security incidents in the moment. All of them potentially are.

In each case, the failure mode is invisible. No error message. No alert. No indication that anything sensitive just left the building.

AI Gateway’s firewall with Data Loss Prevention scanning intercepts this at the infrastructure layer — scanning requests against DLP profiles before they reach any model, and scanning responses on the way back. Sensitive categories can be flagged or blocked. Every match is logged with the profile that triggered and the action taken.

I want to be honest about the limits here: DLP is not magic. Custom patterns need to be configured thoughtfully, and there will always be edge cases that slip through. But there is a significant difference between “we have no visibility into what data is going to AI models” and “we have a scanning layer that catches the most common and most serious exposure patterns.” The latter is a defensible security posture. The former isn’t.

Routing Intelligence — Cost Control Without Limiting Access

One of the most practically valuable capabilities in AI Gateway is Dynamic Routes: the ability to define traffic routing logic based on request attributes, user segments, spending thresholds, or traffic splits between models.

In practice, this answers the question I hear most often after organizations get over their initial concerns about cost: “How do we give everyone access to AI without the spending scaling out of control?”

The answer isn’t to restrict access — it’s to route intelligently. Internal productivity tools and experimentation get routed to capable but cost-efficient models. Customer-facing features requiring the highest output quality get access to premium models. Free-tier users in your product get rate-limited or directed to a lighter model. High-volume batch workloads get throttled when they don’t need to be real-time.

None of this requires engineering once it’s configured. It’s policy, not code. And it means you can say yes to broad AI adoption — across teams, use cases, and user tiers — while maintaining real control over where the budget actually flows.

You can also use Dynamic Routes to run model comparisons in production: split 10% of traffic to a newer model, compare output quality and cost, and promote or roll back based on real data rather than benchmarks. This kind of experimentation infrastructure would take meaningful engineering effort to build internally. Here it is a configuration.

Resilience — Because Production Can’t Depend on a Single Provider

The last piece is the one that only becomes urgent after something goes wrong. AI providers experience outages. Model performance degrades after updates. Latency spikes under load. If your production application has a hard dependency on one model endpoint, you find out how important fallbacks are when a customer-facing feature stops responding at 2pm on a busy Tuesday.

Dynamic Routes handles fallback logic as a native capability. Define your primary model, your fallback, and the conditions for switching. Cloudflare manages the execution at the edge. Your application continues to function. The provider disruption gets absorbed at the infrastructure layer before it reaches your users.

For anyone building AI-powered features in production, this transforms reliability from a custom engineering project into a configuration decision.

What Democratization Actually Requires

“Democratizing AI” gets used loosely. Usually it means giving everyone access — a chatbot here, an API key there. That’s the easy part, and it’s not enough.

Access without governance doesn’t last. The first security incident, unexpected invoice, or compliance finding triggers a blanket ban — and you’re back where you started, except now you’ve burned organizational trust in AI tools on top of it. I’ve seen this play out. The technology gets banned not because it was wrong for the organization, but because the infrastructure to run it safely wasn’t in place when it mattered.

Real democratization means broad access and the controls that make that access sustainable. Not restricting who can use AI, but governing how it’s used — automatically, at the infrastructure layer, regardless of which team or tool or model is involved.

AI Gateway, combined with Workers AI for running models natively on Cloudflare’s network, is that infrastructure layer. And Dynamic Workers closes the loop in a way worth naming explicitly: in part one I described it as the secure execution environment for AI agents acting on the open web. The same technology serves your internal use case — when your developers want AI to generate and run code inside your own applications, Dynamic Workers is what makes that safe. One product, both directions. That’s what a platform looks like.

The Bottom Line

The AI consumption problem is not a future problem. It exists right now, in most organizations, usually just below the surface. The CISO who doesn’t know how many API keys are active. The developer who hardcoded credentials three months ago and has since moved to another team. The finance team about to be surprised by an invoice.

Cloudflare’s answer is infrastructure that solves this at the layer where it can actually be enforced — the network — rather than relying on policy documents and developer discipline. It works whether or not your teams follow best practices, whether or not they remember to check the provider dashboard, whether or not the model they’re using today still exists next quarter.

In part one, I covered the AI traffic coming at your infrastructure. Here I’ve covered the AI your organization reaches out to. Together they represent the two sides of the same challenge — and the same platform answering both.

The infrastructure exists on both sides. The question is whether you’re using it.

I’m a Solutions Engineer at Cloudflare. This post reflects my own field perspective and synthesis of publicly available Cloudflare product information — it does not represent an official Cloudflare position.

← Part 1: The Internet Was Built for Humans. AI Didn’t Get the Memo.

References: AI Gateway — August 2025 · Workers AI · Cloudflare Secrets Store · Dynamic Routes · Dynamic Workers