Skip to Content

The Loop Asia

Insights

Analysis and commentary from The Loop Asia — AI, APIs, and technology leadership across Asia-Pacific.

Thanks for subscribing!

No spam. Unsubscribe any time.

Why your API gateway can't just be extended to handle AI

F5 built AI Gateway on a different architecture than its API gateway — not as a feature gap, but because a scripting model built for discrete events doesn't map onto a continuous flow of language.
June 22, 2025 by
Why your API gateway can't just be extended to handle AI
Jon Scheele

This deep dive expands on the Loop Asia conversation with Buu Lam, Director of Community Evangelism at F5, recorded at F5 AppWorld in Singapore.


Every technical leader deploying LLMs into production eventually asks the same question: we already have an API gateway, why not just add rules to it for AI traffic? F5's Buu Lam gets asked this often enough that he had a ready answer at F5 AppWorld — and it's about a category error, not a product feature.

The instinct to extend is the wrong instinct

F5's BigIP does its programmability through TCL scripting: event-based, a request happens, a rule fires. That's served gateways well for two decades — APIs are well-behaved by design, a fixed protocol, "good" traffic defined with confidence.

F5 built AI Gateway as a separate product on a separate architecture — processors compiled into WebAssembly, not TCL scripts. That's a tell: a company that's sold API gateways for twenty years concluded that extending the old model wasn't viable. If you're securing your organisation's LLM traffic, take that signal seriously before asking your platform team to bolt AI rules onto the gateway you already run. The architecture underneath matters more than the label on top.

Why WebAssembly, specifically

Buu's phrase: "you can just write whatever you want." Event-based scripting is good at "if X, do Y." It's a poor fit for a flow that has to reason about an entire request-response cycle — read the prompt, understand what's being asked semantically, let it through, then evaluate the response before it goes back out, often against different checks than the ones run on the way in. That's a small application with state and judgment built in, not a single triggered rule. WebAssembly gives you a portable, sandboxed place to run it, independent of language. It's the right primitive for a processor that has to think, not a rule that has to fire.

Ask what's underneath any control point you build or buy for LLM traffic. "AI security" bolted onto an existing WAF rule engine is a signature-matching problem applied to a semantics problem. It'll catch attacks that look like last year's attacks. It won't catch the one your red team hasn't thought of — which, per Buu's account of a prompt injection game where "simple adjectives" bypassed system prompts, is most of them.

What this means for the context layer

There's a second-order question here that goes beyond the show: if AI Gateway decides what data an agent's request and response can touch, it sits next to a problem API teams are quietly wrestling with — how do you expose enterprise data to an agent safely, without rebuilding your API estate?

Treat it as a layering problem, not a rip-and-replace one. Existing system and process APIs mostly don't need to change. What changes is the top layer — the interface built for a human-facing app now needs a version built for an agent's context needs, sitting behind a gateway that can inspect both what's asked for and what's about to come back. Get that wrong, and you either lock the agent out of the data it needs, or hand it a wide-open door and hope the security processor polices it after the fact.

This is where a technical champion benefits from a second set of eyes that isn't selling the gateway — someone who's sat on both sides of this kind of decision, and can separate what the vendor wants you to buy from what your actual risk requires. It's a governance conversation, not a procurement one — worth having before the AI mandate turns into a live incident.


Extending your API gateway to cover AI traffic instead of questioning the architecture?

Separating what a vendor wants you to buy from what your actual risk requires — before the AI mandate turns into a live incident — is exactly where I help.

See how I can help →