Agentic CNAI: Orchestrating Cloud-Native AI Agents at Scale

TL;DR - AI agents spawn dozens of concurrent tools - retrieval, APIs, code execution, long tasks - and without cloud-native discipline that becomes an unscalable, undebuggable tangle. This is the agentic CNAI stack: Kubernetes as substrate, kagent as runtime, MCP/A2A as protocol, queues for orchestration, OpenTelemetry for observability, and policy for governance. Here's how each layer works - with real CRDs and configs.

CNAI stands for Cloud Native Artificial Intelligence - the set of approaches and patterns for building and deploying AI applications and workloads using the principles of Cloud Native. Agentic CNAI is the most demanding case of that definition: not one model behind a gateway, but many tools, agents, and long-running tasks fanning out at once. New to the term? Start with the history of CNAI and the production reference architecture.

A chatbot answers. An agent acts - it calls tools, browses the web, runs code, queries a vector database, waits on a slow API, and chains all of it into a multi-step task.

That shift from answering to acting is also a shift from one process to a distributed system. And distributed systems are exactly what cloud native was built to run.

Why do AI agents need CNAI at all?#

Run one agent on your laptop and it feels simple. Run a thousand agent sessions in production and the truth arrives fast: an agent is a fan-out engine.

A single request can trigger retrieval, three external API calls, a vector search, a code-execution sandbox, and a long-running job - each with different latency, cost, failure modes, and blast radius. Wire that together with scripts and you get something nobody can scale, secure, or debug at 2am.

Cloud-native discipline answers every one of those problems: isolate each tool as a service, scale each independently, trace every call, and contain the dangerous ones. Agentic AI isn't a reason to abandon CNAI - it's the most demanding CNAI workload yet.

What's actually new in agentic CNAI (2026)?#

The ecosystem moved faster than anyone expected:

• kagent - a CNCF Sandbox project (accepted May 22, 2025) that brings agentic AI to Kubernetes. Agents, tools, and sessions are defined as Kubernetes CRDs, so you get GitOps, RBAC, and admission control for free. It supports any LLM and any framework, with native MCP (Model Context Protocol), Agent-to-Agent (A2A), and OpenAI-compatible endpoints - no lock-in.
• MCP and A2A are becoming the standard way agents talk to tools and to each other. Both now sit under the Agentic AI Foundation (AAIF) - a Linux Foundation umbrella formed December 2025, anchored by MCP (Anthropic), goose (Block), and AGENTS.md (OpenAI).
• The Certified Kubernetes AI Conformance Program (launched November 2025) now validates agentic workloads too - a signal that agents-on-Kubernetes is becoming standardized infrastructure, not a science project.

The agentic CNAI stack#

Name the layers and the chaos becomes an architecture:

┌─────────────────────────────────────────────┐
│  Governance    RBAC · NetworkPolicy · policy │
│  Observability OTel · Prometheus · Grafana   │
│  Orchestration queues / event bus (Kafka…)   │
│  Protocol      MCP · A2A · OpenAI-compatible │
│  Runtime       kagent (agents/tools as CRDs) │
│  Substrate     Kubernetes + GPU scheduling   │
└─────────────────────────────────────────────┘

Read it bottom-up: Kubernetes schedules the compute, kagent runs the agents as first-class objects, MCP/A2A standardize how they communicate, queues decouple the long-running work, observability makes it debuggable, and governance keeps it safe. Every layer is swappable. That is the whole point of doing this the cloud-native way.

How do you define an agent as a Kubernetes object?#

With kagent, an agent and its tools are declarative resources - reviewable in Git, promotable through environments, governed by RBAC.

apiVersion: kagent.dev/v1alpha1
kind: Agent
metadata: { name: ops-troubleshooter, namespace: agents }
spec:
  description: "Diagnoses cloud-native incidents"
  modelRef: { name: gpt-class-model }      # any LLM backend
  systemPrompt: "You are a Kubernetes SRE. Diagnose, propose, never mutate without approval."
  tools:
    - { type: mcp, ref: prometheus-query }
    - { type: mcp, ref: kubectl-readonly }
---
apiVersion: kagent.dev/v1alpha1
kind: Tool
metadata: { name: prometheus-query, namespace: agents }
spec:
  protocol: mcp
  endpoint: http://mcp-prometheus.tools.svc:8080
  timeoutSeconds: 20

Because these are CRDs, kubectl get agents is a real command, a code review gates every prompt change, and admission controllers can block an agent that requests a tool it isn't allowed to use.

How do you keep dangerous tools contained?#

Some tools are safe (read a metric). Some are not (browse the web, execute code). The cloud-native answer is default-deny isolation: a high-risk tool gets its own namespace, its own restricted service account, and a NetworkPolicy that lets it reach only what it must.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: { name: code-exec-lockdown, namespace: sandboxed-tools }
spec:
  podSelector: { matchLabels: { tool: code-exec } }
  policyTypes: [Ingress, Egress]
  ingress:
    - from: [{ namespaceSelector: { matchLabels: { role: agent-runtime } } }]
  egress: []          # no outbound network at all - pure compute sandbox

Pair that with runAsNonRoot, a read-only root filesystem, and CPU/memory limits, and a compromised or hallucinating agent can't reach your databases or the open internet.

How do you observe a multi-agent system?#

You cannot debug what you cannot trace. Every tool call gets a correlation ID threaded from the originating request, distributed tracing via OpenTelemetry, and metrics scraped by Prometheus, visualized in Grafana.

Track four numbers per tool: latency, error rate, token usage, and GPU utilization.

# OpenTelemetry Collector (excerpt) - fan agent spans into your backends
receivers: { otlp: { protocols: { grpc: {}, http: {} } } }
processors: { batch: {} }
exporters:
  prometheus: { endpoint: "0.0.0.0:8889" }
  otlp/tempo:  { endpoint: "tempo.observability.svc:4317", tls: { insecure: true } }
service:
  pipelines:
    traces:  { receivers: [otlp], processors: [batch], exporters: [otlp/tempo] }
    metrics: { receivers: [otlp], processors: [batch], exporters: [prometheus] }

Now a slow session isn't a mystery - it's a trace that shows exactly which tool call ate the eight seconds.

How do you make agentic CNAI affordable?#

Agents are expensive by default because tokens and GPUs are expensive. Cloud-native gives you the levers:

• Independent autoscaling. A cheap retrieval tool and a GPU-bound reasoning tool should scale on different signals. Separate deployments, separate HPAs.
• Caching & batching. Cache identical tool calls and embeddings; batch vector queries. Most agent workloads are surprisingly repetitive.
• Model-class routing. Route easy steps to a small, cheap model and reserve the expensive frontier model for genuinely hard reasoning. This single lever often halves the bill.

For a SaaS, the unit that matters is cost per completed task, not cost per token - and cloud-native autoscaling is what lets that number fall as you optimize, instead of scaling linearly with users.

A reference implementation you can start from#

agentic-cnai/
  infra/            # Terraform: managed K8s, GPU node pools, spot config
  charts/
    kagent/         # runtime install
    tools/          # one Helm subchart per tool (mcp servers)
    observability/  # OTel Collector, Prometheus, Grafana, Tempo
  agents/           # Agent + Tool CRDs, promoted via GitOps
  policies/         # NetworkPolicies, RBAC, admission rules

Bootstrap the substrate with Terraform, install kagent and the observability stack via Helm, define agents and tools as CRDs in agents/, and let GitOps reconcile them. That's a production-grade agentic CNAI platform - not a notebook.

How do multiple agents coordinate without falling over?#

One agent is a fan-out engine; a team of agents is a distributed system with a coordination problem. The cloud-native pattern here - call it agents-as-a-mesh

• is to let agents talk over A2A (Agent-to-Agent) for synchronous handoffs and a queue/event bus for anything long-running, so no agent blocks on another's slow work.

        A2A (sync handoff)
Planner ───────────────► Researcher ──► [MCP tools]
   │                          │
   │ enqueue long task        │ enqueue long task
   ▼                          ▼
┌──────────────────────────────────────┐
│  Queue / event bus (Kafka, NATS…)     │  ← decouples slow work
└──────────────┬───────────────────────┘
               ▼
          Worker agents  (scale on queue depth)
               │
               ▼   spans + correlation ID
        OTel → Prometheus / Grafana

The rule of thumb: synchronous when you need an answer now, asynchronous when you don't. A planner agent can hand a sub-task to a researcher over A2A and wait; but a "re-index the whole knowledge base" task belongs on a queue, with worker agents that autoscale on queue depth. This is the same decoupling the production reference architecture uses for batch training - applied to agent work.

Where do the emerging standards fit?#

Two Linux Foundation efforts anchor this layer, and it's worth knowing which does what:

Building on these - rather than a proprietary agent framework - is what keeps an agentic CNAI platform portable. For the conformance side, see AI-Ready Kubernetes; for the wider AAIF/MCP context, kagent is the concrete CNCF project where these protocols land.

Frequently asked#

What is agentic Cloud Native AI (CNAI)?#

Agentic CNAI is the practice of running AI agents - systems that call tools and take multi-step actions - on cloud-native infrastructure like Kubernetes. It applies containers, orchestration, isolation, observability, and autoscaling to agent workloads so they are scalable, secure, and debuggable in production.

How do you run AI agents on Kubernetes?#

Use a runtime like kagent (a CNCF Sandbox project) that models agents, tools, and sessions as Kubernetes custom resources (CRDs). Agents communicate over standards like MCP and A2A, high-risk tools are isolated with NetworkPolicy and RBAC, and the whole system is observed with OpenTelemetry, Prometheus, and Grafana.

Why do AI agents need cloud native infrastructure?#

Because an agent fans a single request out into many concurrent tool calls with different latency, cost, and risk profiles. Cloud-native infrastructure lets you scale each tool independently, isolate dangerous ones, trace every call, and control cost - which ad-hoc scripts on a single VM cannot do at scale.

What is the difference between MCP and A2A?#

MCP (Model Context Protocol) standardizes how an agent talks to tools and data sources - the "hands." A2A (Agent-to-Agent) standardizes how agents talk to each other - the "handoffs." In an agentic CNAI stack they compose: an agent uses MCP to call a tool and A2A to delegate a sub-task to another agent, with both governed by RBAC and observed with OpenTelemetry.

Written by Aman Mundra - founder of Welzin and the Welzin Open Source Software (WOSS), which contributes upstream to kagent and the cloud-native AI ecosystem. Part 6 of an ongoing CNAI series - Part 1, Part 2, Part 3, Part 4 - History of CNAI, Part 5 - CNAI in Production, Part 7 - AI-Ready Kubernetes.