Open Source Projects · Sub-chapter 8 · 4 min read

kagent

Sub-chapter 8 of Open Source Projects · AI agents as Kubernetes citizens

kagent is the youngest project in this chapter - open-sourced at KubeCon Europe in March 2025 - and the one most aligned with where Welzin operates. The premise: platform and DevOps engineers should be able to define and run AI agents as Kubernetes resources, so the same agents that read logs and query metrics can also take action under policy.

If you're going to contribute to one project from this chapter, this is the one with the highest career-asymmetry: small enough that your name will be on the early contributor list, ambitious enough that the project matters in 2026 and beyond.

TL;DR

What - Open-source, Kubernetes-native framework for running and operating AI agents that automate cloud-native operations.
License - Apache 2.0.
Governance - CNCF Sandbox project (donated March 2025).
Origin - Built at Solo.io, open-sourced at KubeCon EU March 2025.
Velocity - 300+ contributors within a year (from Microsoft, Amazon, Oracle, Amdocs, Orange, and more).

Why it exists

Platform and DevOps engineers are drowning in alerts, dashboards, and runbooks. Chatbots can talk about a cluster. Most "AIops" tools can recommend fixes. Very few systems can actually take action inside a cluster under safe, policy-checked conditions.

kagent treats AI agents as first-class Kubernetes citizens - managed by CRDs, observable via OpenTelemetry, governed by policy. Agents can:

Diagnose CrashLoopBackOff by reading pod logs and Kubernetes events.
Query Prometheus for latency spikes.
Read Istio mesh config and propose a fix.
Open a PR with a remediation, gated on human approval.
Hand off to another agent when out of scope (agent-to-agent delegation).

It's the agent layer of Sub-chapter 5 made cloud-native and shippable.

Architecture - three layers

text

   ┌────────────────────────────────────────┐
   │  Agents (declarative, CRDs)            │
   │  - planner agents                      │
   │  - specialist agents (k8s-agent, …)    │
   └─────────────────┬──────────────────────┘
                     │
   ┌─────────────────▼──────────────────────┐
   │  Runtime / Framework                   │
   │  - Go & Python runtimes                │
   │  - OTel tracing                        │
   │  - Postgres memory + vector store      │
   │  - Agent-to-agent delegation           │
   │  - Human-in-the-loop approval gates    │
   └─────────────────┬──────────────────────┘
                     │
   ┌─────────────────▼──────────────────────┐
   │  Tools (MCP-style)                     │
   │  - pod logs, events                    │
   │  - Prometheus queries                  │
   │  - kubectl actions (gated)             │
   │  - Istio / Argo / Helm operations      │
   └────────────────────────────────────────┘

Tools - MCP-style functions an agent can call. Includes safe (read-only) and dangerous (write/delete) tools.
Agents - autonomous reasoners that plan multi-step tasks. Defined declaratively as Kubernetes CRDs. Built on Google's ADK (Agent Development Kit).
Framework / Runtime - Go + Python runtimes, OTel tracing, Postgres memory, vector store integration, agent-to-agent delegation, human-in-the-loop gates.

A web UI ships out of the box. Agents can be invoked via CRD, CLI, or API.

This is the architecture that Chapter VI's Agentic Workflows sub-chapter sketches - made concrete on Kubernetes.

License and governance

License: Apache 2.0.
Foundation: CNCF Sandbox (March 2025).
Originating maintainer: Solo.io. Now a broader maintainer set with 300+ contributors.

Install and hello world

You need kind, kubectl, and helm.

bash

kind create cluster --name kagent

helm install kagent-crds oci://ghcr.io/kagent-dev/kagent/helm/kagent-crds \
  -n kagent --create-namespace

helm install kagent oci://ghcr.io/kagent-dev/kagent/helm/kagent \
  -n kagent \
  --set providers.openAI.apiKey=$OPENAI_API_KEY

kubectl port-forward -n kagent svc/kagent-ui 8080:8080

Open http://localhost:8080. Talk to the default k8s-agent:

"Why is my pod CrashLoopBackOff in namespace demo?"

It will fetch logs, events, recent rollouts, and propose a fix. With confirmation gates wired up, it can apply the fix.

Try also creating a custom agent as a CRD:

yaml

apiVersion: kagent.dev/v1alpha1
kind: Agent
metadata:
  name: latency-investigator
  namespace: kagent
spec:
  systemPrompt: |
    You investigate p99 latency spikes on the gateway service.
    Use the prometheus tool to find which routes are slow.
    Propose a fix; do not apply it.
  tools:
    - prometheus
    - k8s-read
  modelRef:
    name: gpt-4o

kubectl apply -f and you have a new agent.

How to contribute

Repo: github.com/kagent-dev/kagent
Docs: kagent.dev/docs
Installation guide: kagent.dev/docs/kagent/introduction/installation
CNCF Sandbox application: github.com/cncf/sandbox/issues/360
Solo.io donation post: solo.io/blog/bringing-agentic-ai-to-kubernetes
Discord: bit.ly/kagentdiscord
CNCF Slack: slack.cncf.io, channel #kagent.
Weekly community meetings on Google Calendar (linked from docs).

Where to start:

Build a new tool. Tools are small, well-isolated units. Look for missing integrations (a vector DB, a logging system) and add one.
Improve docs. As with every young project, docs are sparse in places.
Try a real workflow in your own kind cluster. File issues with concrete repro steps. Maintainers respond fast.
Write a sample agent (the examples/ directory takes contributions).

Hands-on Checkpoints

Install kagent locally on kind. Run the default k8s-agent against a deliberately broken pod.
Define your own custom agent (CRD) for a focused task. Apply it. Use it.
Build a new tool - pick something kagent doesn't have yet (e.g., a tool for querying your Welzin staging environment's Grafana).
Read the kagent CNCF Sandbox application (issue #360) cover-to-cover.
Join the kagent Discord. Lurk for a week. Then say hello.