For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.
Install kagent
Install kagent 1.0 and Agent Substrate on a Kubernetes cluster.
kagent 1.0 runs every agent on Agent Substrate, so an installation sets up two systems in the same cluster. Agent Substrate provides the sandboxed compute that agents run on. kagent provides the Harness, AgentTemplate, and Agent API that you author against.
Note
These steps install kagent 1.0 fresh. kagent 1.0 does not support an in-place upgrade from the 0.10.x version line, and installing its custom resource definitions (CRDs) replaces the ones that a 0.10.x installation uses. To move an existing installation, start with Upgrade from 0.x.
Before you begin
Prepare a Kubernetes cluster at 1.37 or later. Agent Substrate depends on the
PodCertificateRequestandClusterTrustBundleAPIs. Both are generally available (GA) in 1.37 and served atcertificates.k8s.io/v1by default.For local testing and development, create a kind cluster at Kubernetes 1.37 or later. Use kind v0.32.0 or later. The cluster does not need extra API configuration.
kind create cluster --name kagent --image kindest/node:v1.37.0Set your model provider API key. The examples in this guide use OpenAI. For other providers, see Configure model providers.
export OPENAI_API_KEY="your-api-key-here"Choose an installation method.
- CLI: For an evaluation, use the
kagent installcommand of the kagent CLI. The CLI also deploys a development PostgreSQL database. Follow the steps in Install with the CLI. - Helm: For a production installation that Helm manages from the start, install each Helm chart yourself, against a PostgreSQL database that you provide. Follow the steps in Install with Helm.
- CLI: For an evaluation, use the
Install with the CLI
The kagent CLI installs the CRDs, Agent Substrate, the Agent Substrate PodCertificate controller, a development PostgreSQL database, and kagent in the correct order. The CLI also creates the Agent Substrate identity material and database connection Secrets that the releases need.
Install the following CLI tools. The kagent CLI runs
helmto install each chart.Save a Helm values file that enables gRPC server reflection, so that guides that use
grpcurlcan discover the controller API.cat > kagent-values.yaml <<EOF config: server: grpcReflection: true EOFIf your cluster uses a DNS domain other than
cluster.local, also setglobal.clusterDomainin this file. The chart builds the URLs of the controller and the built-inkagent-tool-serverMCP server from this domain, so agents cannot reach either service when the value is wrong.global: clusterDomain: corp.exampleSave a CLI configuration file that points the installer at those values.
kagent installdoes not take Helm flags of its own, so chart values reach it through this file. A relative path invaluesFilesresolves against the configuration file’s own directory.cat > kagent-config.yaml <<EOF install: kagent: valuesFiles: [kagent-values.yaml] EOFField Description install.kagent.valuesFilesThe Helm values files that the installer passes to the kagent chart, in order. The last file wins. install.bundledPostgresImageOptional. The image of the development PostgreSQL database that the installer deploys, for both the database container and its TLS reloader. Defaults to postgres:18-alpine, pinned by digest. To pull the image from a mirror, such as in an air-gapped cluster, set the full image reference, such asmirror.example.com/library/postgres:18-alpine.Install kagent.
kagent --config kagent-config.yaml install --namespace kagentThe command reads your provider API key, installs Agent Substrate into
ate-system, and deploys PostgreSQL askagent-postgresqlin the kagent namespace. It then connects kagent and Agent Substrate to that database with Pod Certificates. The development database runs the stockpostgresimage. That image does not include thepgvectorextension that long-term memory needs.Wait for the controller to roll out.
kubectl rollout status deployment/kagent-controller -n kagent --timeout=300sExample output:
deployment "kagent-controller" successfully rolled outFor production database requirements, see Operational considerations.
Install with Helm
Install the CRDs, the identity material, the database, Agent Substrate, and kagent. Follow these sections in that order, because each section creates objects that a later section needs. Note that as opposed to a CLI installation, a Helm installation connects to a PostgreSQL database that you provide. The charts do not deploy a database.
Install the following CLI tools. The Helm installation does not need the kagent CLI.
helm, the Kubernetes package manager. Use Helm 3.kubectl, the Kubernetes command line tool.jq, to read the cluster’s token issuer and the root certificate out of the generated certificate authority (CA) pool.openssl, to convert the root certificate to Privacy Enhanced Mail (PEM) format.psqlversion 15 or later, to run the database setup scripts.kubectl-ate, the Agent Substrate command line tool, published as akubectlplugin with each Agent Substrate release.curl -fsSL -o kubectl-ate \ "https://github.com/kagent-dev/substrate/releases/download/v1.0.0-rc1/kubectl-ate-$(uname -s | tr '[:upper:]' '[:lower:]')-$(uname -m | sed 's/x86_64/amd64/; s/aarch64/arm64/')" chmod +x kubectl-ate sudo mv kubectl-ate /usr/local/bin/ kubectl ate --help
Prepare a PostgreSQL instance at version 15 or later, and create an empty database for kagent, such as
kagent. kagent and Agent Substrate share the database, each in its own schema. The instance must accept connections from pods in thekagentandate-systemnamespaces. Save the connection string of a database administrator.export ADMIN_CONNECTION_STRING="postgresql://<admin-user>:<admin-password>@<host>:5432/kagent?sslmode=verify-full"
Install the CRDs
The kagent and Agent Substrate custom resource definitions (CRDs) install from their own Helm releases, separate from the charts that run the controllers.
Install the kagent CRDs.
helm upgrade --install kagent-crds \ oci://ghcr.io/kagent-dev/kagent/helm/kagent-crds \ --version 1.0.0-rc1 \ --namespace kagent --create-namespace --waitInstall the Agent Substrate CRDs.
helm upgrade --install substrate-crds \ oci://ghcr.io/kagent-dev/substrate/helm/substrate-crds \ --version 1.0.0-rc1 \ --namespace ate-system --create-namespace --wait
Set up certificates and identity
Agent Substrate authenticates its components with mutual Transport Layer Security (mTLS). You create the CA and JSON Web Token (JWT) pools that sign and verify those identities, and then install the controller that issues certificates from them. No Helm chart creates the pools. The kubectl-ate plugin writes each pool to a Kubernetes Secret, and the components that mount those Secrets stay in ContainerCreating until the pools exist.
Create the CA and JWT pools. Each command writes one pool Secret that Agent Substrate reads when it signs or verifies an identity.
make-ca-poolandmake-jwt-poolfail when the pool Secret already exists. To rotate a pool, delete its Secret and run the command again.Create the namespace that holds the signer pools, then the CA pools that sign service DNS certificates, database login certificates, and pod identity certificates.
kubectl create namespace podcertificate-controller-system --dry-run=client -o yaml | kubectl apply -f - for pool in service-dns-ca-pool postgres-ca-pool pod-identity-ca-pool; do kubectl ate admin make-ca-pool --ca-id=1 --key-type=ED25519 \ --name=${pool} --secret-namespace=podcertificate-controller-system doneCreate the actor identity pools that Agent Substrate uses to issue and verify actor credentials.
kubectl ate admin make-jwt-pool --key-id=1 \ --name=actor-id-jwt-pool --secret-namespace=ate-system kubectl ate admin make-ca-pool --ca-id=1 --key-type=ED25519 \ --name=actor-id-ca-pool --secret-namespace=ate-systemCreate the egress CA pool. Agent Substrate terminates outbound TLS at its egress gateway to inject model provider and Model Context Protocol (MCP) credentials. Without this pool, every outbound HTTPS call from an agent fails.
kubectl ate admin make-ca-pool --ca-id=1 --key-type=ECDSAP256 \ --name=egress-mitm-ca-pool --secret-namespace=ate-system
Configure the authentication that the Agent Substrate API server performs. The
kubernetesprovider accepts Kubernetes ServiceAccount tokens that are issued for the Agent Substrate API server audience.Extract the actor identity root certificate, and store it in the Secret that the Agent Substrate API server reads.
actor_id_ca_root="$(kubectl get secret actor-id-ca-pool -n ate-system \ -o jsonpath='{.data.pool}' | base64 --decode \ | jq -r '.CAs[0].RootCertificateDER' | base64 --decode \ | openssl x509 -inform der -outform pem)" kubectl create secret generic actor-id-ca-certs -n ate-system \ --from-literal=ca.crt="${actor_id_ca_root}"Read the issuer from the cluster, and note which kind it is. Kubernetes distributions advertise different issuers, and the
ate-api-authenticationConfigMap takes a different form for each kind of issuer. An issuer that does not match the cluster’s own is accepted when you create the ConfigMap, and later surfaces astoken issuer ... not trustedon everykubectl atecall.k8s_issuer="$(kubectl get --raw /.well-known/openid-configuration | jq -r .issuer)" echo "${k8s_issuer}"- An in-cluster issuer, such as
https://kubernetes.default.svc.cluster.localon a kind cluster, does not publish a discovery document that a public client can reach. - An external issuer, such as the
https://container.googleapis.com/...issuer that a GKE cluster advertises, is reachable on the public internet.
- An in-cluster issuer, such as
Create the authentication configuration for your issuer.
- For an in-cluster issuer,
certificateAuthorityFileanddiscoveryTokenFileconfigure the API server to use its own projected ServiceAccount certificate authority and token.kubectl create configmap ate-api-authentication -n ate-system \ --from-literal=authentication.yaml="jwtProviders: - name: kubernetes issuer: ${k8s_issuer} audiences: [api.ate-system.svc] certificateAuthorityFile: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt discoveryTokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token " - For an external issuer, omit both lines.
certificateAuthorityFilereplaces the client’s root certificate authorities with the cluster’s own, so the TLS handshake with an external issuer fails when the field is set.kubectl create configmap ate-api-authentication -n ate-system \ --from-literal=authentication.yaml="jwtProviders: - name: kubernetes issuer: ${k8s_issuer} audiences: [api.ate-system.svc] "
- For an in-cluster issuer,
Install the Agent Substrate PodCertificate controller, which signs the certificates that components present to each other.
helm upgrade --install substrate-podcert \ oci://ghcr.io/kagent-dev/substrate/helm/substrate-podcert \ --version 1.0.0-rc1 \ --namespace podcertificate-controller-system --create-namespace \ --wait --timeout 5m
Prepare the PostgreSQL database
kagent and Agent Substrate each connect with an owner login, which applies migrations, and a read/write login, which serves runtime queries. Each project ships a psql script that creates its roles, logins, and schema. Each script reads the passwords from environment variables and must run in one transaction.
Create the kagent roles and the
kagentschema.curl -sSLo kagent-setup.sql https://raw.githubusercontent.com/kagent-dev/kagent/refs/tags/v1.0.0-rc1/go/core/pkg/postgressetup/setup.sql export KAGENT_OWNER_PASSWORD=<kagent-owner-password> KAGENT_READWRITE_PASSWORD=<kagent-readwrite-password> psql "$ADMIN_CONNECTION_STRING" --no-psqlrc --set=ON_ERROR_STOP=1 --single-transaction \ --set=kagent_schema=kagent \ --set=kagent_owner_role=kagent_owner \ --set=kagent_owner_user=kagent_owner_user \ --set=kagent_readwrite_role=kagent_readwrite \ --set=kagent_readwrite_user=kagent_readwrite_user \ --command='\getenv kagent_owner_password KAGENT_OWNER_PASSWORD' \ --command='\getenv kagent_readwrite_password KAGENT_READWRITE_PASSWORD' \ --file=kagent-setup.sqlCreate the Agent Substrate roles and the
substrateschema in the same database.curl -sSLo substrate-setup.sql https://raw.githubusercontent.com/kagent-dev/substrate/refs/tags/v1.0.0-rc1/pkg/postgressetup/setup.sql export SUBSTRATE_OWNER_PASSWORD=<substrate-owner-password> SUBSTRATE_READWRITE_PASSWORD=<substrate-readwrite-password> psql "$ADMIN_CONNECTION_STRING" --no-psqlrc --set=ON_ERROR_STOP=1 --single-transaction \ --set=substrate_schema=substrate \ --set=substrate_owner_role=substrate_owner \ --set=substrate_owner_user=substrate_owner_user \ --set=substrate_readwrite_role=substrate_readwrite \ --set=substrate_readwrite_user=substrate_readwrite_user \ --command='\getenv substrate_owner_password SUBSTRATE_OWNER_PASSWORD' \ --command='\getenv substrate_readwrite_password SUBSTRATE_READWRITE_PASSWORD' \ --file=substrate-setup.sqlStore the connection string of each login in a Secret. Both charts read these Secret names and keys, and the role and schema names that the setup scripts create, by default, so the releases do not need database values. Leave role settings out of the connection strings, because each component assumes its role after it connects.
kubectl create secret generic kagent-postgres-owner -n kagent \ --from-literal=ownerConnectionString="postgresql://kagent_owner_user:${KAGENT_OWNER_PASSWORD}@<host>:5432/kagent?sslmode=verify-full" kubectl create secret generic kagent-postgres-readwrite -n kagent \ --from-literal=readWriteConnectionString="postgresql://kagent_readwrite_user:${KAGENT_READWRITE_PASSWORD}@<host>:5432/kagent?sslmode=verify-full" kubectl create secret generic substrate-postgres-owner -n ate-system \ --from-literal=ownerConnectionString="postgresql://substrate_owner_user:${SUBSTRATE_OWNER_PASSWORD}@<host>:5432/kagent?sslmode=verify-full" kubectl create secret generic substrate-postgres-readwrite -n ate-system \ --from-literal=readWriteConnectionString="postgresql://substrate_readwrite_user:${SUBSTRATE_READWRITE_PASSWORD}@<host>:5432/kagent?sslmode=verify-full"For the other database settings, such as the connection pool and the
pgvectorextension that long-term memory needs, see Use an external PostgreSQL instance.
Install Agent Substrate
The substrate release installs the Agent Substrate control plane and data plane. The release name must be substrate, and the namespace must be ate-system.
Install Agent Substrate. The command sets the following values.
previewturns on thePerTemplateSnapshotLocationgate, so that Agent Substrate stores the snapshots of each agent at the location that its Harness names.ateApi.authorization.bootstrapOwnersgrants owner access to the Agent Substrate API to the kagent controller and to theate-clientServiceAccount. The API server authorizes every call, and without the controller entry every Agent reportsRuntimePreparationFailed (PermissionDenied).atenetRouter.ingressAllowedSPIFFEIDslets the kagent controller send traffic to agents through the Agent Substrate router.credentialProvider.namespacePolicieslets thekagentatespace read Secrets in thekagentnamespace, where the agents and the Secrets that they read both live. The provider denies every request until a policy allows it.
helm upgrade --install substrate \ oci://ghcr.io/kagent-dev/substrate/helm/substrate \ --version 1.0.0-rc1 \ --namespace ate-system \ --wait --timeout 10m \ -f - <<EOF preview: - PerTemplateSnapshotLocation ateApi: authorization: bootstrapOwners: - spiffe://cluster.local/ns/kagent/sa/kagent-controller - system:serviceaccount:ate-system:ate-client atenetRouter: ingressAllowedSPIFFEIDs: - spiffe://cluster.local/ns/kagent/sa/kagent-controller credentialProvider: namespacePolicies: - atespace: kagent allowedNamespaces: [kagent] EOFVerify that Agent Substrate is running.
kubectl get pods -n ate-systemIn the output, verify that every pod reports
Running. Therustfs-bucket-initpod belongs to a Job and reportsCompleted.NAME READY STATUS RESTARTS AGE ate-api-server-d8c6d5b57-lzcbg 2/2 Running 0 74s ate-api-server-d8c6d5b57-twzg8 2/2 Running 0 74s ate-controller-64ff6c8445-mjz5x 1/1 Running 0 75s ate-idp-server-66fbf4d6c5-9hgmj 1/1 Running 0 75s ate-idp-server-66fbf4d6c5-vhrjl 1/1 Running 0 75s atelet-m6pkj 2/2 Running 0 75s atenet-egress-787d98dc77-s92ws 1/1 Running 0 74s atenet-router-556c5ff957-9f4jt 1/1 Running 0 75s k8s-credential-provider-5c8b8fb5b6-5gb6f 1/1 Running 0 75s rustfs-545745bdf6-kc8g6 1/1 Running 0 75s rustfs-bucket-init-v6sf4 0/1 Completed 0 75s
Install kagent with Helm
Install the kagent control plane into the kagent namespace. The controller applies its database migrations when it starts.
Store your model provider API key in the Secret that the chart reads by default. You create this Secret, so Helm does not own it, and you rotate the key by updating the Secret.
kubectl create secret generic kagent-openai -n kagent \ --from-literal=OPENAI_API_KEY="${OPENAI_API_KEY}"Label the
kagentnamespace so that the kagent controller can reach the Agent Substrate API server. Agent Substrate installs NetworkPolicies that deny ingress to theate-systemnamespace by default. The policy of the API server admits only the namespaces that carry theate.dev/api-access=truelabel. Without the label, the controller fails to connect at startup withcontext deadline exceeded.kubectl label namespace kagent ate.dev/api-access=trueInstall kagent. The command sets the following values.
controller.substrateClientCertificate.enabledmounts the certificates that the controller presents to the Agent Substrate API server, and the trust bundle that verifies the server.substrateWorkerPoolcreates thekagent-defaultWorkerPool.workerImagetags the Worker image with the Agent Substrate version, because the Agent Substrate controller starts each Worker with the flags of its own version.
helm upgrade --install kagent \ oci://ghcr.io/kagent-dev/kagent/helm/kagent \ --version 1.0.0-rc1 \ --namespace kagent --timeout 12m \ -f - <<EOF controller: substrateClientCertificate: enabled: true substrateWorkerPool: create: true workerImage: "ghcr.io/kagent-dev/substrate/ateworker-gvisor:v1.0.0-rc1" EOFWait for the controller to roll out.
kubectl rollout status deployment/kagent-controller -n kagent --timeout=300sExample output:
deployment "kagent-controller" successfully rolled out
Verify the installation
Before you create an agent, confirm that kagent and Agent Substrate are running, and find the address that clients use to reach the kagent API.
Confirm that the kagent pods are running.
kubectl get pods -n kagentIn the output, verify that every pod reports
Running. A Helm installation uses your own database, so it does not list thekagent-postgresqlpod.Example output for a CLI installation:
NAME READY STATUS RESTARTS AGE kagent-controller-56c67b6cd7-hqj5z 1/1 Running 3 2m kagent-default-774d4496bd-sfx4b 1/1 Running 0 2m kagent-postgresql-856b475f59-n258n 2/2 Running 0 2m kagent-tools-54959b659c-lfkrz 1/1 Running 0 2m kagent-ui-5fdb6fd85c-vzb2z 1/1 Running 0 2mConfirm that the WorkerPool reports a ready replica.
kubectl get workerpools -n kagentExample output:
NAME DESIRED REPLICAS READY SANDBOXCONFIG RESOLVED AGE kagent-default 1 1 1 gvisor-default True 2mGet the address to reach the kagent gRPC API, which serves Agent, AgentTemplate, Session, and conversation calls. The guide to create your first agent assumes port-forwarding.
Forward the controller port and leave the command running. The API is then available at
localhost:8083.kubectl port-forward -n kagent svc/kagent-controller 8083:8083Optional: Open the kagent UI. The UI is a separate
kagent-uiservice, so it needs its own port-forward in a second terminal. The UI is then available at http://localhost:8082. For thekagent dashboardshortcut and a tour of the UI, see Launch the UI.kubectl port-forward -n kagent svc/kagent-ui 8082:8080To serve the UI behind a reverse proxy under a prefix such as
/ui(configurable withui.basePath), see Serve the UI under a sub-path.