GCP Terraform architecture

Understand what the GCP Terraform modules provision and how the pieces fit together, so you can size, secure, and customize your LangSmith deployment before running make apply. Use this page as a reference while planning a rollout or troubleshooting an existing one. It covers:

Platform layers and deployment tiers (light versus production).
Module descriptions and dependencies.
Networking, Workload Identity, and traffic flow.
Add-ons: LangSmith Deployment, Fleet, Insights, and Polly.
GCP managed services and Secret Manager integration.

If you are ready to install, start with the deployment walkthrough.

Platform layers

LangSmith on GCP deploys in up to five stages. Each stage adds a capability layer on top of the previous. All layers share the same GKE cluster and langsmith namespace.

LangSmith on GCP deployment stages and service layout

Stage	Layer	What it adds
1	GCP infrastructure	VPC, GKE, Cloud SQL, Memorystore, GCS, K8s bootstrap, cert-manager, KEDA, Envoy Gateway
2	LangSmith base	frontend, backend, platform-backend, queue, ace-backend, clickhouse, playground
3	LangSmith Deployment	host-backend, listener, operator + per-deployment pods
4	Fleet	standalone-fleet-api-server, standalone-fleet-tool-server, standalone-fleet-trigger-server, standalone-fleet-queue
5	Insights + Polly	Clio analytics (ClickHouse-backed), Polly eval agent

Fleet (chart v0.15+) is the current form of the feature formerly called Agent Builder. Enable it with enable_fleet. Unlike the deprecated enable_agent_builder path, it does not require the LangSmith Deployment layer. The two flags are mutually exclusive and share the same encryption key. See Enable add-ons in the deployment guide.

Module descriptions

Module	Path	Purpose
`networking`	`infra/modules/networking/`	VPC, subnet with secondary ranges, Cloud Router, Cloud NAT, private service connection for Cloud SQL and Memorystore
`k8s-cluster`	`infra/modules/k8s-cluster/`	GKE Standard or Autopilot cluster, private nodes, node pool with autoscaling, Workload Identity enabled
`postgres`	`infra/modules/postgres/`	Cloud SQL PostgreSQL instance, regional HA standby, private IP, deletion protection
`redis`	`infra/modules/redis/`	Memorystore Redis STANDARD_HA tier, private IP within VPC
`storage`	`infra/modules/storage/`	GCS bucket with versioning and lifecycle rules for `ttl_s/` (14 days) and `ttl_l/` (400 days) prefixes
`k8s-bootstrap`	`infra/modules/k8s-bootstrap/`	`langsmith` namespace, Kubernetes Secrets for Postgres and Redis URLs, cert-manager and KEDA Helm releases
`ingress`	`infra/modules/ingress/`	Envoy Gateway Helm release, GatewayClass, HTTPRoute, optional HTTPS Gateway listener
`iam`	`infra/modules/iam/`	GCP service account and Workload Identity bindings for GCS access (wired by default)
`dns`	`infra/modules/dns/`	Cloud DNS managed zone and managed cert (optional, enable with `enable_dns_module`)
`secrets`	`infra/modules/secrets/`	Secret Manager secret bundle (optional, enable with `enable_secret_manager_module`)

Deployment tiers

Light deploy (all in-cluster)

VPC
└── subnet (10.0.0.0/20, GKE nodes only)
    No Cloud SQL or Memorystore; chart pods handle both

GKE Cluster
├── langsmith namespace
│   ├── frontend, backend, platform-backend, queue, ace-backend, playground
│   ├── clickhouse (in-cluster)
│   ├── postgres   (in-cluster)
│   └── redis      (in-cluster)
├── cert-manager
├── keda
└── envoy-gateway-system

GCS Bucket (trace payloads, always external)

Set in terraform.tfvars:

postgres_source   = "in-cluster"
redis_source      = "in-cluster"
clickhouse_source = "in-cluster"

Production (external managed services)

VPC
├── subnet (10.0.0.0/20, GKE nodes, pods, services)
│   └── Secondary ranges: pods 10.4.0.0/14, services 10.8.0.0/20
└── Private service connection (VPC peering to Google managed network)
    ├── Cloud SQL PostgreSQL  (private IP, regional standby)
    └── Memorystore Redis     (private IP, STANDARD_HA tier)

GKE Cluster
├── langsmith namespace
│   ├── frontend, backend, platform-backend, queue, ace-backend, playground
│   └── clickhouse (in-cluster; use LangChain Managed for production scale)
├── cert-manager
├── keda
└── envoy-gateway-system

GCS Bucket (Workload Identity, no static keys)

Application core services

Service	Purpose	Port	HPA	Workload Identity	Depends on
`langsmith-frontend`	React UI	3000	1 to 10	No	`backend`, `platform-backend`
`langsmith-backend`	Main API (traces, runs, projects, API keys, feedback)	1984	3 to 10	Yes (GCS)	Postgres, Redis, ClickHouse, GCS
`langsmith-platform-backend`	Org and user management, auth, billing, settings	1986	1 to 10	Yes (GCS)	Postgres, Redis, GCS
`langsmith-playground`	LLM prompt playground UI	3001	1 to 10	No	`backend`
`langsmith-queue`	Trace ingestion worker (Redis to ClickHouse + GCS)	—	3 to 10 + KEDA	Yes	Redis, ClickHouse, GCS
`langsmith-ingest-queue`	Dedicated high-throughput ingestion worker	—	3 to 10 + KEDA	Yes	Redis, GCS
`langsmith-ace-backend`	Async compute (dataset runs, evaluations, background jobs)	—	1 to 5	No	Postgres, Redis
`langsmith-clickhouse`	Columnar store (trace spans, run metadata, eval results)	—	StatefulSet, single replica	No	500Gi `premium-rwo` PVC

In-cluster ClickHouse is dev/POC only (single pod, no replication, no backups). For production, use LangChain Managed ClickHouse or a self-managed external cluster.

SmithDB is LangSmith’s purpose-built observability backend, available for Self-hosted starting with self-hosted version 0.16.0 (see deployment support). These Terraform modules provision ClickHouse, so the guidance in the previous sections applies to current deployments.

One-time jobs

Job	Purpose
`langsmith-backend-migrations`	PostgreSQL schema migrations
`langsmith-backend-ch-migrations`	ClickHouse schema migrations
`langsmith-backend-auth-bootstrap`	Creates the initial org and admin account

LangSmith Deployment add-on

Service	Purpose	Workload Identity
`langsmith-host-backend`	LangGraph control plane API. Manages deployment lifecycle, serves deployment metadata.	Yes (GCS)
`langsmith-listener`	Watches host-backend for state changes, creates and updates `LangGraphPlatform` CRDs.	Yes (GCS)
`langsmith-operator`	Kubernetes operator. Reconciles `LangGraphPlatform` CRDs, creates and deletes Deployments and Services.	RBAC for Deployments and Services

Each LangGraph deployment created in the UI produces a Kubernetes Deployment in the langsmith namespace, with pods running as the langsmith-ksa ServiceAccount. That ServiceAccount must carry the iam.gke.io/gcp-service-account annotation, which deploy.sh applies idempotently.

GCP managed services

When postgres_source = "external" and redis_source = "external" (the recommended production setting), Terraform provisions:

Cloud SQL PostgreSQL

Default size db-custom-2-8192 (2 vCPU, 8 GB), private IP, port 5432.
REGIONAL availability with automatic failover.
Holds orgs, users, projects, API keys, settings.
Terraform writes the connection URL directly to the langsmith-postgres-credentials Kubernetes Secret.

Memorystore Redis

Default 5 GB, STANDARD_HA tier, private IP, port 6379.
Trace ingestion queue, pub/sub, short-lived cache.
No auth token required. Access is controlled by VPC private IP only.
Terraform writes the connection URL directly to the langsmith-redis-credentials Kubernetes Secret.

Cloud Storage bucket

Trace payloads: large inputs and outputs, attachments.
The shipped Helm values use native GCS mode (engine: GCS, apiURL: https://storage.googleapis.com), authenticated through Workload Identity with no HMAC keys.
An S3-compatible mode (engine: S3) is also supported, shown in helm/values/examples/langsmith-values.yaml. It requires HMAC keys: create one under Cloud Storage → Settings → Interoperability and pass them to Helm via config.blobStorage.accessKey and config.blobStorage.accessKeySecret.
Lifecycle rules: ttl_s/ prefix (14 days default), ttl_l/ prefix (400 days default).
The GCS bucket is always required, regardless of tier.

Secret Manager (optional module)

Stores a single JSON secret bundle (generated LangSmith secret key, Postgres password, Redis password) when enable_secret_manager_module = true.
Core secrets (langsmith-postgres-credentials, langsmith-redis-credentials) are always stored in Kubernetes Secrets by k8s-bootstrap regardless of this module. Secret Manager provides an additional durable store for secrets that must survive cluster recreation.

Cluster infrastructure

Service	Namespace	Installed by	Required for
Envoy Gateway	`envoy-gateway-system`	`ingress` module (`install_ingress = true`, default)	All ingress
KEDA	`keda`	`k8s-bootstrap` module when `enable_langsmith_deployment = true`	LangSmith Deployment add-on and later
cert-manager	`cert-manager`	`k8s-bootstrap` module when `tls_certificate_source = "letsencrypt"` or `install_cert_manager = true`	Let’s Encrypt TLS

The Gateway resource is managed by Terraform; the HTTPRoute is managed by Helm. Do not delete the Gateway resource manually. GCP releases the external IP when the Gateway is deleted, then issues a new IP on recreate.

Workload Identity

GKE pods access GCS through Workload Identity. The Kubernetes ServiceAccount is bound to a GCP service account via an IAM binding; pods receive temporary credentials with no static keys in Secrets or environment variables.

GKE pod
  └── Kubernetes ServiceAccount (annotated with iam.gke.io/gcp-service-account)
        └── IAM binding: roles/iam.workloadIdentityUser
              └── GCP Service Account
                    └── roles/storage.objectAdmin on the GCS bucket

Component	Annotation	Permissions
`langsmith-backend`	`iam.gke.io/gcp-service-account: <gsa>`	GCS `storage.objectAdmin` on the LangSmith bucket
`langsmith-platform-backend`	Same	GCS `storage.objectAdmin`
`langsmith-queue`	Same	GCS `storage.objectAdmin`
`langsmith-ingest-queue`	Same	GCS `storage.objectAdmin`
`langsmith-host-backend`	Same	GCS `storage.objectAdmin`
`langsmith-listener`	Same	GCS `storage.objectAdmin`
`langsmith-ksa` (operator pods)	Same	GCS `storage.objectAdmin`

The GSA is defined by the iam module and output as workload_identity_annotation. init-values.sh writes these annotations into values-overrides.yaml automatically. In native GCS mode (the shipped default), the GSA bindings above are sufficient. The optional S3-compatible mode (engine: S3) also requires HMAC keys: create one under Cloud Storage → Settings → Interoperability and pass it to Helm.

Network topology

Range	CIDR	Used by
Subnet	`10.0.0.0/20`	GKE nodes
Pods	`10.4.0.0/14`	GKE pod IPs (secondary range)
Services	`10.8.0.0/20`	GKE ClusterIP services (secondary range)
Private service connection	`/16` allocated by Google	Cloud SQL, Memorystore private IPs

Cloud SQL and Memorystore are accessed exclusively via private IP. The networking module establishes a private service connection (VPC peering to Google’s managed network) whenever postgres_source = "external" or redis_source = "external".

Traffic flow

Internet (HTTPS :443)
  ↓
Envoy Gateway  (envoy-gateway-system, external LoadBalancer IP)
  TLS terminated: cert-manager + Let's Encrypt or existing certificate
  │
  ├── /                     → frontend:80
  ├── /api/*                → backend:1984
  └── /api/v1/deployments/* → host-backend:1985  (LangSmith Deployment add-on)

Internal traffic (private IPs, never leaving VPC):
  backend       → Cloud SQL:5432    via private IP
  backend       → Memorystore:6379  via private IP
  backend       → GCS               via Workload Identity (native GCS mode)
  host-backend  → K8s API           reads deployment pod status
  listener      → K8s API           reconciles Deployment CRDs
  operator      → K8s API           creates and manages deployment pods

Component to storage mapping

Component	PostgreSQL	Redis	ClickHouse	GCS
`backend`	Org config, run metadata	Ingestion queue	—	Trace objects
`platform-backend`	—	—	—	Blob routing
`queue`	—	Pops jobs	—	Writes trace blobs
`clickhouse`	—	—	Trace search index	—
`host-backend`	Deployment lifecycle state	—	—	—

Secret Manager integration

Without Secret Manager:

terraform.tfvars → terraform apply → kubernetes_secret (postgres, redis)

With Secret Manager:

terraform.tfvars → terraform apply ─┬─→ kubernetes_secret (postgres, redis)
                                    └─→ Secret Manager (durable copy, survives cluster recreation)

Terraform writes the Kubernetes Secrets directly in both cases. Enabling Secret Manager adds a durable copy of the Postgres password, Redis password, and generated secret key outside the cluster. Nothing syncs Secret Manager back into the cluster, so no External Secrets Operator is installed on GCP (unlike the AWS modules, which use it to sync from SSM Parameter Store).

Terraform module graph

google_project_service (APIs enabled)
  └── module.networking
        ├── module.gke_cluster
        │     └── time_sleep.wait_for_cluster
        │           ├── module.cloudsql      (count = postgres_source == "external")
        │           ├── module.redis         (count = redis_source    == "external")
        │           ├── module.storage
        │           ├── module.iam           (count = enable_gcp_iam_module)
        │           ├── module.secrets       (count = enable_secret_manager_module)
        │           ├── module.dns           (count = enable_dns_module)
        │           ├── module.k8s_bootstrap
        │           └── module.ingress       (count = install_ingress)
        └── (private_service_connection when external services)

The infra layer does not install the LangSmith chart. The application stage installs it one of two ways, both consuming the same layered values files under helm/values/:

Deploy script: make init-values && make deploy runs helm upgrade --install.
Terraform app layer: make init-values && make init-app && make apply-app manages the chart as a helm_release resource. make init-app pulls the infra outputs (cluster, bucket, Workload Identity annotation) into app/infra.auto.tfvars.json, so the app layer reads them without a remote-state data source.

Verification commands

# Cluster connectivity
gcloud container clusters get-credentials <cluster-name> --region <region> --project <project-id>
kubectl cluster-info
kubectl get nodes -o wide

# All LangSmith pods
kubectl get pods -n langsmith

# Envoy Gateway
kubectl get pods -n envoy-gateway-system
kubectl get svc -n envoy-gateway-system

# cert-manager
kubectl get pods -n cert-manager
kubectl get certificate -n langsmith

# KEDA (LangSmith Deployment add-on)
kubectl get pods -n keda

# Cloud SQL connectivity test
kubectl run psql-test --rm -it --image=postgres:15 -n langsmith -- \
  psql "postgresql://langsmith:<password>@<cloud-sql-private-ip>:5432/langsmith" -c "SELECT version();"

# Memorystore connectivity test
kubectl run redis-test --rm -it --image=redis:7 -n langsmith -- \
  redis-cli -h <redis-private-ip> ping

# GCS connectivity test
kubectl run gcs-test --rm -it --image=google/cloud-sdk -n langsmith -- \
  gsutil ls gs://<bucket-name>

Connect these docs to Claude, VSCode, and more via MCP for real-time answers.

Edit this page on GitHub or file an issue.

Get started by cloud provider

Deploy with Terraform

Setup guides

Configuration

Connect external services

Platform auth & access control

Self-hosted observability

Hybrid

Scripts

Reference

Platform layers

Module descriptions

Deployment tiers

Light deploy (all in-cluster)

Production (external managed services)

Application core services

One-time jobs

LangSmith Deployment add-on

GCP managed services

Cloud SQL PostgreSQL

Memorystore Redis

Cloud Storage bucket

Secret Manager (optional module)

Cluster infrastructure

Workload Identity

Network topology

Traffic flow

Component to storage mapping

Secret Manager integration

Terraform module graph

Verification commands

​Platform layers

​Module descriptions

​Deployment tiers

​Light deploy (all in-cluster)

​Production (external managed services)

​Application core services

​One-time jobs

​LangSmith Deployment add-on

​GCP managed services

​Cloud SQL PostgreSQL

​Memorystore Redis

​Cloud Storage bucket

​Secret Manager (optional module)

​Cluster infrastructure

​Workload Identity

​Network topology

​Traffic flow

​Component to storage mapping

​Secret Manager integration

​Terraform module graph

​Verification commands

Platform layers

Module descriptions

Deployment tiers

Light deploy (all in-cluster)

Production (external managed services)

Application core services

One-time jobs

LangSmith Deployment add-on

GCP managed services

Cloud SQL PostgreSQL

Memorystore Redis

Cloud Storage bucket

Secret Manager (optional module)

Cluster infrastructure

Workload Identity

Network topology

Traffic flow

Component to storage mapping

Secret Manager integration

Terraform module graph

Verification commands