Are you an LLM? Read llms.txt for a summary of the docs, or llms-full.txt for the full context.
Skip to content

GCP

Deploy rflow on Google Kubernetes Engine (GKE) using the repository's Helm chart and a Postgres database, such as Cloud SQL. This guide uses a GKE Standard cluster and a direct TLS database connection.

Prerequisites

  • Google Cloud CLI, authenticated to a project with billing enabled and permissions to manage GKE and networking.
  • kubectl and the GKE authentication plugin, Helm, Git, and Docker.
  • A reviewed rflow image from release 0.2.1 or later, pinned by digest.
  • A Postgres 14+ database dedicated to this project. The chart does not create Postgres or a Cloud SQL Auth Proxy.

The commands below create billable infrastructure. They use one AMD64 worker node as a starting point; size the cluster for your workflows and availability requirements. The published rflow release image targets linux/amd64.

1. Create a GKE cluster

Set your actual Google Cloud project ID, then enable the APIs:

export RFLOW_GCP_PROJECT=YOUR_PROJECT_ID
gcloud auth login
gcloud config set project "$RFLOW_GCP_PROJECT"
gcloud services enable container.googleapis.com sqladmin.googleapis.com

Create a cluster and configure access:

gcloud container clusters create rflow-cluster \
  --project "$RFLOW_GCP_PROJECT" \
  --zone us-west1-a \
  --machine-type e2-standard-2 \
  --num-nodes 1 \
  --enable-ip-alias \
  --enable-network-policy
 
gcloud container clusters get-credentials rflow-cluster \
  --project "$RFLOW_GCP_PROJECT" \
  --zone us-west1-a
 
kubectl get nodes
kubectl create namespace rflow

The command uses the default VPC. Set --network and --subnetwork for an existing VPC, including one used by Cloud SQL. The network-policy flag enables enforcement of the chart's ingress rules. See the GKE cluster command reference for private clusters, regional clusters, and node sizing. If using an existing cluster, select its context and create the namespace if needed.

2. Prepare Postgres and the image

Create or select a Cloud SQL for PostgreSQL instance with private connectivity to the GKE VPC. Configure private services access or Private Service Connect, create an rflow database and a login that can create its schemas and tables, and enable backups. Follow Google's GKE connection guide for network setup.

For this guide's direct TLS connection, use a certificate with a DNS name that rflow can verify. Cloud SQL's shared CA mode (GOOGLE_MANAGED_CAS_CA) supplies a DNS name in the certificate; configure private DNS so that name resolves to the instance's private address. Download the issuing CA chain and install it in the container's operating-system trust store using the database CA Dockerfile, one certificate per .crt file. See Cloud SQL certificate modes and hostname verification.

Use that certificate hostname in the database URL:

postgresql://USER:URL_ENCODED_PASSWORD@YOUR-CLOUD-SQL-DNS-NAME:5432/rflow?sslmode=require

rflow verifies the certificate and hostname; a raw private IP will fail if it is not in the certificate. Configure encrypted connections on Cloud SQL. This example uses a database password and server certificate verification, not client-certificate authentication.

If your existing Cloud SQL setup requires the Auth Proxy, follow Google's proxy instructions and supply the proxy deployment and identity configuration yourself. The bundled chart has no proxy sidecar or Workload Identity configuration. Do not point this direct-TLS URL at a local plaintext proxy or disable TLS on the remote connection to bypass a certificate error.

Build the derived image for linux/amd64, push it to a registry your nodes can pull from (for example Artifact Registry), and record its published digest. Grant the GKE node identity access to that registry. For another Postgres provider whose CA is already trusted by the base image, you can use the published rflow image directly.

3. Create the Kubernetes Secrets

Create a local file outside your Git repository, readable only by you, named rflow-env.production. It should contain the database URL from the previous step:

DATABASE_URL=postgresql://USER:URL_ENCODED_PASSWORD@YOUR-CLOUD-SQL-DNS-NAME:5432/rflow?sslmode=require

Export its path and create the Secrets. The API token is read from a file so it is not placed in a command argument:

export RFLOW_ENV_FILE=/absolute/private/path/rflow-env.production
chmod 600 "$RFLOW_ENV_FILE"
kubectl -n rflow create secret generic rflow-env --from-env-file="$RFLOW_ENV_FILE"
 
RFLOW_TOKEN_FILE=$(mktemp)
openssl rand -hex 32 | tr -d '\n' > "$RFLOW_TOKEN_FILE"
kubectl -n rflow create secret generic rflow-api \
  --from-file=rflow-api-token="$RFLOW_TOKEN_FILE"
rm "$RFLOW_TOKEN_FILE"

You can also provision these Secrets through your existing secret manager. They must exist in namespace rflow before installation. Never put resolved secrets in Helm values, rflowConfig, or projectFiles.

4. Install the Helm chart

git clone https://github.com/joshstevens19/rflow.git
cd rflow

Use a repository revision matching the release you reviewed. Create gcp-values.yaml, replacing the repository and digest with your image from step 2:

replicaCount: 1
image:
  repository: YOUR_IMAGE_REPOSITORY
  digest: sha256:REPLACE_WITH_REVIEWED_DIGEST
service:
  type: ClusterIP
  port: 3940
ingress:
  enabled: false

Install the chart with its production defaults:

helm upgrade --install my-rflow ./helm/rflow \
  --namespace rflow \
  -f helm/rflow/values-production.yaml \
  -f gcp-values.yaml \
  --wait --timeout 10m

The default project is a keyless heartbeat: it writes a console notification once a minute and only requires DATABASE_URL. Production values configure bearer authentication using rflow-api, a non-root container, resource limits, and a read-only root filesystem with writable project and temporary volumes.

Use your own workflows

Once the heartbeat is healthy, supply your project's rflow.yaml with --set-file rflowConfig=/absolute/path/to/project/rflow.yaml on the same Helm command. Before introducing a different project name, use a fresh database; the heartbeat database is already bound to my-rflow-project.

For example, a project using ${ETH_RPC} and abis/token.json also needs these entries in gcp-values.yaml:

env:
  DATABASE_URL: ""
  ETH_RPC: ""
projectFiles:
  abis/token.json: |
    REPLACE_WITH_THE_COMPLETE_ABI_JSON

Add ETH_RPC to the rflow-env Secret; env lists which keys the chart reads from that Secret. Supply every required ABI and command script under its relative path in projectFiles. ConfigMap data is limited to 1 MiB; larger projects need a custom image/volume deployment. Run rflow validate --preflight and rflow doctor against the project with the intended environment before installing it. See self-hosting and the chart README for the full chart configuration.

5. Verify and monitor

kubectl -n rflow get pods
kubectl -n rflow rollout status deployment/rflow-my-rflow --timeout=10m
kubectl -n rflow logs deployment/rflow-my-rflow -c rflow --tail=100
kubectl -n rflow port-forward service/rflow-my-rflow 3940:3940

Leave the port-forward running. In another terminal:

curl --fail http://127.0.0.1:3940/live
curl --fail http://127.0.0.1:3940/health
 
RFLOW_API_TOKEN=$(kubectl -n rflow get secret rflow-api \
  -o jsonpath='{.data.rflow-api-token}' | openssl base64 -d -A)
curl --fail -H "Authorization: Bearer $RFLOW_API_TOKEN" \
  http://127.0.0.1:3940/api/workflows

Open http://127.0.0.1:3940/ for the history explorer and use the token stored in rflow-api to log in. /live is the startup/liveness probe; /health is readiness and returns 503 with reasons when degraded. The chart already configures these probes. Scrape the bearer-protected /metrics endpoint with your monitoring stack and collect container logs; see observability.

The service remains private. For external access, configure an ingress controller, TLS, and the chart's ingress values. Extend networkPolicy.ingressFrom to allow the controller and monitoring namespaces. Protect webhook routes with their own HMAC authentication.

6. Upgrade and recover

Back up Postgres, update the image digest/project files, and rerun the same helm upgrade --install command with all values and --set-file options used for your project. The chart uses Recreate, stopping the old worker before starting its replacement. Expect a short interruption. Secret changes need a restart:

kubectl -n rflow rollout restart deployment/rflow-my-rflow
kubectl -n rflow rollout status deployment/rflow-my-rflow --timeout=10m

Keep replicaCount: 1, or configure both replicaCount and ha.enabled: true for leader/standby failover. Additional replicas do not divide the workflow load. Waiting standbys are not HTTP-ready, so Helm's --wait and Deployment rollout checks can time out even with a healthy leader; check leader readiness and standby logs separately for that topology.

Project and /tmp volumes are ephemeral. Back up Postgres, retain project files separately, and store any durable script artifacts externally. Follow the production runbook before rolling back a release or restoring a database; Helm rollback does not undo database migrations.

7. Clean up

After taking any required backup, remove the application and the example cluster:

helm uninstall my-rflow --namespace rflow
gcloud container clusters delete rflow-cluster \
  --project "$RFLOW_GCP_PROJECT" \
  --zone us-west1-a

Only delete a cluster you created for this example. Cloud SQL, registry images, backups, and separately provisioned networking are not removed by Helm; review and delete unused resources separately to stop their charges.