Skip to content
wfnexus

Kubernetes with Helm

The chart lives at infra/helm/wfnexus/ and is published as an OCI chart with every release. It installs one server (UI + API + engine), a Service, an optional Ingress, a data volume, and any number of worker pools.

What it creates

Section titled “What it creates”
Object Why
Deployment (server, 1 replica, Recreate) The engine holds running steps in memory; scale with workers, not replicas.
PersistentVolumeClaim at /data Run workspaces, the env-store key file, and the SQLite file / artifact folder. Kept on helm uninstall.
ConfigMap The plain settings under config: in values, as env.
Service, optional Ingress Probes and traffic on GET /api/health.
Deployment per runner pool A worker is a machine with a toolchain: one pool per label set.
ServiceAccount No API token mounted; nothing calls the Kubernetes API.

Everything runs as UID 10001, non-root, all capabilities dropped, RuntimeDefault seccomp.

Secrets: by reference, never as values

Section titled “Secrets: by reference, never as values”

values.yaml holds no secret. You create Secrets, the chart references them by name and key. A config: key that looks like a secret (*_KEY, *_TOKEN, *_SECRET, *_PASSWORD) fails the render.

Terminal window
kubectl create namespace wfnexus
# The model key — read by NAME at call time, never written anywhere.
kubectl -n wfnexus create secret generic wfnexus-llm --from-literal=OPENROUTER_API_KEY=...
# The env store's encryption key. Back this Secret up: lose it and stored values are unreadable.
kubectl -n wfnexus create secret generic wfnexus-key --from-literal=WFX_SECRET_KEY="$(openssl rand -hex 32)"

Smallest install: SQLite on a volume

Section titled “Smallest install: SQLite on a volume”
Terminal window
helm install wfnexus oci://ghcr.io/muthuishere/charts/wfnexus --version <chart-version> -n wfnexus \
  --set secretKey.existingSecret=wfnexus-key \
  --set 'extraSecretEnv[0].name=OPENROUTER_API_KEY' \
  --set 'extraSecretEnv[0].secretName=wfnexus-llm' \
  --set 'extraSecretEnv[0].key=OPENROUTER_API_KEY'

From a checkout, use ./infra/helm/wfnexus instead of the OCI reference.

First boot: the admin token

Section titled “First boot: the admin token”

A fresh install has no users, and the server will not serve an unauthenticated API on a reachable address. On its first boot it creates an admin, prints that admin’s token once to stdout, and exits; Kubernetes restarts it and the second boot serves with authentication required. Read the token from the first container’s output:

Terminal window
kubectl -n wfnexus logs deploy/wfnexus --previous | grep 'admin token'

Then wfx login --url https://wfx.example.com and approve with it.

Postgres and S3

Section titled “Postgres and S3”
Terminal window
kubectl -n wfnexus create secret generic wfnexus-db \
  --from-literal=DATABASE_URL='postgres://user:pass@pg:5432/wfx?sslmode=require'
kubectl -n wfnexus create secret generic wfnexus-s3 \
  --from-literal=S3_ACCESS_KEY=... --from-literal=S3_SECRET_KEY=...
my-values.yaml
database:
  driver: postgres
  postgres: { existingSecret: wfnexus-db }
artifacts:
  driver: s3
  s3: { endpoint: minio.minio.svc:9000, bucket: wfx-artifacts, useSSL: false, existingSecret: wfnexus-s3 }
config:
  WFX_PUBLIC_URL: https://wfx.example.com
ingress:
  enabled: true
  className: nginx
  annotations:
    nginx.ingress.kubernetes.io/proxy-buffering: "off"      # run logs are SSE
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
  hosts: [{ host: wfx.example.com, paths: [{ path: /, pathType: Prefix }] }]
  tls: [{ secretName: wfx-tls, hosts: [wfx.example.com] }]

The volume is still mounted: run workspaces live there.

Workers

Section titled “Workers”
Terminal window
kubectl -n wfnexus create secret generic wfnexus-runner \
  --from-literal=WFX_RUNNER_TOKEN=wfx_$(openssl rand -hex 24)
runnerToken: { existingSecret: wfnexus-runner }   # the server and every worker read the same key
runners:
  enabled: true
  pools:
    - name: linux
      replicas: 2
      labels: self-hosted,linux
    - name: jdk
      replicas: 1
      labels: self-hosted,jdk
      image: { repository: ghcr.io/you/wfx-runner-jdk, tag: "1.0" }

Each pod joins under its own pod name, over the in-cluster Service unless runners.serverUrl says otherwise. Workers outside the cluster join with the URL in config.WFX_PUBLIC_URL.

carry

Section titled “carry”

Both images start as carry run --. It is a no-op until CARRY_REPO is set — see Tools and config with carry.

carry:
  enabled: true
  repo: s3://my-bucket/carry
  s3Endpoint: s3.eu-west-1.amazonaws.com
  toolPubkey: "<public key from carry tool keygen>"
  awsCredentialsSecret: carry-aws     # keys AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
  keyFileSecret: carry-key            # mounted read-only at /etc/carry; CARRY_KEY_FILE points at it
  keyFileKey: carry.key

The server restores carry.state (or its project default); each worker restores its own pod name.

Settings

Section titled “Settings”

Any setting wfx-server reads from the environment can go under config: — WFX_MODEL, LLM_BASE_URL, WFX_DEFAULT_PROVIDER, WFX_FALLBACK_PROVIDERS, WFX_MAX_CONCURRENT_RUNS, WFX_CLASSIFIER_MODEL, WFX_TEMPLATES, WFX_MOUNT_ROOTS and the rest; values.yaml lists them. The chart itself sets WFX_MODE=server, the storage and artifact drivers, WFX_WORKDIR and the key location from the sections above.