Skip to main content

Deployment

Cloudflare Integration

ScalarLM's Kubernetes deployment includes a built-in Cloudflare tunnel sidecar (cloudflare_deployment.yaml). This routes public HTTPS traffic to your inference API without exposing a raw IP or configuring an ingress controller — no hostPort needed in the Helm chart.

This guide walks through creating a Cloudflare account, registering a domain, creating a tunnel, and wiring the tunnel token into your ScalarLM deployment.

Prerequisites

  • A running ScalarLM Kubernetes deployment (see Kubernetes)
  • A domain name you own or can purchase (e.g. yourdomain.com)

Step 1: Create a Cloudflare Account

Go to cloudflare.com and sign up for a free account. The free tier is sufficient for ScalarLM deployments.

Step 2: Add Your Domain to Cloudflare

If you already own a domain, add it to Cloudflare by following these steps:

  1. In the Cloudflare dashboard, click Add a domain.
  2. Enter your domain name and click Continue.
  3. Select the Free plan and click Continue.
  4. Cloudflare will scan your existing DNS records. Review them and click Continue.
  5. Cloudflare will give you two nameserver addresses (e.g. ava.ns.cloudflare.com). Log in to your domain registrar (e.g. Namecheap, GoDaddy) and replace the existing nameservers with these two values.
  6. Click Done, check nameservers. Propagation can take a few minutes to a few hours.

If you don't have a domain, you can purchase one directly through Cloudflare under Domain Registration.

Step 3: Create a Tunnel

Once your domain is active in Cloudflare:

  1. In the left sidebar, go to Zero Trust → Networks → Tunnels.
  2. Click Create a tunnel.
  3. Select Cloudflared as the connector type and click Next.
  4. Give the tunnel a name — e.g. gemma3-4b-it — and click Save tunnel.
  5. On the next screen, Cloudflare will display a tunnel token. Copy this token — you will need it in the next step. It looks like:
eyJhIjoiNDYyNTI5ZDY5NTEyZjg5...
  1. Skip the connector install step (ScalarLM handles this via the cloudflared pod) and click Next.

Step 4: Configure a Public Hostname

Still in the tunnel configuration screen:

  1. Click Add a public hostname.
  2. Fill in the fields:This tells the tunnel to forward incoming requests to the ScalarLM API service inside the cluster. The resulting public URL will be https://gemma3-4b-it.yourdomain.com.
    • Subdomain: e.g. gemma3-4b-it
    • Domain: select your domain from the dropdown (e.g. yourdomain.com)
    • Type: HTTP
    • URL: scalarlm-api-service:8000
  3. Click Save hostname.

Step 5: Add the Tunnel Token to values.yaml

Open your deployment's values.yaml file:

vim deployment/helm/gemma3_4b_it/scalarlm/values.yaml

Paste the tunnel token into the cloudflared section:

cloudflared:
  tunnelToken: eyJhIjoiNDYyNTI5ZDY5NTEyZjg5...
Note: You do not need to set hostPort anywhere in the Helm chart when using Cloudflare. The tunnel handles external routing entirely.

Step 6: Deploy or Redeploy

If this is a fresh install:

helm -n <namespace> install scalarlm scalarlm

If you are updating an existing deployment:

helm -n <namespace> upgrade scalarlm scalarlm

Monitor the rollout:

watch kubectl -n <namespace> get pods

You should see the scalarlm-cloudflared-* pod reach Running status alongside the other pods:

NAME                                    READY   STATUS    RESTARTS   AGE
scalarlm-6675679b96-sbhq2               1/1     Running   0          1m
scalarlm-cloudflared-644f75496b-snszr   1/1     Running   0          1m
scalarlm-megatron-0                     1/1     Running   0          1m
scalarlm-vllm-6b78fcdbf-ql4rh           1/1     Running   0          1m

Step 7: Verify the Tunnel

Back in the Cloudflare dashboard under Zero Trust → Networks → Tunnels, your tunnel's status should show Healthy.

You can now send requests directly to your public hostname:

curl https://gemma3-4b-it.yourdomain.com/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Hello!"}
        ]
    }'

Troubleshooting

The cloudflared pod is running but the tunnel shows as unhealthy in the dashboard. Check that the tunnel token in values.yaml matches the token shown in the Cloudflare dashboard exactly. Redeploy after correcting it.

Requests return a 502 or "Bad Gateway" error. Verify the public hostname URL in the Cloudflare tunnel config points to the correct internal service name and port (scalarlm-api-service:8000). You can confirm the service name by running:

kubectl -n <namespace> get services

The cloudflared pod is crash-looping. Check its logs for an invalid or expired token:

kubectl -n <namespace> logs -f <cloudflared-pod-name>

If the token is invalid, generate a new one from the Cloudflare dashboard and update values.yaml.