Deployment

This page describes where the platform runs and how a release reaches it. The pipeline covers the Bitbucket → Jenkins flow up to the point of a green build; this page picks up from there: what each host runs, the systemd units behind each service, and the deploy step that puts new code on the box. There are no containers for the application services — Backend, Console, Voice fleet, and Dialler all run directly under systemd, with code synced from Bitbucket and dependencies installed in place. The only Docker on the platform is the LiveKit stack.

Host roles

The platform is deployed by role, not by machine. A small deployment co-locates several roles on one host; a busy one splits them out and adds more fleet hosts. The roles, and what runs on each, are:

Role	What runs on it	systemd / process
API / App host	Backend API, Console static build (via nginx), MongoDB, Redis	`voxbridge.service` (uvicorn :8080), nginx
Fleet host(s)	N single-worker Voice fleet processes behind nginx, local MinIO	`voxcore@1…N.service`, nginx, MinIO
SIP / LiveKit host	LiveKit + `livekit-sip`, the single Dialler	docker-compose, `voxdialler.service`
WSS ingress host (optional)	HAProxy fronting multiple fleet hosts	HAProxy

The Ori repo names map to the service names once, here, and the friendly names are used everywhere else: Backend API = vox-backend, Console = vox-frontend, Voice fleet = vox-agents, Dialler = vox-dialler. All four live in the oriserve1 Bitbucket workspace. See the repository map for the full breakdown.

How Jenkins deploys to a host

Every role follows the same shape once the build is green: Jenkins reaches the target host, checks out the release, installs dependencies in place, and restarts the relevant systemd unit. The only differences are the build command (uv sync for Python, npm for the Console) and which unit gets restarted.

API / App host

The control-plane host. The Backend API serves admin and runtime-config APIs on :8080; the Console is a static build served by nginx; MongoDB and Redis back the Backend. Small deployments co-locate all four.

What runs

# scripts/run.sh — what the unit executes
uvicorn voxbridge.app:app \
  --host 0.0.0.0 \
  --port 8080 \
  --app-dir src
# host/port from VOXBRIDGE_HOST (0.0.0.0) / VOXBRIDGE_PORT (8080)

systemd unit (Backend)

# /etc/systemd/system/voxbridge.service
[Unit]
Description=Backend API
After=network.target mongod.service redis-server.service

[Service]
Type=simple
WorkingDirectory=/opt/voxbridge
EnvironmentFile=/opt/voxbridge/.env
ExecStart=/opt/voxbridge/scripts/run.sh
Restart=always

[Install]
WantedBy=multi-user.target

Jenkins deploy step

Backend

Check out the release to /opt/voxbridge, then uv sync and restart.

uv sync
sudo systemctl restart voxbridge
curl -fsS http://127.0.0.1:8080/health

Console

Build the static bundle and let nginx serve dist/. No service to restart — only the static files change.

npm ci
npm run build
# publish dist/ to the nginx web root

The Backend boots only when its required settings are present. If voxbridge.service flaps on restart, check MONGODB_URI, REDIS_URL, and JWT_SECRET in /opt/voxbridge/.env before anything else — see configuration & secrets.

Fleet host(s)

The runtime host. Each fleet host runs N single-worker uvicorn processes, one per call slot, each bound to its own Unix socket. nginx fans incoming calls across the sockets with least_conn and max_conns=1, so each worker handles exactly one call at a time. Recordings are uploaded to a MinIO instance local to the host.

The worker model

This is the part that does not look like an ordinary web service. Instead of one uvicorn process with many workers, the fleet uses one systemd instance per worker, each pinned to its own socket. That makes a single call the unit of failure and lets you restart one worker without touching the others.

Property	Value
Process per worker	1 uvicorn `--workers 1`
Socket per worker	`/tmp/voxcore_<i>.sock`
Calls per worker	1 (`MAX_CONCURRENT_CALLS=1`)
Worker count	N per host (commonly 16 on a 4-CPU / 8 GB host)
Host capacity	= number of enabled `voxcore@<i>` instances

# scripts/run_worker.sh — one worker on one socket
uv run uvicorn voxcore.app:app \
  --uds "$VOXCORE_SOCKET" \
  --workers 1 \
  --app-dir src

Templated systemd unit

A single template unit, voxcore@.service, is enabled once per worker as voxcore@1, voxcore@2, … voxcore@N. The instance number %i selects the socket.

# /etc/systemd/system/voxcore@.service
[Unit]
Description=Voice fleet worker %i
After=network.target

[Service]
Type=simple
WorkingDirectory=/opt/voxcore
EnvironmentFile=/opt/voxcore/.env
Environment=VOXCORE_SOCKET=/tmp/voxcore_%i.sock
ExecStart=/opt/voxcore/scripts/run_worker.sh
Restart=always

[Install]
WantedBy=multi-user.target

# Enable N workers
for i in $(seq 1 16); do sudo systemctl enable --now "voxcore@$i"; done

nginx in front of the sockets

nginx terminates public traffic and balances across the worker sockets. Two details are non-negotiable, both shipped in the repo template at infra/nginx/voxcore-fleet.conf.template:

map $http_upgrade for the Connection header. A hardcoded Connection "upgrade" works for WebSocket calls but breaks the HTTP POST routes (dialout returns 422).
429-retry to the next upstream for the short POST routes (/attach, /livekit/dialout, /livekit/widget). These return before the call pipeline finishes, so least_conn can route a later request to a worker that is logically busy; retrying a 429 lets nginx try the next free socket.

# map: pick the right Connection header per request type
map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

upstream voxcore_workers {
    least_conn;
    server unix:/tmp/voxcore_1.sock  max_conns=1 fail_timeout=10;
    server unix:/tmp/voxcore_2.sock  max_conns=1 fail_timeout=10;
    # … one line per worker …
    server unix:/tmp/voxcore_16.sock max_conns=1 fail_timeout=10;
}

server {
    listen 443 ssl;

    # WebSocket call entry
    location /ws/ {
        proxy_pass http://voxcore_workers;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
    }

    # Short POST routes — retry 429 to the next worker
    location = /attach {
        proxy_pass http://voxcore_workers;
        proxy_next_upstream error timeout http_429 non_idempotent;
        proxy_next_upstream_tries 16;
    }
}

Do not apply the 429-retry to /livekit/dispatch. Its 429 path removes the inbound SIP participant, so a retry would tear down a live inbound call. The retry is only for the short POSTs that spawn a background call and return immediately.

Jenkins deploy step

Sync and install

Check out the release to /opt/voxcore and install dependencies.

uv sync

Restart all workers

The glob restarts every worker on the host. For a true zero-downtime roll you can restart workers one at a time while the rest keep serving.

sudo systemctl restart 'voxcore@*'
# or, one worker at a time:
sudo systemctl restart voxcore@2

Verify

Hit the aggregate health endpoint, which queries every worker over its socket.

curl -fsS https://fleet.example.com/health/fleet

Scaling the fleet

More workers on a host

Add a socket line to the nginx upstream, systemctl enable --now voxcore@<i> for the new instances, then nginx -t && systemctl reload nginx.

More fleet hosts

Deploy the fleet to a new host, then add its URL to the Backend’s fleet list (and to HAProxy if used). No code change on existing hosts — fleet selection picks it up.

SIP / LiveKit host

The telephony host. It runs the LiveKit media server and the livekit-sip bridge as a docker-compose stack (the only Docker on the platform), and it hosts the single Dialler instance.

LiveKit

LiveKit and livekit-sip are brought up with the docker-compose stack from the repo (infra/livekit/setup.sh). It holds the SIP trunks per carrier, needs use_external_ip on cloud hosts, and needs a webhook configured for inbound SIP dispatch. LiveKit is the one component that is not redeployed by the app pipeline — it is brought up once and left running.

# Brought up once on the SIP host
docker compose up -d   # livekit + livekit-sip

Dialler

The Dialler is a background asyncio worker (with a small health server), not a web app. It runs from /opt/voxdialler and ticks every LOOP_INTERVAL_SECONDS (2.0): query running campaigns → predictive pacing → fair-share → lease calls atomically in MongoDB → place SIP calls via LiveKit → on answer, attach to a free fleet worker through POST /attach.

# /opt/voxdialler/deploy/voxdialler.service
[Unit]
Description=Dialler
After=network.target

[Service]
Type=simple
WorkingDirectory=/opt/voxdialler
EnvironmentFile=/opt/voxdialler/.env
ExecStart=/opt/voxdialler/scripts/run.sh   # uv run python -m voxdialler.main
Restart=always

[Install]
WantedBy=multi-user.target

Exactly one Dialler per database. The Dialler paces as if it is the only one dialling that MongoDB. Two Dialler instances pointed at the same voxbridge database will each pace to the full target and over-dial — abandoning calls and breaching carrier limits. Run the Dialler on the SIP host and nowhere else. If you stand up a second copy, leave it stopped and disabled.

Jenkins deploy step

Sync and install

Check out the release to /opt/voxdialler and install dependencies.

uv sync

Restart the Dialler

A single unit, Restart=always, so it comes straight back up.

sudo systemctl restart voxdialler

Verify

The Dialler exposes /health and /metrics on HEALTH_PORT (8090). A companion voxdialler-healthcheck.timer runs check_health.sh / smoke_check.py periodically.

curl -fsS http://127.0.0.1:8090/health
curl -fsS http://127.0.0.1:8090/metrics

WSS ingress host (optional)

When more than one fleet host serves a single public domain, an HAProxy ingress fronts them. Carrier and CRM WSS traffic hits HAProxy, which spreads it across the fleet hosts; each fleet host still runs its own nginx and worker sockets unchanged. The templates live in the fleet repo under infra/haproxy/ (render.sh, add-fleet.sh, haproxy.cfg.template). Adding a fleet to the ingress is config-only — render the config with the new backend and reload HAProxy; no fleet-host code changes.

A release, end to end

# Illustrative bitbucket-pipelines.yml — build + test on push, deploy from main.
# (Snippet only; not committed to the repos.)
image: python:3.12

pipelines:
  branches:
    main:
      - step:
          name: Install, lint, test
          script:
            - pip install uv
            - uv sync
            - uv run pytest -v
      - step:
          name: Deploy
          deployment: production
          trigger: manual
          script:
            - ssh deploy@fleet-host 'cd /opt/voxcore && git pull && uv sync && sudo systemctl restart "voxcore@*"'

// Illustrative Jenkinsfile (declarative) — same flow, Jenkins-driven.
// (Snippet only; not committed to the repos.)
pipeline {
  agent any
  stages {
    stage('Install & test') {
      steps {
        sh 'uv sync'
        sh 'uv run pytest -v'      // Console: npm ci && npm run lint && npm run build && npm test
      }
    }
    stage('Deploy') {
      when { branch 'main' }
      steps {
        sh '''
          ssh deploy@$TARGET_HOST "cd $APP_DIR && git checkout $RELEASE_TAG && uv sync"
          ssh deploy@$TARGET_HOST "sudo systemctl restart $UNIT"
        '''
      }
    }
    stage('Health check') {
      steps {
        sh 'curl -fsS $HEALTH_URL'  // /health, /health/fleet, or dialler :8090/health
      }
    }
  }
}

Rollback uses the same job in reverse: redeploy the previous release (check out the prior tag → uv sync / rebuild → restart the unit). Keep the last-known-good tag handy. The full set of restart, log, and recovery commands lives in the runbook.

Where to go next

The pipeline

The Bitbucket → Jenkins flow: branch strategy, build, lint, tests, and the gate to deploy.

Operations runbook

Restart commands, health checks, logs, scaling, and rollback for day-to-day ops.

Configuration & secrets

Every .env var per service and where each one belongs.

Repository map

The four repositories and the contracts between them.

​Host roles

​How Jenkins deploys to a host

​API / App host

​What runs

​systemd unit (Backend)

​Jenkins deploy step

​Fleet host(s)

​The worker model

​Templated systemd unit

​nginx in front of the sockets

​Jenkins deploy step

​Scaling the fleet

More workers on a host

More fleet hosts

​SIP / LiveKit host

​LiveKit

​Dialler

​Jenkins deploy step

​WSS ingress host (optional)

​A release, end to end

​Where to go next

The pipeline

Operations runbook

Configuration & secrets

Repository map

Host roles

How Jenkins deploys to a host

API / App host

What runs

systemd unit (Backend)

Jenkins deploy step

Fleet host(s)

The worker model

Templated systemd unit

nginx in front of the sockets

Jenkins deploy step

Scaling the fleet

SIP / LiveKit host

LiveKit

Dialler

Jenkins deploy step

WSS ingress host (optional)

A release, end to end

Where to go next