Tutorial 4: Deploy to Proxmox LXC

This tutorial deploys the 12 v1 services as Proxmox LXC containers — one container per service, isolated networking, Vault-managed secrets. This is the maintainer's primary deployment target ⭐.

Time: ~45 minutes. Prerequisites: Proxmox VE 8.0+, network access to the Proxmox node, a Proxmox API token.

Why LXC over VMs

Property	LXC	VM
Cold start	≤ 5 s	30-60 s
Memory overhead	10-50 MB	500 MB+
Density (per host)	30+	5-10
Isolation strength	namespace-level	hypervisor-level
GPU passthrough	yes (cgroup device)	yes (vfio)
Suitable for prod trading	✅	✅ but heavier

LXC gives you near-native performance with strong-enough isolation for the trust boundary. We're not multi-tenanting strangers; we're isolating our own services.

Step 1 — Create a Proxmox API token

In the Proxmox web UI:

Datacenter → Permissions → API Tokens → Add
User: root@pam
Token ID: mts1b-deploy
Uncheck Privilege Separation (the deploy tool needs the user's perms)
Add → copy the token value (shown once)

export PROXMOX_API_URL=https://pve.local:8006/api2/json
export PROXMOX_API_TOKEN_ID=root@pam!mts1b-deploy
export PROXMOX_API_TOKEN_SECRET=<the_secret_value>

Step 2 — Bootstrap Vault on the Proxmox host (recommended)

For production we want secrets in Vault, not .env files. mts1b-deploy can bootstrap a Vault for you:

mts1b-deploy vault bootstrap \
  --target proxmox-lxc \
  --hostname vault.local \
  --storage local-lvm

This:

Creates an LXC container vault on Proxmox.
Installs Vault + storage backend (raft).
Initializes with 5 unseal shares (Shamir).
Outputs the root token + unseal shares to ~/.mts1b/vault-init.json (move this off-host immediately).
Auto-unseals on subsequent boots via Proxmox API.

You can skip this step if you already have a Vault. Just set VAULT_ADDR + VAULT_TOKEN and mts1b-deploy will use it.

Step 3 — Configure with `menuconfig`

mts1b-deploy menuconfig

Pick:

Target: proxmox-lxc
Profile: foundational-12
Asset classes: pick what you want (equities, crypto, ...)
Optional services: frontends (so you have a UI)
Secrets: external-vault (the one we just bootstrapped or your existing one)

Then there's a Proxmox section:

[*] Proxmox
    Node:           pve1
    Storage:        local-lvm
    Network:        vmbr0
    Template:       ubuntu-22.04-standard
    Bridge VLAN:    20             (optional VLAN ID)
    IP range:       192.168.20.0/24
    Reserve IDs:    100-130        (LXC IDs)
    SSH keys:       ~/.ssh/id_ed25519.pub

Save.

Step 4 — Validate the config

mts1b-deploy validate --config mts1b.config

Checks:

Proxmox API reachable + token valid
Vault reachable + paths populated for each service
Template image exists on the storage pool
Requested LXC IDs not already in use
Sufficient host resources (RAM + disk)

Fix any errors, then continue.

Step 5 — Install

mts1b-deploy install --config mts1b.config

What happens (~10-20 minutes):

For each of the 12 services, mts1b-deploy creates an LXC container on Proxmox.
Configures network (bridge + IP from the configured range).
Mounts a shared /data volume backed by ZFS.
Installs the MTS1B image into each container.
Renders per-service env from Vault.
Starts the service and waits for /healthz green.
Configures Caddy reverse proxy to expose frontends on port 443 with a self-signed cert.

Progress is streamed:

[mts1b-foundation]  creating LXC 100... OK
[mts1b-platform]    creating LXC 101... OK
[mts1b-marketdata]  creating LXC 102... OK
[mts1b-quantkit]    creating LXC 103... OK
[mts1b-portfolio]   creating LXC 104... OK
[mts1b-riskengine]  creating LXC 105... OK
[mts1b-oms-algos]   creating LXC 106... OK
[mts1b-oms]         creating LXC 107... OK
[mts1b-brokers]     creating LXC 108... OK
[mts1b-GPUbacktester] creating LXC 109... OK (CUDA passthrough enabled)
[mts1b-deploy]      creating LXC 110... OK
[mts1b-docs]        creating LXC 111... OK

Installing services and rendering Vault secrets...
[12/12] Done.

Healthz probe:
  ✓ mts1b-platform      /healthz green
  ✓ mts1b-marketdata    /healthz green
  ✓ mts1b-portfolio     /healthz green
  ✓ mts1b-riskengine    /healthz green
  ✓ mts1b-oms           /healthz green
  ✓ mts1b-GPUbacktester /healthz green   cuda=available
  ... (12/12 green)

Caddy reverse proxy: https://mts1b.local (self-signed)
Grafana:             https://mts1b.local/grafana (admin/<random>)
Vault:               https://vault.local:8200

Step 6 — Smoke test

mts1b-deploy demo backtest-equities

This runs end-to-end through all 12 services in their LXC containers. Should complete in ≤ 1 minute on a modest 8-core host.

Step 7 — Configure observability

Grafana dashboards are pre-imported:

Fund Overview — per-fund NAV, P/L, exposure
OMS Throughput — orders/sec, fills/sec, reject rate
Risk Gates — gate failures by gate name
Data Lake — ingest lag, parquet write rate, storage growth
Service Health — /healthz status, CPU/mem/disk per container

Browse to https://mts1b.local/grafana and the admin password is in ~/.mts1b/grafana-admin.txt.

For Telegram/Slack alerts, add bot credentials to Vault:

vault kv put secret/mts1b/messaging \
  telegram_bot_token=<your_bot_token> \
  telegram_chat_id=<your_chat_id> \
  slack_webhook_url=<your_slack_webhook>

mts1b-platform/messaging will pick them up on its next reload (≤ 60s).

Step 8 — Backups

mts1b-deploy backup configure \
  --target nfs://nas.local/backups/mts1b \
  --schedule "0 3 * * *" \
  --retention 30d \
  --encryption-key /etc/mts1b/restic.key

Daily restic backups (deduped + AES-256 + zstd) of:

Postgres dumps
DuckDB databases
MinIO buckets
Vault snapshot

Restore tested via mts1b-deploy restore --to-staging (writes to a separate staging Proxmox node).

Step 9 — Updates

When a new MTS1B release lands:

mts1b-deploy upgrade --dry-run
# Shows: foundation 0.1.0 → 0.1.1
#        platform   0.1.0 → 0.1.2
#        oms        0.1.0 → 0.1.0 (unchanged)
#        ...

mts1b-deploy upgrade

The upgrade is rolling: each LXC container is updated in dependency order (foundation first, frontends last), with health checks between each step. Failures roll back to the previous version.

Step 10 — Going live (real broker)

When you're ready to flip from paper to a live broker:

# Add broker creds to Vault
vault kv put secret/mts1b/brokers/ibkr \
  username=<your_ibkr_username> \
  password=<your_ibkr_password> \
  account_id=<your_account_id> \
  trading_mode=live           # was 'paper'

# Restart the brokers service
mts1b-deploy restart mts1b-brokers

# Update the fund to use the live broker
mts mts1b-treasury fund edit --fund-id production-momentum --broker ibkr

⚠️ Before going live, double-check the risk envelope is tight. The daily_loss_halt_pct is the single most important production safety setting.

Common issues

Symptom	Likely cause	Fix
LXC container won't start	Storage pool full	`pvesm status` then resize or clean
`/healthz` timeout	NATS not reachable from container	Check bridge VLAN config
Vault `permission denied`	Service token has wrong policy	`vault token lookup` + repair the policy
Grafana login fails	Default password rotated	Look in `~/.mts1b/grafana-admin.txt`
GPUbacktester says CUDA unavailable	LXC needs `cgroup2 devices.allow` for `/dev/nvidia*`	`pct set <id> --features nesting=1` + nvidia device add

What's next

Tutorial 5: Add a broker — connect a new broker
Concept: Risk envelopes — production risk hardening
mts1b-deploy repo spec — full CLI reference

Why LXC over VMs​

Step 1 — Create a Proxmox API token​

Step 2 — Bootstrap Vault on the Proxmox host (recommended)​

Step 3 — Configure with menuconfig​

Step 4 — Validate the config​

Step 5 — Install​

Step 6 — Smoke test​

Step 7 — Configure observability​

Step 8 — Backups​

Step 9 — Updates​

Step 10 — Going live (real broker)​

Common issues​

What's next​