Nebula

Deploy to Hetzner Cloud

Run the Nebula control plane, orchestrator, gateway, and workers on hosted infrastructure.

Run nebula deploy to deploy Nebula to Hetzner Cloud. The command creates one virtual machine (VM) for the orchestrator and gateway and lets the orchestrator create one VM per worker. The orchestrator hosts the control plane and keeps it, with every saved entity, in a SQLite file on its VM or in a PostgreSQL database you point it at.

The deployment uses the same game build as a local mesh. It starts the orchestrator with -nebula-host hetzner instead of -nebula-host process. To run the mesh on the hosted service instead, see Deploy to Nebula Cloud.

Prerequisites

  • A Hetzner Cloud project and a Read & Write API token for it (console.hetzner.cloud, project, Security, API tokens). The token is checked against the API when you configure it.
  • Optionally, a PostgreSQL server the orchestrator VM can reach, for a world that must outlive the VM. Without one the orchestrator uses SQLite on its own disk.
  • Unity with Linux Dedicated Server support installed for the project's Unity version. nebula deploy builds Builds/Linux64/Nebula.x86_64 with the Server subtarget; without the module the build fails.
  • ssh and scp on your PATH. The CLI creates and uses ~/.ssh/nebula_hetzner.
  • The nebula CLI installed and a project with nebula.json (see Install).

Configure the deployment

Two one-time steps. Both write to ~/.nebula-cli/config.json (0600 on Unix) and to the project's nebula.json.

nebula config hetzner
nebula config database

nebula config hetzner asks for the token, project label, region, worker VM type, and orchestrator VM type. Pass --token to supply the token as an option. The command verifies the token and sets deploy.target to hetzner in nebula.json.

Set HCLOUD_TOKEN to override the saved token, such as in continuous integration (CI).

nebula config database is optional. It stores a database URL (--url) for deployed meshes in the CLI config: postgres://user:password@host:5432/nebula for PostgreSQL, or default for SQLite on the orchestrator VM. deploy.database in nebula.json and NEBULA_DATABASE_URL in the environment take precedence over it.

The deploy section of nebula.json

{
  "deploy": {
    "target": "hetzner",
    "meshName": "nebula",
    "database": "postgres://nebula:secret@db.example.com:5432/my-game",
    "workers": 4,
    "workerType": "ccx13",
    "orchestratorType": "cpx21",
    "location": "ash"
  }
}
FieldDefaultMeaning
targethetznerDeploy target: hetzner, or cloud for Nebula Cloud.
meshNamenebulaLabel every cloud resource carries (nebula-mesh=<meshName>) and the base for resource names: network and ssh key <meshName>, server <meshName>-orchestrator, firewalls <meshName>-orchestrator and <meshName>-worker. One mesh per Hetzner project.
databasesqlite:/opt/nebula/data/nebula.dbWhere the deployed orchestrator keeps the control plane and saved entities. Falls back to the CLI config's database.url, then to SQLite on the VM.
workers4Worker VMs the orchestrator keeps running. --workers overrides per deploy.
workerTypeCLI config (cpx21)Hetzner server type for worker VMs.
orchestratorTypeCLI config (cpx21)Server type for the orchestrator VM.
locationCLI config (ash)Region. Changing it after the network exists needs nebula destroy --all first, because the subnet is bound to a network zone.
envnoneEnvironment variables every worker gets, as { "KEY": "value" }. Edit them with nebula env. Not for secrets: see Pass settings and secrets to workers.

The mesh section (dashboardPort 7080, gatewayPort 7000) is shared with the local mesh and also sets the ports the firewall opens.

Deploy

Build and deploy

nebula deploy --workers 4 --open-ui

--workers N runs a fixed mesh of N worker VMs. --min N --max M opens an autoscaling band instead: the orchestrator starts at --min, adds VMs while workers are busy and retires idle ones into an idle pool, where a VM stays for the hour Hetzner already billed (deploy.idlePoolSeconds in nebula.json shortens that). --min 0 allows scaling to zero. Without any of these flags the mesh uses deploy.workers, deploy.minWorkers and deploy.maxWorkers from nebula.json.

deploy refuses to run until nebula config hetzner has been completed. Then, in order:

  1. Build. Runs Nebula.Editor.NebulaBuild.BuildLinuxServerBatch in batchmode (from a mirrored copy under ~/.nebula-cli/scratch/<project> if the Editor has the project open) and packs Builds/nebula-linux.tar.gz. --skip-build reuses the existing tarball.
  2. Provision what is missing (idempotent): the ssh key ~/.ssh/nebula_hetzner (generated if absent) uploaded as <meshName>; the private network <meshName> on 10.0.0.0/16 with subnet 10.0.1.0/24 in the region's zone; the two firewalls; and the orchestrator VM, created with the key, the network and the orchestrator firewall, then waited on until ssh answers (up to 240 s).
  3. Upload. scp of the tarball to /opt/nebula/artifacts/nebula-linux.tar.gz.uploading, then over ssh: stop nebula-orchestrator (the orchestrator deletes its worker VMs on shutdown), move the tarball into place, unpack it into /opt/nebula/bin. --skip-upload skips the build and upload and only rewrites the service.
  4. Write the service. The secrets go over stdin into /etc/nebula/env (HCLOUD_TOKEN, NEBULA_DATABASE_URL, a freshly generated NEBULA_MESH_TOKEN, and NEBULA_AUTH_KEY; mode 0600, never on a command line), and /etc/systemd/system/nebula-orchestrator.service is written with the full command line (see below). NEBULA_AUTH_KEY, the key anonymous identities and session tokens are signed with, is generated the first time and carried over from the existing environment file on every later deploy, so players keep their anonymous identities across deployments. The mesh token is new on every deploy. systemctl daemon-reload, enable, restart. --reset-persistence adds -nebula-reset-persistence, which deletes every saved entity as the orchestrator starts.
  5. Wait for the dashboard at http://<public ip>:7080/ for up to 80 s. On timeout it prints the last journalctl and log lines and fails with a pointer to nebula logs --cloud orchestrator.

The orchestrator then creates one VM per worker. Deploy reports the dashboard URL and the gateway address as soon as the orchestrator answers; the workers take about 30 s more each to register.

Check it

nebula status --cloud
nebula logs --cloud orchestrator
nebula logs --cloud w1

status --cloud lists every Hetzner server labeled with the mesh. It then shows the orchestrator's view from /api/state, including workers, containers, players, bots, server-driven entities, tick time, heartbeat age, and recent events.

Connect a client

Builds\Win64\Nebula.exe -nebula-role client -nebula-gateway <orchestrator public ip>:7000 -nebula-name Player

If your scene includes NebulaTitleScreen, you can start the client without -nebula-gateway and type the address on the title screen. The title screen remembers the last address you used. Bots work the same way with -nebula-bot.

To play in a browser, host Builds/Web yourself (the deployment never includes it) and point it at the gateway. The gateway gets its HTTPS certificate from Let's Encrypt a few seconds after it starts, so web builds hosted on HTTPS sites such as Unity Play can connect to it. See Build for the web.

Redeploy after a code change

Run nebula deploy again. It rebuilds, re-uploads and restarts the orchestrator, which deletes its worker VMs on the way down and recreates them from the new tarball. Players are disconnected during a redeploy; there is no rolling update. Their anonymous identities survive it, because the player signing key is kept on the VM.

Topology

            you (client / bot)                     Hetzner project my-game, location ash
                    │ udp/7000                     ┌─────────────────────────────────────────────────────┐
                    ▼                              │  nebula-orchestrator VM (public IP)                  │
  ┌───────────────────────────────┐   HCLOUD API   │   Nebula.x86_64 -nebula-role orchestrator            │
  │ api.hetzner.cloud             │◄───────────────│     -nebula-host hetzner  (creates/deletes worker VMs)│
  └───────────────────────────────┘                │     dashboard :7080, serves /build/nebula-linux.tar.gz│
                                                   │   gateway process  udp/7000                          │
  ┌───────────────────────────────┐                │   control plane + saved entities at :7080/api/    │
  │ PostgreSQL (optional)         │◄───────────────│     in /opt/nebula/data/nebula.db, or PostgreSQL     │
  │ my-game                       │  NEBULA_DATABASE_URL           │ private net   │ 10.0.1.0/24
  └───────────────────────────────┘                └───────────────▲───────────────▲─────────────────────┘
                                                          ┌────────┴───────┬───────┴─────────┐
                                                          │ orch1-w1-1 VM  │ orch1-w2-2 VM   │ ...
                                                          │ worker udp/7101│ worker udp/7102 │
                                                          └────────────────┴─────────────────┘

The orchestrator VM runs two processes: the orchestrator (dashboard, worker host) and the gateway it launches through the process host. The systemd unit's command line is:

Nebula.x86_64 -batchmode -nographics -nebula-role orchestrator -nebula-host hetzner
  -nebula-workers 4 -nebula-min-workers 4 -nebula-max-workers 4 -nebula-idle-pool 0
  -nebula-dashboard-port 7080 -nebula-dashboard-bind + -nebula-build-dir /opt/nebula/artifacts
  -nebula-advertise <private ip> -nebula-gateway <public ip>:7000
  -nebula-cloud-mesh nebula -nebula-cloud-location ash -nebula-cloud-type cpx21 -nebula-cloud-image ubuntu-24.04
  -nebula-cloud-network nebula -nebula-cloud-sshkey nebula -nebula-cloud-firewall nebula-worker
  -logFile /var/log/nebula/orchestrator.log

-nebula-advertise is the orchestrator's private IP: workers reach the gateway, the control plane, and the build download over the private network. -nebula-gateway is the public address the gateway registers so clients can find it. The database URL, the mesh token, and the player signing key come from /etc/nebula/env, not the command line. --verbose on nebula deploy adds -nebula-verbose.

How worker VMs boot

HetznerWorkerHost creates each worker VM (<mesh>-<workerId>-<generation>, labels nebula-mesh, nebula-role=worker, nebula-worker=wN) with an Ubuntu 24.04 image, the ssh key, the private network, the worker firewall, a public IPv4 and a cloud-init script (HetznerWorkerHost.BuildCloudInit) that:

  1. Logs to /var/log/nebula-bootstrap.log.
  2. Reads its private IP from the Hetzner metadata service (169.254.169.254/hetzner/v1/metadata/private-networks), falling back to the first 10.* address, retrying for up to two minutes.
  3. Downloads http://<orchestrator private ip>:7080/build/nebula-linux.tar.gz from the orchestrator's /build/ route, retrying up to 90 times.
  4. Unpacks it into /opt/nebula, writes the deployment's environment variables (deploy.env) to /etc/nebula/worker.env, readable only by root, and writes nebula-worker.service, which loads that file and runs Nebula.x86_64 -batchmode -nographics -nebula-role worker -nebula-worker-id wN -nebula-worker-index N -nebula-port 710N -nebula-advertise <private ip> ... with -logFile /var/log/nebula-worker.log.
  5. Starts the unit. Restart=no: if the worker dies, the orchestrator notices through heartbeats and replaces the VM.

From the create call to a registered, container-owning worker takes about 30 s. Killing a worker (dashboard Kill, POST /api/workers/kill) deletes its VM; removing one drains it first and then deletes it. When the orchestrator starts it deletes every VM labelled nebula-mesh=<mesh>,nebula-role=worker before launching fresh ones, so a crashed or redeployed orchestrator never leaves machines billing.

Ports

WherePortPurpose
Orchestrator VM, publicudp/7000Gateway, the address UDP clients use.
Orchestrator VM, publictcp/7000, udp/7001Gateway for web clients: WebRTC offers over HTTPS on TCP, WebRTC traffic on UDP.
Orchestrator VM, publictcp/80Let's Encrypt validation of the gateway's HTTPS certificate.
Orchestrator VM, publictcp/7080Dashboard and API. Unauthenticated.
Orchestrator VM, publictcp/22ssh for nebula logs --cloud.
Worker VMs, publictcp/22ssh for nebula logs --cloud wN. Nothing else is open inbound.
Private network 10.0.0.0/16udp/710N, tcp/7080Worker to worker, worker to gateway, build download.

The firewalls are <meshName>-orchestrator (22, the dashboard port, the gateway port over UDP and TCP, the gateway port plus one over UDP, and TCP port 80, from anywhere) and <meshName>-worker (22 only). Each deployment adds missing required rules and keeps rules you added. Traffic on the private network is not filtered by Hetzner firewalls.

Keep the worker ports private

Nebula encrypts only the client-to-gateway link, and only when you turn it on. Worker traffic carries entity state and player input in the clear, so the worker ports must stay reachable from the private network alone. Do not expose the configured worker ports to the internet. See Encrypt client connections.

Choose VM types

Worker and orchestrator VMs default to cpx21. A four-worker mesh uses five VMs: four workers and one orchestrator.

Use a dedicated-vCPU VM type when you need stable performance measurements. Set deploy.workerType in nebula.json, then run nebula deploy again. Check Hetzner's current pricing before you deploy.

Operate the deployment

CommandWhat it does
nebula status --cloudHetzner server table plus the orchestrator's /api/state. --json prints the raw state.
nebula logs --cloud orchestratorjournalctl -u nebula-orchestrator (last 10 lines) then /var/log/nebula/orchestrator.log.
nebula logs --cloud gateway/opt/nebula/bin/Logs/gateway.log on the orchestrator VM.
nebula logs --cloud w2On that worker's VM: the tail of /var/log/nebula-bootstrap.log, then /var/log/nebula-worker.log. Worker VMs are found by their nebula-worker label.
nebula logs --cloud ... -n 200More lines. --follow is local only.

Open the Nebula Dashboard at http://<public-ip>:7080/. Copy the gateway address for your client, change the worker count, add or remove workers, stop a worker to test recovery, and edit your game's shared settings. Worker changes create or delete virtual machines. See Orchestrator, Nebula Dashboard and worker hosts.

Log file paths on the VMs, for when you ssh in yourself (ssh -i ~/.ssh/nebula_hetzner root@<ip>):

VMPath
Orchestrator/var/log/nebula/orchestrator.log, /opt/nebula/bin/Logs/gateway.log, journalctl -u nebula-orchestrator
Worker/var/log/nebula-bootstrap.log (cloud-init), /var/log/nebula-worker.log, journalctl -u nebula-worker

Remove the deployment

nebula destroy
nebula destroy --all

destroy lists the servers labelled with the mesh, asks for confirmation (--yes skips it) and deletes them all, workers and orchestrator. --all also deletes the firewalls, the private network and the ssh key after a short wait for the servers to detach. A SQLite database on the orchestrator VM goes with the VM, and with it every saved entity; a PostgreSQL database is left in place. The ssh key file in ~/.ssh is left alone.

Stray workers

The orchestrator deletes worker VMs when it stops cleanly and sweeps leftovers when it starts. If it is killed hard and never restarted, its workers keep running. nebula status --cloud shows them; nebula destroy removes them.

Security and operations

  • The control plane is protected by the mesh token only. Workers and the gateway present the token nebula deploy generated; without it the orchestrator answers 401, and the gateway and workers refuse a peer that cannot present a credential derived from it. The token lives in /etc/nebula/env on the orchestrator VM and on the worker VMs' command lines.
  • The player signing key lives on the orchestrator VM. NEBULA_AUTH_KEY in /etc/nebula/env signs anonymous identities and session tokens. nebula deploy keeps it across deployments; nebula destroy deletes it with the VM, and every anonymous player then becomes a new player. Copy the value out of the file first if you rebuild the VM and want to keep them.
  • The dashboard is unauthenticated. Anyone who can reach tcp/7080 can resize or kill the mesh. Restrict the orchestrator firewall's dashboard rule to your own IP (the firewall is created by nebula deploy; edit it in the Hetzner console).
  • SQLite lives and dies with the VM. Point the mesh at PostgreSQL (nebula config database) when the saved world must survive nebula destroy or a lost VM.
  • Replacing a worker takes about 30 s: a fresh VM boots, downloads the build and registers. Its transient entities are gone; players respawn through OnSpawnPlayer.
  • nebula deploy restarts the mesh. Every deploy restarts the orchestrator and recreates every worker.
  • One mesh per Hetzner project, labelled with meshName.

On this page