.pink

POST · INFRASTRUCTURE

Why I run every service on its own VM

Published . About 7 minutes to read.

All posts apoptotic case study
Premise

The default modern advice

Open any "deploy your side project" guide written in the last five years and the shape is roughly the same: containers, a registry, an orchestrator, a service mesh, a reverse proxy, and at least one cloud-native primitive that didn't exist when you started reading. It's not bad advice; it's optimized for a specific set of problems — multi-tenant, multi-developer, multi-region, "we might need to autoscale this on Black Friday".

None of those problems describe what I actually run. The apoptoses portfolio is half a dozen small services, all run by one person, with predictable load. Optimizing the deployment story for problems I do not have is a perfect way to spend an afternoon every week reading kubectl errors.

Shape

What I actually run

The shape is boring on purpose. Each service is a single Rust binary. Each binary lives on its own Debian VM. Each VM has exactly one systemd unit pointing at that binary. TLS is terminated by the binary itself, with rustls, bound directly to port 443.

  • main_site — apex, runs on its own VM.
  • spl_site — SP-Legends, on its own VM.
  • dc_bot + dashboard_site — Discord bot and the management UI, co-located on one VM, two systemd units.
  • emu_service — the Roblox AFK farm, on its own VM with a Docker container farm beneath it.
  • cloudflared — the tunnel itself, on its own VM.

There is no nginx between Cloudflare and the binary. There is no reverse proxy on the VM at all. There is nothing to misconfigure between the request and the code that handles it.

Wins

What this gives me

The wins compound. They are not individually impressive; the value is that all of them apply, all the time, on every service:

  • One process to look at. "Is the bot up?" is systemctl status dc_bot. There is no orchestrator-says-yes-but-the-pod-says-no ambiguity.
  • One log to tail. journalctl -u dc_bot -f. Structured logs go through log_store for retention; everything else lands in the journal.
  • One blast radius. If the bot's process gets into a bad state, the website doesn't care. If the website's TLS layer corrupts itself, the bot doesn't notice. The blast radius of a bug is the VM the bug lives on.
  • One dependency closure. Each VM has the apt packages it needs and nothing else. Updates on one VM cannot break a different service.
  • One mental model for deploys. Every service deploys the same way: tar the source, scp, build on the VM, swap the binary, systemctl restart. The admin desktop app automates this; the script form (deploy.ps1) is the same five steps, in the same order.
Costs

What it doesn't give me

Honest accounting is important. Things this shape does not give me:

  • Density. A handful of binaries do not need separate VMs. Co-location would be more efficient. I trade efficiency for isolation, and at this scale the trade is cheap.
  • Auto-scaling. Each service is sized to its peak. If peak changed by an order of magnitude tomorrow, I would rewrite this paragraph.
  • Built-in HA. Each service has exactly one home. If that VM goes down, the service is down. Cloudflare in front buys some shielding; for the workloads in question, the right answer to "what is the SLO?" is "best-effort hobbyist" and that is fine.
  • A platform team's worth of features. No service mesh, no canary deploys, no internal mTLS, no centralized config plane. None of these would help me deliver anything faster.
Updates

What about updates and rollouts?

The thing the container-and-orchestrator world does best is rollout: zero downtime, traffic shifting, easy rollback. I do not get that for free, and most of the time I do not need it. Restart times for these services measured in seconds; restarts happen at off-peak; users on the receiving end of a four-second blip on a hobby site do not care.

For the bot specifically, rolling restart matters more — gateway connections are not free. The bot handles its own graceful shutdown: stop accepting new work, drain in-flight events, write the last bit of state to SQLite, and exit. Systemd waits, sees a clean exit, and starts the new binary.

Honesty

When I would change my mind

There are real triggers that would push me away from this shape:

  • A second engineer. "One person who knows everything" is a load-bearing constraint here. Two engineers want a shared platform.
  • Real SLOs. If any of these services started carrying actual revenue or user expectations of "always up", HA stops being optional.
  • Tens of services. The marginal cost of a new VM is small but not zero. At ten or twenty services, container-per-process on a small fleet of bigger VMs probably wins.

None of these is true today. So the shape stays.