Networking

The problem it solves

Two copies of the same stack cannot both publish port 8000. The usual answer is to give the second copy different ports and then teach every front-end, every callback URL and every developer about them. multiversectl does not allocate ports: every universe is reached through one shared proxy on one set of ports, and requests are told apart by the Host header.

The consequence is that adding the tenth universe costs no ports and changes no configuration anywhere.

The shared proxy

One Traefik container for the whole machine, in its own runtime project multiverse-proxy, joined to one external network — multiversenet by default. The first up brings it up; it is not per workspace and not per universe.

It discovers routes from the container runtime, so starting a universe is all the configuration it needs. multiversectl writes the labels for it into a generated compose fragment, one per app, which is why your own compose files say nothing about Traefik:

# universes/3/.multiverse-routes-api.yml, rewritten on every up
services:
  web:
    ports: !reset []
    labels:
      traefik.enable: "true"
      traefik.docker.network: "multiversenet"
      traefik.http.services.api-u3.loadbalancer.server.port: "8001"
      traefik.http.routers.api-u3.rule: "Host(`api-u3.example.test`)"
      traefik.http.routers.api-u3.entrypoints: "web"
    networks: [default, multiversenet]

Edit the manifest or your compose files, never the generated file: it is replaced on every start.

Entrypoints

An entrypoint is a named port the proxy listens on. Every universe shares all of them.

spec:
  proxy:
    entrypoints:
      - name: web
        port: 80
      - name: devapi
        port: 8001

One is usually enough. The second exists for a case worth spelling out: a browser application built to call its API on a fixed port. Point the proxy at that port too, and the API answers there under every universe's own hostname — so the front-end's hard-coded :8001 keeps working, in every universe, unchanged.

A host rule picks its entrypoint by name, defaulting to web. Because a hostname tells the proxy which universe is meant, the ports never have to be unique.

Hostnames

Every address is <host-key>-<slug>.<domain>. The host key names the address, the slug names the universe:

api-u3.example.test         # host api, universe 3
adm-u3.example.test         # host adm, universe 3
api-fix-login.example.test  # host api, universe fix/login

The scheme is deliberately flat rather than nested. api-u3.example.test is one DNS label under the domain, so a single wildcard certificate for *.example.test covers every address of every universe. api.u3.example.test would need a certificate per universe, because a wildcard matches one label.

A service can also claim more than one hostname, which is how one API answers under each front-end's own name — see Labels.

Domains

spec.domain is the wildcard domain every hostname is built under. auto, the default, derives one from how the workspace is exposed.

Who can reach a universe

One setting, because the two halves have to agree:

spec:
  expose: lan     # default
  # expose: local

lan publishes the proxy's ports on every interface and derives a publicly resolvable domain: a phone, a tablet or a colleague on the same network opens a universe with nothing to install. local publishes on 127.0.0.1 only and derives localhost: nothing about the project is looked up anywhere, and nothing off the machine can connect at all.

They are one setting rather than two knobs because either alone is a half-measure. A .localhost name with the proxy still on 0.0.0.0 leaves the port open to anyone who sends the right Host header; a loopback bind with public hostnames still puts your naming in a stranger's DNS logs.

doctor reports where the proxy actually listens, and warns when that is not what the workspace asks for.

What lan actually exposes

Worth reading once before choosing it. A development stack usually runs with debug pages on, seeded data and no authentication in front of its mail catcher — so anyone on the network can read every message a universe sends, password resets included. That is fine on a home network and not fine in a café, a hotel or a shared office.

Those hostnames are also public DNS names pointing at a private address, which means a web page you happen to open can make your browser send requests to a universe. Reading the response needs permissive CORS, which development stacks often have. Under localhost the browser blocks that class of request outright.

Why nothing publishes a port

Isolating project names, networks and volumes is not enough while two universes both map a host port. So every service's published ports are dropped by default — that is the ports: !reset [] in the generated fragment — and a universe is reached through the proxy instead.

A service that genuinely needs its host port back asks for it, either in the manifest's publish: list or with a label on itself:

services:
  db:
    labels:
      multiverse.publish: "true"

Only one universe can hold such a port at a time, which is the trade you are making. ls -o wide has a PUBLISHED column so you can see at a glance that the answer is none everywhere it should be.

Talking between services

Inside one app, services reach each other by compose service name on the app's own network, exactly as they would without multiversectl: db, redis, mailpit. Nothing about that changes.

Between apps of the same universe, use the universe's hostnames through the proxy. Note the one asymmetry with expose: local: a .localhost name resolves to the calling container's own loopback, not to the proxy, so container-to-container calls by hostname do not work there. If you need both privacy and that kind of call, keep a resolvable domain and bind to loopback instead — domain: 127-0-0-1.traefik.me with expose: local gives you a name that only ever answers 127.0.0.1.

One proxy, several workspaces

The proxy is per machine, so two workspaces with different entrypoints or a different expose compete for it: starting a universe in the second reconfigures it, and the first workspace's universes stop answering until something is started there again. Starting says so explicitly:

! the shared proxy is being reconfigured: published on 127.0.0.1 instead of
  0.0.0.0, entrypoints web:80 instead of devapi:8001 web:80 — universes of any
  other workspace on this machine stop answering until it is started from
  there again

Running several workspaces side by side is therefore best done with the same entrypoints and the same exposure. A proxy that merges what each workspace asks for is an open question, not a settled design.