Skip to main content

Layout, Validation, and Operations Runbook

Keep each service under its own inspected directory. For a new project, first inspect /srv and confirm that the intended project directory does not already exist:

ls -ld /srv
sudo find /srv -mindepth 1 -maxdepth 2 -printf '%M %u:%g %p\n'
# Example: only create this directory if it is absent, not a symlink or mount.
sudo mkdir -- /srv/example-api &&
sudo chown -- "$VPS_USER:$VPS_USER" /srv/example-api

Here mkdir deliberately omits -p: an existing path makes it fail, so the && prevents the ownership change. Confirm that $VPS_USER names the intended deployment account and that its same-named group exists. On a shared host, directory creation also needs protection against concurrent changes.

Never run chown -R over /srv. Recursive ownership changes reach service data as well as configuration; container volumes may need a different numeric UID/GID, and backups may need root-only access. For an existing project, inspect its service-specific ownership requirements before changing any named path. An empty /srv/backups directory is not a backup system; recovery needs schedules, versions, off-site copies, and restore tests.

systemctl is-active asks whether a service runs now; is-enabled asks whether it starts on boot. Diagnose with:

systemctl --failed --no-pager
sudo journalctl -u docker --since today --no-pager
sudo journalctl -u tailscaled --since today --no-pager

Redact logs before sharing them. For updates, run apt update, inspect apt list --upgradable, and then apt upgrade. Simulate risky automated changes with apt-get -s. Phased ordinary updates may be deferred intentionally.

Check /var/run/reboot-required; before rebooting, verify Tailscale access, keep a session, reboot deliberately, reconnect, and recheck the kernel and failed units.

Validation checklist​

hostnamectl; timedatectl
nproc; free -h; df -h /
ip -brief addr; ip route
tailscale status; tailscale ip -4
sudo ufw status verbose
sudo sshd -t
sudo sshd -T | grep -E \
'^(permitrootlogin|passwordauthentication|kbdinteractiveauthentication|pubkeyauthentication|authenticationmethods) '
docker --version; docker compose version
systemctl is-active docker tailscaled
docker info
docker ps --format 'table {{.Names}}\t{{.Image}}\t{{.Ports}}\t{{.Status}}'
docker compose -f /srv/caddy/compose.yaml ps
docker compose -f /srv/uptime-kuma/compose.yaml ps
docker compose -f /srv/rsshub/compose.yaml ps
docker network inspect infra-edge
systemctl --failed --no-pager
find /srv -maxdepth 2 -printf '%M %u:%g %p\n' | sort

Compare the SSH values with the key-only policy, including any applicable Match blocks. From WSL2, a fresh Tailscale SSH transport must pass and a fresh public-IP SSH attempt must fail. Both public HTTPS health URLs must pass normal certificate validation, while direct public and Tailscale requests to RSSHub port 1200 must fail.

A bare 1200/tcp in docker ps is only an internal image port. A host publication looks like 0.0.0.0:1200->1200/tcp and would fail this design. The accepted state is Caddy on host 80/443, Kuma exactly on 127.0.0.1:3001, and no RSSHub host binding.

After every deployment, check the related service, UFW, and docker ps; weekly check updates, failed units, disk, and containers; monthly review tailnet devices, keys, exposed ports, and log growth.

During an incident: preserve access, stop widening the change, identify the failing layer, collect non-secret evidence, restore the nearest safe state, and verify recovery with a new connection or full request. Use Tmux for long work, but remember that it survives SSH disconnects, not host reboots.

The three current Compose projects—Caddy, Uptime Kuma, and RSSHub—have independent lifecycles. Operate from the matching /srv/<service> directory, inspect docker compose ps and logs first, and never use a global Docker prune as routine troubleshooting. The RSSHub shadow deployment owns image, ACCESS_KEY, canary, and rollback details. The existing feed consumer, its persistent data, and its scheduler have not migrated yet.

Explore connectionsOpen network