Summary: Using the Tailscale package’s “Serve On Tailscale” action on the StartOS UI interface (packageId start-os, hostId admin, interfaceId admin-ui) produces a serve entry whose tailnet endpoint accepts TLS (valid Let’s Encrypt cert, ALPN negotiates h2) but then hangs forever — zero response bytes, every request times out (curl: HTTP 000 / exit 28). The same action on a package-bound interface (Public Pool UI) works perfectly.
Environment:
- StartOS 0.4.x on x86_64 DIY hardware
- Tailscale package: tailscaled running with
--tun=userspace-networkinginside a subcontainer of the service LXC - Serve entry as created by the UI action:
start-os-admin-ui-https-8443, mode https → socatTCP-LISTEN:20001→TCP:10.0.3.1:57854 - Working comparison:
public-pool-ui-https-443→ socat 20000 →10.0.3.1:61331— works from tailnet devices
Root cause (verified on the host):
-
The admin UI’s assigned port (57854 here) is host-bound, and the net controller creates NAT rules in
table ip startos, chainpreroutinglike:ip saddr 10.0.3.0/24 ip daddr 10.0.3.1 meta l4proto { tcp, udp } th dport 57854 dnat to 127.0.0.1:80 -
Traffic from the tailscale service container (10.0.3.215, eth0 on lxcbr0) to
10.0.3.1:57854is bridge-originated, not locally-originated. After DNAT to127.0.0.1, the kernel drops these packets as martians (loopback destination on a non-loopback ingress path). The connection is accepted at the tailnet front door by tailscaled, forwarded to socat, then dies silently at the NAT layer — hence the infinite hang with no bytes. -
Host-originated traffic works fine (
curl http://10.0.3.1:57854/from the host returns 200 in ~3ms), which makes the bug invisible from the StartOS box itself — it only manifests through the container path thattailscale serveuses. -
Package-bound interfaces (e.g. Public Pool’s UI) don’t hit this because their DNAT target is a real container IP (
10.0.3.174:80here) — routable, no martian drop.
Reproduction:
- Services → Tailscale → sign into tailnet
- System → StartOS UI → Tailscale section → “+ Serve On Tailscale”, mode HTTPS, published port 8443
- From any tailnet device:
curl -v https://<node>.<tailnet>.ts.net:8443/→ TLS completes, request hangs indefinitely - Same result in all three serve modes (https / http / tcp) — failure is below the protocol layer
Workaround I’m running (one additive nft rule on the host, inserted above the generated rules; survives my use case and is removable):
nft insert rule ip startos prerouting ip saddr 10.0.3.0/24 ip daddr 10.0.3.1 tcp dport 57854 dnat to 192.168.6.121:80
With that rule in place, https://<node>.<tailnet>.ts.net:8443/ returns the StartOS UI immediately (HTTP 200, ~250ms) from tailnet devices. I wrapped it in a small oneshot systemd unit that re-derives the assigned port + LAN IP at boot since both are dynamic.
Suggested fix directions (maintainer’s call):
- For host-bound interfaces, DNAT bridge traffic to the host’s LAN address (or an actual listener on 10.0.3.1) instead of 127.0.0.1; or
- Enable
route_localneton the bridge path if loopback DNAT is intentional (with the usual caveats); or - Have the serve action detect host-bound targets and wire socat directly to the host LAN IP instead of the bridge VIP.
Possibly related noise seen in start-core logs during the same window (likely a separate issue): the gateway watcher for the host NIC loops on Network Error: Usage: ip route { list | flush } SELECTOR and intermittent RTNETLINK answers: No such device from start_core::net::gateway::watch_ip (shared-libs/crates/start-core/src/net/gateway.rs:1704/2064) — looks like a malformed ip route invocation when the interface has no usable gateway address.
Happy to provide the full nft ruleset dump, serve-routes.json, or start-core logs if useful.
Thanks!