Fresh 0.4.0 install: LND stuck on "Syncing graph" - Solved thanks to Kimi K3

Services Support
1

[SOLVED] Fresh 0.4.0 install: LND stuck on “Syncing graph” for 18+ hours (Neutrino backend) — BOLT-0010 bootstrap gets zero peers; fixed by manually connecting a peer

Environment: DIY x86_64 box, StartOS 0.4.0 (stable, clean install), LND package v0.21.1-beta, Neutrino backend, Tor service installed. Server name and keys redacted where relevant.

TL;DR

On a fresh 0.4.0 install, LND’s chain sync finished in ~12 minutes but the graph sync never started: the node had zero Lightning peers for 18+ hours. The BOLT-0010 DNS-seed bootstrap was failing on every attempt (“no addresses found”, looping every ~6 minutes). Restarting Tor did not fix it. Manually connecting a single Lightning peer via lncli connect broke the loop immediately — the full graph (~37,000 channels) downloaded in 13 minutes and the node reported synced.

Symptoms

  • LND service health check stuck on “Syncing to graph” for 18+ hours on a brand-new install.
  • Neutrino (Bitcoin) side completely healthy:
[INF] BTCN: Fully caught up with cfheaders at height 959745, waiting at tip for new blocks
[INF] LTND: Chain backend is fully synced!
  • Meanwhile, the Lightning bootstrap looped endlessly with zero successful peer connections:
[INF] DISC: Attempting to bootstrap with: Authenticated Channel Graph
[ERR] SRVR: Unable to retrieve initial bootstrap peers: no addresses found
[INF] DISC: Attempting to bootstrap with: BOLT-0010 DNS Seed: [[nodes.lightning.wiki soa.nodes.lightning.wiki] [lseed.bitcoinstats.com ]]

This repeated 158 times over 18 hours. (“no addresses found” from the Channel Graph bootstrapper is expected on a fresh node with an empty graph — the failure is that the DNS-seed fallback also returned nothing, every time.)

What I ruled out first

  1. Tor not running. On 0.4.0 Tor is a separate marketplace service, and it wasn’t up initially (tor host is unreachable in early logs). LND’s “Enable Tor” defaults to on. Fixing Tor did NOT fix the bootstrap — but it was verifiably working afterwards, because neutrino started connecting to .onion Bitcoin peers (New valid peer ....onion:8333 (outbound)). So Tor was not the (remaining) blocker.
  2. General connectivity. Fine — LND’s external fee estimator (nodes.lightning.computer) fetched successfully the whole time, and Bitcoin P2P on 8333 (clearnet + onion) worked.
  3. Pi-hole / router DNS filtering. Not in play: the router doesn’t hand out the Pi-hole as DNS, and SRV lookups via the system resolver work fine from another machine on the same LAN (dig SRV nodes.lightning.wiki returns ~25 node records in 10 ms).

Analysis

The BOLT-0010 bootstrapper doesn’t use the system DNS resolver — it opens a direct TCP connection to the seed’s authoritative nameserver on port 53 (that’s the soa.nodes.lightning.wiki second field in the log line) and issues SRV queries there. Each attempt in my logs hangs ~6 minutes and yields nothing, which is consistent with direct outbound TCP:53 being blocked or redirected somewhere between the container and the internet. Candidates: router/ISP forced-DNS, or StartOS 0.4.0’s new built-in DNS/container networking policy. I could not fully isolate which one — see “For the devs” below for the discriminating test.

Since discovery was dead but outbound 9735 was presumably fine, the workaround was to skip discovery entirely.

The fix

1. Get a real, current Lightning peer. On any machine with working DNS, resolve the same seed LND uses:

dig +short SRV nodes.lightning.wiki     # gives bech32-encoded hostnames
dig +short A <one of those hostnames>   # gives the node's IP

The pubkey is the bech32 string in the hostname (ln1q... decodes to the 33-byte node pubkey). Alternatively just grab any big, reliable node’s URI (clearnet or onion) from amboss.space.

2. Connect manually. SSH into the server (ssh start9@<server>.local, docs: SSH - StartOS), attach to the container (Accessing Service Containers - StartOS), and connect:

sudo start-cli package attach lnd
lncli --rpcserver=127.0.0.1:10009 connect 0355157b4260b70c7f407a720c527a84e9522cd948e7a8ad92ae00773be52488e3@71.255.78.29:9735
lncli --rpcserver=127.0.0.1:10009 connect 02dfe525d9c5b4bb52a55aa3d67115fa4a6326599c686dbd1083cffe0f45c114f8@91.232.115.39:9735

(One peer is enough; two is insurance. These two were resolved from the official seed and verified reachable minutes earlier.)

Note the --rpcserver=127.0.0.1:10009: plain lncli inside the container fails with x509: certificate is not valid for any names, but wanted to match localhost — the StartOS-managed tls.cert has an IP SAN for 127.0.0.1 but no DNS SAN for localhost, and lncli dials localhost by default.

3. Watch it work:

[INF] SRVR: Established outbound connection to: 0355157b...@71.255.78.29:9735
[INF] DISC: Creating new GossipSyncer for peer=0355157b...
[INF] DISC: GossipSyncer(...): buffering chan range reply of size=4000
[INF] DISC: GossipSyncer(...): filtering through 37295 chans
[INF] DISC: GossipSyncer(...): querying for 500 new channels
...
[INF] DISC: GossipSyncer(...): no more chans to query
[INF] DISC: GossipSyncer(...): applying gossipFilter(...)
Launching synced-true...

18:47 first peer connected → 19:00 fully synced. 37,295 channels in 13 minutes, after 18 hours of nothing. From then on the Authenticated Channel Graph bootstrapper also works (“Obtained 3 addrs to bootstrap network with”), so this is a one-time fix — peers and graph persist across restarts.

Verify with:

lncli --rpcserver=127.0.0.1:10009 getinfo | grep -E 'synced|num_peers'

Secondary issue found on the way (possible wrapper bug)

The very first start with Neutrino selected crashed hard:

[ERR] LTND: unable to create partial chain control: --fee.url parameter required when running neutrino on mainnet
error creating wallet config: unable to create partial chain control: --fee.url parameter required when running neutrino on mainnet

On the next start the wrapper had configured the external fee estimator (https://nodes.lightning.computer/fees/v1/btc-fee-estimates.json) and it went away — so it looks like a first-run ordering bug where fee.url isn’t written before LND starts with a fresh Neutrino config. Worth a look: GitHub - Start9Labs/lnd-startos: wrapper for building lnd.s9pk · GitHub

For the devs / open questions

  1. Why does BOLT-0010 fail in this environment? Discriminating test (on the Start9 host shell): dig SRV nodes.lightning.wiki (system resolver) vs dig +tcp SRV nodes.lightning.wiki @soa.nodes.lightning.wiki (direct TCP:53). If the first works and the second times out/refuses, direct external TCP:53 is blocked — either by the user’s network or by 0.4.0’s container DNS policy. If it’s the container policy, every fresh LND install on 0.4.0 will hit this.
  2. Resilience suggestion: if DNS-seed bootstrap returns nothing after N attempts, the LND package could fall back to a few baked-in well-known bootstrap peers (or expose “add peer” in the UI), so a fresh node can never be stranded with zero peers.
  3. Consider making the wrapper retry/fee.url write ordering deterministic on first Neutrino start (issue above).

Lessons for other DIY users hitting “syncing graph forever”

  • Chain sync finishing fast (minutes) + graph sync never finishing = it’s a peer discovery problem, not a sync problem. Check for Lightning (9735) peer connections in the logs, not Bitcoin (8333) ones.
  • Restarting Tor alone may not be enough — the DNS-seed path can be blocked independently.
  • One manual lncli connect to any reliable node fixes it permanently.