Mempool on StartOS: Lightning Section Not Updating (Kraken API 403 Error)

[Avranches]

Hello,
I’m running into an issue with my Mempool service on StartOS and I’m hoping someone can point me in the right direction.
The Lightning Network section of my Mempool instance is no longer updating. I can’t see new nodes, and looking up existing node information doesn’t work. The data seems to be completely stale. The rest of the service (blockchain explorer, mining info, etc.) is working perfectly fine.
After checking the service logs, I found these recurring error messages:

ERR: <lightning> Could not connect to https://api.kraken.com/0/public/Ticker?pair=XBTUSD. All 1 attempts failed
WARN: <lightning> Could not connect to https://api.kraken.com/0/public/Ticker?pair=XBTEUR (Attempt 1/1). Reason: Request failed with status code 403

I’m not sure, but is it possible that these logs are connected with the fact that some information in the lightning section doesn’t work?
Has anyone had the same problem and knows how to solve it?

Thanks for your help!

[Jesse]

Those error messages look to be primarily focused on the ticker / price for BTC vs. USD / EUR. So not likely to be the cause of your issue with missing data. I’d look at your Lightning Node logs and Electrum logs (Maybe even Bitcoin logs) for errors there that might be causing issues.

If you haven’t yet, you could just restart those services (and then mempool service) and see if after a restart the information is picked up.

[Avranches]

Thanks for the suggestion! I already tried restarting all the related services, and even the full node itself, but unfortunately nothing changed. Below are some excerpts from the logs you mentioned.

#LND
2025-11-03T17:17:32+01:00  [INF] DISC: GossipSyncer([NODEID_1]): applying gossipFilter(start=0001-01-01 00:00:00 +0000 UTC, end=0001-01-01 00:00:00 +0000 UTC)
2025-11-03T17:17:32+01:00  [INF] DISC: GossipSyncer([NODEID_2]): applying gossipFilter(start=2025-11-03 16:17:32.925821984 +0000 UTC m=+4823.033637753, end=2161-12-10 22:45:47.925821984 +0000 UTC)
2025-11-03T17:18:15+01:00  [INF] GRPH: Processed channels=0 updates=73 nodes=0 in last 59.999877601s
2025-11-03T17:19:07+01:00  [WRN] GRDB: Channel=[CHANNEL_ID_1] not found in graph cache
2025-11-03T17:19:07+01:00  [WRN] GRDB: Channel=[CHANNEL_ID_1] not found in graph cache
2025-11-03T17:19:07+01:00  [WRN] GRDB: Channel=[CHANNEL_ID_2] not found in graph cache
2025-11-03T17:19:07+01:00  [WRN] GRDB: Channel=[CHANNEL_ID_2] not found in graph cache
2025-11-03T17:19:15+01:00  [INF] GRPH: Processed channels=0 updates=121 nodes=38 in last 59.999620782s
2025-11-03T17:20:07+01:00  [WRN] GRDB: Channel=[CHANNEL_ID_3] not found in graph cache
2025-11-03T17:20:07+01:00  [WRN] GRDB: Channel=[CHANNEL_ID_3] not found in graph cache
2025-11-03T17:20:15+01:00  [INF] GRPH: Processed channels=0 updates=70 nodes=3 in last 1m0.000473315s
2025-11-03T17:21:03+01:00  [WRN] GRDB: Channel=[CHANNEL_ID_4] not found in graph cache
2025-11-03T17:21:03+01:00  [WRN] GRDB: Channel=[CHANNEL_ID_4] not found in graph cache
2025-11-03T17:21:15+01:00  [INF] GRPH: Processed channels=0 updates=156 nodes=3 in last 59.999666096s
2025-11-03T17:22:15+01:00  [INF] GRPH: Processed channels=0 updates=183 nodes=29 in last 59.999525976s
2025-11-03T17:22:27+01:00  [WRN] GRDB: Channel=[CHANNEL_ID_5] not found in graph cache
2025-11-03T17:22:27+01:00  [WRN] GRDB: Channel=[CHANNEL_ID_5] not found in graph cache
2025-11-03T17:23:15+01:00  [INF] GRPH: Processed channels=0 updates=88 nodes=15 in last 59.999945249s
2025-11-03T17:24:15+01:00  [INF] GRPH: Processed channels=0 updates=116 nodes=34 in last 1m0.000164866s
2025-11-03T17:24:33+01:00  [INF] HSWC: ChannelLink([CHANNEL_TXID_1]:0): updating commit fee to 299 sat/kw
2025-11-03T17:25:15+01:00  [INF] GRPH: Processed channels=0 updates=148 nodes=18 in last 59.999336547s
2025-11-03T17:25:37+01:00  [WRN] GRDB: Channel=[CHANNEL_ID_6] not found in graph cache
2025-11-03T17:25:37+01:00  [WRN] GRDB: Channel=[CHANNEL_ID_6] not found in graph cache
2025-11-03T17:26:15+01:00  [INF] GRPH: Processed channels=0 updates=54 nodes=44 in last 1m0.000674479s
2025-11-03T17:26:30+01:00  [WRN] GRDB: Channel=[CHANNEL_ID_7] not found in graph cache
2025-11-03T17:26:30+01:00  [WRN] GRDB: Channel=[CHANNEL_ID_7] not found in graph cache
2025-11-03T17:27:05+01:00  [INF] DISC: GossipSyncer([NODEID_1]): filtering chan range: start_height=0, num_blocks=922069
2025-11-03T17:27:05+01:00  [INF] DISC: GossipSyncer([NODEID_1]): sending range chunk of size=4000
2025-11-03T17:27:05+01:00  [INF] DISC: GossipSyncer([NODEID_1]): sending range chunk of size=4000
2025-11-03T17:27:05+01:00  [INF] DISC: GossipSyncer([NODEID_1]): sending range chunk of size=4000
2025-11-03T17:27:06+01:00  [INF] DISC: GossipSyncer([NODEID_1]): sending range chunk of size=4000
2025-11-03T17:27:06+01:00  [INF] DISC: GossipSyncer([NODEID_1]): sending range chunk of size=4000
2025-11-03T17:27:06+01:00  [INF] DISC: GossipSyncer([NODEID_1]): sending range chunk of size=3999
2025-11-03T17:27:06+01:00  [INF] DISC: GossipSyncer([NODEID_1]): sending range chunk of size=4000
2025-11-03T17:27:06+01:00  [INF] DISC: GossipSyncer([NODEID_1]): sending range chunk of size=3994
2025-11-03T17:27:06+01:00  [INF] DISC: GossipSyncer([NODEID_1]): sending range chunk of size=3997
2025-11-03T17:27:06+01:00  [INF] DISC: GossipSyncer([NODEID_1]): sending range chunk of size=4000
2025-11-03T17:27:06+01:00  [INF] DISC: GossipSyncer([NODEID_1]): sending range chunk of size=4000
2025-11-03T17:27:06+01:00  [INF] DISC: GossipSyncer([NODEID_1]): sending range chunk of size=4000
2025-11-03T17:27:06+01:00  [INF] DISC: GossipSyncer([NODEID_1]): sending final chan range chunk, size=2365
2025-11-03T17:27:06+01:00  [INF] DISC: GossipSyncer([NODEID_1]): fetching chan anns for 10 chans
2025-11-03T17:27:08+01:00  [WRN] GRDB: Channel=[CHANNEL_ID_8] not found in graph cache
2025-11-03T17:27:08+01:00  [WRN] GRDB: Node=[NODEID_3] not found in graph cache
2025-11-03T17:27:08+01:00  [WRN] GRDB: Channel=[CHANNEL_ID_7] not found in graph cache
2025-11-03T17:27:08+01:00  [WRN] GRDB: Channel=[CHANNEL_ID_7] not found in graph cache
2025-11-03T17:27:15+01:00  [INF] GRPH: Processed channels=0 updates=117 nodes=35 in last 59.999529528s
2025-11-03T17:28:15+01:00  [INF] GRPH: Processed channels=0 updates=144 nodes=8 in last 59.99986287s
2025-11-03T17:28:47+01:00  [WRN] HSWC: ChannelLink([CHANNEL_TXID_2]:1): insufficient bandwidth to route htlc: 100052100 mSAT is larger than 164230 mSAT
2025-11-03T17:28:47+01:00  [ERR] HSWC: insufficient bandwidth to route htlc
2025-11-03T17:28:47+01:00  [ERR] HSWC: Unhandled error while reforwarding htlc settle/fail over htlcswitch: insufficient bandwidth to route htlc
2025-11-03T17:29:06+01:00  [WRN] BTWL: Received non-standard input sig=[SIGNATURE_DATA], witness=[WITNESS_DATA]
2025-11-03T17:29:06+01:00  [WRN] BTWL: Received non-standard input sig=[SIGNATURE_DATA], witness=[WITNESS_DATA]
2025-11-03T17:29:06+01:00  [WRN] BTWL: Received non-standard input sig=[SIGNATURE_DATA], witness=[WITNESS_DATA]
2025-11-03T17:29:06+01:00  [WRN] BTWL: Received non-standard input sig=[SIGNATURE_DATA], witness=[WITNESS_DATA]
2025-11-03T17:29:06+01:00  [WRN] BTWL: Received non-standard input sig=[SIGNATURE_DATA], witness=[WITNESS_DATA]
2025-11-03T17:29:06+01:00  [WRN] BTWL: Received non-standard input sig=[SIGNATURE_DATA], witness=[WITNESS_DATA]
2025-11-03T17:29:06+01:00  [INF] GRPH: Pruning channel graph using block [BLOCK_HASH] (height=922070)
2025-11-03T17:29:07+01:00  [INF] GRPH: Block [BLOCK_HASH] (height=922070) closed 0 channels
2025-11-03T17:29:07+01:00  [INF] NTFN: New block: height=922070, sha=[BLOCK_HASH]
2025-11-03T17:29:07+01:00  [INF] CNCT: ChannelArbitrator([CHANNEL_TXID_2]:1) no close event
2025-11-03T17:29:07+01:00  [INF] CNCT: ChannelArbitrator([CHANNEL_TXID_3]:1) no close event
2025-11-03T17:29:07+01:00  [INF] CNCT: ChannelArbitrator([CHANNEL_TXID_4]:1) no close event
2025-11-03T17:29:07+01:00  [INF] CNCT: ChannelArbitrator([CHANNEL_TXID_5]:1) no close event
2025-11-03T17:29:07+01:00  [INF] CNCT: ChannelArbitrator([CHANNEL_TXID_1]:0) no close event
2025-11-03T17:29:07+01:00  [INF] CNCT: ChannelArbitrator([CHANNEL_TXID_6]:1) no close event
2025-11-03T17:29:07+01:00  [INF] HSWC: Garbage collected 2 shared secret hashes at height=922070
2025-11-03T17:29:15+01:00  [INF] GRPH: Processed channels=0 updates=53 nodes=47 in last 59.999442323s

#Mempool
2025-11-03T15:58:30+01:00  Lightning tab enabled...
2025-11-03T15:58:30+01:00  Running on LND...
2025-11-03T15:58:30+01:00  [i] mysqld not found, creating....
2025-11-03T15:58:30+01:00  No upgrade needed or already completed.
2025-11-03T15:58:30+01:00  All processes initalized
2025-11-03T15:58:30+01:00  wait-for-it.sh: waiting 720 seconds for localhost:3306
2025-11-03T15:58:30+01:00  2025-11-03 14:58:30 0 [Note] Starting MariaDB 10.4.32-MariaDB-1:10.4.32+maria~deb10
2025-11-03T15:58:30+01:00  2025-11-03 14:58:30 0 [Note] InnoDB: Using Linux native AIO
2025-11-03T15:58:30+01:00  2025-11-03 14:58:30 0 [Note] InnoDB: Initializing buffer pool, total size = 128M
2025-11-03T15:58:30+01:00  2025-11-03 14:58:30 0 [Note] InnoDB: Completed initialization of buffer pool
2025-11-03T15:58:30+01:00  2025-11-03 14:58:30 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=2448853218
2025-11-03T15:58:30+01:00  2025-11-03 14:58:30 0 [Note] InnoDB: 10.4.32 started; log sequence number 2448853227
2025-11-03T15:58:30+01:00  2025-11-03 14:58:30 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2025-11-03T15:58:31+01:00  2025-11-03 14:58:31 0 [Note] mariadbd: ready for connections.
2025-11-03T15:58:31+01:00  Version: '10.4.32-MariaDB-1:10.4.32+maria~deb10'
2025-11-03T15:58:31+01:00  wait-for-it.sh: localhost:3306 is available after 1 seconds
2025-11-03T15:58:33+01:00  Nov 3 14:58:33 [244] NOTICE: <lightning> Starting Mempool Server... (32fadb4)
2025-11-03T15:58:33+01:00  Nov 3 14:58:33 [244] INFO: <lightning> Connected to Electrum Server at electrs.embassy:50001
2025-11-03T15:58:33+01:00  Nov 3 14:58:33 [244] INFO: <lightning> Database connection established.
2025-11-03T15:58:34+01:00  Nov 3 14:58:34 [244] INFO: <lightning> Restoring mempool and blocks data from disk cache
2025-11-03T15:58:34+01:00  Nov 3 14:58:34 [244] INFO: <lightning> Loaded mempool from disk cache in 310 ms
2025-11-03T15:58:34+01:00  Nov 3 14:58:34 [244] INFO: <lightning> Restoring rbf data from disk cache
2025-11-03T15:58:34+01:00  Nov 3 14:58:34 [244] INFO: <lightning> Starting statistics service
2025-11-03T15:58:37+01:00  Nov 3 14:58:37 [244] INFO: <lightning> [Lightning] Starting lightning network sync service
2025-11-03T15:58:37+01:00  Nov 3 14:58:37 [244] NOTICE: <lightning> Mempool Server is running on port 8999
2025-11-03T15:58:37+01:00  Nov 3 14:58:37 [244] INFO: <lightning> 23 blocks since tip. Fast forwarding to the 8 recent blocks
2025-11-03T15:58:37+01:00  Nov 3 14:58:37 [244] INFO: <lightning> Re-indexing skipped blocks and corresponding hashrates data
2025-11-03T15:58:38+01:00  Nov 3 14:58:38 [244] INFO: <lightning> Latest BTC fiat averaged price: {"time":1762181918,"USD":108018,"EUR":93828,"GBP":82333,"CAD":151809,"CHF":87251,"AUD":165116,"JPY":16662217}
2025-11-03T15:58:40+01:00  Nov 3 14:58:40 [244] NOTICE: <lightning> [Mining] Inserted 2 daily historical prices into the db
2025-11-03T15:59:02+01:00  Nov 3 14:59:02 [244] ERR: <lightning> [Lightning] $runTasks() error: Cannot read properties of undefined (reading 'split')
2025-11-03T15:59:02+01:00  Nov 3 14:59:02 [244] INFO: <lightning> [Lightning] Starting Lightning Stats service
2025-11-03T16:00:06+01:00  Nov 3 15:00:06 [244] INFO: <lightning> Latest BTC fiat averaged price: {"time":1762182006,"USD":108020,"EUR":93828,"GBP":82339,"CAD":151912,"CHF":87289,"AUD":165200,"JPY":16660390}
2025-11-03T16:00:06+01:00  Nov 3 15:00:06 [244] NOTICE: <lightning> The mempool is now in sync!
2025-11-03T16:00:06+01:00  Nov 3 15:00:06 [244] INFO: <lightning> Core index 'txindex' is synced. Best block height is 922061
2025-11-03T16:00:06+01:00  Nov 3 15:00:06 [244] INFO: <lightning> Core index 'basic block filter index' is synced. Best block height is 922061
2025-11-03T16:00:07+01:00  Nov 3 15:00:07 [244] INFO: <lightning> [Mining] Indexing 3 blocks from #922061 to #912062
2025-11-03T16:00:08+01:00  Nov 3 15:00:08 [244] NOTICE: <lightning> [Mining] Block indexing completed: indexed 3 blocks
2025-11-03T16:00:09+01:00  Nov 3 15:00:09 [244] INFO: <lightning> [Mining] Indexing blocks prices completed. Indexed 18
2025-11-03T16:00:09+01:00  Nov 3 15:00:09 [244] INFO: <lightning> migrated 0 audits to version 1
2025-11-03T16:38:24+01:00  Nov 3 15:38:24 [244] INFO: <lightning> Starting lightning network forensics service
2025-11-03T16:38:24+01:00  Nov 3 15:38:24 [244] INFO: <lightning> [Lightning] Caching funding txs for currently existing channels
2025-11-03T16:38:24+01:00  Nov 3 15:38:24 [244] INFO: <lightning> Lightning topology folder is not set. Not importing historical LN stats
2025-11-03T16:58:54+01:00  Nov 3 15:58:54 [244] ERR: <lightning> [Lightning] $runTasks() error: Cannot read properties of undefined (reading 'split')
2025-11-03T17:00:03+01:00  Nov 3 16:00:03 [244] INFO: <lightning> Latest BTC fiat averaged price: {"time":1762185604,"USD":105736,"EUR":91769,"GBP":80463,"CAD":148827,"CHF":85401,"AUD":162003,"JPY":16322000}
2025-11-03T17:00:10+01:00  Nov 3 16:00:10 [244] INFO: <lightning> Core index 'txindex' is synced. Best block height is 922066
2025-11-03T17:00:10+01:00  Nov 3 16:00:10 [244] INFO: <lightning> Core index 'basic block filter index' is synced. Best block height is 922066
2025-11-03T17:00:12+01:00  Nov 3 16:00:12 [244] INFO: <lightning> migrated 0 audits to version 1

The logs below only include Mempool and LND data. I didn’t include the Bitcoin Core or Electrs logs since they showed no errors and seemed to be running fine. I’ve redacted all sensitive details (node IDs, channel IDs, IPs, transaction hashes), so if something looks a bit strange, it’s probably because of that. The main error I found is [Lightning] $runTasks() error: Cannot read properties of undefined (reading ‘split’)

Thanks a lot for your help!

[StuPleb]

The only time I’ve seen something similar is when someone had LND installed, with no channels and failing to sync to graph.

Can you confirm the status of your channels in LND and of LND more generally?

[Avranches]

I checked the LND app on my system, and everything looks fine. The node is running, synced to the graph, and the channels all appear to be in good condition. I also checked through Ride The Lightning, and everything looks okay there as well.

[StuPleb]

Well there’s nothing of any use in that snippet of the logs either.

If you want to give it the ultimate “turn if off and on again”, feel free to uninstall it and then install it again. It’ll start over and begin syncing from scratch. If there’s something small in the background that has gone wrong, we’ll see if this makes any difference and if it goes wrong again.

[Avranches]

Got it, thanks! Just to confirm are you referring to reinstalling Mempool or LND?

[StuPleb]

Mempool. Definitely not LND.