Backup Restore LND issue, potential lost channel and sats after upgrading LND

General
1

Hello,

I had a hardware failure but luckily I made a network backup of my Start9 and restored it on a new device. All of my lightning channels closed except for 1 which was in the Thunderhub Channels Pending tab as my channel partner was offline. While I was waiting for my channel partner to come back online or for the force close, I stupidly decided to upgrade from LND 0.19.1 to LND 0.19.3 and this is where I suspect my problem came from. Fast forward a week or so and I was able to contact my channel partner and they got back online and closed our channel with many confirmations on chain now. However, I have not received any additional sats in my on-chain balance.

Before the LND upgrade I could see all of my recently closed channels and the pending one in ThunderHub. After the upgrade, in Thunderhub my on-chain balance was still there and the chain transaction history going back years is still there but no channel history in closed or pending tabs. I confirmed this with the lncli listchannels and lncli pending channels commands which showed nothing. In Amboss.space the last update received from my node was before the LND upgrade. Before I got in contact with my channel partner, I tried using lncli to force close the channel manually but it said channel not recognized. Finally after the channel closed I did the LND Reset Wallet Transactions command and after completion it still showed the same amount of sats as before.

Now that the missing sats are on-chain but not recognized by my node I’m worried to do anything drastic. For the next step, my thoughts are to sweep my current on-chain LND balance to a cold wallet, uninstall LND, then restore LND from the network backup. Is this a good idea at this point or does anyone have any other suggestions? Thank you!

2

Some aspect of the upgrade was likely the initiation of a series of fluke occurrences. That there was a pending close, or a channel with an offline partner (I’m not sure which from your text) shouldn’t mean anything should go wrong with the upgrade itself. Something clearly happened though, with this as the starting point.

So let’s work with what we know.

You say there’s NO channel history now after the upgrade, confirmed with lncli. What I’m not clear about is whether there are on-chain funds and on-chain history of if those are all gone too.

If not, it makes me wonder whether you actually somehow have a fresh install of LND. I suppose the easiest way to tell would be to look at your pubkey/nodeID on Amboss when you look up your node (which interestingly shows it being offline) and then comparing it to the node ID inside Thunderhub. Is it the same? Is it different?

If on the other hand you do have funds on chain that have not changed from before and after the upgrade, then it’s possible that you did run into some kind of a bug. But it’s certainly not a common one or one I’ve seen before. We could look into it and direct you to the LND team for further help.

Now, a bit of a disclaimer… I can’t tell you what to do with regards to your funds, you’re fully responsible for your choices and actions. I can simply tell you what should be the case after taking certain actions, and what I would do if I was in your shoes.

And in my view, regardless of which of the two situations you’re facing above (and I can’t know if you’re facing a third situation that I haven’t envisioned), the only copy of the information you need is in that older pre-upgrade backup.

So, in theory, you could restore that backup, request force closes and hopefully get everything you need back in order to monitor the return of funds and then to move those funds to safety.

In particular and with some safety steps (again this is not recovery advice)…

  1. Make a new fresh backup of what you currently have to a new separate location.
  2. Uninstall LND
  3. Restore the old backup to your server to get your channel DB and history back

This should let you see your channel with this peer who abandoned their node. It won’t help you get the funds back from that node unless they ever come back online. But you’ll at least be able to monitor them.

Hopefully this helps at least a little.

3

Hey Stu thank you for responding,

Yes I believe this is a bug with LND because my Node ID and pubkey still match in Amboss and on Thunderhub. In Thunderhub it still shows my on-chain funds from all of the other channel closures and the chain history going back years. However it does not show any channels under any tab (open, pending, and closed), its all blank. Before upgrading LND, Thunderhub showed the pending channel closure and under closed tab it showed all of my recently closed channels. After the upgrade it doesn’t show anything for channels.

Also my channel partner did come back online and the channel successfully closed but my node has not recognized the closed channel, UTXO or chain history of that closing transaction. However, it does show closed in Amboss.space and confirmed in mempool.space. Now those sats are stranded in an address that my node doesn’t see for some reason.

Disclaimer from me… I understand and accept that I am fully responsible for every action that I take and I will not hold you or any one else in this forum responsible if I lose any funds no matter what is said or recommended.

That being said my first concern is sweeping the significant on-chain balance I currently have to cold storage. I have not done that yet as I’m not sure if that would interfere with restoring the old backup (ie if there would be an error or something because it’s expecting to see on-chain balances for all of the successful closures.)

I am inclined to think this wouldn’t happen and that it would be safe to sweep the funds before restoring, but I don’t want to do anything else until I at least have some input from some more technical and experienced plebs like yourself. I appreciate your thoughts. Also what’s the best way to get in touch with LND about this potential bug? I’d hate for any more plebs to have to go through this as well.

Thanks again!

4

If you’re keen on taking it to the LND team, you’d probably want to preserve your current state and post here after trying and failing to find a similar case.

If instead the priority is removing funds from this instance of LND and starting over… yes, I would say you can sweep the bulk of your funds elsewhere now. I’d leave just a bit behind so LND doesn’t complain. After that, I’d follow the steps I recommended earlier, making a new backup to a DIFFERENT location not overwriting your old good backup then restoring the old good backup. This should now seen the channels, see the resulting closes, and hopefully the onchain balance. It should see the missing balance, once everything is synced, and should also see your sweep out once it catches up.

Lots of shoulds, but that’s my understanding and should be right.

1 Like