I’m experiencing issues with opening LND, and the log error suggests the problem lies with LND itself, not StartOS. The error message is: “LTND: unable to open graph DB: ReadVarBytes; sig is larger than the max allowed size [count 179, max 80]”.
After reaching out to the Lightning Labs team, they recommended using chantools_compactdb. I installed chantools and attempted to run the command:
However, this resulted in an error: “Error: error opening source DB: open /root/embassy-data/package-data/volumes/lnd/data/main/data/graph/mainnet/channel.db: no such file or directory”.
To troubleshoot, I used the command sudo find / -name channel.db (from the default SSH location, not root), which returned the path /embassy-data/package-data/volumes/lnd/data/main/data/graph/mainnet/channel.db. This suggests that channel.db is indeed located in the expected location.
I’m unsure where I’m going wrong. Should I not be executing chantools from the root user? Any assistance would be appreciated.
Has anyone out there successfully run chantools? If so, please confirm if you execute chantools commands as root (sudo /usr/lib/startos/scripts/chroot-and-upgrade).
I found that, after using the (sudo /usr/lib/startos/scripts/chroot-and-upgrade) script, it makes linux all weird and makes it not be able to find the /embassy-data/package/volumes/lnd/data/main/data/graph/mainnet/channel.db file.
So after using that script, I exit the startOS and exit SSH.
I go back in and use the following commands sudo -i curl -L https://github.com/lightninglabs/chantools/releases/download/v0.12.2/chantools-linux-amd64-v0.12.2.tar.gz | tar -zxv -f- --strip-components=1 -C /usr/local/bin
(you might need a slightly different URL for a different chantools version and/or for different OS/processor).
Then run the chantool compactdb command with: chantools compactdb \ --sourcedb /embassy-data/package-data/volumes/lnd/data/main/data/graph/mainnet/channel.db \ --destdb /embassy-data/package-data/volumes/lnd/data/main/data/graph/mainnet/compacted.db
What I’m stuck on now though is what do I do next? How do I get LND to use this newly compacted file named “compact.db” instead of the original un-compacted file “channel.db”?
Is the best way by simply renaming both files and “trick” LND into using the newly compacted file?
Right now it’s considering the newly compacted file as successfully compacted.
Now it’s back to trying to sync the rest of the graph.
What service log output lines should I be expecting to see when it has proper/healthy syncing? Even if it takes a whole week from here, I would love to at least have a peace of mind that it’s making syncing progress.
Thanks for letting me know! Well that knocks out “compacting” as a suspect.
Just googling “syncing to chain and graph takes a long time”, I get the following response:
Understanding “Syncing to Chain and Graph Long Time” in Blockchain Nodes
When you see a message like “Syncing to chain and graph long time” when running a blockchain node, it generally means your node is in the process of downloading and verifying the entire history of the blockchain, also known as the ledger. This process involves fetching block data from other nodes in the network and validating each block and its transactions to ensure consistency and integrity.
Why Does it Take So Long?
Several factors contribute to the time it takes to sync a blockchain node:
Size of the blockchain: As blockchains grow and gain more users and transactions, the amount of data the node needs to download and verify increases, leading to longer sync times.
Hardware and network capabilities: The speed of your processor (CPU), disk (especially using an SSD instead of an HDD), and network connection significantly impact syncing speed.
Syncing method: Different sync methods exist, with varying levels of efficiency. For example, syncing from the genesis block requires processing all historical data, while checkpoint sync allows a node to quickly sync to a recent state.
Peers: The number and speed of peers your node connects to also play a role in how quickly it can download blockchain data.
Common Reasons for Delays
Slow Internet Connection: An unstable or slow internet connection can hinder the download and verification of blockchain data.
Insufficient Resources: Lack of adequate CPU power or insufficient disk space (especially with HDDs) can slow down the syncing process.
Firewalls and Antivirus: Firewalls and antivirus software can sometimes block necessary ports or connections, causing syncing issues.
Node Overload: A node that’s being overloaded with too many transactions can struggle to keep up with syncing.
Software Incompatibility: Using outdated software or encountering issues during software updates can lead to sync failures.
Potential Solutions to Speed Up Syncing
Improve Internet Connectivity: Ensure a stable and fast internet connection for your node.
Use an SSD: Upgrade from an HDD to an SSD for faster data access and verification.
Adjust Cache Size: If applicable, increase the cache size in your node’s configuration to allow more data to be stored in RAM, reducing disk I/O.
Increase Peer Count: Manually adding well-connected peers can help improve syncing speed.
Use Checkpoint Sync: If supported by the blockchain protocol, use checkpoint sync to speed up the initial sync process.
Consider Node Snapshots: Some blockchain projects offer node snapshots that can drastically reduce sync time.
Periodically Restart Your Node: This can help resolve temporary syncing issues and improve peering.
I don’t think “slow internet connection” is the issue. i’m at 275 Mbps download and 161 Mbps upload per speed test just now.
I don’t think “insufficient resources” is the issue since the Start9 Pure is pretty strong.
“Firewalls and Antivirus” might be an issue. I have to confess I don’t know much about this at all. How can I make sure that I have all firewalls and antivirus configured properly using StartOS or bash?
“Node overload” might be an issue. I had over 10 pending closure transactions that didn’t quite close before all this started.
“Software incompatibility” is not likely a problem since both my Bitcoin Core and LND are latest versions on Start9 Marketplace and they state in the version notes that they are compatible with each other.
None of these are 100% valid because your syncing isn’t taking a “long time”, it’s just not happening. There are two potential reasons why…
You have no peers or channels
You have no internet connection
If you’re saying your 100+ channels are ALL ONLINE and you can make payments, and yet it’s still syncing, I’m not sure I’ll ever have an answer for you unless perhaps I can see the logs.
The last time it was synced (more than 2 weeks ago) and I opened thunderhub, I had 100+ channels.
Of course, it’s hard to tell now because I can’t even open up thunderhub anymore since LND is not synced. Can you show me how I can check if I have peers even if LND doesn’t sync and thunderhub can’t start?
Is it possible to add peers without LND fully synced? Can you show me?
No, right now, I cannot make LN payments.
My bitcoin core is running well on the same start9 server, so I just assumed the internet is working.
It won’t let me upload text files here (and I can’t copy and paste because logs are too long) I’ll sent you the logs via email.
There’s nothing here that jumps out at me, other than the fact that it looks like it takes a really really long time for Bitcoin Core to report back that it’s fully synced after a restart. The only question that brings up for me is… is this under-powered hardware? I don’t think we’ve discussed the hardware you’re using.
With a Pure, you’d have no issues with resources, or with 100 channels.
I’m at a bit of a loss at this stage. We can do StartOS support, and part of that is making sure LND runs on StartOS… the nitty gritty of LND support is more Lightning Labs team’s space. Have they been able to suggest anything else? If they do, I’d be able to show you how to do it on StartOS.
Are you on Telegram? If you look me up there, I can probably suggest a couple of other resources where someone might be able to help: Telegram: Contact @StuPleb
I am not the lightning expert here, and would trust StuPleb on this one, but one simple thing is to try and reset your Tor connection or make sure that Tor is not having issues before you bail on it
If all your channels are connected and online, you should be able to trigger cooperative closes just fine. Definitely try this first, it’s cheaper and less scary.