Bitcoin Core sync seems to be getting slower every day

DIY, Hacking, & How-To
1

I’m using an old Lenovo Thinkpad X61x from 2007 with a brand new external 2TB SanDisk SSD for the Bitcoin Core data. At first the sync seemed to be fairly reasonable (15% in the first 24h), but now after 3 full days I’m still crawling along at 21%.

I looked at the Bitcoin Core logs and I see no only am I only adding around 7 blocks per minute, but 3 days ago I was doing the same. So the impression of the sync percentage slowing down seems to be due to an inconsistency of how the progress is calculated.

On the other hand, my log says I’m at block 448904, which is above 50% of the current blockchain, which AFAIK ist at block height 803876. So then why does ./services/bitcoind tell me “Progress: 21.87%” ?

Here’s a row from the log entries (progress=0.218710):

2023-08-19T12:26:57+02:00 2023-08-19T10:26:57Z UpdateTip: new best=000000000000000002eae7eff92f872275c1951dca63b311ede7dcab036db7b1 height=448938 version=0x20000002 log2_work=85.841913 tx=188991345 date=‘2017-01-19T11:37:48Z’ progress=0.218710 cache=404.4MiB(2938222txo)

The StartOS metrics say RAM ist around 50%, CPU is around 50%. When I tried syncing a pruned node on the internal HDD, my CPU ran at 95% and syncing was a lot faster. So I’m assuming maybe the bottleneck is the old USB 2.0 port on my Laptop. Any other ideas ?

Thanks

Chris

2

PS: sometimes I see this in the log:

2023-08-19T12:40:51+02:00 2023-08-19T10:40:51Z Socks5() connect to z7a2gum…azqd.onion:8333 failed: host unreachable

But I assume this is isn’t enough to cause a significant decrease in sync spee.

3

one more observation I cannot explain. My bitcoin debug.log file and also the log I get in the web GUI shows the first synced block to be at height=407115. Why not height 0 or 1 ? This would explain how I get to 50% of the blockchain after only 3 days of only 7 blocks per minute, but it doesn’t explain how I got the first 400k blocks! Does Start9 not start the block sync from block 0 ? That would be concerning.

PS: I’m talking about the debug.log file inside /embassy-data/package-data/volumes/bitcoind/data/main

4

Hi @chris

It can take some time to sync the whole chain, and it isn’t dependent entirely on only having an SSD, or only having fast internet. With old hardware it can also be slow going to validate each block. You mention your SSD is external and connected via USB 2.0, which is a valid observation as to why things might be slow. We also have no real way to guarantee there’d not be any unexpected issues with DIY hardware.

Your logs are easier to explain, and aren’t dependent on dozens of factors difficult to determine…

023-08-19T12:26:57+02:00 2023-08-19T10:26:57Z UpdateTip: new best=000000000000000002eae7eff92f872275c1951dca63b311ede7dcab036db7b1 height=448938 version=0x20000002 log2_work=85.841913 tx=188991345 date=‘2017-01-19T11:37:48Z’ progress=0.218710 cache=404.4MiB(2938222txo)

The reported progress of 0.218710 corresponds to the cumulative difficulty of the downloaded blocks. The cumulative difficulty is a measure of the total computational effort expended by the network to mine all the blocks up til then. It’s not a linear measure based on block height, but rather a logarithmic representation of the difficulty.

2023-08-19T12:40:51+02:00 2023-08-19T10:40:51Z Socks5() connect to z7a2gum…azqd.onion:8333 failed: host unreachable

This is just a peer, attempted to be connected to over Tor, not being reachable. It gets ignored. But if you’re downloading blocks only over Tor, and this happens a lot, this could slow things down too. (Check config)

My bitcoin debug.log file and also the log I get in the web GUI shows the first synced block to be at height=407115. Why not height 0 or 1 ? This would explain how I get to 50% of the blockchain after only 3 days of only 7 blocks per minute, but it doesn’t explain how I got the first 400k blocks! Does Start9 not start the block sync from block 0 ? That would be concerning.

The Bitcoin Core service on StartOS is Bitcoin Core, nothing more. Start9 doesn’t make a decision as to what is synced and why. This is decided by your config options, which you should ideally be accessing via the GUI. Likely, in your case, the earlier logs have been wiped, but you might also want to check you’re not running a pruned node without realizing.

On old hardware, it really wouldn’t be strange for the syncing to take many more days.

1 Like
5

This seems to have quite a lot to do with the hardware. I have a Raspberry Pi 4 with 4GB memory and so far in four weeks it has almost hit 40% with the next 60% getting progressively slower I am told. As it is, I am unlikely to see this hardware sync the whole Bitcoin core before 2024!

G

1 Like
6

We highly recommend against the 4GB pi for exactly this reason. We have been unable to discover the exact reason, but continue to investigate.

2 Likes