[Ethereum] How to manually configure the geth fast sync pivot block

fast-syncgo-ethereumsynchronization

WARNING – Running Ethereum Wallet 0.8.7 without manually starting geth 1.5.2 will end up clobbering your blockchain data!

UPDATE Nov 22 2016 – New geth 1.5.2 binaries will be downloaded by Ethereum Wallet / Mist (source) so this problem should go away. You will have to restart Ethereum Wallet / Mist for the new binaries to be downloaded.

I have been using the "Cry Uncle" 1.5.2-stable geth version which includes the latest Hard Fork No. 4: Spurious Dragon.

I started the current latest Ethereum Wallet 0.8.7 which packages geth 1.4.18-stable – see mist/clientBinaries.json, lines 39-55:

"mac": {
  "x64": {
    "download": {
      "url": "https://bintray.com/artifact/download/karalabe/ethereum/geth-1.4.18-stable-c72f545-darwin-10.6-amd64.tar.bz2",
      "type": "tar",
      "sha256": "1f7ac168a4502a9e88474f74b3cef2dda4843f350d7f9fcdd9ef10dff30b7282",
      "bin": "geth-1.4.18-stable-c72f545-darwin-10.6-amd64"
    },
    "bin": "geth",
    "commands": {
        "sanity": {
        "args": ["version"],
        "output": [ "Geth", "1.4.18" ]
      }
    }
  }
},

As I was not running geth 1.5.2 manually then, Ethereum Wallet started geth 1.4.18 and this ended up clobbering my chain data as stated in the geth 1.5.2 release notes:

Database Upgrade

The 1.5.0 release changes the structure of the blockchain database. Geth will upgrade the database
during normal operation, but you cannot revert to the previous 1.4.x releases. If you
do want to revert, you'll need to keep a backup of the chaindata directory or resync.

I then had to decide whether to download the fast sync blockchain, but I need the debug.traceTransaction(...) capabilities from block #2,394,190 as I am using this information for some transaction analysis, but fast syncing exclude this information until a very recent block (currently 2661841).

My blockchain on my mining node is a full archive node and is currently at 87 Gb and I do not want to use this amount of space on my notebook:

user@Rasterbator:~$ du -hs .ethereum/chaindata/
87G .ethereum/chaindata/

How do I fast sync the Ethereum blockchain up until a specified block?

Best Answer

Summary

I downloaded the geth source, modified the source code to specify the fast sync pivot block, compiled the code, removed the old chaindata and started the fast syncing. Once this is complete, I'll be back to running the regular geth binaries.

UPDATE This experiment failed. There were a few different errors with my hack that prevented the blockchain to fast sync to the specified block and then normal sync after the specified block. Back to full archive node sync.

Anyone has any suggestions?


Details

I downloaded the source code for geth and modified the source code for section that calculates the fast sync pivot point eth/downloader/downloader.go, lines 419-441:

case FastSync:
    // Calculate the new fast/slow sync pivot point
    if d.fsPivotLock == nil {
        pivotOffset, err := rand.Int(rand.Reader, big.NewInt(int64(fsPivotInterval)))
        if err != nil {
            panic(fmt.Sprintf("Failed to access crypto random source: %v", err))
        }
        if height > uint64(fsMinFullBlocks)+pivotOffset.Uint64() {
            pivot = height - uint64(fsMinFullBlocks) - pivotOffset.Uint64()
        }
    } else {
        // Pivot point locked in, use this and do not pick a new one!
        pivot = d.fsPivotLock.Number.Uint64()
    }
    // If the point is below the origin, move origin back to ensure state download
    if pivot < origin {
        if pivot > 0 {
            origin = pivot - 1
        } else {
            origin = 0
        }
    }
    glog.V(logger.Debug).Infof("Fast syncing until pivot block #%d", pivot)

I modified the last line above to change the Debug into Info and added the following two lines below the code above:

    glog.V(logger.Info).Infof("Fast syncing until pivot block #%d", pivot)
    if (pivot >= 2394190) {
      pivot = 2394190;
    }
    glog.V(logger.Info).Infof("Fast syncing until modified pivot block #%d", pivot)

I recompiled and started off the fast sync process using the modified binaries:

Iota:go-ethereum user$ make geth
...
Done building.
Run "build/bin/geth" to launch geth.

I checked the version of the modified geth:

Iota:go-ethereum user$ build/bin/geth version
Geth
Version: 1.5.3-unstable

I removed the old damaged chaindata:

Iota:go-ethereum user$ build/bin/geth removedb
/Users/bok/Library/Ethereum/chaindata
Remove this database? [y/N] y

Removing...
Removed in 35.242291ms

I started the fast sync:

Iota:go-ethereum user$ build/bin/geth --fast --cache=1024 console
I1120 23:44:44.870142 ethdb/database.go:83] Allotted 1024MB cache and 1024 file handles to /Users/user/Library/Ethereum/geth/chaindata
I1120 23:44:44.878926 ethdb/database.go:176] closed db:/Users/user/Library/Ethereum/geth/chaindata
...
I1121 08:33:51.340811 eth/downloader/downloader.go:441] Fast syncing until pivot block #2664150
I1121 08:33:51.340847 eth/downloader/downloader.go:445] Fast syncing until modified pivot block #2394190

After the fast syncing is complete, I'll go back to using the regular geth binaries.

Related Topic