[Ethereum] Geth takes more than 26 Gb in –fast mode!

Summary

I downloaded the geth source, modified the source code to specify the fast sync pivot block, compiled the code, removed the old chaindata and started the fast syncing. Once this is complete, I'll be back to running the regular geth binaries.

UPDATE This experiment failed. There were a few different errors with my hack that prevented the blockchain to fast sync to the specified block and then normal sync after the specified block. Back to full archive node sync.

Anyone has any suggestions?

Details

I downloaded the source code for geth and modified the source code for section that calculates the fast sync pivot point eth/downloader/downloader.go, lines 419-441:

case FastSync:
    // Calculate the new fast/slow sync pivot point
    if d.fsPivotLock == nil {
        pivotOffset, err := rand.Int(rand.Reader, big.NewInt(int64(fsPivotInterval)))
        if err != nil {
            panic(fmt.Sprintf("Failed to access crypto random source: %v", err))
        }
        if height > uint64(fsMinFullBlocks)+pivotOffset.Uint64() {
            pivot = height - uint64(fsMinFullBlocks) - pivotOffset.Uint64()
        }
    } else {
        // Pivot point locked in, use this and do not pick a new one!
        pivot = d.fsPivotLock.Number.Uint64()
    }
    // If the point is below the origin, move origin back to ensure state download
    if pivot < origin {
        if pivot > 0 {
            origin = pivot - 1
        } else {
            origin = 0
        }
    }
    glog.V(logger.Debug).Infof("Fast syncing until pivot block #%d", pivot)

I modified the last line above to change the Debug into Info and added the following two lines below the code above:

    glog.V(logger.Info).Infof("Fast syncing until pivot block #%d", pivot)
    if (pivot >= 2394190) {
      pivot = 2394190;
    }
    glog.V(logger.Info).Infof("Fast syncing until modified pivot block #%d", pivot)

I recompiled and started off the fast sync process using the modified binaries:

Iota:go-ethereum user$ make geth
...
Done building.
Run "build/bin/geth" to launch geth.

I checked the version of the modified geth:

Iota:go-ethereum user$ build/bin/geth version
Geth
Version: 1.5.3-unstable

I removed the old damaged chaindata:

Iota:go-ethereum user$ build/bin/geth removedb
/Users/bok/Library/Ethereum/chaindata
Remove this database? [y/N] y

Removing...
Removed in 35.242291ms

I started the fast sync:

Iota:go-ethereum user$ build/bin/geth --fast --cache=1024 console
I1120 23:44:44.870142 ethdb/database.go:83] Allotted 1024MB cache and 1024 file handles to /Users/user/Library/Ethereum/geth/chaindata
I1120 23:44:44.878926 ethdb/database.go:176] closed db:/Users/user/Library/Ethereum/geth/chaindata
...
I1121 08:33:51.340811 eth/downloader/downloader.go:441] Fast syncing until pivot block #2664150
I1121 08:33:51.340847 eth/downloader/downloader.go:445] Fast syncing until modified pivot block #2394190

After the fast syncing is complete, I'll go back to using the regular geth binaries.

(1) What's the geth default blockchain sync type when installing via command line tools? Full?

Just tested with geth 1.5.6, the default is full.

(2) Do my settings for geth automatically transfer over to Mist -- and visa versa? (e.g., if I'm running geth --fast will Mist also refer to the same size blockchain?)

Yes, mist uses (in most cases) geth as Ethereum node, so if you run geth --fast, mist will work in fast mode. However, if you stop a geth --fast node, and restart it, it will resume in full mode as far as I remember. This means, after shutting down geth, and starting mist, it will start a full node. But that terminology is misleading in some cases, and you should probably read on here:

• What is Geth's “fast” sync, and why is it faster?

(3) Is it possible to run Mist in "light" mode? I saw from this answer that Mist can be in "full" or "fast" sync but didn't see "light". I tried it, and it looks like I'm not getting any errors yet, but curious if there are known issues.

Light client was just very recently released and you should expect hiccups. The same goes for mist if you use a geth node in light mode. As if Ethereum Stack Exchange isn't awesome already, check out this post:

• What is Geth's “light” sync, and why is it so fast?

(4) Is it possible to have multiple copies of the blockchain on my computer? For example, if I first set up geth using geth -full and then I run geth -light, will the light version overwrite the full version or will I need additional space to sync light?

Yes, that is possible. Running a full node, i.e.,

$ geth             #full node (default),

creates a full copy of the blockchain in ~/.ethereum/geth/chaindata/.

Running a light node, i.e.,

$ geth --light     #,

creates a directory for the state in ~/.ethereum/geth/lightchaindata/. To run both clients at the same time, you need some additional adjustments such as IPC path, ports, etc.

• How to run two nodes on the same device?

However, if you want to run a --fast sync, this only works on the first run of geth. If you already synced the full chain, you will get a message like this if you run geth in fast mode:

I0112 21:09:00.024747 eth/handler.go:119] blockchain not empty, fast sync disabled

If you insist on keeping a full and a fast copy of the blockchain on the same device, you can use the --datadir switch.