A week ago I started a synchronization of a geth node with these parameters:
--fast --cache=1024 --rpc --rpcaddr "0.0.0.0" --ws --wsaddr "0.0.0.0" --wsorigins="*"
The result that I have obtained is that after a week running it has spent 69.9049 GB. The fast synchronization needed a total of 39.1735 GB (More info about Ethereum ChainData Size Growth w/FAST Sync) so the normal use of geth for a week has needed 30.7314 GB.
This table shows the evolution of the free space on the disk at specific points:
| DATE | SIZE(GB) | DIFF (GB) |
|---------------|-------------|-------------|
| 27/12/17 8:30 | 93.23209763 | |
| 28/12/17 8:30 | 46.39630127 | 46.83579636 |
| 29/12/17 8:30 | 41.57238007 | 4.823921204 |
| 30/12/17 8:30 | 36.69866943 | 4.873710632 |
| 31/12/17 8:30 | 31.91919327 | 4.779476166 |
| 1/1/18 8:30 | 27.54598618 | 4.373207092 |
| 2/1/18 8:30 | 25.38702011 | 2.158966064 |
| 3/1/18 8:30 | 23.32711411 | 2.059906006 |
and this is the graph of the evolution:
With these results ~3GB/day (~90GB/month) it seems very difficult to keep a production node synchronized in a VPS without having to perform resynchronizations to clean space or spend a lot of money in disk space
Do you get similar results to me?
Do you know if there is any way to optimize it?
Does geth plan to optimize this behavior?
UPDATE #1:
About the behavior of Geth Péter Szilágyi commented the following on a go-ethereum issue: why my chaindata size up to 400GB?
After it's initial sync, Geth switches to "full sync" where all historical state form that point onward is retained. If you resync, then only the latest state is downloaded. The latest state with the blockchain data is worth about 50GB, but since we don't have state pruning yet, after a sync the data just keeps accumulating.
UPDATE #2:
I'm currently having trouble synchronizing with the most recent block. It seems that the performance has worsened due to the increasing number of transactions in the last days. After one hour running it has only imported 228 blocks:
chain_1 | INFO [01-09|10:05:48] Imported new chain segment blocks=1 txs=256 mgas=7.973 elapsed=15.996s mgasps=0.498 number=4852965 hash=219fed…eed6cc
chain_1 | INFO [01-09|11:05:54] Imported new chain segment blocks=1 txs=338 mgas=7.993 elapsed=28.800s mgasps=0.278 number=4853193 hash=44e3bd…1bb02f
Assuming that a new block is generated every 15 seconds approx. Involve that 240 blocks are generated per hour, which means that every hour the node will be 12 blocks behind the network.
I think the problem is due to the lack of ram because it is using the whole system ram plus 3.00 GB of SWAP
As a conclusion it seems that nowadays a VPS with two processors at 2.2GHz and 4GB of RAM does not seem enough to have a node synchronized with geth running inside a docker.
Does anyone experience a similar behavior?
Do you have the same problems?
NOTE: Detailed characteristics of the node:
Digital Ocean droplet:
# cat /proc/cpuinfo | grep "processor\|model name"
processor : 0
model name : Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
processor : 1
model name : Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
# free -h
total used free shared buff/cache available
Mem: 3.9G 3.7G 112M 108K 85M 26M
Swap: 8.0G 3.7G 4.3G
# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.3 LTS"
# df -klh | grep "^/dev/"
/dev/vda1 58G 18G 41G 31% /
/dev/vda15 105M 3.4M 102M 4% /boot/efi
/dev/sda 99G 83G 12G 89% /mnt/mounted-drive
# docker version
Client:
Version: 17.12.0-ce
API version: 1.35
Go version: go1.9.2
Git commit: c97c6d6
Built: Wed Dec 27 20:11:19 2017
OS/Arch: linux/amd64
Server:
Engine:
Version: 17.12.0-ce
API version: 1.35 (minimum version 1.12)
Go version: go1.9.2
Git commit: c97c6d6
Built: Wed Dec 27 20:09:53 2017
OS/Arch: linux/amd64
Experimental: false
# docker ps | grep geth
f9cdb05cd876 ethereum/client-go:v1.7.3 "geth --fast --cache…" 2 hours ago Up 2 hours 0.0.0.0:8545-8546->8545-8546/tcp, 0.0.0.0:30303->30303/tcp, 30303-30304/udp installpath_chain_1
command:
--fast --cache=1024 --rpc --rpcaddr "0.0.0.0" --ws --wsaddr "0.0.0.0" --wsorigins="*"
Best Answer
In short, when node is syncing it executes transactions, that are program code, that needs to read and write data (variables, arrays etc.), when executes. The data is stored at disk, and read\write accesses are random in general (depens of what code transactions include). As bigger blockchain, as more area to perform random reads\writes. I'm not sure, but I think make syncing slower when blockchain growing. So the first problem of syncing is IO. When I was syncing my node I had about 1.2K read and 0.15K write operations per second in average (SSD). I tryed to use HDD first, but syncing was too slow and I read that many people could not sync their nodes at all with HDD. The problem for me was that in production I have only HDD.
I've solved my problem with two steps:
P.S. I can not say anything abot RAM beacause I have 32GB and did not monitor it. You can give more info about RAM usage by geth process and by docker.