Parity Peer Count – How to Optimize

paritypeers

I would like to approx the max amounts of peers my box can handle and whether this max could be too much. Therefore:

a) What is the main machine resource consumed by more peers besides network? CPU? RAM?

b) Is there a downside to having a lot of peers?

Parity//v1.7.0-beta-5f2cabd-20170727/x86_64-linux-gnu/rustc1.18.0

Best Answer

Note that all the machines I run a node on have AES-NI extensions enabled, so I can't speak to whether CPU would be a bottleneck without it.

Having a lot of peers means a lot of network activity which can lead to an increase in latency and bandwidth usage; this is the primary constraint I have when running a high peer count on a VPS (I have a capped monthly transfer limit and the higher my peer count, I think the more likely I am to connect to a peer that needs to download the entire blockchain). The VMs I use for parity have 256 to 512 MiB of RAM and I run into network constraints before RAM constraints. They all have a single virtual core.

At home, I run a few nodes, all behind a firewall (which limits my peer count). One of the machines is a mid-range i3 with parity 1.6 running in a VM (VirtualBox) that is allocated 2 GiB of RAM and one core; it has four spinning disks locally configured to use RAID 10 over ZFS. This machine also is also does light duty as an iSCSI target and a few other things. This system has about 10 GiB of RAM left over for use as the ZFS ARC cache. The main bottleneck on this system is the disk I/O latency speed, even with a large DB cache set for parity. This computer has trouble with 50 connections due to disk access. This is why I built...

Another one of my machines is a lowly G4620 and runs both the Ethereum Classic and the main Ethereum chain using Parity/v1.7.0-unstable-6b16fe3-20170627/x86_64-linux-gnu/rustc1.18.0 in, each, a 2-virtual-core VM (VirtualBox) allocated 6 GiB of RAM. It has a pair of low-end SSDs running in RAID 1 that are de facto dedicated to it. The classic chain has trouble getting up to even 80 peers while the main chain node is pretty happy with a steady 128 peers (128 peer min, 256 peer max). I suspect the difference between the two chains may have to do with the peer discovery process.

I also run both the classic and main chains on Parity/v1.7.0-unstable-4900e9011-20170626/x86_64-macos/rustc1.18.0 on MacOS 10.12.5 with an NVMe drive directly on the host. Unfortunately, I haven't synced in a while, so my computer needs to fully check the chains after doing the snapshot sync and I can't collect more data. However, I do know that I can get a higher peer count when I'm not behind a firewall than when I am (it's a laptop, so I'm also on different physical networks in those situations; unfortunately, I can't rule out that particular aspect as being a driving factor).

I see a large spike in CPU usage whenever a new block comes in; I assume this is parity validating the node and updating the blockchain state and thus does not vary with number of peers. Based on all this, I run into issues with peer discovery due to my firewalling before I hit any system resource constrains except when I'm using a spinning disk. If I had to guess, on a decently spec'ed machine, you'd run into some kind of OS-based limit (such as max open file descriptors) or whatever your OS consumes the most of for network connections before you hit some parity-specific bottleneck.

Theoretical downsides to having a lot of peers include

  • More bandwidth consumed uploading to peers;
  • More latency? (Verifying competing chains?);
  • More TCP connections consumed. This might be a problem for some routers (I've seen a source for GBe home routers, but I can't find it at the moment) and
  • More disk thrashing for HDDs (a very real downside) which decreases your ability to process new blocks leading to latency in accepting new blocks.
Related Topic