Txpool – Max Size of Transactions Clients Like Geth Can Keep in Txpool

txpool

What is the limit of transactions Ethereum clients can keep in transaction pool?

Geth can keep 64 transactions per address according to this answer, but what is the max size of total transactions in txpool?

Best Answer

Summary

It is limited by your computer's memory.

From my basic testing in What happens when a transaction nonce is too high?, I crashed geth with 400 x 64 transactions with a data payload of 4,500 bytes when running in 4 Gb RAM.

I am assuming that your question is asking what is the maximum number of transactions that can be stored in the transaction pool. If you are asking what is the maximum size of a single transaction, the answer is at Is there a limit for transaction size?.

Details

There is a pool for pending transactions and a queue for transactions that are not ready to be processed.

The data structure representing the transaction pool in geth can be seen at go-ethereum - tx_pool.go, lines 54 to 77:

// TxPool contains all currently known transactions. Transactions
// enter the pool when they are received from the network or submitted
// locally. They exit the pool when they are included in the blockchain.
//
// The pool separates processable transactions (which can be applied to the
// current state) and future transactions. Transactions move between those
// two states over time as they are received and processed.
type TxPool struct {
    config       *ChainConfig
    currentState stateFn // The state function which will allow us to do some pre checks
    pendingState *state.ManagedState
    gasLimit     func() *big.Int // The current gas limit function callback
    minGasPrice  *big.Int
    eventMux     *event.TypeMux
    events       event.Subscription
    localTx      *txSet
    mu           sync.RWMutex
    pending      map[common.Hash]*types.Transaction // processable transactions
    queue        map[common.Address]map[common.Hash]*types.Transaction

    wg sync.WaitGroup // for shutdown sync

    homestead bool
}

From stackoverflow.com - Maximum number of elements in map, the maximum number of elements that can be stored in a Go map data structure is 2147483647 for 32-bit architecture and 9223372036854775807 for 64-bit architecture.

Pending Transaction Pool

Pending transactions are stored in TxPool.pending that is a map(transaction hash -> pointer to transaction). In go-ethereum - tx_pool.go, there is no limit check to the number of transaction when adding transactions to this data structure.

The number of transactions in the pending transaction pool is effectively limited by memory.

Transaction Queue

Queued transactions are stored in TxPool.queue that is a map(from address -> map(transaction hash -> pointer to transaction)). In go-ethereum - tx_pool.go, there is a check that the number of transactions from the same From address does not exceed maxQueued = 64.

The number of transactions in the transaction queue is effectively limited by memory, with the limitation that there can only be a maximum of 64 transactions for transactions with the same From address.

What is the difference between a pending transaction and a queued transaction?

Pending transactions are transactions that are ready to be processed and included in the block.

Queued transactions are transactions where the transaction nonce is not in sequence. The transaction nonce is an incrementing number for each transaction with the same From address.

For example:

Transaction from account 0xaaaa...aaaa with nonce 0 has been included in the blockchain.
Transaction from account 0xaaaa...aaaa with nonce 1 has been included in the blockchain.
10 transactions from account 0xaaaa...aaaa with nonces 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 as sent to an Ethereum node. These are placed in the transaction queue as the transaction from account 0xaaaa...aaaa with nonce 2 has not been seen by the Ethereum node.
Once the transaction from account 0xaaaa...aaaa with nonce 2 is added to the transaction pool, the 10 transactions with nonces 3, 4, 5, 6, 7, 8, 9, 10, 11 and 12 will be moved from the queue to the pending transaction pool and all 11 transactions are ready to be processed and inserted into into the blockchain (provided there is enough gas).

Related Solutions

[Ethereum] How to configure max txpool memory size

Geth

The default config values associated with the transaction pool can now be changed from Geth's CLI:

TRANSACTION POOL OPTIONS:
  --txpool.pricelimit value    Minimum gas price limit to enforce for acceptance into the pool (
default: 1)
  --txpool.pricebump value     Price bump percentage to replace an already existing transaction (default: 10)
  --txpool.accountslots value  Minimum number of executable transaction slots guaranteed per account (default: 16)
  --txpool.globalslots value   Maximum number of executable transaction slots for all accounts (default: 4096)
  --txpool.accountqueue value  Maximum number of non-executable transaction slots permitted per account (default: 64)
  --txpool.globalqueue value   Maximum number of non-executable transaction slots for all accounts (default: 1024)
  --txpool.lifetime value      Maximum amount of time non-executable transaction are queued (default: 3h0m0s)

While this doesn't allow you to limit exactly how much memory is used, increasing, for example, accountslots or globalslots will lead to an increase in used memory, and vice versa.

Parity

Similarly, Parity has the following CLI option:

 --tx-queue-size LIMIT          Maximum amount of transactions in the queue (waiting
                                 to be included in next block) (default: 1024).

Transactions Propagation – Maximum Safe Transactions Per Second

here is how you handle this issue in a practiced and proven way.

First, before planning your organization to be too large-scale, please make yourself familiar with the characteristics of the Ethereum blockchain. Transactions per second is a bad measure, as the critical part here instead is the block and the time it takes to mine it. Also consider the block gas limit, which is around 4.7 Million gas right now. Depending on the gas costs of your transactions, and even ignoring all other users, there are only so many transactions you can fit in a block - the others have to wait. And here comes the technique of concurrency into play.

As we are dealing with async computing here, simply firing hundreds of transactions will make most of them just fail, as you correctly noticed. Therefore, you set a limit for the number of transaction promises being generated, and do not create another one until at least one has resolved. Compare it to juggling: Not more than X balls at once in the air, or it will end in a mess. Technically, this is being done with the concurrency feature of the Promise.map function in the Bluebird Promise Library. The number you set for the concurrency limit depends on many factors, prepare to make it variable and adjust it to the state of the network.

Have fun.